How to Tell If a Backtest Is Lying to You: Overfitting, Look-Ahead Bias & More

A backtest can show a beautiful equity curve and still be worthless. The chart goes up and to the right, the return looks great, and the strategy falls apart the moment you trade it live. The backtest was not wrong. It was lying, and you let it.

This guide covers the ways a backtest deceives you, overfitting, look-ahead bias, ignored costs, and cherry-picked periods, and how to tell if a backtest is lying to you before you fund it.

Lie 1: Overfitting

Overfitting is the most common and most seductive. You tweak the rules, the thresholds, the periods, until the strategy fits the past almost perfectly. The backtest looks incredible. It is also useless.

A strategy tuned to every wiggle of historical data has memorised the past, not learned a rule. When new data arrives that does not match, the edge vanishes.

How to catch it:

• Be suspicious of perfection. A near-flawless equity curve is a red flag, not a green one.

• Count the parameters. The more knobs you turned to get the result, the more likely it is fitted.

• Test on unseen data. Tune on one period, then test on a different one the strategy has never seen.

Lie 2: Look-Ahead Bias

Look-ahead bias is when a strategy uses information it could not have known at the time. It is easy to introduce by accident and it makes results look magical.

Classic examples: acting on a daily closing price before the day has closed, or using a signal calculated from data that only became available later. The backtest quietly trades with tomorrow's information.

How to catch it:

• Ask what the strategy knew, and when. Every decision must use only data available at that exact moment.

• Distrust suspiciously smooth wins. If the strategy always seems to enter just before a move, check your data timing.

• Use a platform that models timing correctly, so signals are evaluated on completed bars.

Lie 3: Ignoring Costs

A strategy that looks profitable at zero cost can be a loser once real trading fees are applied. High-frequency strategies are especially vulnerable, because many small trades accumulate many small fees.

How to catch it:

• Always include fees. A backtest without them is a fantasy, not a test.

• Watch the trade count. The more trades, the more fees erode the result.

On CoinQuant, backtests include trading fees by default, so the result reflects real costs rather than an idealised one.

Lie 4: The Cherry-Picked Period

A strategy tested only on a bull market will look brilliant. The same strategy in a bear market may be a disaster. Testing one favourable window is one of the easiest ways to fool yourself.

How to catch it: test across regimes, a bull phase, a bear phase, and a sideways stretch. A strategy that survives all three is robust. One that only wins in a single regime is a bet on that regime.

Lie 5: The Metric That Hides the Risk

This one is subtle because the numbers are real. A strategy can show a genuine positive return and a high win rate while hiding a punishing drawdown.

A real example from CoinQuant makes it concrete. A simple RSI strategy on Bitcoin, daily, 2022 to 2026, produced these honest numbers:

HTML

HTML

Read only the return and win rate and this looks like a winner. Read the drawdown and Sharpe and it looks nearly untradeable. The backtest did not lie. The trader who ignored half the metrics lied to themselves.

The Honest-Backtest Checklist

Before you trust any backtest, yours or an AI agent's, run it through this:

• Fees are included.

• The result holds on data the strategy was not tuned on.

• It was tested across bull, bear, and sideways periods.

• The drawdown is one you could actually endure.

• The sample of trades is large enough to mean something.

• No decision uses information from the future.

A strategy that passes all six is worth considering. One that fails any is a story, not a strategy.

The Takeaway

Backtests do not lie on their own. They mislead when you overfit, peek at the future, skip costs, cherry-pick the period, or read only the flattering metric. Test honestly, across regimes, with fees, and read every number. The goal is not a pretty curve. It is a result you can actually trade.

Run honest backtests on CoinQuant