forex-scalping.com

The backtest that was too good

A scalping system showed a smooth equity curve across three years of one-minute data. Win rate above 70 percent. Tiny drawdowns. The kind of curve that makes a developer nervous.

The test was rerun on tick data. The curve flattened and then sloped down.

What had happened? The bar-based engine assumed that when both the target and the stop fell within the same candle, the target was hit first. On three-pip targets and two-pip stops, a huge share of trades had both levels inside a single bar. Each of those was counted as a win.

Real ticks showed that about half of them hit the stop first.

It is a simple error and an extremely common one. Strategies with tight stops and targets are the most vulnerable, because they depend on the order of events inside bars.

The check takes an afternoon. Take a sample of trades from the backtest, find the ones where target and stop sat inside the same bar, and look at the real tick path for each.

How many trades in the backtest had both levels inside one bar, and what did the tick path say?

After a few weeks of live trading, a simple analysis pays off. Take each live trade and find where the backtest would have placed it. Compare entry price, exit price, and result.

The differences fall into categories. Entry slippage: the live fill was worse than the signal price. Exit slippage: the stop or target fill differed. Missed trades: signals that never became orders because of rejections or latency. Phantom trades: positions the live account opened that the backtest didn't.

Add them up and the gap between the backtest's equity curve and the live one becomes explainable. A scalper might discover that most of the gap comes from a handful of hours or a specific order type.

The point isn't to be disappointed by the difference. Almost every strategy shows one. The point is to measure it, since each component can be addressed separately: better routing, different order types, filters for bad hours.

A spreadsheet with a row per trade and columns for each difference is enough. After a hundred trades, patterns jump out.

Sometimes the difference is a systematic bias. If live results are consistently worse by about 0.4 pips per trade, a cost has been missed. If they are worse mostly during certain hours, a liquidity issue is likely. If they are worse only on winning trades, something is cutting profits short, such as a spread widening at the exit.

Those clues point to specific fixes. A cost issue can be added to the model. A liquidity issue can be addressed by filtering hours. An exit problem can be handled by changing order types.

It's also worth noting the opposite case. If live results are better than the backtest, the cause deserves scrutiny, since good surprises can mean a problem in the test.

A weekly review with the same layout each time, difference in entry, difference in exit, missed trades, and extra trades, builds the habit and exposes drift as soon as it starts.

Which component of the gap between backtest and live trading turned out to be the biggest?