Backtest results that vanish when real spread is applied

[Fairman]

Backtest results that vanish when real spread is applied

A common story: the backtest looks beautiful, the equity curve rises smoothly, and the live results fall apart. Often the cause is a fixed spread assumption in the test. Real spreads vary, widening at rollover, during news, and in thin hours.

A system trading five or six times an hour will hit these widened windows regularly. Each trade at a wider spread carries a hidden penalty that the test never recorded.

The remedy is using variable spread data in the backtest, ideally actual historical spread from tick data for the account type being used. If that is unavailable, apply a stress test: double the spread, then triple it, and see how the equity curve responds. If profit disappears at 1.5 times the average spread, the system lacks robustness.

Also test session by session. A strategy that makes money only in the London session with tight spreads should be restricted to that window.

Costs decide scalpers' fate, and the test should reflect it honestly.

At what spread multiple does the system in your testing stop being profitable?

[Fairman]

My range scalper on EUR/USD stopped being profitable at about 1.6 times the average spread. The breakout version survived up to around 2.2 times. That told me the range scalper had a thin edge and the breakout one had more room. I only run the breakout one live now. The range scalper looked better at 1 times spread, which is exactly why fixed spread testing is dangerous: it ranks strategies in the wrong order. So I'd suggest always ranking strategies at a realistic spread multiple, not at the average, since the average flatters systems with a thin edge.

[Amanhasnoname]

I found this out with a small scalper I built on GBPJPY. On a fixed spread of 2 pips, the test showed a profit factor of 1.6 over six months, which looked wonderful. When I loaded variable spread data from my broker, the profit factor dropped to 1.05, and in the hours between 9pm and 1am it was clearly below 1.

The trades during the wide spread window were only about 11 percent of the total, but they produced almost a third of the losses, because my target was only 6 pips and the spread was sometimes 4 or 5 at that time. A blocking filter that skipped entries when the spread exceeded 2.5 pips brought the profit factor back to about 1.3.

I now always run three versions of any test: average spread, 1.5 times the average, and the real variable data. If the middle one does not stay above 1.1, I discard the idea without looking at the equity curve any further.