A backtest that looks ahead by one bar: the shift(1) fix, shown on six close prices

[Fairman]

A backtest that looks ahead by one bar: the shift(1) fix, shown on six close prices

The most common bug in vectorised backtests is trading on a signal using the same bar that produced it. Here are six closes and a toy rule: be long if the bar's close is above the previous close.

Code: Select all

close = pd.Series([1.0850, 1.0856, 1.0852, 1.0861, 1.0858, 1.0866, 1.0860])
ret = close.diff() * 1e4              # pips per bar
sig = (close.diff() > 0).astype(int)
bad = (sig * ret).sum()               # +23.0 pips
good = (sig.shift(1) * ret).sum()     # -13.0 pips

In the bad version, the signal at bar 1 says the close rose by 6 pips, and the same 6 pips are booked as profit. Up bars give 6, 9 and 8, so 23 pips from three trades with a hundred percent win rate. That's what a leak looks like.

With shift(1) the signal can only be acted on during the next bar. The three positions then earn minus 4, minus 3 and minus 6, a total of minus 13 pips. The strategy went from a genius to a loser by one line.

The check I use now is to shuffle the future: change the last bar's close to a random value and confirm the position and result on earlier bars don't change. If they do, something is reading ahead. Another test is to compare results from running on a series cut at bar N with the same series run on all data, looking at the decisions up to bar N.

If a result looks too good, assume a leak before you assume an edge.

[Fairman]

Indicators can leak too. A rolling max or min computed with the current bar included lets a breakout rule compare a close with a high that contains that same close. For a "close above the 20 bar high" breakout, the 20 bar high has to be calculated on the previous bars: high.rolling(20).max().shift(1). Without the shift, a close can never exceed a high that already includes it, so the rule would never trigger. The symptom isn't always a profit. It can be zero trades, which people often read as an overly strict filter.

[Fairman]

Higher timeframe data is a subtler leak. Suppose you resample M1 to H1 and then merge the H1 close back onto the M1 data. An M1 bar at 10:15 would see the close of the 10:00 hour bar, which isn't known until 11:00. The fix is to shift the higher timeframe series by one H1 bar before merging, so that at 10:15 you can only see the 09:00 bar's close, the last one that had finished by then. Some multi-timeframe results change a lot when this single shift is applied, which tells you what they were measuring.

[Fairman]

Execution realism is the next step after removing the leak. With shift(1) the trade happens at the next bar, but you still need a price. The cleanest choice is the next bar's open, plus half the spread for the side you cross. For the example above the next open is almost the same as the last close, so I used close to close, but on real data the difference can matter. Make the choice explicit in code, as a named function, so any change in the fill assumption is visible in the diff.

[Fairman]

A small habit that helps: print the first ten rows of the frame side by side, with the close, the signal, the position and the return, and read them with a pencil. For the six closes above, row 2 should show a signal of 1 at 1.0856, a position of 0 on that same row, and a position of 1 on the row after. If a position appears on the same row as its signal, you have the bug. Reading ten rows takes two minutes, and it finds mistakes that a hundred line test suite can miss.