Advertisement IC Markets

Python backtesting libraries: the traps behind the convenience

Discuss 1-minute to 15-minute price action setups, fading intraday momentum, key support/resistance zones, and proven short-term trading methodologies.
Post Reply
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Python backtesting libraries: the traps behind the convenience

Post by Shadow Trader »

Several Python libraries make backtesting easy: vectorised frameworks that process whole arrays at once, and event-driven ones that simulate bar by bar. Both can produce beautiful equity curves in a few lines of code. Beauty is not validity.

Vectorised backtests are fast but make look-ahead mistakes easy. A signal computed on a bar's close and traded at the same bar's close isn't realistic. Shifting the signal by one bar is a common fix, and forgetting it is a classic error.

Event-driven frameworks mimic live trading more closely, handling order types, fills and slippage models, but they're slower and more complex.

Both depend on data quality, the cost model and assumptions about fills. Default settings, like zero commission or zero slippage, produce flattering results.

Checks that help: reproduce a handful of trades by hand, test with deliberately pessimistic costs, run a shifted-signal test to detect look-ahead, and compare against a simple benchmark.

A library is a tool for organising the work, not a source of truth.

A short list of sanity checks before believing any result: does the number of trades look plausible? Is the average holding time consistent with the design? Do the largest wins and losses make sense when inspected on a chart? Are costs included and set to realistic values? Does shifting the signal by one bar change the result dramatically?

Five questions, ten minutes, and they filter out the majority of flattering mistakes.

Which backtesting framework is used, and what assumptions were overridden from the defaults?
Recommended broker for automated trading & scalping IC Markets
FXS Prop Desk
Posts: 1690
Joined: Wed Oct 07, 2026 9:52 am

Re: Python backtesting libraries: the traps behind the convenience

Post by FXS Prop Desk »

Shadow Trader wrote: Shifting the signal by one bar is a common fix, and forgetting it is a classic error.
The look-ahead trap in vectorised tests is easy to miss: a signal on the bar's close, filled at the same close. Shifting signals by one bar before computing returns is the standard fix, and checking that results change after the shift is a quick sanity test.
FXS Official Team | The Idea Lab & Strategy
Amanhasnoname
Posts: 2060
Joined: Fri Oct 30, 2026 9:45 am

Re: Python backtesting libraries: the traps behind the convenience

Post by Amanhasnoname »

I learned this the hard way with a vectorised library when I was testing a moving average pullback idea on GBP/USD. The curve looked wonderful, close to a straight line, and I was ready to put real money behind it. Out of curiosity I rebuilt about a dozen of the trades by hand on the chart and found that the entries were using the close of the same candle that produced the signal. Once I lagged the signal properly, the edge shrank to almost nothing.

The other default that caught me was the spread. The library assumed a fixed tiny spread around the clock, so all the late evening trades looked profitable when in reality they were taken in the widest part of the day.

I now use an event-driven framework for anything I plan to trade live, set commission to what my broker actually charges, and add an extra half pip of slippage to every fill. If the strategy still survives that, I take it seriously.
Post Reply