Python backtesting libraries: the traps behind the convenience
Posted: Wed Oct 28, 2026 6:47 am
Several Python libraries make backtesting easy: vectorised frameworks that process whole arrays at once, and event-driven ones that simulate bar by bar. Both can produce beautiful equity curves in a few lines of code. Beauty is not validity.
Vectorised backtests are fast but make look-ahead mistakes easy. A signal computed on a bar's close and traded at the same bar's close isn't realistic. Shifting the signal by one bar is a common fix, and forgetting it is a classic error.
Event-driven frameworks mimic live trading more closely, handling order types, fills and slippage models, but they're slower and more complex.
Both depend on data quality, the cost model and assumptions about fills. Default settings, like zero commission or zero slippage, produce flattering results.
Checks that help: reproduce a handful of trades by hand, test with deliberately pessimistic costs, run a shifted-signal test to detect look-ahead, and compare against a simple benchmark.
A library is a tool for organising the work, not a source of truth.
A short list of sanity checks before believing any result: does the number of trades look plausible? Is the average holding time consistent with the design? Do the largest wins and losses make sense when inspected on a chart? Are costs included and set to realistic values? Does shifting the signal by one bar change the result dramatically?
Five questions, ten minutes, and they filter out the majority of flattering mistakes.
Which backtesting framework is used, and what assumptions were overridden from the defaults?
Vectorised backtests are fast but make look-ahead mistakes easy. A signal computed on a bar's close and traded at the same bar's close isn't realistic. Shifting the signal by one bar is a common fix, and forgetting it is a classic error.
Event-driven frameworks mimic live trading more closely, handling order types, fills and slippage models, but they're slower and more complex.
Both depend on data quality, the cost model and assumptions about fills. Default settings, like zero commission or zero slippage, produce flattering results.
Checks that help: reproduce a handful of trades by hand, test with deliberately pessimistic costs, run a shifted-signal test to detect look-ahead, and compare against a simple benchmark.
A library is a tool for organising the work, not a source of truth.
A short list of sanity checks before believing any result: does the number of trades look plausible? Is the average holding time consistent with the design? Do the largest wins and losses make sense when inspected on a chart? Are costs included and set to realistic values? Does shifting the signal by one bar change the result dramatically?
Five questions, ten minutes, and they filter out the majority of flattering mistakes.
Which backtesting framework is used, and what assumptions were overridden from the defaults?