Walk-forward testing for systems that trade in seconds

[Fairman]

Walk-forward testing for systems that trade in seconds

Walk-forward testing means optimizing on one window of data, trading the next window blind, then sliding forward and repeating. It mimics how a strategy gets used in real life: fit on the past, run on the future.

For short-horizon systems, window length is the tricky choice. Too short, and the optimization chases noise. Too long, and it averages across regimes that no longer apply.

A reasonable starting point is a few weeks of tick data for fitting and one week for testing, rolled forward in weekly steps. That gives dozens of independent out-of-sample periods over a year.

What matters is how stable the results are across those periods. A strategy that makes money in 40 of 52 test weeks is more convincing than one that earns a huge profit in three weeks and loses in the rest.

Also watch how much the optimal parameters drift from window to window. If the best stop distance jumps from 3 pips to 9 and back, the system is fitting noise, and the live version will wander too.

For scalping systems, how wide are the training windows, and do the chosen parameters stay stable?

[Amanhasnoname]

I use a slightly longer training window than you suggest, roughly six weeks of tick data, testing on the following week and rolling forward weekly. Shorter windows kept handing me parameters that worked brilliantly for one week and then fell apart, which is the noise-chasing you describe. Longer than about two months and the system stopped adapting to volatility changes at all.

Parameter stability became my main pass or fail test. I chart the chosen value of each parameter across all the test windows. If a take-profit setting bounces around randomly, I fix it at a sensible middle value and stop optimising it. Most of my systems ended up with only two or three parameters that are actually allowed to move.

One thing I'd add for second-level systems is to include realistic costs in every test window, not just the final check. I once had a system pass forty-odd walk-forward weeks on raw prices, then lose money in most of them once I added a variable spread model. That was a painful but useful afternoon.