Walk-forward testing optimises a strategy on one period and tests it on the next, then rolls forward. The window lengths affect the result.
Considerations:
In-sample length: long enough for enough trades (e.g. 200+), short enough to reflect current conditions.
Out-of-sample length: typically 20–30% of in-sample.
Number of steps: more steps give a more reliable picture.
Example: 6 months in-sample, 2 months out-of-sample, rolled forward monthly over three years.
What the result shows: whether parameters chosen on past data still work on unseen data. A big gap between in-sample and out-of-sample results suggests overfitting.
A simple extra check is to record which parameter set was chosen in each window. If it changes completely from window to window, the strategy may be fitting noise; if it stays similar, the edge is more likely to be real.
What window lengths do you use for walk-forward testing?
For scalping systems with a few trades a day I have settled on shorter windows than your six and two month example.
My current setup is 8 weeks in-sample and 2 weeks out-of-sample, rolled every 2 weeks. That gives enough trades on an M5 EURUSD system, usually 180 to 250 tickets in the in-sample block, without stretching so far that London session behaviour from last quarter dominates.
Where I push back on the 20 to 30% out-of-sample rule is when the strategy is session specific. Two weeks of London opens can be distorted by one CPI week. In that case I keep the out-of-sample at 2 weeks but I require three consecutive passing windows before I trust a parameter set.
Recording which parameters were chosen each window is the check that caught my worst overfit. One version of my spread filter flipped between 0.3 and 0.7 every window. That was noise. The version that stayed near 0.4 survived.
Do you optimise continuous parameters inside each window, or only choose from a small fixed grid?
FXS Prop Desk wrote: Example: 6 months in-sample, 2 months out-of-sample, rolled forward monthly over three years.
One thing to watch with that example. If the out-of-sample block is two months but the window rolls forward one month at a time, each out-of-sample month gets tested twice by two different parameter sets. Stitching those results together double counts the data and makes the combined out-of-sample curve look smoother than it really is. Rolling by the same length as the out-of-sample block, so 2 months here, keeps every month tested once. Over three years with a 6 month in-sample that still gives 15 clean out-of-sample windows, which is a reasonable number. The parameter tracking idea at the end is a good one too, especially if you plot the chosen values over time.
Fairman's warning on overlapping out of sample months is the fix I'd lock in first. I already roll by the same length as my 2 week holdout, but now I know it was right for a reason and not by luck.
For M1 and M5 scalps I'd also add a short embargo between the end of training and the start of the holdout. Three trading days is enough for me, and it stops trades that straddle the boundary from leaking the training period into the test.
The other number I watch is how many windows I end up with. A year of data with an 8 week training block and a 2 week roll gives about 22 holdout blocks, and I want at least 20 before I read anything into the stitched curve.
Fairman's idea of plotting the chosen values over time is worth doing literally. I chart each parameter as a line across windows, and a set that drifts slowly in one direction tells me something different from one that jumps back and forth.
FXS Prop Desk, do you leave any gap between training and holdout, or run them back to back? The embargo has been the quieter but more useful change for me.
Walk-forward testing mimics real-life use of a strategy. Parameters are optimized on one period, then tested on the following unseen period. The window then moves forward and the process repeats, producing a series of out-of-sample results.
For scalping, the approach is especially valuable because market microstructure changes over time. Spreads, volatility, and liquidity patterns shift, and parameters tuned to one regime may lose effectiveness in another.
A typical setup might optimize on three months of data and test on the following month, rolling forward by a month at a time. The combined out-of-sample results approximate what live trading would have produced, with periodic re-optimization. If performance holds across many windows, confidence rises. If it collapses outside the optimization period, the strategy is likely overfitted. Walk-forward analysis also helps decide how often parameters should be updated. Frequent re-optimization is not always better, and testing different intervals reveals the best balance.
The extra effort is considerable, but the reward is a far more honest estimate of how a short-timeframe system is likely to behave when real money is involved.
Fairman wrote: Rolling by the same length as the out-of-sample block, so 2 months here, keeps every month tested once.
Correct, the example as written double counts, thanks for spotting it. If someone still wants monthly re-optimisation for live use, a workable compromise is to re-fit monthly for trading but build the evaluation curve only from non-overlapping windows, so each month appears in the result exactly once.
That compromise makes sense to me. Re-fitting monthly for live but scoring only on non-overlapping windows keeps the live behaviour honest without counting any month twice. The part I'd still want to see is how much the parameters move between those monthly re-fits. If the best lookback jumps from 20 to 55 and back, I don't care how clean the stitched curve looks, because the system is basically chasing noise and the next fit won't carry over.
A quick example of what I mean: 12 months of data, 2 month out-of-sample blocks, so six test windows. If five of them are mildly positive at around +1R each and one is -4R, the total is +1R, which looks flat but hides that one regime killed it. I'd read each window on its own before looking at the sum.
Do you set a minimum number of trades per in-sample window before you trust a fit? I've been using around 150 and wonder if that's too low for a scalper.