forex-scalping.com

Plan a Walk-Forward Test Over Three Years of Data

You have 36 months of data and a strategy with one parameter you want to optimize. You choose a walk-forward design: optimize on 12 months, then test on the next 3 months, then slide forward 3 months.

Count the windows. The first optimization uses months 1 to 12, and the test uses 13 to 15. The second optimization uses months 4 to 15, and the test uses 16 to 18. Each step moves forward 3 months. The last test must end at month 36, so the last optimization ends at month 33, covering months 22 to 33, with the test on 34 to 36.

The number of windows is the number of 3-month steps from the start: the first test begins at month 13, the last at month 34. That is 13, 16, 19, 22, 25, 28, 31, 34: eight windows.

Out-of-sample coverage: eight tests of 3 months, which is 24 months of results never used for parameter choice. The first 12 months are used only for the first optimization.

What do you look at? The combined out-of-sample result, and the stability of the chosen parameter. If the best parameter is 14 in some windows and 31 in others, the optimization is chasing noise.

The cost: fewer trades per window. If the strategy takes 5 trades a month, each test window has about 15 trades, which is small, so judge the combined 120 trades, not each window.

Try this: sketch the window diagram for your own data length.

The count works out to eight windows: 24 months available for testing after the first 12, in steps of 3. That gives 24 months of out-of-sample results in total, which is the real benefit of the design.

One thing we always check after a walk-forward run is how much the chosen parameter jumps between windows. If it goes 10, 35, 12, 40, the optimization is mostly fitting noise even if the combined out-of-sample curve looks fine. A stable parameter, or at least a stable region, is more convincing than a good total.

Also be careful with the very first trades of each test window when the strategy uses indicators with long lookbacks. They need warm-up data from the optimization period, otherwise the first days of each test are calculated on incomplete values.

The window count is correct, and laying it out explicitly avoids a common mistake. Test windows start at months 13, 16, 19, 22, 25, 28, 31, and 34, which gives eight windows and 24 months of out of sample results. Each optimisation uses only the 12 months before its test, so no test window ever sees data used to choose its parameter. That's the core discipline of walk forward testing. Getting the boundaries wrong, for example letting an optimisation window overlap its own test period by a month, would leak information and make the results look better than they should.

I ran my first walk forward test on a breakout system with a rolling window like yours and found the window count was the least interesting part. The surprise was the parameter chosen in each optimization. For a channel length I got 18, 42, 20, 55, 19 and 48 across six windows, and the combined out of sample curve looked pretty decent even though the choice was jumping around. That told me I was mostly fitting noise.

What helped was changing the question from which value is best to which region is stable. I took the optimization results as a table, found the range where profit stayed above about eighty percent of the peak in most windows, and fixed the parameter in the middle of that range. The out of sample result dropped a little but I trusted it far more.

I also kept a spreadsheet with the trade count for each window. Three of mine had under ten trades, so I judged the combined total only, as you suggest, and I did not read anything into a single bad quarter.

Parameter stability is the most revealing output. If the chosen parameter stays in a narrow range across all eight windows, say between 18 and 22, the strategy's behaviour is consistent, and the optimisation is finding something real. If it jumps around, from 14 to 31 and back, the optimisation is fitting noise in each window, and the out of sample results will likely be poor. Plotting the chosen parameter for each window on a simple chart makes the pattern obvious. A stable parameter is often more reassuring than a good combined result, since the result might still be luck.

The small sample per window is a real constraint. Fifteen trades per test window gives very noisy results, so individual windows will vary a lot even if the strategy is sound. Some will look excellent and some poor. Judging the combined 120 out of sample trades, as the post says, is the right approach. With a standard deviation of 1.5R, 120 trades give a standard error of about 0.14R, which is enough to detect an edge of 0.3R or more with reasonable confidence. Smaller edges would need more data, either a longer history or more trades per month.

An alternative design uses an anchored walk forward, where each optimisation window starts at month 1 and grows over time, rather than sliding. That uses more data for later optimisations, which can give more stable parameters, but it gives equal weight to old and recent data. The sliding window adapts faster to changing conditions but uses less data each time. Both are valid, and comparing them can be informative. Has anyone here run a walk forward test on their own strategy? I'd be interested in how stable the parameter was and how the out of sample result compared with the full period optimisation.