Walk-forward testing for strategies with very small edges

[FXS London Scalper]

Walk-forward testing is usually explained with strategies that have a clear edge. For scalping systems where the edge per trade is a fraction of a pip after costs, I find it much harder to interpret.

The problems:

• Noise dominates. In a three-month out-of-sample window, the difference between a real edge of 0.05R and zero can easily be hidden by variance.
• Cost assumptions decide the result. A change of 0.1 pip in assumed spread can flip a window from positive to negative.
• Parameter stability matters more than performance. If optimal parameters jump around from window to window, that tells you more than the P&L does.

What I do now:

1. Long in-sample windows (12 months) and shorter out-of-sample (3 months), rolled forward.
2. Costs set slightly worse than my live average.
3. I track how often the out-of-sample result is positive, not the total.
4. I look at the parameter surface: a smooth plateau is good, a single sharp peak is a warning.

If 70% or more of out-of-sample windows are positive with conservative costs, I consider live testing at tiny size. Below that, back to the drawing board.

How do others judge tiny-edge systems in walk-forward? Any statistical tests you find useful for this?