Overfitting: how tuning parameters creates fantasy results

[Shadow Trader]

You test a setup using a fifteen candle lookback for swings and get mediocre results. You change it to seventeen, and the results improve. Then you tweak the buffer, the stop distance, and the session filter until the equity curve looks beautiful. The problem is that you have probably fit the rules to the noise of that specific sample. When you trade live, the noise is different, and the performance disappears. To avoid overfitting, I limit the number of adjustable parameters, ideally to three or four. I choose values that make logical sense rather than the ones that maximize past profit. And I check whether neighboring values give similar results. If a lookback of fifteen works but fourteen and sixteen fail badly, the result is fragile. A robust setup shows a plateau of acceptable performance across a range of values. Finally, I reserve a portion of the data that I never touch until the end, and test the final rules on it once. If the performance collapses, I know the earlier results were an illusion.

[FXS HongKong Scalper]

My limit is three tunable parameters per setup. Beyond that, the backtest starts finding the past rather than an edge. A quick check I use: nudge each parameter by 10 to 20 percent. If the result collapses, the original values were fitted to noise.