forex-scalping.com

Curve Fitting: How Indicator Settings Become a Trap

You test an indicator with the usual settings, and it loses money. You change the period from 14 to 11, and suddenly the equity curve looks beautiful. This is not a discovery. It is curve fitting.

Curve fitting means tuning parameters until a strategy matches past data. With enough knobs, any strategy can look brilliant on history, because you are fitting noise, not signal. The settings that worked best on the past often fail on new data.

The warning signs are familiar. The strategy performs well only at very specific settings. A small change, like moving a period by one, ruins the results. The rules have many conditions that seem to fit particular past events. And the number of trades in the test is small.

A robust strategy works across a range of settings. If periods from 10 to 20 all give similar results, the idea probably captures something real. If only 13 works, it is likely luck.

Good habits include limiting the number of parameters, keeping the logic simple and explainable, and testing on data not used for the design.

Remember that more complexity does not equal more edge. Often it equals more overfitting.

Practical step: take any strategy you are testing, and run it with five neighboring parameter values. If results swing wildly, do not trust it.

One parameter that gets curve-fitted without people noticing is the time filter. "Only trade between 08:15 and 10:40" looks like a session rule, but if the window was found by trying start and end times until the backtest improved, it's the same problem as changing 14 to 11. The window often ends up neatly excluding a handful of bad trades that happened to land on release days.

The neighbouring-values test works here too. Shift the window by fifteen or thirty minutes either side and see whether the results hold. If you want a time filter that survives, base it on something with a reason behind it, like excluding a fixed window around scheduled releases, rather than whatever start time scored best.

A test I'd suggest: if changing one parameter from 14 to 13 or 15 makes the results collapse, the setting isn't trustworthy. Settings worth trusting tend to hold up when you nudge them a little either way. They don't depend on one magic number. I'd also limit the number of parameters overall. A strategy with two settings is much harder to curve fit than one with eight. And always keep part of your data aside, maybe the most recent year, and only test on it once at the very end. If the results there are much worse than in the data you optimised on, the strategy was fitted to the past and probably won't hold up live.

Out-of-sample testing is the other half. Optimise on one chunk of data, then test on a period you haven't touched. If results fall apart on the new data, the settings were just memorising the past. A simple split is to use the first 70 percent of your data for testing and keep the last 30 percent aside until the end. Look at how results change with nearby settings too. If a 14 period setting works but 13 and 15 lose money, it's fragile. Settings worth trusting show similar results across a range of nearby values.