Overfitting looks like skill until the market changes
Posted: Fri Oct 30, 2026 12:50 pm
A strategy has 14 parameters. After testing thousands of combinations, one set produces a beautiful equity curve. It feels like discovery.
More likely, the optimisation found the noise in the data and memorised it. The parameters fit the past precisely, and the future does not resemble the past precisely.
Signs of overfitting include extreme sensitivity, where a small change in one parameter wrecks the results, a very large number of rules for a small number of trades and performance that varies wildly between years.
Defences exist. Keep the rule set simple. Use out-of-sample data, reserving 30 percent of history that the optimisation never saw. Check that nearby parameter values also produce reasonable results, forming a plateau instead of a spike.
A robust strategy tends to look slightly mediocre in testing but holds up in live conditions. A stunning backtest is often a warning.
How many parameters does the main strategy use, and was it tested on unseen data?
More likely, the optimisation found the noise in the data and memorised it. The parameters fit the past precisely, and the future does not resemble the past precisely.
Signs of overfitting include extreme sensitivity, where a small change in one parameter wrecks the results, a very large number of rules for a small number of trades and performance that varies wildly between years.
Defences exist. Keep the rule set simple. Use out-of-sample data, reserving 30 percent of history that the optimisation never saw. Check that nearby parameter values also produce reasonable results, forming a plateau instead of a spike.
A robust strategy tends to look slightly mediocre in testing but holds up in live conditions. A stunning backtest is often a warning.
How many parameters does the main strategy use, and was it tested on unseen data?