Overfitting: When a Strategy Is Built to Explain the Past

[Fairman]

Overfitting: When a Strategy Is Built to Explain the Past

Overfitting happens when a strategy is tailored so closely to historical data that it captures noise instead of real patterns. It looks excellent in testing and fails in live trading.

How does it occur? You test an idea, see losses, and add a filter. You add another rule for a specific bad period. You adjust the settings until the equity curve smooths out. Each tweak seems reasonable, but together they carve a strategy around the quirks of the sample.

The more parameters and rules you have relative to the number of trades, the greater the risk. A strategy with ten adjustable settings tested on 80 trades has essentially memorized them.

Signs of overfitting include sensitivity to small parameter changes, rules that lack a plausible explanation, an unusually smooth equity curve, and sharp deterioration on new data.

Defenses are straightforward. Keep strategies simple with few parameters. Prefer rules with a logical reason. Test on multiple pairs and time periods. Hold back part of the data that you never look at while designing, and test on it at the end.

Expect live results to be worse than backtested results. A common rule of thumb is to assume a meaningful haircut.

If a result looks too good, it probably is.

Practical step: count the number of adjustable rules and parameters in your strategy, and compare with your number of tested trades.

[Fairman]

Easy check for this: nudge every setting 10 to 20 percent each way and rerun. A real idea gets a bit worse. An overfit one falls off a cliff.

[Fairman]

The trap for me was always "just one more filter" after a bad month. Now any new rule has to make sense on its own before I look at whether it improves the numbers, not after.