When Optimisation Becomes Curve Fitting

[Shadow Trader]

Every optimiser will find a combination of inputs that looks brilliant on historical data. With enough parameters and enough runs, some setting will always fit past noise. The question is whether that setting describes a repeatable behaviour or merely memorises one stretch of price history.

Warning signs are easy to list. A very high number of tunable inputs relative to the number of trades, a single sharp peak in the results surface, and settings with oddly specific values such as a 13.7 period all suggest memorisation. Rules that make economic sense, such as a session filter tied to liquidity, tend to need fewer arbitrary numbers.

One practical discipline is to count degrees of freedom before testing. Each added filter, threshold or time window consumes some of the data's evidential value. A modest number of trades spread across many parameters leaves very little to support any conclusion.

Preferring broad plateaus over isolated peaks helps. A system whose results degrade gently when inputs shift slightly is more plausible than one that works only at exact values. Simpler logic with fewer inputs usually generalises better, even if the headline backtest looks less spectacular.

[FXS HongKong Scalper]

The 13.7 period example is spot on. Another warning sign we see often is a strategy that only works in one specific year of the test. If the profit came mostly from 2022, when the dollar trended strongly, the optimiser has learned "buy dollars in 2022".

A practical step: split results by year and by session before trusting the total. A parameter set that is slightly profitable in every year is usually more robust than one with one great year and four flat ones. The total may look the same, but the second one is much more likely to fail live.