Advertisement IC Markets

Curve Fitting: How Indicator Settings Become a Trap

Share, develop, and backtest custom MQL4/MQL5 Expert Advisors, Python data-scraping scripts, trading bots, and automated market alert systems.
Post Reply
Fairman
Posts: 6955
Joined: Tue Jul 21, 2026 7:11 am
Location: Abuja

Curve Fitting: How Indicator Settings Become a Trap

Post by Fairman »

Curve Fitting: How Indicator Settings Become a Trap

You test an indicator with the usual settings, and it loses money. You change the period from 14 to 11, and suddenly the equity curve looks beautiful. This is not a discovery. It is curve fitting.

Curve fitting means tuning parameters until a strategy matches past data. With enough knobs, any strategy can look brilliant on history, because you are fitting noise, not signal. The settings that worked best on the past often fail on new data.

The warning signs are familiar. The strategy performs well only at very specific settings. A small change, like moving a period by one, ruins the results. The rules have many conditions that seem to fit particular past events. And the number of trades in the test is small.

A robust strategy works across a range of settings. If periods from 10 to 20 all give similar results, the idea probably captures something real. If only 13 works, it is likely luck.

Good habits include limiting the number of parameters, keeping the logic simple and explainable, and testing on data not used for the design.

Remember that more complexity does not equal more edge. Often it equals more overfitting.

Practical step: take any strategy you are testing, and run it with five neighboring parameter values. If results swing wildly, do not trust it.
It’s Fairman :geek:
Recommended broker for automated trading & scalping IC Markets
LondonNewsTrader
Posts: 421
Joined: Mon Sep 21, 2026 9:30 am

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by LondonNewsTrader »

One parameter that gets curve-fitted without people noticing is the time filter. "Only trade between 08:15 and 10:40" looks like a session rule, but if the window was found by trying start and end times until the backtest improved, it's the same problem as changing 14 to 11. The window often ends up neatly excluding a handful of bad trades that happened to land on release days.

The neighbouring-values test works here too. Shift the window by fifteen or thirty minutes either side and see whether the results hold. If you want a time filter that survives, base it on something with a reason behind it, like excluding a fixed window around scheduled releases, rather than whatever start time scored best.
Fairman
Posts: 6955
Joined: Tue Jul 21, 2026 7:11 am
Location: Abuja

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by Fairman »

A test I'd suggest: if changing one parameter from 14 to 13 or 15 makes the results collapse, the setting isn't trustworthy. Settings worth trusting tend to hold up when you nudge them a little either way. They don't depend on one magic number. I'd also limit the number of parameters overall. A strategy with two settings is much harder to curve fit than one with eight. And always keep part of your data aside, maybe the most recent year, and only test on it once at the very end. If the results there are much worse than in the data you optimised on, the strategy was fitted to the past and probably won't hold up live.
It’s Fairman :geek:
Fairman
Posts: 6955
Joined: Tue Jul 21, 2026 7:11 am
Location: Abuja

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by Fairman »

Out-of-sample testing is the other half. Optimise on one chunk of data, then test on a period you haven't touched. If results fall apart on the new data, the settings were just memorising the past. A simple split is to use the first 70 percent of your data for testing and keep the last 30 percent aside until the end. Look at how results change with nearby settings too. If a 14 period setting works but 13 and 15 lose money, it's fragile. Settings worth trusting show similar results across a range of nearby values.
It’s Fairman :geek:
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by Shadow Trader »

The neighbouring values test gets more convincing when the results are drawn as a curve across the whole range rather than read off a table. A spike and a plateau look very different, and the plateau is often the one with the lower best figure.

In the made up chart below, version A hits a profit factor of 1.62 at period 13, while 12 and 14 give 0.94 and 1.01. Version B never goes above 1.25 but stays between 1.15 and 1.25 for every period from 10 to 20.
img_12938_r1.png
img_12938_r1.png (75.35 KiB) Viewed 275 times
Version A is the one an optimiser report puts at the top, and version B is the one more likely to survive live data. The search itself adds to the problem.

Test 100 combinations of two settings on a strategy with no edge at all, and the best one will typically sit around two and a half standard deviations above the average purely by luck. So the more combinations tried, the higher the bar the winner has to clear on the held out data.

Do you count the combinations tried and write that number next to the result? It is a simple habit that makes it much harder to fool yourself later.
FXS Prop Desk
Posts: 1689
Joined: Wed Oct 07, 2026 9:52 am

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by FXS Prop Desk »

Plateau testing is the best defence. If the strategy only works with a period of 11 and fails at 10 and 12, the result is probably noise. If it works across a range of values, the edge is more likely real.

Testing on data the optimiser never saw is the other key check. A curve-fitted setting usually falls apart quickly on new data.
FXS Official Team | The Idea Lab & Strategy
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by Shadow Trader »

Another check that catches fitted strategies is how they react to higher costs. A version tuned on history often has a small average trade, and a small edge breaks fast when spread or slippage gets worse.

Say a scalping strategy makes 2.6 pips gross per trade and pays 1.0 pip in spread and commission, so it nets 1.6 pips. Add 0.5 pip a trade, which is normal on a busy news week, and the net drops to 1.1, losing about 31 percent of the edge.

At 1.0 pip extra only 0.6 is left. The chart below draws that made up decline step by step.
img_12938_r2.png
img_12938_r2.png (41.47 KiB) Viewed 248 times
A useful habit is to run every finalist at base cost, plus half a pip and plus one pip. If it only survives at the cheapest spread the test ever saw, live trading will find the weakness quickly.

It also helps to check where the profit comes from. If most of it sits in trades under 3 pips, the result depends on fills more than on the idea.

Does anyone here set a minimum average trade size before trusting a backtest? Something like three times the round trip cost would rule out a lot of fragile results.
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Re: Curve Fitting: How Indicator Settings Become a Trap

Post by Shadow Trader »

Gripe time. Someone posts an EA with a stop of 13.7 pips, a take profit of 8.3, an RSI period of 11 and a start hour of 7. The equity curve looks like a ruler. Everyone claps.

Those odd numbers are the giveaway. The optimizer ran thousands of combinations over one stretch of history and kept the winner, which means it kept the settings that fit that stretch's noise. Nudge the stop to 14 and the curve sags. That's curve fitting, and it's why so many EAs die in the first month live.

A sturdier set of settings has a neighbourhood. If stops of 10, 12 and 14 pips all give similar results, the edge probably isn't riding on one lucky number. If only 13.7 works, it's coincidence wearing a suit.

Forward testing on data the optimizer never saw is the other check, and plenty of sellers skip it because it spoils the sales page.

How do you split data for out-of-sample testing, and what share of the history do you keep hidden?
Post Reply