Some things should raise suspicion immediately when reviewing a backtest or a shiny system someone else is selling.
- A perfectly smooth equity curve with almost no drawdown.
- A huge number of rules, filters and exceptions.
- Results only on one pair and one timeframe.
- Stellar performance in a specific year and mediocre elsewhere.
- Optimised values that look oddly precise.
- No mention of costs, or costs set unrealistically low.
- A short test period with a small number of trades.
None of these proves a problem alone, but several together are a strong warning.
A useful rule is to ask what the strategy is exploiting. If the answer is vague, such as "the market tends to go up after this pattern", digging deeper usually exposes a weak foundation. A believable edge has a story: a forced flow, a behavioural bias, a structural feature.
Which of these warning signs showed up in the last system tested or purchased?
Curve fitting smells: a quick checklist
-
Shadow Trader
- Posts: 5594
- Joined: Wed Sep 09, 2026 5:19 pm
-
FXS Prop Desk
- Posts: 1690
- Joined: Wed Oct 07, 2026 9:52 am
Re: Curve fitting smells: a quick checklist
Of that list, "stellar in one year, mediocre elsewhere" is the one most often explained away. A quick test is to drop the best year entirely and see if the remaining equity curve still justifies the risk. If it doesn't, the system is one regime's luck, and sizing it for a prop evaluation would be betting on that regime coming back.
FXS Official Team | The Idea Lab & Strategy
-
Amanhasnoname
- Posts: 2060
- Joined: Fri Oct 30, 2026 9:45 am
Re: Curve fitting smells: a quick checklist
The oddly precise optimised values one is what caught me the first time I bought a system, years ago. It was a moving average crossover with a period of 37 and an RSI filter at 63.5, and the seller had an equity curve that looked like a staircase. I should have asked why 37 and not 35 or 40, but I was too impressed by the curve to think about it.
When I tested it myself, shifting the average by just two periods either way turned a nice profit into a steady loss. That is now the first thing I do with any strategy, including my own. I nudge every parameter a little in both directions and look at whether the results form a gentle hill or a single sharp spike. A hill I can live with. A spike means I have found noise.
I would add one more smell to your list, which is a system that never shows its losing months in detail. If someone only ever shares the yearly total, there is usually a stretch they would rather you didn't see.
When I tested it myself, shifting the average by just two periods either way turned a nice profit into a steady loss. That is now the first thing I do with any strategy, including my own. I nudge every parameter a little in both directions and look at whether the results form a gentle hill or a single sharp spike. A hill I can live with. A spike means I have found noise.
I would add one more smell to your list, which is a system that never shows its losing months in detail. If someone only ever shares the yearly total, there is usually a stretch they would rather you didn't see.