forex-scalping.com

Trusting a backtest more than it deserves

A good backtest feels like proof. You see a smooth equity curve over a few years, and it's natural to believe the strategy works. The trouble starts when that belief gets stronger than the evidence.

Backtests have limits. Spreads might be modelled as fixed when they actually widen. Slippage might be ignored. The data might have gaps. The strategy might have been tuned on the same data it's tested on. Each of those makes the backtest look better than live trading will be.

Then live trading starts and results are worse. If you trust the backtest too much, you don't question the strategy. You question yourself, the broker, or the market. You keep trading something that might not work, because the test said it would.

The opposite also happens. A strategy that's fine has a bad live month and gets abandoned, because it didn't match the backtest's smooth curve, which was never realistic.

What helps me is treating a backtest as a rough estimate with a wide margin. Expect live results to be worse, by a fair amount. Set in advance what level of underperformance would make you stop and review.

Forward testing on a demo or small size for a while bridges the gap. It's less satisfying than a backtest, but more honest.

Fairman wrote: Sun Oct 18, 2026 8:17 am Trusting a backtest more than it deserves

A good backtest feels like proof. You see a smooth equity curve over a few years, and it's natural to believe the strategy works. The trouble starts when that belief gets stronger than the evidence.

Backtests have limits. Spreads might be modelled as fixed when they actually widen. Slippage might be ignored. The data might have gaps. The strategy might have been tuned on the same data it's tested on. Each of those makes the backtest look better than live trading will be.

Then live trading starts and results are worse. If you trust the backtest too much, you don't question the strategy. You question yourself, the broker, or the market. You keep trading something that might not work, because the test said it would.

The opposite also happens. A strategy that's fine has a bad live month and gets abandoned, because it didn't match the backtest's smooth curve, which was never realistic.

What helps me is treating a backtest as a rough estimate with a wide margin. Expect live results to be worse, by a fair amount. Set in advance what level of underperformance would make you stop and review.

Forward testing on a demo or small size for a while bridges the gap. It's less satisfying than a backtest, but more honest.

Hello,

A flawless equity curve in a backtest is one of the most dangerous illusions in trading. It provides a false sense of certainty in an environment governed entirely by probabilities, effectively convincing you that you have unlocked a cheat code to the market. When that belief hardens into dogma, the gap between historical simulation and live execution becomes a trap.

To expand on your thoughts, treating a backtest as a blueprint rather than a guarantee requires understanding exactly how and why historical data lies to us—and how to construct psychological and mechanical fail-safes before flipping the switch to live capital.

The Seduction of Hindsight

The primary flaw in trusting a backtest lies in how it is built. In our pursuit of a high Sharpe ratio and minimal drawdowns, we naturally tweak parameters. We adjust the moving average lengths, filter out specific days of the week, or add volatility thresholds. Without realizing it, we transition from discovering a market edge to memorizing historical noise.

A strategy highly optimized for the past five years is essentially a bespoke suit tailored for a market regime that has already ended. When the market inevitably shifts its behavior—moving from a low-volatility bull run to a choppy, mean-reverting environment—the over-fitted model collapses because it lacks the robustness to handle the unknown.

The Invisible Friction of Reality

Your point about spreads and slippage highlights the mechanical limits of backtesting. Algorithms assume seamless execution: if the price hits your limit, you get filled instantly. Reality operates with friction.

Liquidity mirages: A backtest will happily fill a 1,000-share market order at the exact closing price. In live trading, especially in thinner markets, that same order eats through the order book, resulting in an average fill price significantly worse than the signal price.

Survivorship bias: If you test a strategy on today’s S&P 500, you are only testing it on the winners. You are ignoring the companies that went bankrupt, delisted, or merged during your test period, artificially inflating historical win rates.

Execution latency: The fraction of a second it takes your broker to route an order during a high-impact news event is often the difference between a profitable scalp and a devastating loss.

Time Dilation and the Psychology of Drawdowns

Perhaps the most under-discussed difference between a backtest and live trading is the psychological experience of time.

In a backtest, a six-month, 20% drawdown is just a slight dip on a chart that you scroll past in three seconds. You see the recovery immediately on the right side of the screen. In live trading, a six-month drawdown is 180 consecutive days of waking up to a shrinking account balance. You do not get to see the right side of the chart.

This time dilation causes the exact errors you mentioned. If you trust the backtest too much, you stubbornly ride a broken strategy into the ground, convinced the recovery is just around the corner. If you trust it too little, the sheer stress of a normal, statistically expected losing streak causes you to pull the plug on a perfectly viable system right before it rebounds.

Engineering a Bridge to Live Markets

To survive the transition from the laboratory to the arena, a backtest must be aggressively stress-tested and handicapped.

Apply a mandatory "Reality Haircut": Automatically assume your live strategy will generate 30% to 50% less profit and experience a maximum drawdown 1.5 to 2 times deeper than your worst historical drawdown. If the strategy cannot survive this haircut on paper, it will not survive live capital.

Use Out-of-Sample Data: Never test your strategy on the entirety of your available data. Build it on data from 2015 to 2020, lock the rules, and then test it blindly on 2021 to 2024. If the performance falls off a cliff, you didn't find an edge; you curve-fit the data.

Define the "Kill Switch" in Advance: Before a single live trade is placed, write down the exact conditions that will invalidate the strategy. (e.g., "If the live drawdown exceeds 25%, or if there are 12 consecutive losing trades, the system is halted immediately for review.") This removes emotion from the decision-making process when you are in the middle of a losing streak.

Incubate with Micro-Stakes: Paper trading is useful, but it doesn't replicate the execution infrastructure of your broker. Forward-test with micro-lots or a tiny fraction of your bankroll. This validates your execution speed, API stability, and real-world spread costs while keeping the tuition fees low.

What specific invalidation metrics or "kill switches" do you currently use to separate a normal statistical drawdown from a fundamentally broken strategy?

Looking at the worst periods in the backtest rather than the overall result helps calibrate expectations. If the backtest had a three month flat stretch or a drawdown of a certain size, expect at least that live, probably worse. If you'd be uncomfortable living through the backtest's worst period, you'll be more uncomfortable with the real version. That's a better test of whether you can trade the strategy than the total profit figure, which is the part people tend to focus on. Planning for that worst period, with a smaller size if needed, makes it more likely you'll stick with the strategy when it comes.

Out of sample testing is a partial fix for overfitting. Tune on one period, test on another you haven't looked at. If results hold up in the untouched period, that's encouraging. But it only works once. If you look at the out of sample results, change the strategy and test again, that period is no longer out of sample. A lot of people do this without realising it, and end up with a strategy fitted to all the data they have. Keeping a final slice of data completely untouched, and only testing on it once at the end, is the cleanest way to avoid that.

The point about using the out of sample period only once is often missed. A practical way to protect it is to save the final slice of data in a separate file that is not opened until the end. It sounds simple, but it stops casual peeking. Another option is to forward test on a demo account for a few months, which gives fresh data that could not have been used in design. Results from that period are usually a better guide to live performance than any backtest.

A single bad tick, such as a quote that prints 50 pips away from the market for a millisecond, can trigger a stop or a target in a backtest that never could have been hit in reality. A scalping test with thousands of trades can be skewed by a handful of those.

They come from feed errors, stale quotes, or liquidity provider glitches. Some are obvious, like a price of zero. Others are subtle, a few pips off, and invisible without a filter.

Cleaning methods vary. A simple one flags any tick that differs from its neighbours by more than a multiple of the recent volatility and that reverts immediately. A spread check removes ticks where bid exceeds ask, or where the spread is implausibly large.

Care is needed in both directions. Real spikes do happen, especially around news, and a filter that's too aggressive removes them, making the backtest look better than reality.

Keeping a log of every removed tick, and looking through it, avoids discarding the real events.

It's also worth running the strategy on both raw and cleaned data. If the results differ a lot, the strategy depends on specific ticks, which is a warning in itself.

How has bad data been found in a dataset, and did it change any results?