forex-scalping.com

Same-bar exits and entries: what your backtest assumes happened first

Here's a backtesting problem that's easy to miss. On a given bar, price hits both your stop loss and your take profit. Which one happened first? Bar data alone doesn't tell you.

Many simple backtesters make an assumption. Some assume the stop hit first, which is pessimistic. Some assume the target hit first, which is optimistic. Some use the bar's open and close direction to guess. If you test on M5 bars with tight stops and targets, a lot of trades end up on bars where both levels were touched, and that assumption can change the results a lot.

The same problem appears with entries. If your signal triggers on a bar and your stop is also within that bar's range, did the stop get hit after entry, or before?

The fix is either to use smaller bars or tick data for those cases, or to set the assumption to pessimistic and see if the strategy survives. MT5's real tick mode handles this properly because it processes ticks in order. Python backtesters often need you to deal with it explicitly.

I'd check how many trades in a backtest are resolved on a bar where both levels were reached. If it's a big share, the results depend heavily on that assumption.

A quick way to measure it is to add a flag in the backtest for trades where the exit bar's high and low both crossed the stop and target. Then run the test once assuming stop first and once assuming target first. The gap between the two results tells you how much the assumption matters. If the gap is small, fine. If it's large, the strategy's targets and stops are too close relative to the bar size, and you either need finer data or a different setup. Most of the time, wider targets and stops relative to the bar size remove most of the problem.

Spread complicates this further. With bid and ask data, a long stop is hit by the bid, a long target by the bid too, but a short stop is hit by the ask. If your data only has bid prices and your strategy shorts with tight stops, some stop hits will be missing from the test. Adding the spread to the high when checking short stops is a rough fix. Tick data with both sides is better if you can get it, especially for strategies where stops are only a few pips away.

The order assumption can change the result a lot for scalping strategies, where stop and target are often close. Some testers assume the worst case, the stop first, which is safer.

Using tick data, or at least M1 data inside the bar, reduces the problem. For strategies with targets of a few pips, testing on higher timeframe bars alone can produce results that are far from reality.

Using lower timeframe data inside the bar can resolve the ambiguity in many cases. If the backtest is on M15 and both the stop and target were hit in the same bar, checking the M1 bars within it shows which came first. MT5's tester does something similar in 'every tick' mode, generating ticks from M1 data. In Python, you'd need to load the lower timeframe data and look it up yourself. It adds work, but for strategies where same bar exits are common, it makes results much more trustworthy. Even a small sample of checked bars shows how often the guess was wrong.

A one-minute bar contains four numbers: open, high, low, close. A scalping strategy with a target of 3 pips and a stop of 3 pips needs to know which came first inside the bar. The bar doesn't say.

Most backtesters fill that gap with an assumption. Some assume the high came first on a bullish bar, some assume the worst case, some interpolate ticks artificially. Each choice changes the results, sometimes by a huge margin.

On strategies with tight targets, this is the whole ballgame. A system that looks like a 62 percent win rate on bar data can fall to 48 percent on real tick data, because the stop and target both sat inside the same bar and the backtester guessed wrong.

Modelled ticks are no fix. They are generated from the bar, so they inherit the same blind spot while looking convincing.

Real tick data has its own problems. Different brokers have different ticks, different spreads, different timestamps. A test run on one broker's history doesn't map perfectly onto another's execution.

Still, a strategy that survives real ticks with realistic spreads and a pinch of slippage is worth more than one that shines on bars.

A practical way to see the damage without buying a new data set: find all trades in a backtest where the bar's range was larger than both the stop distance and the target distance. Those are the trades where order matters. Count them. If they make up 15 percent of all trades, then 15 percent of the results are a coin flip dressed up as a number.

Then rerun the same test twice, once assuming the stop was always hit first on such bars and once assuming the target was. The gap between the two outcomes is the uncertainty band of the whole test. When that band is wider than the claimed edge, the backtest has not proven anything yet, however smooth the equity curve looks.

What source of tick data has held up best when comparing backtest results to live fills?

How common are these ambiguous bars in your backtest? I'd count them before worrying too much. If only one or two percent of trades have both levels inside the same bar, the assumption barely affects results. If it's twenty percent, the assumption is driving a big part of the outcome, and the strategy might look much better or worse depending on a guess. That's a sign the stop and target are too tight for the timeframe being tested, and testing on a lower timeframe is really necessary before trusting anything.