Multi timeframe filter in my EA and the repainting bug that fooled my backtest

[Fairman]

Multi timeframe filter in my EA and the repainting bug that fooled my backtest

I added a higher timeframe filter to my M5 EA, using an M15 moving average to decide the direction. The backtest improved a lot, which should have made me suspicious. It turned out I'd made a classic mistake.

When the EA read the M15 value on the current bar, it was reading a value that wasn't final yet. In the tester, depending on how it's set up, that could include information from the future within the bar. The M15 candle was still forming, but the test was using values that only exist after it closed. Live, the EA only sees what's actually available at that moment, so the results didn't match.

The fix was to read the M15 values from the last closed bar, not the current one. That means the filter reacts one bar later, but it's what actually happens in real time. After that change, the backtest improvement mostly disappeared, which told me it wasn't real.

Whenever a result looks too good after a small change, I go looking for a mistake like this before I celebrate. Using only closed bar data for anything that comes from another timeframe is a rule I follow now.

[Fairman]

Reading from the last closed bar is the fix, and I would add that it applies to anything from a higher timeframe, not just moving averages. Highs, lows and candle patterns on M15 all have the same problem if you read the forming bar. I made it a habit to always use shift one for any higher timeframe value unless I specifically want the live one, and if I do, I make sure the tester is set to model it correctly.

[Fairman]

Another thing worth checking is whether the higher timeframe data is actually available in the tester for the dates you test. If the M15 history is missing or incomplete, some testers quietly build it from M5 or return zero, and the filter behaves strangely. I check the history before testing and print the filter value on a few bars to make sure it matches what I see on a normal chart. That caught a problem once where the filter was reading empty values.

[Fairman]

The suspicious improvement point is the real lesson here. When a simple filter makes a backtest dramatically better, I now assume a bug first and an edge second. A real filter usually helps a bit, by cutting some bad trades and some good ones. A filter that removes nearly all losers is almost always reading information it should not have. Running the EA on demo for a couple of weeks next to the tester for the same period is the quickest way to confirm.