forex-scalping.com

Gaps in tick history

Download a year of tick data and look for holes. Missing minutes. Missing hours. Entire sessions with no ticks.

Gaps appear for many reasons: server maintenance, connection drops at the data provider, or low activity periods that simply produced few ticks. Some are harmless. Others hide real market moves.

A backtest running through a gap treats the missing time as no price change. A trade held through the gap shows no profit or loss until the next tick, which then jumps. The strategy may seem to catch a big move that the real market spread over many ticks.

Some data providers fill gaps by copying the last known price forward. That creates flat lines in what was actually a moving market.

Checking involves a quick script that finds time gaps longer than some threshold, such as 30 seconds during active hours, and listing them. A dozen gaps in a year is fine. Hundreds suggest poor data.

How clean is the tick history being used, and has anyone scanned it for gaps during active hours?

An expert advisor in MetaTrader runs its logic every time a tick arrives, but if the previous run isn't finished, the next tick is dropped. Under heavy load, an EA can miss many ticks without any warning.

During quiet periods nothing looks wrong. At the moments when speed matters, such as news or fast breakouts, the EA runs behind. Decisions are made on old information, and in the worst cases, it acts on a price that's no longer there.

Backtests don't reveal it. The strategy tester feeds ticks one at a time at its own pace, and every tick is processed in order. Live trading is chaotic by comparison.

An easy diagnostic is to log timestamps for each OnTick call and measure the gaps. Big gaps during active periods point to missed ticks. Another is to print the time spent inside the function.

Fixes include moving heavy computation to a timer, caching indicator values, and avoiding history loops on every tick.

The deeper lesson is that live performance includes the platform's own scheduling behavior, which no tester simulates.

There is a pattern in how EAs written for MT4 handle ticks on MT5. MT5 queues events differently, and some EAs that ran fine on the older platform start to skip or double-process ticks after a port. Differences in how OnTick is called, and in the number of ticks the terminal delivers, can alter strategy behavior without any error.

Strategies that depend on specific tick counts, such as "act on the third tick after a signal," are especially fragile. If the broker's feed changes, the count changes, and the strategy shifts.

The more robust design uses timestamps, not tick counts, and reacts to price conditions rather than the number of updates. It also keeps a record of the last processed time, so skipped ticks can be detected.

Printing a counter of ticks received versus ticks processed, even for a single day, can be shocking.

Has an EA ever been caught running behind live, and what turned out to be the cause?