forex-scalping.com

Timestamps and the millisecond problem

Tick data carries timestamps. Some in seconds, some in milliseconds. The resolution matters.

Second-resolution timestamps make multiple ticks within the same second indistinguishable in order. A backtest has to guess the sequence, which creates the same kind of uncertainty as bar-based data, just on a smaller scale.

Millisecond timestamps preserve the order. For scalping systems with fast entries and exits, this resolution is needed.

Another issue is the timestamp source. Is it the time the broker's server generated the tick, the time the platform received it, or the time the data provider recorded it? The differences can be tens or hundreds of milliseconds.

Comparing data from two sources can show systematic offsets. Aligning them properly is a boring job that decides whether a latency-sensitive test means anything.

A quick sanity check is to count how many ticks share an identical timestamp in the dataset. On second-resolution data, a busy minute can have dozens of ticks stamped with the same second, and the order among them is invisible. If that count is high, any tick-level conclusion drawn from the file deserves heavy scepticism.

What timestamp resolution does the tick data being used provide, and where do the timestamps come from?

MetaTrader 4 records ticks with a one-second resolution. If a hundred ticks arrive within the same second, they all carry the same timestamp, and the sequence between them is preserved only by their order in the data.

MT5 improved this with millisecond timestamps. Still, many brokers' history exports and many third-party data sets stick to seconds.

The consequence matters for analysis. A stop and a target inside the same second can't be sequenced from the timestamps. Questions like "did price hit the stop first or the target first" have to be answered by order alone, and some exports lose even that.

For very short-term strategies, one-second resolution hides the structure that matters. A burst of fifty ticks that moves price six pips in half a second looks like a single data point with the same timestamp.

Studying execution quality also suffers. Request and fill times need to be measured in milliseconds to see differences between brokers or between orders, and a platform that rounds to seconds blurs the picture.

If the data source offers millisecond stamps, they're worth using for any serious short-term analysis.

For execution analysis, the lack of finer timestamps blurs one of the most useful measures: how long an order took. If the request is stamped 10:02:15 and the fill is stamped 10:02:15, the delay is anywhere between zero and a second. Any number of broker differences hide inside that second.

MT5 keeps millisecond stamps in its tick history and in deal records, which makes a big difference. Third-party tools and APIs can give microsecond precision. For a serious look at latency, it is worth using one of them, or at least logging from an EA that calls the system clock at the moment of sending and at the moment of confirmation.

The same applies to news analysis. Whether the first spike came before or after the release stamp can't be determined at one-second resolution if the market moved within the first half second.

How granular is the timestamp in the data currently being used for testing?