The 90% modeling quality number and what it conceals

[Shadow Trader]

MetaTrader's strategy tester shows a "modeling quality" percentage in its report. Seeing 90% or 99% feels reassuring. It shouldn't be taken at face value.

In MT4, the figure reflects how much of the test period was covered by real one-minute data to generate ticks. The ticks themselves are synthesized by an algorithm from the bars. The process interpolates price paths that never happened. So even 99% quality means 99% of the bars were available, not that the ticks were real.

For slower strategies, that's acceptable. For scalpers, the interpolated path determines whether stops or targets were hit first, and the interpolation is a guess.

Some brokers provide higher-quality tick history, and third-party tools can import real tick data from external sources. MT5 has a "real ticks" mode that uses actual recorded ticks when the broker provides them.

Even then, one broker's ticks aren't another's. A strategy tested on tick data from provider A may behave differently on broker B's feed.

A related shortcut is using a tool that imports real tick data from an external source and runs the tester on it. Doing so often drops the "quality" figure to a lower number but raises the realism. A lower number honestly earned is worth more than a high number built on synthesized ticks.

What data source gives the most trustworthy results for tests of short-term strategies?