Backtesting Basics: Testing an Idea Against History
Backtesting means applying your rules to past data to see how they would have performed. Done carefully, it filters out weak ideas before they cost real money.
The manual version works for beginners. Scroll back on a chart, find each instance of your setup according to your written rules, record the entry, stop, target, and result, and continue forward bar by bar without looking ahead. Cover the right side of the chart if necessary.
Record a sufficient number of trades, at least 100 if possible, across different market conditions, including trends, ranges, and high-volatility periods. A test limited to one favorable stretch proves little.
Include costs. Subtract the spread and any commission from each trade. Many strategies that look profitable fall below zero after realistic costs.
Calculate metrics beyond total profit: win rate, average win and loss in R, expectancy, maximum drawdown, and the longest losing streak.
Be honest about limits. A backtest is an estimate. It cannot capture changing conditions, emotional pressure, or execution problems like slippage.
Treat positive results as a reason to move to forward testing on demo or very small live size, not as proof.
Automated backtesting tools exist and are faster, but they require clear rules and careful data, and they make overfitting easier.
Practical step: manually backtest fifty occurrences of your main setup, with costs included, and calculate expectancy.
Backtesting Basics: Testing an Idea Against History
Backtesting Basics: Testing an Idea Against History
It’s Fairman 
Re: Backtesting Basics: Testing an Idea Against History
For the manual version, log it in a spreadsheet as you go, not afterwards from memory. Date, session, entry, stop, target, result in R, and one extra column: would I really have taken this live? That last column is humbling.
It’s Fairman 
Re: Backtesting Basics: Testing an Idea Against History
Worth splitting the results by period too. If 70 of your 100 trades came from one trending month, the overall number is mostly that month. Break it down by quarter and see if it holds up in each one.
It’s Fairman 
Re: Backtesting Basics: Testing an Idea Against History
Small thing, but take breaks when you test by hand. After an hour of clicking bar by bar you start seeing setups that aren't really there, and the sample quietly gets worse.
It’s Fairman 
-
Shadow Trader
- Posts: 5594
- Joined: Wed Sep 09, 2026 5:19 pm
Re: Backtesting Basics: Testing an Idea Against History
One trap the manual method makes easy is testing many versions of the same idea and keeping the best. Each tweak to the stop, the session or the target is in effect a fresh backtest on the same history.
Say an idea has no edge at all and you try 20 variations of it. If each has a 5 percent chance of looking good by luck, the chance that at least one does is 1 minus 0.95 to the power 20, which is about 64 percent.
So the winning variant after a long evening of tweaking is more likely luck than not. A simple guard is to set aside the most recent quarter of the data before starting and not look at it at all.
Do all the tweaking on the older part, pick one version, then run it once on the held back quarter. If the expectancy falls by half or more there, the earlier result was mostly fitted to the past.
Look-ahead creeps in too, even with the right side covered. A swing low used for the stop is often only obvious two or three bars later, so check that every stop level was visible at the moment of entry.
Say an idea has no edge at all and you try 20 variations of it. If each has a 5 percent chance of looking good by luck, the chance that at least one does is 1 minus 0.95 to the power 20, which is about 64 percent.
So the winning variant after a long evening of tweaking is more likely luck than not. A simple guard is to set aside the most recent quarter of the data before starting and not look at it at all.
Do all the tweaking on the older part, pick one version, then run it once on the held back quarter. If the expectancy falls by half or more there, the earlier result was mostly fitted to the past.
Look-ahead creeps in too, even with the right side covered. A swing low used for the stop is often only obvious two or three bars later, so check that every stop level was visible at the moment of entry.
-
FXS Prop Desk
- Posts: 1689
- Joined: Wed Oct 07, 2026 9:52 am
Re: Backtesting Basics: Testing an Idea Against History
In MT4 the manual version gets easier if you switch off auto-scroll and press F12 to move the chart forward one bar at a time, so the next candle never appears before the decision is written down.
I would also add a column to the log for every setup where you were unsure the rules applied. If more than a handful out of fifty end up there, the rules need tightening before the expectancy number means much, because live those are exactly the trades you take or skip depending on mood. Did writing down the entry time show results bunching in one part of the session?
I would also add a column to the log for every setup where you were unsure the rules applied. If more than a handful out of fifty end up there, the rules need tightening before the expectancy number means much, because live those are exactly the trades you take or skip depending on mood. Did writing down the entry time show results bunching in one part of the session?
FXS Official Team | The Idea Lab & Strategy
-
Shadow Trader
- Posts: 5594
- Joined: Wed Sep 09, 2026 5:19 pm
Re: Backtesting Basics: Testing an Idea Against History
Before splitting results by quarter, it helps to know whether the total is big enough to mean anything. The number to look at is the average result in R next to how much the results swing around it.
Say 100 trades average plus 0.15R with a standard deviation of 1.1R per trade. The standard error is 1.1 divided by the square root of 100, which is 0.11R, so the average sits only about 1.4 standard errors above zero.
To reach roughly two standard errors at the same figures you need about 215 trades, because 2 times 1.1 divided by 0.15 is about 14.7, and 14.7 squared is about 215. Splitting 100 trades into four quarters leaves 25 per slice, which can flag a disaster but never confirm an edge.
This is why a manual test often ends inconclusive rather than wrong. That is still worth knowing, since it says keep collecting before risking money on it.
Costs belong in before this sum, not after. An edge of 0.15R falls to 0.05R once a spread worth 0.1R comes off each trade, and then the sample needed grows ninefold.
Say 100 trades average plus 0.15R with a standard deviation of 1.1R per trade. The standard error is 1.1 divided by the square root of 100, which is 0.11R, so the average sits only about 1.4 standard errors above zero.
To reach roughly two standard errors at the same figures you need about 215 trades, because 2 times 1.1 divided by 0.15 is about 14.7, and 14.7 squared is about 215. Splitting 100 trades into four quarters leaves 25 per slice, which can flag a disaster but never confirm an edge.
This is why a manual test often ends inconclusive rather than wrong. That is still worth knowing, since it says keep collecting before risking money on it.
Costs belong in before this sum, not after. An edge of 0.15R falls to 0.05R once a spread worth 0.1R comes off each trade, and then the sample needed grows ninefold.
-
Shadow Trader
- Posts: 5594
- Joined: Wed Sep 09, 2026 5:19 pm
Re: Backtesting Basics: Testing an Idea Against History
Holding back a test segment sounds simple, yet it is easily spoiled. Each time a held-out period is checked and the strategy is then adjusted, information from that period leaks into the design. After several rounds, the out-of-sample set has effectively become part of the training data.
Data snooping also happens across a whole research process. Testing fifty ideas on the same history and keeping the one that looks best guarantees a flattering result, because chance alone will produce a winner. The more variations tried, the higher the bar the survivor should clear.
Good practice includes writing down the hypothesis before running any test, logging every variant attempted, and reserving a final segment that is examined only once. If that single look disappoints, the honest response is to discard the idea, not to tweak and look again.
Time-ordered holdout is preferable to random sampling for market data, since random splits mix neighbouring bars and let a model see conditions adjacent to the test points. Ideally the final validation comes from genuinely new data, meaning forward testing, which cannot be snooped because it did not exist when the rules were written.
Data snooping also happens across a whole research process. Testing fifty ideas on the same history and keeping the one that looks best guarantees a flattering result, because chance alone will produce a winner. The more variations tried, the higher the bar the survivor should clear.
Good practice includes writing down the hypothesis before running any test, logging every variant attempted, and reserving a final segment that is examined only once. If that single look disappoints, the honest response is to discard the idea, not to tweak and look again.
Time-ordered holdout is preferable to random sampling for market data, since random splits mix neighbouring bars and let a model see conditions adjacent to the test points. Ideally the final validation comes from genuinely new data, meaning forward testing, which cannot be snooped because it did not exist when the rules were written.