Why Your Scalping Backtest is Lying to You (And How Monte Carlo Fixes It)

[PTScalper]

Hey everyone,
If you are running a high-frequency scalping strategy, relying on a standard MT4 backtest equity curve is a massive trap. Because scalping relies on a high volume of trades with tight profit margins, it is incredibly vulnerable to sequence risk—the exact order in which your wins and losses happen.
This is exactly why you need Monte Carlo simulation. Instead of trusting the single historical path MT4 spits out, a Monte Carlo test takes your actual backtest data and reshuffles the trade order 1,000 to 10,000 times. 
Here is why this is an absolute game-changer for scalpers:
1. Uncovers True Drawdown: Your base backtest might show a comfortable 8% max drawdown. But what if those 7 random losses that happened months apart suddenly trigger back-to-back? Monte Carlo reshuffling exposes your realistic 95th-percentile worst-case scenario.
2. Exposes "Perfect Condition" Bias: By configuring the simulation to randomly skip 5% to 10% of trades, you instantly simulate the real-world chaos of slippage, market gaps, or platform disconnects—factors that destroy fragile scalping EAs. 
3. Builds Psychological Armor: If your simulation proves that an 8-trade losing streak is mathematically expected within your win rate, you won't panic and turn off your strategy when it inevitably happens in live markets.
Stop risking capital on a single lucky timeline. Stress-test your systems. Has anyone else run their EAs through Monte Carlo recently? What was the difference between your base MT4 drawdown and your simulated one?

[PTScalper]

How to Prepare a Monte Carlo Test for MT4
MetaTrader 4 does not have a native Monte Carlo simulator built into its default Strategy Tester. To stress-test your strategy, you need to export your raw trade data and run it through a dedicated analyzer (like QuantAnalyzer, BacktestBase, or a custom Excel sheet).
Here is the exact workflow to prepare and execute the test:
Step 1: Generate High-Quality MT4 Data
1. Open the MT4 Strategy Tester.
2. Select your scalping Expert Advisor (EA) and set the Model to Every tick (this is mandatory for scalping to ensure tick-data accuracy).
3. Run the backtest over a significant period. For scalping, you want a sample size of at least 300 to 500 trades to give the Monte Carlo algorithm enough statistical weight.
Step 2: Export the Trade Sequence
1. Once the backtest finishes, go to the Results tab at the bottom of the Strategy Tester.
2. Right-click anywhere inside the results window and select Save as Report.
3. Save the file as an HTML document. (This HTML file contains the precise sequence of your wins, losses, and dollar amounts).
Step 3: Run the Simulation
Import your MT4 HTML report into your chosen Monte Carlo software and configure the following parameters specifically for scalping:
Iterations: Set to a minimum of 1,000 simulations. 
Exact Trade Reshuffling (Sequence Risk): Turn this on. It will scramble the order of your trades to calculate the probability of severe losing streaks.
Missed Trade Simulation (Trade Skipping): Set the software to randomly skip 5% to 10% of your historical trades. Scalping is highly sensitive to missed entries due to latency or spread widening; if your strategy falls apart when 5% of its trades are missed, it is over-optimized and will fail in live markets.
Step 4: Analyze the Output
Ignore the absolute best and worst simulations. Look strictly at the 95th percentile confidence interval. If the maximum drawdown at the 95th percentile exceeds your risk tolerance, you must reduce your lot sizing in MT4 before taking the strategy live.

[PTScalper]

Here is a complete, object-oriented Python implementation of a Monte Carlo trading simulator.
Since simulating thousands of equity curves using standard Python loops is highly inefficient, this script utilizes NumPy's vectorized operations to run 10,000 iterations in a fraction of a second. It handles trade reshuffling (sequence risk), random trade skipping (missed execution risk), calculates maximum drawdowns, and plots the resulting probability distribution.
Requirements
You will need to install the standard data science libraries if you haven't allready:

Code: Select all

 pip install numpy pandas matplotlib

[PTScalper]

And this is my implementation in Python:

Code: Select all

 import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

class MonteCarloSimulator:
    def __init__(self, trades: list[float], initial_balance: float = 10000.0):
        """
        Initializes the simulator with a list of historical trade results (P&L in currency).
        """
        self.trades = np.array(trades)
        self.initial_balance = initial_balance

    def run(self, iterations: int = 1000, skip_probability: float = 0.05) -> np.ndarray:
        """
        Runs the Monte Carlo simulation using NumPy vectorization for high performance.
        
        :param iterations: Number of equity curves to simulate.
        :param skip_probability: Probability (0.0 to 1.0) of a trade being skipped.
        :return: A 2D numpy array of shape (iterations, number_of_trades) containing equity curves.
        """
        n_trades = len(self.trades)
        
        # 1. Reshuffle trades randomly for all iterations at once
        # Create a matrix of random indices of shape (iterations, n_trades)
        random_indices = np.random.randint(0, n_trades, size=(iterations, n_trades))
        simulated_trades = self.trades[random_indices]
        
        # 2. Simulate missed trades (e.g., due to slippage, platform disconnects)
        if skip_probability > 0:
            skip_mask = np.random.rand(iterations, n_trades) < skip_probability
            simulated_trades[skip_mask] = 0.0  # Zero P&L for skipped trades
            
        # 3. Calculate equity curves
        # Cumulative sum across the columns (axis=1), offset by initial balance
        equity_curves = np.cumsum(simulated_trades, axis=1) + self.initial_balance
        
        return equity_curves

    def calculate_metrics(self, equity_curves: np.ndarray):
        """
        Calculates drawdowns and ending balances across all simulations.
        """
        # Ending balances are the last column of the matrix
        ending_balances = equity_curves[:, -1]
        
        # Calculate Drawdowns
        # Running maximum for each curve up to each point
        running_max = np.maximum.accumulate(equity_curves, axis=1)
        # Ensure running max doesn't drop below initial balance for accurate DD math
        running_max = np.maximum(running_max, self.initial_balance) 
        
        # Drawdown percentage matrix
        drawdowns = (running_max - equity_curves) / running_max * 100
        max_drawdowns = np.max(drawdowns, axis=1)
        
        # Calculate Percentiles
        results = {
            "Ending Balance (Median)": np.percentile(ending_balances, 50),
            "Ending Balance (5th Percentile)": np.percentile(ending_balances, 5),
            "Ending Balance (95th Percentile)": np.percentile(ending_balances, 95),
            "Max Drawdown (Median)": np.percentile(max_drawdowns, 50),
            "Max Drawdown (95th Percentile / Worst Case)": np.percentile(max_drawdowns, 95),
            "Risk of Ruin (< 50% Balance)": np.mean(ending_balances < (self.initial_balance * 0.5)) * 100
        }
        return results

    def plot_simulation(self, equity_curves: np.ndarray, sample_size: int = 100):
        """
        Plots a spaghetti chart of the simulated equity curves.
        Limits to sample_size to prevent memory/rendering overload.
        """
        plt.figure(figsize=(12, 6))
        
        # Plot a subset of lines so it remains visually readable
        subset = equity_curves[:sample_size, :]
        for curve in subset:
            plt.plot(curve, color='blue', alpha=0.1, linewidth=1)
            
        # Plot the median curve in a strong color
        median_curve = np.median(equity_curves, axis=0)
        plt.plot(median_curve, color='red', linewidth=2, label='Median Equity Curve')
        
        plt.title(f"Monte Carlo Trading Simulation ({len(equity_curves)} Iterations)")
        plt.xlabel("Trade Number")
        plt.ylabel("Account Equity")
        plt.axhline(self.initial_balance, color='black', linestyle='--', label='Initial Balance')
        plt.legend()
        plt.grid(True, alpha=0.3)
        plt.tight_layout()
        plt.show()

# ==========================================
# Example Execution
# ==========================================
if __name__ == "__main__":
    # 1. Generate some dummy backtest data for a scalping strategy
    # Let's say: 60% win rate, average win = $15, average loss = -$20
    np.random.seed(42) # For reproducible dummy data
    dummy_trades = np.where(np.random.rand(400) < 0.60, 15.0, -20.0)

    # 2. Initialize the simulator
    starting_capital = 5000.0
    simulator = MonteCarloSimulator(trades=dummy_trades, initial_balance=starting_capital)

    # 3. Run 10,000 iterations with a 5% chance of skipped trades
    iterations = 10000
    curves = simulator.run(iterations=iterations, skip_probability=0.05)

    # 4. Analyze and print metrics
    metrics = simulator.calculate_metrics(curves)
    
    print("-" * 40)
    print("MONTE CARLO SIMULATION RESULTS")
    print("-" * 40)
    for key, value in metrics.items():
        if "Balance" in key:
            print(f"{key}: ${value:,.2f}")
        else:
            print(f"{key}: {value:.2f}%")
    print("-" * 40)

    # 5. Plot the distribution
    simulator.plot_simulation(curves, sample_size=200)

[PTScalper]

And how to adapt it for that MT4?

To feed your real MT4 backtest into this engine, simply use pandas to parse the HTML or CSV export. Assuming you have exported your trades into a CSV with a column named ⁠Profit⁠, you can load it into the engine like this:

Code: Select all

 df = pd.read_csv("mt4_results.csv")
real_trades = df['Profit'].tolist()

simulator = MonteCarloSimulator(trades=real_trades, initial_balance=10000.0)

[PTScalper]

The beauty of the Monte Carlo simulation engine provided above is that the core mathematics do not care where the data comes from. The only thing you need to change is the extraction and data-parsing layer (the ETL pipeline).
Your goal for both MetaTrader 5 and IC Markets cTrader (which is typically what "IC trader" refers to) is exactly the same: extract the history, parse the file using ⁠pandas⁠, filter out non-trade operations like deposits or withdrawals, and pass a flat list of ⁠float⁠ profit/loss values into the ⁠MonteCarloSimulator⁠ class.
Here is exactly how to extract and parse the data for both platforms.
1. Adapting for MetaTrader 5 (MT5)
MT5 handles data export slightly differently than MT4. Instead of an HTML file, the cleanest way to extract data for Python is via an Excel (OpenXML) export.

[PTScalper]

How to export from MT5:
1. Open the MT5 Strategy Tester (for backtests) or the Toolbox -> History tab (for live trading history).
2. Right-click anywhere in the list of trades.
3. Select Report -> Open XML (Excel).
4. Save the file (e.g., ⁠mt5_report.xlsx⁠).
Python Parsing Code for MT5:
MT5 reports often include balance operations (deposits/withdrawals) mixed in with the trades. You must filter these out before running the simulation.

Code: Select all

 import pandas as pd

def load_mt5_history(filepath: str) -> list[float]:
    """
    Parses an MT5 Open XML (Excel) report and extracts pure trade P&L.
    """
    # Load the Excel file
    # Note: You may need to install openpyxl: pip install openpyxl
    df = pd.read_excel(filepath)
    
    # MT5 exports often label deposits/withdrawals as 'Balance' in the 'Type' or 'Action' column.
    # We want to filter those out so we only simulate actual trading performance.
    # Note: Column names might vary slightly based on your MT5 language settings.
    
    # Assuming standard English MT5 export:
    if 'Type' in df.columns:
        df = df[df['Type'] != 'Balance']
        
    # Extract the Profit column, drop any empty rows, convert to float
    # If your export includes swaps and commissions as separate columns, 
    # you may want to sum them: df['Total P&L'] = df['Profit'] + df['Swap'] + df['Commission']
    real_trades = df['Profit'].dropna().astype(float).tolist()
    
    return real_trades

# Usage:
# mt5_trades = load_mt5_history("mt5_report.xlsx")
# simulator = MonteCarloSimulator(trades=mt5_trades, initial_balance=10000.0)

[PTScalper]

2. Adapting for IC Markets cTrader
If you are using cTrader through IC Markets, the data pipeline is actually much cleaner than MetaTrader, as cTrader natively exports to a highly structured CSV format.
How to export from cTrader:
1. Navigate to the History tab at the bottom of the cTrader terminal.
2. Right-click on any trade in the list.
3. Select Export to Excel (this will actually download a CSV file).
4. Save the file (e.g., ⁠ctrader_history.csv⁠).
Python Parsing Code for cTrader:
cTrader’s CSV uses column names like ⁠Net P&L⁠ (which includes commissions and swaps, making it highly accurate for backtesting).

Code: Select all

 import pandas as pd

def load_ctrader_history(filepath: str) -> list[float]:
    """
    Parses a cTrader CSV history export and extracts pure trade P&L.
    """
    # cTrader exports as a standard CSV
    df = pd.read_csv(filepath)
    
    # cTrader typically labels non-trade transactions (deposits) differently.
    # You can filter by 'Closing Direction' or similar columns to ensure it's a real trade.
    if 'Closing Direction' in df.columns:
        df = df[df['Closing Direction'].notna()]
        
    # The 'Net P&L' column in cTrader already accounts for commissions and swaps, 
    # which is perfect for an accurate Monte Carlo simulation.
    real_trades = df['Net P&L'].dropna().astype(float).tolist()
    
    return real_trades

# Usage:
# ctrader_trades = load_ctrader_history("ctrader_history.csv")
# simulator = MonteCarloSimulator(trades=ctrader_trades, initial_balance=10000.0)

[PTScalper]

The Complete Pipeline
To bring it all together, you simply replace the dummy data generation in the original script's ⁠if __name__ == "__main__":⁠ block with the appropriate loader function:

Code: Select all

 if __name__ == "__main__":
    # Choose your platform loader
    # real_trades = load_mt5_history("mt5_report.xlsx")
    real_trades = load_ctrader_history("ctrader_history.csv")
    
    starting_capital = 10000.0
    
    # Pass the sanitized list of floats to the engine
    simulator = MonteCarloSimulator(trades=real_trades, initial_balance=starting_capital)
    
    # Run the iterations
    curves = simulator.run(iterations=10000, skip_probability=0.05)
    metrics = simulator.calculate_metrics(curves)
    
    # Print and plot
    for key, value in metrics.items():
        print(f"{key}: {value}")
        
    simulator.plot_simulation(curves, sample_size=200)

[PTScalper]

And remember, this is like basic version.

I will work little bit to make it better

In my point of view most important for great simulation Monte Carlo is to have as random generator as possible.