Logging real slippage on market orders (MT5 + a dumb Python script)

[LondonScalper]

Broker comparison threads love screenshots of "0.0 spread." Cool. Show me your **slippage distribution** on market orders during London open and I'll care.

I got tired of vibes, so I log every scalp fill:

- Requested price (tick at send)
- Fill price
- Symbol, side, size
- Server time + local VPS time
- Spread at send
- Comment tag (session: london_open / overlap / news)

MT5 deal history gives you most of this if you pull it via Python (`MetaTrader5` package) after the session. I dump CSV, then a 30-line pandas script: mean/median/p95 slippage in points, split by hour.

**Ugly truths from doing this:**
- Average slippage can look fine while p95 ruins your expectancy on the days that matter.
- Some brokers are saints at 10:00 and monsters at 08:05.
- "Raw account" does not automatically mean honest fills when liquidity is thin.

I'm not naming-and-shaming in the OP on purpose — numbers first, brand wars second. If you've got a clean method (or better fields I should log), pile on.

What slippage stats do you actually track — average only, or percentiles?
Anyone logging slippage live via deal listener instead of end-of-day export?
Does your broker look different on EURUSD vs XAUUSD under the same logging?

[LondonScalper]

This is the right instrumentation.

Vibes don’t survive a histogram. Logging requested vs fill by session turns “my broker is fine” into something you can demote or keep.

I’d add two fields if you don’t already have them:
• Spread at send
• Reject / requote flag (even if rare)

Then review p95 slippage, not only the average. Averages hide the London-open tail that murders a week of M-targets.

If the Python side is “dumb,” good — dumb and consistent beats elegant and abandoned.

Are you storing this to CSV per day, or pushing into one running table for the month?

[PTScalper]

Broker comparison threads love screenshots of "0.0 spread." Cool. Show me your **slippage distribution** on market orders during London open and I'll care.

I got tired of vibes, so I log every scalp fill:

- Requested price (tick at send)
- Fill price
- Symbol, side, size
- Server time + local VPS time
- Spread at send
- Comment tag (session: london_open / overlap / news)

MT5 deal history gives you most of this if you pull it via Python (`MetaTrader5` package) after the session. I dump CSV, then a 30-line pandas script: mean/median/p95 slippage in points, split by hour.

**Ugly truths from doing this:**
- Average slippage can look fine while p95 ruins your expectancy on the days that matter.
- Some brokers are saints at 10:00 and monsters at 08:05.
- "Raw account" does not automatically mean honest fills when liquidity is thin.

I'm not naming-and-shaming in the OP on purpose — numbers first, brand wars second. If you've got a clean method (or better fields I should log), pile on.

What slippage stats do you actually track — average only, or percentiles?
Anyone logging slippage live via deal listener instead of end-of-day export?
Does your broker look different on EURUSD vs XAUUSD under the same logging?

Hi LondonScalper,

Spot on. When pushing heavy volume through algorithmic scalping, "0.0 spread" marketing is essentially meaningless. Spread is what they advertise; slippage is what you actually pay. Getting away from vibes and tracking actual tick-level fills is the only way to protect a strategy's expectancy.

Here is a breakdown of how those tracking metrics play out in live environments:

1. The Percentile Reality

Averages are deceptive because slippage is rarely distributed normally—it is heavily skewed by tail events.

The Trap: An average slippage of 0.2 pips looks acceptable on paper, but it easily hides a distribution where 90% of trades get zero slippage and 10% absorb 2.0+ pips during the London open.

What to Track: Median (p50) establishes the honest baseline, but p90, p95, and p99 are the survival metrics. When executing thousands of lots over time, those p95 tail events will mathematically chew through a scalper's edge. Tracking percentiles reveals exactly when a broker's liquidity provider (or B-book plugin) is struggling with volatile bursts.

2. Live Deal Listeners vs. EOD Export

End-of-day CSV dumps paired with a pandas script are perfect for historical auditing and weekly broker comparisons. However, EOD data is strictly post-mortem.

Proactive Logging: Moving to a live deal listener—like utilizing OnTradeTransaction in MT5 or hooking into the cTrader API—shifts tracking from reactive to proactive.
Circuit Breakers: By tracking fill quality live in memory, an algorithmic script can calculate a rolling p90 slippage metric over the last 20 trades. If the broker's execution degrades past a hard threshold, the listener can act as a circuit breaker and automatically halt trading before the next batch of orders is sent into the woodchipper.

3. Asset Differentiation

You will absolutely see stark differences between assets under the same logging setup, primarily due to order book depth.

Deep Liquidity (EURUSD): Top-tier fiat pairs can usually absorb market orders efficiently even during the overlap, keeping the slippage variance tight.

Thin Liquidity (Metals): Silver, for instance, has a notoriously fragmented and thinner book compared to major forex pairs. A market order that gets filled at the requested tick on EURUSD might get dragged through three different price levels on silver, resulting in a significantly wider and uglier slippage distribution—even on a true "raw" account.

[PTScalper]

To protect you is important use python listener:

To build a live deal listener natively in Python, we have to navigate a specific architectural quirk of the MetaTrader5 package: it does not support asynchronous event callbacks.

Unlike MQL5, which has the native OnTradeTransaction() event handler, the Python library is synchronous. However, because the Python API communicates directly with the local MT5 terminal's memory via a high-speed C++ bridge, we can build a high-frequency polling loop that acts as a real-time listener without incurring network lag or CPU spikes.

Here is how to set up a robust listener that catches fills as they happen and calculates slippage on the fly.

The Implementation Strategy

Bypass Timezone Headaches: MT5 logs history using the broker's server time, which causes endless timezone offset bugs in Python. We bypass this entirely by pulling a massive 48-hour rolling window (now - 1 day to now + 1 day).

Deduplication: On script startup, we pull all deals in that window and dump their ticket IDs into a Python set().

High-Frequency Polling: We loop every 50–100ms, pulling the history again. Any ticket ID not in our set() is a fresh fill that occurred milliseconds ago.

[PTScalper]

The Python Listener

Code: Select all

import MetaTrader5 as mt5
import time
from datetime import datetime, timedelta

def live_deal_listener():
    # 1. Initialize MT5 Connection
    if not mt5.initialize():
        print("MT5 initialization failed. Ensure terminal is open.")
        return

    print("Syncing historical deals...")
    
    # 2. Define a wide time window to completely ignore broker timezone offsets
    time_from = datetime.now() - timedelta(days=1)
    time_to = datetime.now() + timedelta(days=1)
    
    # 3. Pre-populate seen tickets to ignore past deals
    seen_deals = set()
    initial_deals = mt5.history_deals_get(time_from, time_to)
    
    if initial_deals:
        for deal in initial_deals:
            seen_deals.add(deal.ticket)

    print(f"Listener active. Tracking {len(seen_deals)} prior deals. Waiting for new fills...")

    try:
        # 4. High-frequency polling loop
        while True:
            # Re-fetch deals (reads from local memory, extremely fast)
            current_deals = mt5.history_deals_get(time_from, time_to)
            
            if current_deals:
                for deal in current_deals:
                    if deal.ticket not in seen_deals:
                        seen_deals.add(deal.ticket)
                        
                        # Filter for actual market entries/exits (ignore deposits/withdrawals)
                        if deal.type not in (mt5.DEAL_TYPE_BUY, mt5.DEAL_TYPE_SELL):
                            continue
                            
                        # Fetch the parent order to get the requested price
                        orders = mt5.history_orders_get(ticket=deal.order)
                        if not orders:
                            continue
                        order = orders[0]
                        
                        # Calculate Slippage
                        symbol_info = mt5.symbol_info(deal.symbol)
                        if symbol_info:
                            point = symbol_info.point
                            req_price = order.price_open
                            
                            # Handle Market Execution edge case (see notes below)
                            if req_price == 0.0:
                                print(f"Warning: Broker logged 0.0 requested price for ticket {deal.ticket}")
                                continue
                                
                            # Calculate difference in points (positive = negative slippage against you)
                            if deal.type == mt5.DEAL_TYPE_BUY:
                                slippage = (deal.price - req_price) / point
                            else:
                                slippage = (req_price - deal.price) / point
                                
                            # Output or route to your tracking database
                            timestamp = datetime.now().strftime('%H:%M:%S.%f')[:-3]
                            side = 'BUY' if deal.type == mt5.DEAL_TYPE_BUY else 'SELL'
                            
                            print(f"[{timestamp}] NEW FILL | {deal.symbol}")
                            print(f"  Side: {side} | Vol: {deal.volume}")
                            print(f"  Req: {req_price} | Fill: {deal.price}")
                            print(f"  Slippage: {slippage:.1f} points\n")
                            
            # Poll every 100ms
            time.sleep(0.1)
            
    except KeyboardInterrupt:
        print("Listener stopped by user.")
    finally:
        mt5.shutdown()

if __name__ == "__main__":
    live_deal_listener()

[PTScalper]

Two Edge Cases to Watch

The 0.0 Requested Price Trap: Depending on whether your broker uses Instant Execution or strict Market Execution, the MT5 terminal may log the historical price_open of a market order as 0.0. If you execute your trades directly from this same Python environment using mt5.order_send(), the best workaround is to store your requested price (the top of book ask/bid at the moment of calculation) into a local Python dictionary keyed by the order_ticket, and reference that dictionary in the listener instead of relying on mt5.history_orders_get().

Scaling Up (ZeroMQ): Polling every 100ms is perfectly fine for an isolated local setup. However, if you eventually orchestrate multiple Python agents or route fills to remote databases, the polling loop can become a bottleneck. At that stage, the enterprise solution is to abandon Python polling entirely: write a lightweight MQL5 Expert Advisor that leverages the native OnTradeTransaction() hook, and push the fill data over a local ZeroMQ socket directly into a listening Python microservice.

[PTScalper]

To build an effective circuit breaker, we need to track a rolling window of recent fills and isolate the logic by symbol. Pushing high volume means the acceptable slippage threshold for a major fiat pair will be drastically tighter than the tolerance for thinner order books like silver.

We can achieve this efficiently by utilizing collections.deque for a fast, fixed-size memory window, and numpy to calculate the p90 percentile on the fly.

Here is how to integrate a symbol-specific circuit breaker into the listener loop.

The Circuit Breaker Implementation

Code: Select all

import MetaTrader5 as mt5
import time
import numpy as np
from datetime import datetime, timedelta
from collections import deque

# --- Circuit Breaker Configuration ---
WINDOW_SIZE = 20      # Number of recent trades to track per symbol
MIN_TRADES = 10       # Minimum trades required before evaluating the breaker
TRADING_ACTIVE = True # Global kill switch

# Define maximum acceptable p90 slippage in points per symbol
SLIPPAGE_THRESHOLDS = {
    "EURUSD": 5.0,    # Halt if p90 > 5 points (0.5 pips)
    "XAGUSD": 25.0,   # Wider tolerance for thinner books like Silver
}

# Initialize rolling windows for each tracked symbol
rolling_windows = {symbol: deque(maxlen=WINDOW_SIZE) for symbol in SLIPPAGE_THRESHOLDS}

def halt_trading(symbol, current_p90, threshold):
    """Executes the kill sequence when thresholds are breached."""
    global TRADING_ACTIVE
    TRADING_ACTIVE = False
    
    print("\n" + "="*50)
    print("🚨 CIRCUIT BREAKER TRIGGERED 🚨")
    print("="*50)
    print(f"Symbol: {symbol}")
    print(f"Metric: p90 Slippage hit {current_p90:.1f} points")
    print(f"Limit : {threshold:.1f} points")
    print("Action: TRADING_ACTIVE flag set to False. Routing halted.")
    print("="*50 + "\n")
    
    # Add external alerts here (Discord webhook, SMS, etc.)

def live_deal_listener_with_breaker():
    global TRADING_ACTIVE
    
    if not mt5.initialize():
        print("MT5 initialization failed.")
        return

    time_from = datetime.now() - timedelta(days=1)
    time_to = datetime.now() + timedelta(days=1)
    
    seen_deals = set()
    initial_deals = mt5.history_deals_get(time_from, time_to)
    if initial_deals:
        seen_deals.update(deal.ticket for deal in initial_deals)

    print("Circuit Breaker armed. Tracking real-time fills...")

    try:
        while True:
            # If the breaker tripped, stop the polling loop (or keep logging without trading)
            if not TRADING_ACTIVE:
                print("System halted. Awaiting manual reset.")
                break 

            current_deals = mt5.history_deals_get(time_from, time_to)
            
            if current_deals:
                for deal in current_deals:
                    if deal.ticket not in seen_deals:
                        seen_deals.add(deal.ticket)
                        
                        if deal.type not in (mt5.DEAL_TYPE_BUY, mt5.DEAL_TYPE_SELL):
                            continue
                            
                        # Only track symbols we have configured thresholds for
                        if deal.symbol not in SLIPPAGE_THRESHOLDS:
                            continue

                        orders = mt5.history_orders_get(ticket=deal.order)
                        if not orders:
                            continue
                            
                        req_price = orders[0].price_open
                        if req_price == 0.0:
                            continue
                            
                        point = mt5.symbol_info(deal.symbol).point
                        
                        # Calculate slippage (positive = bad for you)
                        if deal.type == mt5.DEAL_TYPE_BUY:
                            slippage = (deal.price - req_price) / point
                        else:
                            slippage = (req_price - deal.price) / point
                            
                        # 1. Update the rolling window
                        rolling_windows[deal.symbol].append(slippage)
                        
                        # 2. Evaluate Circuit Breaker
                        window = rolling_windows[deal.symbol]
                        if len(window) >= MIN_TRADES:
                            current_p90 = np.percentile(window, 90)
                            limit = SLIPPAGE_THRESHOLDS[deal.symbol]
                            
                            print(f"[{deal.symbol}] Fill: {slippage:.1f} pts | Rolling p90: {current_p90:.1f} pts")
                            
                            # 3. Trip the breaker if breached
                            if current_p90 > limit:
                                halt_trading(deal.symbol, current_p90, limit)
                                break # Exit the for-loop immediately
                                
            time.sleep(0.1)
            
    except KeyboardInterrupt:
        print("Listener stopped.")
    finally:
        mt5.shutdown()

if __name__ == "__main__":
    live_deal_listener_with_breaker()

[PTScalper]

Architecture Considerations

Enforcing the Halt

The code above uses a global TRADING_ACTIVE boolean. If your execution logic (the code calculating signals and calling mt5.order_send()) lives in the same Python script, it simply checks this flag before firing the next batch of orders.

If your execution logic lives in standalone MQL5 Expert Advisors, flipping a Python boolean won't stop them. To bridge that gap, you can modify the halt_trading() function to write an instruction to a local text file, or use Python to programmatically strip the "AutoTrading" permission from the terminal, though communicating via local file reads in the EA is generally the most bulletproof route.

Handling Negative Slippage

The calculation intentionally appends negative slippage (price improvements) to the deque. When evaluating the health of an execution venue, unexpected extreme price improvements are just as indicative of a destabilized or thin book as negative slippage. Keeping them in the array ensures the p90 calculation reflects the true variance of the feed.

[PTScalper]

For a high-volume setup, relying on file I/O to bridge Python and MQL5 introduces unnecessary disk latency and potential read/write lock contention.

The most robust, lowest-latency method to broadcast a halt command securely to all running EAs is utilizing MT5 Terminal Global Variables.

Global Variables are memory-mapped within the MT5 terminal. They can be instantly written by the Python MetaTrader5 package and read by any MQL5 Expert Advisor in microseconds. Because they are stored as double precision floats, we can use 1.0 to represent a tripped breaker and 0.0 (or the absence of the variable) for safe trading.

Here is how to implement the bridge.

[PTScalper]

1. The Python Trigger

Modify your halt_trading function to instantiate the global variable when the slippage threshold is breached.

Code: Select all

import MetaTrader5 as mt5

def halt_trading(symbol, current_p90, threshold):
    """Executes the kill sequence by setting a terminal-wide Global Variable."""
    
    # 1. Define the kill switch variable name (can be global or symbol-specific)
    kill_switch_name = "GLOBAL_KILL_SWITCH"
    
    # 2. Set the global variable to 1.0 (True)
    # This writes directly to the terminal's memory
    success = mt5.global_variable_set(kill_switch_name, 1.0)
    
    if success:
        print("\n" + "="*50)
        print("🚨 CIRCUIT BREAKER TRIGGERED 🚨")
        print(f"Symbol: {symbol} | p90: {current_p90:.1f} pts > {threshold:.1f} pts")
        print(f"Terminal Global Variable '{kill_switch_name}' set to 1.0")
        print("All EAs will reject new execution requests.")
        print("="*50 + "\n")
    else:
        # Fallback logging if the API bridge fails
        print("CRITICAL ERROR: Failed to set MT5 Global Variable. EAs may still be active.")