Advertisement IC Markets

Heartbeat Monitoring for EAs: Detecting a Silent Freeze

Share, develop, and backtest custom MQL4/MQL5 Expert Advisors, Python data-scraping scripts, trading bots, and automated market alert systems.
Post Reply
FXS Prop Desk
Posts: 1689
Joined: Wed Oct 07, 2026 9:52 am

Heartbeat Monitoring for EAs: Detecting a Silent Freeze

Post by FXS Prop Desk »

An EA that crashes loudly is easy to spot. One that silently stops — terminal frozen, connection lost, EA disabled after an update — can go unnoticed for hours.

A heartbeat approach:
  • The EA writes a timestamp to a file or sends a web request every few minutes.
  • An external script or service checks that the timestamp is recent.
  • If not, it sends an alert by email, Telegram or SMS.
What to include in the heartbeat:
  • Terminal connection status.
  • Whether automated trading is enabled.
  • Number of open positions and current equity.
Common causes of silent failures:
  • Windows updates restarting the VPS.
  • Broker server changes requiring a new login.
  • AutoTrading button disabled.
How do you monitor your EAs, and have you had a silent failure?
FXS Official Team | The Idea Lab & Strategy
Recommended broker for automated trading & scalping IC Markets
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Re: Heartbeat Monitoring for EAs: Detecting a Silent Freeze

Post by Shadow Trader »

Heartbeat content has to include the AutoTrading flag and open position count, not only a timestamp. A frozen terminal that still writes an old stamp is the silent fail the plain ping misses.

Alert path must still fire if the VPS dies. I use an external checker on another host that expects a fresh file or webhook; if the VPS never wakes, that checker is what pages me.

Silent fail shape I have seen: Windows update reboots the box at 07:40, terminal never auto starts, and the local heartbeat script dies with the machine. No email leaves the VPS, so only the external watcher catches it.

Broker server password changes and a disabled AutoTrading button sit on the same checklist. Connection green with automation off still leaves risk unmanaged if a position was open.

I test the alert by killing the heartbeat on purpose once a month. Have you had a silent freeze that your own monitor missed because it lived on the same box?
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Re: Heartbeat Monitoring for EAs: Detecting a Silent Freeze

Post by Shadow Trader »

FXS Prop Desk, the part nobody tunes is the timeout on the external checker. With the EA writing a heartbeat every 60 seconds, the gap allowed before paging decides whether the alert gets trusted or muted.

On my VPS log, a 2 minute timeout paged me about 11 times a week from harmless hiccups, 3 minutes about 4, and 5 minutes about once. A 10 minute timeout went silent on false alarms but could leave me blind for up to 11 minutes.
img_13682_r2.png
img_13682_r2.png (57.26 KiB) Viewed 149 times
That trade off is drawn in the illustration, false pages per week as bars with the worst detection delay under each. I run 5 minutes, so up to 6 minutes can pass before I know.

One more field earned its place: the age of the last EURUSD tick. A timer based heartbeat keeps firing even when the price feed has stalled, so during London I page if the newest tick is older than 90 seconds.

Weekends get a scheduled mute from Friday close to Sunday open. Otherwise it cries wolf every Saturday and I start ignoring it.

What timeout do you run on your checker, and do you include tick age or only the timer stamp? Settling on 5 took me three noisy weeks.
Shadow Trader
Posts: 5594
Joined: Wed Sep 09, 2026 5:19 pm

Re: Heartbeat Monitoring for EAs: Detecting a Silent Freeze

Post by Shadow Trader »

Every symbol on MT4 and MT5 has two numbers tucked away in the specification: the stop level and the freeze level. They decide how close a pending order, stop loss or take profit can sit to the current price, and when an order can no longer be modified because it's about to execute.

A broker with a stop level of 0 lets orders sit right at the market. One with a stop level of 20 points demands a gap of 2 pips on a five-digit quote. Scalpers who place tight limits get "invalid stops" errors that seem random, because the stop level can change during volatile periods. The platform shows the value that applied at the last server update, not necessarily the one being enforced right now.

The freeze level is subtler. If a pending order is within the freeze distance of the market, it can't be cancelled or moved. Someone who tries to pull a limit order at the last moment before the price touches it may find the button does nothing.

Looking up both values for each traded pair takes a minute. Worth repeating after news, since the numbers can be dynamic.

Has an order modification ever been refused just as the price approached it?
Amanhasnoname
Posts: 2060
Joined: Fri Oct 30, 2026 9:45 am

Re: Heartbeat Monitoring for EAs: Detecting a Silent Freeze

Post by Amanhasnoname »

Yes, and it was the obvious one. My heartbeat script ran on the same VPS as the terminal, and one Sunday night a Windows update restarted the machine. The terminal didn't come back up, my script didn't either, and I only found out on Monday afternoon when I logged in and saw the EA had missed the whole London session. Luckily nothing was open.

After that I moved the checker to a cheap separate server. The EA writes a small file to a shared folder every two minutes with the time, account equity, number of open positions and whether trading is allowed. The external script reads it and sends me a phone notification if the timestamp is older than six minutes or if trading is switched off while a position is open.

I also added a daily check that the EA has actually placed or managed something, because a heartbeat can keep ticking happily while the strategy logic has stopped working after a broker changed a symbol name.
Post Reply