Garbage collection pauses and threading in latency-sensitive code
Posted: Sat Oct 31, 2026 7:11 pm
In languages with automatic memory management, the runtime occasionally pauses to reclaim memory. In Python, Java, and C#, such pauses can last milliseconds to tens of milliseconds, appearing at random times.
For a scalping system that reacts to ticks, a pause at the wrong moment delays an order. The effect is rare but real, and it contributes to the tail of the latency distribution.
Reducing the impact involves allocating less in the hot path: reuse objects and buffers, avoid creating many temporary structures per tick, and pre-allocate arrays. Some runtimes allow tuning the collector, or triggering collections at safe times.
Threading adds another layer. Locks and contention can introduce delays. A design with a dedicated thread for the tick handler and a lock-free queue for communication avoids many problems.
Profiling tools show where time goes and how often pauses occur. Logging timestamps at entry and exit of the critical function reveals outliers.
Optimizing the average is less important than controlling the tail.
How large are the latency outliers in the code's hot path, and what causes them?
For a scalping system that reacts to ticks, a pause at the wrong moment delays an order. The effect is rare but real, and it contributes to the tail of the latency distribution.
Reducing the impact involves allocating less in the hot path: reuse objects and buffers, avoid creating many temporary structures per tick, and pre-allocate arrays. Some runtimes allow tuning the collector, or triggering collections at safe times.
Threading adds another layer. Locks and contention can introduce delays. A design with a dedicated thread for the tick handler and a lock-free queue for communication avoids many problems.
Profiling tools show where time goes and how often pauses occur. Logging timestamps at entry and exit of the critical function reveals outliers.
Optimizing the average is less important than controlling the tail.
How large are the latency outliers in the code's hot path, and what causes them?