Resampling With Replacement vs Shuffling: Two Monte Carlo Methods Compared

[FXS Prop Desk]

Monte Carlo tests of trading results usually use one of two methods. They answer slightly different questions.

Shuffling (permutation): reorder the same trades randomly. The final result is always the same; only the path changes. It shows the range of drawdowns possible with exactly these trades.

Resampling with replacement (bootstrap): draw trades randomly, allowing repeats. Final results vary. It estimates the range of outcomes if the future resembles the past distribution.

Python example:

Code: Select all

import numpy as np
r = np.array(trades)
perm_dd = [np.minimum.accumulate(np.cumsum(np.random.permutation(r))).min() for _ in range(10000)]
boot_end = [np.random.choice(r, len(r)).sum() for _ in range(10000)]

Use shuffling for drawdown planning, bootstrap for estimating return uncertainty.

In both cases, the number of trades in the original sample matters. With fewer than about 100 trades, the simulation mostly reflects the randomness of the sample itself, and its results should be treated with caution.

Which method do you use, and what did it show about your strategy?

[Fairman]

Use shuffling for drawdown planning, bootstrap for estimating return uncertainty.

Agree with the split, but the shuffle line in the example measures something slightly different from drawdown. Taking the minimum of the cumulative sum gives the lowest point relative to the starting balance, so a path that runs up 20R and then drops 15R shows no drawdown at all. For peak-to-trough you need equity minus its running maximum, something like eq = np.cumsum(np.random.permutation(r)) and then dd = (eq - np.maximum.accumulate(eq)).min(). The original version is still useful for a static max-loss rule measured from the starting balance, but for trailing limits or for how much pain to expect, the peak-to-trough number is the one that matters.

[Shadow Trader]

FXS Prop Desk, I use both methods, but I lean on shuffling when sizing a soft stop and on bootstrap when I want a feel for return scatter. Fairman's note on peak to trough is the fix I needed after my first scripts understated pain.

With a path that runs up 12R and then gives back 8R, a start to trough measure barely blinks. Equity minus running max shows the 8R draw that actually hits a trailing floor.

I only trust either test once the sample clears about 120 trades. Below that the paths mostly replay sample noise, and I treat the percentiles as sketches rather than rules.

On my London EURUSD book the shuffle peak to trough 95th percentile sat near 6.5R over a 200 trade window. That is what set the soft stop, not the average equity curve.

I still run bootstrap on final P and L to see how often a quarter lands negative. Which percentile do you size from, and do you refresh the sample every month?