Pass Lab — Cross-firm propfirm backtest pass rate

How Pass Lab computes backtest pass rate

For each firm in the catalog, Pass Lab slides a window matching the firm's evaluation period (e.g. 30 days for FTMO 2-step Phase 1) across the user's MT4/MT5 backtest with stride = window/2. A 1-year backtest yields ~24 windows, a 2-year backtest yields ~48. For unlimited-time challenges (FundedNext, FundingPips, FXIFY, Goat) we cap window length at 60 days — long enough for portfolios to compound, short enough for the bootstrap to have enough samples.

For each window we replay the trades 1,000 times with within-day trade order shuffled. Within-day shuffling preserves daily clustering (so daily DD remains realistic) but captures path-dependence: the same trades in different intraday order can pass or fail daily-DD rules differently. The window pass-rate is the fraction of replays that survived every firm rule.

The per-window pass-rates form a sample. Bootstrap that sample 5,000 times to estimate uncertainty: resample N windows with replacement, compute the mean of each resample, then take the 2.5th and 97.5th percentiles of resampled means. That is the 95% confidence interval. The CI captures sampling uncertainty across walk-forward windows; narrower CIs mean tighter agreement across the full backtest period.

Why CI lower bound, not point estimate

Most "propfirm pass-rate" tools rank by point estimate. That is gameable (cherry-pick favorable backtests, claim 95% pass rate) and brittle (small sample sizes inflate point estimates). Pass Lab ranks viable firms by their CI lower bound, not their point estimate — the Verdict it returns is conservative by design.

Concrete example: if Firm A has CI [40%, 95%] and Firm B has CI [55%, 90%], Pass Lab surfaces B as the primary backtest match even though A's point estimate (67.5%) is higher than B's (72.5%). The lower bound for B is more stable in the backtest data than the lower bound for A.

Backtest match strength tiers reflect the CI lower bound: ≥70% is a Strong Backtest Match, ≥50% is Moderate, ≥30% is Weak. Below 30% returns "no firm meets backtest threshold" rather than a misleading high-but-uncertain estimate.

Sample-size guard and failure attribution

Pass Lab requires at least 12 walk-forward windows of data. Below that the bootstrap CI is too unstable to be useful — too few unique resamples to estimate the tails. The Verdict returns NoneViable in that case rather than a misleading high-but-uncertain estimate, with a warning that explains why.

For each firm, Pass Lab also reports failure-mode attribution: which rule fired most often across failed iterations (Daily DD breach, Total DD breach, Profit target not met, etc.). When a single firm-side rule (weekend holding, news trading) accounts for ≥70% of failures, the Verdict surfaces it as "Rule mismatch" instead of a misleading 0% CI-low — telling the user the portfolio is structurally incompatible with that firm rather than statistically inferior.

The Verdict also identifies the worst-EA contributor: which EA in the portfolio was the largest negative contributor on failure-causing days across the most failed iterations. This makes the failure attribution actionable.

What Pass Lab does not do

Pass Lab does not predict your future pass rate on a real account. The Verdict is a backtest pass rate with a 95% confidence interval — a measurement on past simulations, not a forecast. Real trading involves slippage, latency, broker-dependent execution, and live-market dynamics that backtests do not model. Pass Lab is intended as a comparison and research tool, not as financial advice.

Pass Lab does not currently support path-dependent EA strategies (grid, martingale, hedging, recovery). The Monte Carlo trade-shuffle assumes within-day exchangeability, which path-dependent strategies violate categorically. Portfolios containing flagged EAs are excluded from the primary match selection and surfaced separately for transparency.

Pass Lab does not access your broker account or execute trades. All processing is client-side: trade data never leaves your browser. The Verified Badge generator embeds the high-level summary (CI bounds, master seed, issued date) in the URL fragment, which browsers do not transmit to servers.

Reproducibility and audit

Same input + same seed = bit-identical Verdict. Every randomness source is seeded with rand_chacha (deterministic across platforms). The engine is implemented in Rust and compiled to WebAssembly (~300 KB bundle). The full methodology is open and documented at fxoptimize.com/blog/pass-lab-methodology.html.

Pass Lab's test suite includes 95+ unit tests + 7 mandatory integration tests covering determinism, sample-size guards, CI sanity, failure-mode attribution, primary-match selection logic, cross-firm consistency, and edge cases (zero-firm-viable, insufficient samples, rule-mismatch detection).