AlgoXpert IS–WFA–OOS: plateaus, purged walk-forward, deploy gates

A strategy that looks great in a single backtest often fails live because of overfitting, selection bias, or fragile execution assumptions. The AlgoXpert Alpha Research Framework turns research into an auditable Deploy / Reject / Refactor decision: plateau selection in-sample, purged rolling walk-forward with majority-pass and catastrophic-veto gates, then a strict OOS holdout under parameter lock.

Why a protocol, not a peak Sharpe

Peak in-sample performance is cheap to manufacture when the search space is large. Harvey–Liu–Zhu style multiple-testing risk, Bailey et al. on backtest overfitting, and López de Prado’s purging/embargo ideas all point the same direction: deployment is a process decision, not a single equity-curve screenshot.

AlgoXpert (Nguyen, Nguyen, Pham; arXiv:2603.09219) packages that into chronology with pre-committed gates. This guide is pedagogy for that loop. Pair it with walk-forward in pure Python, Nested CV, and the Deflated Sharpe calculator / oos-lab helpers — different layers, same honesty goal.

Three stages at a glance

IS (map plateaus, cliff veto, shortlist)
  → G1
Purged rolling WFA (restricted DoF, majority-pass + catastrophic veto, lock θ★)
  → G2
Strict OOS holdout (no further tuning)
  → G3
Verdict: DEPLOY | REJECT | REFACTOR

Defense-in-depth (structural cliff veto, execution/microstructure guards, equity protection / kill switch) is meant to run throughout, not as an afterthought. Stress envelopes for cost inflation are part of the framework template; the paper’s case study explicitly defers some stress/ablation results.

Stage I — In-Sample plateau selection

Do not lock the single best Sharpe on the IS map. Extreme peaks are usually thin. The stability region in the paper is:

Ω_stable = { θ | SR(θ) ≥ α · SR_opt } with default α = 0.9.

• Viability: require SR_opt ≥ SR_min > 0; otherwise stop with Refactor (Policy A) before plateau work.
• Trade-count filter: drop configs with N_trades(θ) < N_min so thin lucky samples do not enter the shortlist.
• Cliff veto: reject configs whose neighborhood (one grid step) shows Sharpe collapse or MaxDD spike beyond pre-set τ_SR / τ_DD.
• Rank inside the plateau by tail-risk / recovery criteria (e.g. Calmar, recovery, worst-window), not by chasing the absolute SR peak again.
• Parameter lock: after shortlist, freeze cliff-prone / high-leverage dimensions; WFA may only re-select inside the shortlist — not reopen full Θ.

Stage II — Purged rolling walk-forward

Each fold is W_train → purge gap g → W_test. Purge reduces lookback overlap and, for stateful/path-dependent strategies (grid, trailing, inventory), state bleed. At the start of each test window, normalize state (flat / no inventory / reset internals) so forward metrics are not gifts from prior path.

Per fold, choose θ_i only from Ω_stable (restricted degrees of freedom). Forward metrics on W_test are the gate anchors; train metrics are diagnostic-only — not a second tuning loop.

Majority-pass + catastrophic veto

Pre-commit a forward benchmark vector b and a majority threshold q. A fold passes if its forward metric vector meets b. Overall WFA passes if the share of evaluable passing folds is ≥ q. If any fold trips a catastrophic veto (drawdown integrity breach, kill-switch / leverage-cap violation, …), the entire WFA fails immediately.

If WFA fails, OOS is not opened. If it passes, lock θ★ by a rule written before OOS (e.g. best median forward SR, or risk criteria across test windows).

Resilience ratio η_i = SR_test / SR_train and ΔSR are reported as diagnostics only — never as gate inputs after the fact.

Stage III — Strict OOS under parameter lock

Open the holdout once. No retuning, no “just one more grid.” Compare against the same pre-committed b under declared execution assumptions. OOS is the strictest chronological check in the protocol; it is also where selection bias shows up if you cheated earlier.

Deploy / Reject / Refactor (Policy A)

• G1 (IS) fail → Refactor — structural / design-level failure, not a parameter nudge.
• G2 (WFA) fail → Reject — insufficient sequential adaptability.
• G3 (OOS) fail → Reject — failed strict holdout.
• All pass → Deploy — only under the stated execution assumptions; stress envelope still required for full deployment readiness per the paper.

Attributed case study (paper figures only)

Execution caveat the authors state plainly: ideal MT5 tick model (no explicit latency / adverse slippage beyond defaults). Time-robustness under that baseline ≠ execution-robustness under cost inflation. Treat their numbers as the paper’s illustrations, not Quant for Free backtests.

Limitations the paper owns

• Simplified execution assumptions; stress/ablation incomplete in the reported version.
• Single asset/broker case (USDJPY M5 / Exness).
• Comparing multiple post-validation variants increases research-level degrees of freedom — small OOS gaps need DSR / SPA / HAC-style care.
• Trades/day density was flagged as a reporting gap in places where logs could not confirm it.

Questions