Optimization climbs the sharpest peak. Robustness picks the widest plateau. The IS-optimal set is rarely robust. The robust set is rarely IS-optimal. OOS is the true distribution. Pick the plateau.
A backtest is informative about regimes it covers, silent about the rest. Stratify by vol, trend, correlation, macro, microstructure. Report cell Sharpe with CI. Ship when every required cell passes.
The rolling window is a hyperparameter. Picking W by IS Sharpe inflates the estimate. Pick from a structural prior matched to the strategy timescale. Audit for look-ahead. Do not retune on live P&L.
Stationarization destroys signal when the strategy lives in the non-stationary part. 12-1 momentum raw Sharpe 0.55, time-series z-scored 0.18. Find the signal axis, transform only the orthogonal one.
Raw indicators have non-stationary distributions that break threshold rules across decades. Match the violation to the transformation. Eight classes, eight recipes. Verify causally. Do not stack.
Returns are a near-random walk daily. Volatility has long memory: ACF +0.27, variance ratio 1.8 at 252 days. Vol is structurally easier to predict than direction. Build feature hierarchies around it.
Lifetime Sharpe is regime-weighted, not favorable-year. Each strategy has a vol regime where it lives and one where it dies. Gate at deployment, calibrate against the full distribution.
Most OOS failures get blamed on overfitting. Often it is regime change between IS and OOS. Run the diagnostic flow: permutation, pipeline audit, sample size, regime overlap test, in order.
A trading system never dies without warning. Eight diagnostic metrics flash months before the equity curve. Read the metrics. Hard kill switches written at deployment, automatic at threshold.
Every strategy has a finite life. Four decay mechanisms: crowding, regime drift, microstructure, capacity. Size against half-life. Run a shadow book. Write the decommission policy at deployment.
Slow drift is harder to detect than abrupt breaks because standard tests have weak power against it. CUSUM and long-vs-short divergence catch what ADF misses. Trust the alarm when it triggers.
Every trading rule assumes stationarity. Markets violate it constantly. Strategies do not break because the rule fails; they break because the regime moves. Rolling stats first, then ADF and KPSS.