Author
Apply realistic costs from Stage 2, not at the end. HFT strategies are cost-dominated; low-frequency edge-dominated. Teams that fall in love with pre-cost numbers resist the post-cost reality.
A strategy is more than net profit. The evaluation panel has 25 metrics across 7 categories. The panel reveals the failure axis. Headline numbers project away the operational info. Report all 25.
Profit factor is dominated by extreme trades. Two strategies at PF 2.0 can have very different OOS prospects. Report trimmed PF and Gini of trade P&L alongside the headline. Don't optimize on PF.
MAE is the largest unrealized loss; MFE the largest unrealized gain. Plots reveal what aggregates hide: bounceback rates, profit capture, exit timing. Compute before adding stops or profit targets.
A stop loss truncates losses (lowers drawdown) and cuts winners that briefly moved against entry (lowers expectancy). Mean reversion: second effect dominates. Trend following: first does. Test it.
Optimization surfaces are hills, spikes, or cliffs. Hill: pick the center, deploy. Spike: reject. Cliff: pick away from the boundary. Same IS Sharpe; very different OOS. Read the shape.
The IS-optimum is the peak. The plateau is a wide region of similar Sharpes. Pick the plateau center. Peak inherits search-width bias; plateau does not. Lower IS, higher OOS. Pick the plateau.
Walk-forward chains many IS-OOS pairs, producing many OOS realizations and a temporal trend. A single split is one realization. Read the per-step distribution and trend, not the headline alone.
PBO is the probability the IS-optimal parameter ranks below median OOS. CSCV enumerates all S-of-2S splits. Under 0.10 is robust; over 0.50 is pure overfit. Read the histogram, not the number.
Permutation tests build the null by reordering the indicator while keeping returns fixed. No normality or i.i.d. assumed. Use block permutation for autocorrelated indicators. Pair with effect size.
Monte Carlo has two flavors: bootstrap of trades (IS-distribution) and synthetic paths (model-conditional). Set kill-switches at bootstrap 99th percentile, not IS maximum. Not a substitute for OOS.
The 30-trade CLT threshold is about the sampling distribution of the mean, not deployment readiness. SE is 0.27 R for typical sigma. CI is wide. Each strategy property needs 200-1000 trades, not 30.