Walk-forward chains many IS-OOS pairs, producing many OOS realizations and a temporal trend. A single split is one realization. Read the per-step distribution and trend, not the headline alone.
PBO is the probability the IS-optimal parameter ranks below median OOS. CSCV enumerates all S-of-2S splits. Under 0.10 is robust; over 0.50 is pure overfit. Read the histogram, not the number.
Permutation tests build the null by reordering the indicator while keeping returns fixed. No normality or i.i.d. assumed. Use block permutation for autocorrelated indicators. Pair with effect size.
Monte Carlo has two flavors: bootstrap of trades (IS-distribution) and synthetic paths (model-conditional). Set kill-switches at bootstrap 99th percentile, not IS maximum. Not a substitute for OOS.
The 30-trade CLT threshold is about the sampling distribution of the mean, not deployment readiness. SE is 0.27 R for typical sigma. CI is wide. Each strategy property needs 200-1000 trades, not 30.
The standard error of a Sharpe estimate is roughly 1/sqrt(N). At SR=1.0, 100 trades give CI +/-0.24, 500 give +/-0.11. Effective N corrects for hold-time, cross-section, gating. Report the bracket.
The 10% rule: parameters / effective trades stays below 10%. Below 5% preferred. Above 10% is exploratory. Bias-to-noise scales with sqrt(p N). Count all categories.
More parameters raise IS Sharpe and pitch quality. More parameters lower the bias-corrected Sharpe and OOS survival. The curves diverge after a small count. The mandate-winner is not the survivor.
Eight DoF categories, with numeric parameters as one of them. A "simple" strategy often hides thousands or millions of configurations. Count them all, apply the bias correction.
Right order: hypothesis, prototype at structural priors, parameter-stability map, then small validated optimization. Optimization first is search-width bias dressed as rigor. Test first, then tune.
Markets have personalities: efficiency ratio, autocorrelation, skew, vol regime, macro driver. Six families, six framework fits. SPX is not gold, gold is not BTC. Match framework to personality.
"Works on all markets" usually hides per-market tuning, cherry-picking, or long-bias. Real edges are family-specific. Demand the full matrix: fixed parameters, after costs, multi-test corrected.