Retrospective Backtest — escaping the backtest-of-one
Adopted 2026-06-10 as a hard doctrine change (see changelog-2026-06-10-hard-changes). This file holds the harness and the initial run.
The problem this fixes: everything about the kit's quality is currently theoretical. There is no realized track record — only a beautifully specified process and a few weeks of holding cash. The live six-month-test is the first real evidence, and it rests on a sample of zero realized trades. A retrospective backtest converts "well-specified but unproven" into an actual prior, before the live verdict lands, by applying the current doctrine to historical names whose outcomes are already known.
What it tests
Two failure modes the doctrine is most exposed to, and which the backtest is designed to detect:
- Value-trap / permanent-loss avoidance — does the EPV floor + asset value + permanent-capital-loss screen + quality-of-earnings checks actually keep the kit out of the names that went to zero or stayed broken?
- Compounder miss (over-conservatism) — does EPV-anchored deep value systematically pass on durable compounders because they "never look cheap," forgoing large returns? This is the same failure the Shadow-Book and Near-Miss-Ledger track live.
A doctrine that aces (1) but fails (2) is not neutral — it is a specific, correctable bias, and the backtest's job is to quantify which way the kit leans.
Method (the harness)
For each historical name, at a chosen decision date with a known forward outcome:
- Pull the financials as they stood at the decision date (point-in-time, via the data connectors — figure-level cited per data-integrity-gate). No hindsight figures.
- Apply the current doctrine cold: EPV floor, three Greenwald gating tests, the banded-valuation-standard band + confidence tag + ±20% flip test, MoS band, permanent-loss screen.
- Record the verdict the doctrine would have produced (buy / pass-with-trigger / pass / avoid) and the trigger.
- Compare to the actual forward outcome (price path, whether the trigger filled, terminal result).
- Score the call: right / wrong / too-early / noise, and tag the failure mode if wrong.
Scoring is process-first: a pass on a name that later soared is only a "wrong-pass" if the doctrine should have caught it within its own rules; if it was correctly outside the circle or genuinely unknowable at the time, it is "out-of-mandate," not a miss.
Status of the initial run
The initial run below is a directional reconstruction from known historical outcomes, recorded to establish the harness and a first read on the kit's lean. The figures are not yet figure-level T1-sourced — per the data-integrity-gate, precise point-in-time EPV recomputation via the data connectors is the required next step before any of these scores are treated as load-bearing. They are tagged accordingly. This honesty is itself a test of the new doctrine: we do not let a clean-looking table imply precision the sourcing has not earned.
Initial run (directional — pending T1 figure-level pull)
[T3/AS-cal: outcomes from public price history; doctrine-verdict reconstructed, not yet recomputed from point-in-time filings]
| # | Name | Decision date | Setup at the time | Doctrine verdict (reconstructed) | Actual outcome | Call | Failure mode if wrong |
|---|---|---|---|---|---|---|---|
| 1 | Bed Bath & Beyond | 2021 mid | Negative/eroding FCF, melting asset base, leverage; "cheap" on multiples | avoid (fails permanent-loss screen; no EPV floor) | Bankrupt 2023, equity → 0 | right-avoid | — |
| 2 | Peloton | 2021 late | Pandemic-pull-forward demand, no durable moat, cash burn | avoid / pass (moat not nameable; runway = pull-forward, fails gating) | −90%+ collapse | right-pass | — |
| 3 | Valeant/Bausch | 2015 | Roll-up, aggressive accounting, heavy leverage | avoid (QoE flags + leverage permanent-loss path) | −90% collapse | right-avoid | — |
| 4 | SVB Financial | 2022 | Duration mismatch, deposit concentration, thin tangible capital under stress | avoid (balance-sheet permanent-loss screen) | Failed 2023, equity → 0 | right-avoid (if screen applied to HTM marks) | — |
| 5 | Intel | 2021 | Falling ROIC, process-node moat eroding, heavy capex | pass / pass-with-trigger low (moat-collapse risk; growth gating fails) | Multi-year decline, dividend cut | right-pass | — |
| 6 | ExxonMobil | 2020 trough | Capital-cycle trough, asset floor intact, dividend stress but solvent | buy / pass-with-trigger (cyclical-trough, asset floor holds) | ~3x off the lows into 2022 | right-buy (capital-cycle call) | — |
| 7 | Meta Platforms | 2022 ~$90 | Profitable, huge FCF, narrative panic (metaverse spend), buyback capacity | buy on EPV alone (price near EPV-only with optionality free) | ~5x rebound by 2024 | right-buy (if EPV floor trusted over narrative) | — |
| 8 | Moody's | multiple | Exceptional moat (duopoly), but rarely below EPV-only; always "expensive" | pass (too expensive on EPV-only) repeatedly | Long-run compounder, large gains | WRONG-pass | compounder miss / over-conservatism |
| 9 | Mastercard | multiple | Wide-moat network, high ROIC, persistent premium multiple | pass (never cheap on EPV) | Long-run compounder | WRONG-pass | compounder miss / over-conservatism |
| 10 | Domino's Pizza | 2010s | High ROIC franchise, leveraged-recap, rarely cheap on assets | pass | ~20x+ over the decade | WRONG-pass | compounder miss / over-conservatism |
| 11 | General Electric | 2018 | Opaque conglomerate, hidden liabilities (insurance, power), leverage | avoid / pass (complexity + permanent-loss flags) | Deep drawdown then post-breakup recovery | too-early / mixed | complexity discount cut both ways |
| 12 | First Republic Bank | 2022 | "Quality" franchise narrative, but rate/duration + deposit-flight exposure | avoid (balance-sheet screen) if applied; trap if bought on franchise narrative | Failed 2023, equity → 0 | right-avoid (screen) | — |
First read on the kit's lean
From this directional sample, the pattern is the one the critique predicted:
- Permanent-loss avoidance: strong. On the trap/blowup names (BBBY, Peloton, Valeant, SVB, First Republic, Intel) the doctrine's screens point the right way. The kit's core competence — not losing money permanently — appears real.
- Compounder miss: the live weakness. Moody's, Mastercard, Domino's — wide-moat compounders that EPV-only chronically calls "too expensive" — are systematic wrong-passes. This is exactly the over-conservatism the banded-valuation-standard confidence tags and the Shadow-Book waiting-cost metric are meant to surface, and it argues the 2026-05-24 shift to Greenwald-modified (crediting growth value when gating tests pass) was directionally correct and possibly still too timid for verified-exceptional moats.
This is a hypothesis, not a conclusion — N=12, reconstructed, not yet T1-sourced. It does not by itself justify changing the MoS bands.
Required next steps before these scores are load-bearing
- Re-pull each name's point-in-time financials via the data connectors, figure-level cited.
- Recompute the band + confidence tag + flip test properly for at least the three compounder-miss cases (the highest-value lesson).
- Expand to N ≥ 20 across multiple regimes (a rate-up window, a rate-down window, a value cycle, a growth cycle) before the methodology-calibration bar for revising a band is met.
- Add the compounder-miss names to the Near-Miss-Ledger retroactively as documented historical wrong-passes.
Linked
- methodology-calibration — the bar for acting on a detected bias (10+ cases, multi-regime, retroactive test)
- banded-valuation-standard — the confidence/flip machinery the recompute will use
- data-integrity-gate — why the current figures are flagged not-yet-load-bearing
- Shadow-Book / Near-Miss-Ledger — the live analogues of this retrospective
- six-month-test — the forward test this provides a prior for
- changelog-2026-06-10-hard-changes — adoption record