habitatfrontier · quantitative research · backtests
Did the backtest’s state ever see the future?
Look-ahead is the quiet killer of quantitative research: an accumulated state — a covariance, a risk model, a factor history — conditioned, anywhere, on information not yet available at the time being simulated. Everyone in the chain owes the same claim: at every date, the state used only what was known by that date — and here is proof.
Value Given a declared as-of state of a dataset, the receipt reproduces that state's digest byte-for-byte and flags any later restatement of the same closed window.
Finding In the series it caught a real vendor restatement of a closed 2018-2022 window: between two pinned pulls the publisher's file moved from the 202605 to the 202608 CRSP database and 1133 of 1258 digests for the same closed window changed.
Boundary it does NOT certify the as-of data is correct, only that the declared state is reproduced and a restatement of the closed window is detected; for Fama-French the publisher publishes no vintages, so that pair is self-captured between pins.
the pain
Today that claim is an assertion backed by procedure — point-in-time databases, code review, methodology documents — and allocators discount backtests wholesale precisely because the assertion is uncheckable. The fund owes it to its allocator; the model developer owes it to model validation under SR 26-2; the index and risk-model vendor owes it to licensees; the researcher owes it to the journal. None of them can currently hand over anything a skeptic can recompute.
the exhibit — recompute it yourself
corpus: real pinned data — Ken French Data Library, daily research factors (mkt_rf, smb, hml, rf) · 1,259 trading days, 2018-01-02 → 2022-12-30 · the vendor archive ships inside the kit, sha256-pinned (af8aec07…), fetched 2026-07-08 — the vendor restates monthly; this pin is the record · dates gated strictly increasing: the history was ingested in order. Register: detection — the argmax cell (mkt_rf×mkt_rf) is measured, not a claim about which row peeked.
The state at each date is a pure function of the trace prefix V[:i+1] — recomputed on your machine, byte-for-byte, at every index. A backtest reading these states read only what was known at each date.
This red is a deliberate fold-in we run as a brake test. Had the history peeked, the GREEN match was impossible: byte-divergence at the peek and every index after, ringing past the N-matched worst-case cost of re-ingesting a row the state already held.
Declared honestly, beside the headline: a statistically unremarkable future row (the median control, 2022-10-20) also breaks byte-equality at every subsequent index — detection is byte-level — but lands at ~0.032× the duplicate floor, within the corpus’s own null: an uninformative future row changes bytes, not materiality. And a survivorship forge — the 5 worst pre-peek days dropped retroactively — fails to reproduce the digest chain from index 23, the first dropped row, onward. A cleaned-up history cannot forge this receipt.
recompute fingerprint — the final chain digest the kit’s verify step reproduces from the pinned vendor archive alone, byte-for-byte on any engine that follows the kit’s published canonicalization · v3 =
be07f3713ccc406cfd6e880a6b92dadedf6950f503e326ebc2bdfd9f897b2472
kit fingerprint · sha256 =
982ec9f9ca86bc70c9a93f7056f6a67c4e4e05dea39015d511bb9c739db4aeaf
The verifier re-derives the
trace from the raw vendor zip, replays all 1,258 prefixes, and runs the peek, median, and
survivorship controls — in a clean environment (numpy only).
what this does not prove
The receipt covers the substrate’s derived state — the covariance/metric layer a strategy reads — not arbitrary downstream code: a signal computed outside the substrate can still peek, and this certificate says exactly which state it certifies. It is a frontier receipt: the accumulation must run on the substrate from the start — you cannot retro-certify a history assembled elsewhere; the buyer adopts the substrate to get the receipt. RED is detection, not localization: the argmax cell is measured, and which row peeked is a separate claim this kit does not make. Vendor restatements complicate “the record up to i” — that is the revision-materiality case, and the two compose. And it is a snapshot: this trace, these pins — not the strategy’s future conduct.
verify this receipt in your browser
This case's receipt records its verdict and both digests above, and carries its own SHA-256. Recompute it here (WebCrypto, in your tab, nothing sent) and it matches; alter one character of a copy and it fails. It is the smallest version of the whole move: recompute, don't trust. The kit below lets you recompute the geometry digest itself on your bench.
published fingerprint ·
or download this receipt and check it on your bench:
receipt-point-in-time.json.
shasum -a 256 prints the same number. Every receipt is on
the ledger.
request the kit
The point-in-time receipt kit — pinned vendor archive, trace, manifest,
verify.py. A person sends it: the kit and its
verification steps arrive by reply, usually within a business day.