The documents below are reproduced as written, in the order they were produced. They use the first person and record errors as they were found, including two hypotheses that were rejected by tests the author designed to check them. They are retained essentially unedited — sanitising a defect log removes the part worth reading — with dated amendments inserted where later work contradicted them, rather than silent corrections.
The headline results and their inference are in PAPER.md. This appendix exists
to show how they were arrived at, and what was discarded along the way.
Verdict: GO, and the international version is the stronger project.
Jarod van Eijsden · 2026-08-16 · supersedes the auto-generated memo in phase0_out/
Sample: Compustat Global g_fundq (3,573,235 rows) and Compustat North America
fundq (1,681,518 rows as pulled, 1,680,197 after de-duplicating on
(gvkey, datadate), which is the figure the coverage funnel uses), datadate
1990-01 to 2026-07, pulled over the WRDS
PostgreSQL API. Reproduced by fetch_wrds.py → coverage_check.py → diagnose.py.
Compustat Global has no rdq. It has pdateq (preliminary date) and fdateq
(final date). North America has all three, so the US sample calibrates the
international one — and on 350k overlapping US firm-quarters pdateq lands
within one day of rdq at the median and within five days for 80–90% of
observations. A pdateq-based international sample is measuring the same thing
as an rdq-based US one, with a quantified error band.
That makes the cross-country reporting-lag comparison buildable. Here it is — 2015–2024, reported halves only, 792,261 firm-halves, 58 countries with ≥1,500 observations:
| Country | Median lag (days) | p90 | Share not public at 182 days | |
|---|---|---|---|---|
| Fastest | OMN | 20 | 72 | 3.0% |
| USA | 41 | 88 | 1.5% | |
| JPN | 42 | 46 | 0.2% | |
| IND | 58 | 126 | 5.9% | |
| CAN | 59 | 119 | 0.6% | |
| AUS | 68 | 97 | 4.2% | |
| CHN | 74 | 135 | 4.1% | |
| GBR | 78 | 161 | 7.0% | |
| DEU | 84 | 181 | 9.7% | |
| ITA | 87 | 168 | 7.6% | |
| FRA | 89 | 153 | 5.4% | |
| GRC | 100 | 178 | 8.4% | |
| Slowest | BGD | 116 | 230 | 17.4% |
The result that matters is the last column. The Fama-French six-month lag is a single global constant. Against it:
-
In the US it is 98.5% safe — and wildly over-conservative, since the median figure was public after 41 days. Roughly 4.7 months of availability is being discarded by convention.
Amended 2026-08-17 after Phase 2. This originally read "4.7 months of usable signal", which implied a cost. Phase 2 measured that cost in returns and found it is small but not uniformly zero — the difference in four-factor alpha runs from −2.84pp to +1.14pp across the three signals, and only one of the six contrasts clears 1% significance. [Amended 20 August 2026: those endpoints are from the pre-correction run. On the committed tables the four-factor range is −1.84pp to +1.46pp and the raw paired range −2.86pp to +1.34pp; the one-of-six count still holds.] The days are discarded; the alpha is not, because these signals decay slowly enough that five months of staleness does not bite. The convention wastes time, not money — in the US. The claim below, about markets where the convention is not conservative enough, is unaffected and remains the live result.
-
In Bangladesh (17.4%), Romania (15.8%), Spain (14.0%), Vietnam (13.4%), Russia (12.9%), Bulgaria (12.4%), Cyprus (11.6%) it is not conservative enough. Between one in six and one in nine firm-halves were still not public at the six-month mark, so a backtest applying the convention is using data that did not yet exist.
One uniform lag is therefore wrong in both directions at once: it throws away alpha in the fast markets and manufactures look-ahead bias in the slow ones. That is the paper, and it fell out of the availability check before a single signal was built.
Claim 1 — Global's mediocre coverage is row-type dilution. CONFIRMED.
g_fundq returns both HIST_STD (as originally reported) and RST_STD
(restated) rows. The restated rows carry no dates at all — 0% on both
pdateq and fdateq. They are 13.8% of rows and were also the source of a 13.9%
duplicate rate on (gvkey, datadate). Filtering to HIST_STD lifts pdateq
coverage to 97% from 2015 onward and cuts duplicates to 0.09%.
Claim 2 — North America's declining rdq is the same dilution. REFUTED.
fundq returned a single row type (STD/INDL/C/D, 99.997% of rows). There is
nothing to dilute. The decline is real: rdq coverage falls from 73% in 2012 to
53% in 2025 while row counts rise. It is not country-driven either — USA 68%,
Canada 71%, and the spread across the top eight countries of incorporation is
narrow. The remaining explanation is universe composition: Compustat NA has been
adding entities that never announce quarterly earnings. This is the one open
item, and it resolves in Phase 1 — see §4.
Claim 3 — the huge early-decade lags are backfill stamps. CONFIRMED.
Median lag to pdateq was 5,130 days for 1990s Global and 6,749 days for 1990s
NA. Fourteen-year "reporting delays" are bulk-load timestamps on backfilled
history, not availability dates. The first year from which the median lag stays
below 400 days for the rest of the sample is 2007 for Global and 2006 for
North America (rdq itself is clean from 1995).
The first country table used all four fiscal quarters and gave the UK a median lag of 132 days and Australia 134 — roughly 3.5× the US. That would have been the headline, and it would have been wrong.
Both are semi-annual reporting regimes. Their Q1 and Q3 rows carry stale interim figures, dragging the median up. Restricting to the halves actually reported cuts GBR 132 → 78 and AUS 134 → 68, and the honest spread is ~2.8× rather than ~6×. Japan turned out to have a separate quirk in the same test: fiscal Q2 is populated on 0.4% of rows against 92–97% for the other three.
The check that caught it is in §3 of diagnose.py and cost about thirty seconds.
The conclusion survived; its magnitude did not.
Separately, apdedateq was dropped from the timing set — it is the actual period
end date, a period field rather than an availability field. Its apparent median
lag of zero and ~3% negative values were an artefact of my misclassifying it.
The proposal's §7 said "no international extension in v1." That is now overturned by evidence — the international dimension is where the dispersion is, and dispersion is the finding. Revised scope:
| Decision | Basis | |
|---|---|---|
| Universe | Global + North America | pdateq at 97% from 2015; validated against rdq |
| Sample start | 2007 primary, 2015 for the strongest coverage | backfill-onset test, §2 claim 3 |
| Availability field | pdateq (Global), rdq (NA) |
calibrated at median 0–1 days apart |
| Data format | HIST_STD / STD only |
RST_STD carries no dates and duplicates rows |
| Periods | Reported halves for semi-annual regimes | §3 |
| L2 restatement layer | Deferred, likely dropped | RST_STD exists but has no dates, so it cannot be placed in time |
Kill-criteria status: the proposal said L2 would be dropped if point-in-time
history was too thin. RST_STD rows exist but carry no pdateq/fdateq, so
restated values cannot be dated. L2 is provisionally dropped and the project
becomes a two-layer decomposition — timing (L1) and backfill (L3) — with the
cross-country dimension replacing it as the contribution. That is a smaller
claim than proposed and a more interesting one.
Open item — CLOSED 2026-08-16. The 69% was universe composition, as suspected.
Re-measured through the CCM link (phase1_link.py):
| Step | Firm-quarters | Firms | rdq coverage |
|---|---|---|---|
| all Compustat NA | 1,680,197 | 41,414 | 68.8% |
| linked to CRSP | 932,436 | 22,898 | 85.6% |
| + common equity (shrcd 10/11) | 675,128 | 17,532 | 93.6% |
| + major exchange (NYSE/AMEX/Nasdaq) | 671,240 | 17,527 | 93.7% |
The residual gap is entirely pre-2000. On the study universe from 2000 onward,
rdq coverage is 99.2–100% in every single year. No coverage restriction is
needed on the US arm, and the earlier worry about a silent dropna is moot.
Two side-effects worth keeping. The median US lag on the linked universe is 29–38 days, shorter than the 41 measured on the unfiltered file — so the over-conservatism of the six-month convention is slightly larger than §1 states, not smaller. And the lag drifts upward over the sample, from 30 days in 2000–2003 to 38 in 2024; worth a footnote, and worth checking whether it is composition or genuinely slower filing.
The link also passes an external sanity check: firm counts fall from 6,844 in 2000 to 3,693 in 2019 before recovering to 3,914 in 2024 — the well-documented halving of the US listed universe. A broken link would not reproduce that shape.
ficis country of incorporation, not listing or domicile. Cayman (2,674 firms) and Bermuda (771) are visible in the table as incorporation havens and must be reassigned by listing venue before any country claim is published.pdateqis database availability, not the announcement itself. On US data the two coincide to within a day at the median, but the p90 gap runs 5–11 days and drifts over time. Report it as a bound, not an equality.- Coverage is measured on firm-quarters, not on the capitalisation those firms represent. A market can look slow because of many small late filers while its investable large-cap universe reports on time. Weight by size before concluding anything about a tradeable strategy.
- The 2025–2026 tail shows
fdateqcoverage falling (100% → 80%) because recent periods have not finalised yet. Expected; exclude the open edge.
Phase 0 tried: 2 databases × 4 candidate timing fields, 1 row-type filter fixed by
convention before seeing coverage, 2 period definitions (all quarters vs reported
halves). No selection on outcome — the HIST_STD and INDL-preference choices
were made on documentation grounds, and the one time frequency-based selection
was attempted it picked a zero-coverage row type and was replaced.
Jarod van Eijsden · 2026-08-17 · US common equity, 1995–2024, 359 months
Three published signals, four availability rules, identical universes by construction (cross-section size gap: 0.00 names). The only moving part is when the accounting number is allowed to be known, so every alpha difference below is attributable to timing alone.
| Rule | Data usable from | Meaning |
|---|---|---|
lag0 |
fiscal period end | pure look-ahead — wrong, and the naive default |
rdq |
day after the announcement | the truth |
lag6 |
period end + 6 months | the Fama-French convention |
lag12 |
period end + 12 months | a probe, not a candidate convention |
Alphas are annualised, risk-adjusted against FF factors plus momentum, with Newey-West standard errors at six lags.
Gross profitability, equal-weighted, is the cleanest result in the study:
| Rule | Alpha | t |
|---|---|---|
lag0 |
12.3% | 4.36 |
rdq |
9.5% | 3.29 |
lag6 |
8.3% | 2.95 |
lag12 |
6.3% | 2.34 |
Monotone decay across all four rules. That is the signature of a real signal losing value as it ages, and it is exactly what should happen. Look-ahead bias is 2.8 percentage points a year — 23% of the alpha a naive build would report never existed. Value-weighted gives 2.0pp on the same pattern.
Book-to-market agrees on magnitude: look-ahead of 2.4pp (VW) and 4.4pp (EW), though its rule-by-rule path is noisier and not monotone.
Both signals move in the direction they must. Cheating helps. That is the minimum bar a decomposition has to clear before any of its numbers mean anything, and it is why the asset-growth result in §3 is being withheld rather than reported.
| Signal | VW | EW |
|---|---|---|
| gross profitability | −0.9pp | +1.1pp |
| asset growth | −2.4pp | −2.8pp |
| book-to-market | −0.2pp | +0.2pp |
This corrects the Phase 0 memo. That memo said the convention "discards roughly 4.7 months of usable signal", implying a cost. The days really are discarded — the median US figure is public after 38 days and the convention waits 182. But these signals decay slowly enough that five months of staleness costs nothing measurable in alpha.
Both statements are true and they are not in tension: the convention wastes time, not money. For a US-only book built on slow accounting signals, the six-month habit is defensible and the effort of shortening it is better spent elsewhere. The Phase 0 framing over-claimed and has been amended.
Where the convention does matter is where Phase 0 actually found it: not in the US, but in the markets where 11.6–17.4% of firm-halves are still not public at the six-month mark. That remains the live result.
Asset growth shows a negative look-ahead of −1.9pp (VW) and −7.6pp (EW) — the cheating version losing to the honest one. Extra information cannot systematically hurt, so this is a defect signature rather than a finding.
Three explanations were tested. Two are dead:
- Universe composition. The rules were trading different stocks, because the
staleness cutoff runs from
datadateand a later availability date leaves a shorter usable window. Fixed by intersecting firm-months across all rules. The anomaly barely moved (−7.9 → −7.6pp). Refuted. - A longer-horizon effect. If the asset-growth premium lived beyond the
quarter it was measured in, alpha should keep rising with staleness. It does
not — it peaks at
lag6(13.6%) and falls atlag12(13.0%). Refuted. - Fiscal-quarter mix. A December-year firm seen in July gives the June
quarter under
lag0but the December one underlag6; asset growth is a year-over-year measure and the literature builds it from annual data. Tested in §4. Refuted for the calendar rules — Q4 share is 25.1% underlag0and 25.5% underlag6, flat.
Asset growth is therefore excluded from the L1 headline and logged as open. A fourth story could be constructed — pre-announcement drift running against a long-run reversal is the obvious candidate — but three untested stories in a row is how a result gets talked into existence. It needs its own test, not a narrative.
The fiscal-quarter check refuted its own hypothesis and turned up something better. Share of firm-months holding each fiscal quarter, 1995–2024:
fqtr |
lag0 |
rdq |
lag6 |
lag12 |
|---|---|---|---|---|
| 1 | 23.8 | 23.7 | 24.2 | 24.1 |
| 2 | 25.1 | 25.4 | 25.1 | 25.2 |
| 3 | 25.9 | 32.0 | 25.3 | 25.1 |
| 4 | 25.1 | 18.8 | 25.5 | 25.6 |
Every calendar rule delivers a flat 25/25/25/25. The announcement-date rule does not: it holds Q3 data 32% of the time and Q4 data only 18.8%.
The cause is mechanical and easy to miss. Annual results take longer to announce than interim ones — audit, year-end adjustments, the full report — so the Q4 number arrives late and the Q3 number stays "most recent" for longer. A portfolio timed honestly therefore spends a third of its life on third-quarter data and under a fifth on the audited annual figure.
This matters beyond this study. Any fixed calendar lag implicitly assumes reporting speed is uniform, and it is not uniform even within a single firm's year. It is the same assumption Phase 0 showed failing across countries, reappearing across quarters.
Phase 2 ran 24 portfolio builds: 3 signals × 4 rules × 2 weightings. No alternatives were tried and discarded; the rule set and signal set were fixed before results were seen.
Three defects were found and fixed during the phase, all mine:
- Nullable dtypes. WRDS returns
Float64, whose missing value ispd.NA, which has no truth value. The self-test used plain floats, so it passed while the script crashed on contact. Fixed by coercing at the boundary; the self-test now builds fixtures with realpd.NA. - Different universes. See §3. Fixed by exact intersection, with a warning that fires if cross-section sizes ever diverge by more than half a name.
- Phase 0 over-claim. See §2. Corrected in the Phase 0 memo rather than quietly softened.
The harness itself is validated by a planted look-ahead: a signal that is next
month's return must show a screaming alpha under lag0 and nothing under rdq.
It returns t = 182.7 on the plant and t = 0.1 on a noise control. A harness that
cannot detect a deliberately inserted bias cannot be trusted to measure a real
one.
Jarod van Eijsden · 2026-08-17 · US common equity, 1995–2024, 359 months
Compustat does not add a company from the day it is added — it loads the company's history. Measuring the gap between each firm's earliest fiscal period and the earliest date any of its data could be looked up, across 32,914 firms:
| Years of history pre-loaded at entry | |
|---|---|
| p25 | 1.07 |
| median | 1.93 |
| p75 | 2.61 |
| p90 | 5.37 |
46.0% of firms arrive carrying more than two years of history, and 16.2% of all firm-quarters in Compustat predate their own firm's entry date. One row in six was never visible in real time at the date it describes.
This is arithmetic on two dates and it is the raw material for every backfill result below.
Amended 2026-08-18 after the second review. This originally claimed the measurement needs no model and no threshold. It needs both. Entry date is proxied by the firm's earliest
rdq, so a firm whose early quarters carry no announcement date has its entry placed too late and its history scored as backfilled; and gaps under a quarter are floored to zero. On the unrestricted file, whererdqcoverage is 68.8%, 16.2% is an upper bound rather than a point estimate. The paper now says so.Further amended 20 August 2026. "Upper bound" did not survive either. A second, mechanical upward bias was found — a firm's own entry quarter satisfies the test by construction — and against them an unsized downward one: 29% of firms carry no
rdqanywhere and every quarter of theirs is scored as not backfilled. §5, §7, the abstract and the README now all call 16.2% an approximation with biases in both directions, not a bound.
Seasoning discards each firm's first N months of database coverage — a forty-year-old company added last year is unseasoned; a startup in Compustat since its IPO is not. Alphas annualised, FF4 + momentum, Newey-West:
| Signal | Weight | 0 months | 12 months | 24 months | Backfill bias |
|---|---|---|---|---|---|
| gross profitability | VW | 6.9% | 6.4% | 6.0% | +0.9pp |
| gross profitability | EW | 9.7% | 8.9% | 7.7% | +2.0pp |
| book-to-market | VW | −2.0% | −1.6% | −1.0% | −1.0pp |
| book-to-market | EW | 12.9% | 11.7% | 10.9% | +2.0pp |
Every series is monotone in seasoning. More seasoning, less alpha, for all three series carrying positive alpha — the signature of a real effect rather than noise. The fourth, value-weighted book-to-market, moves the other way, but it is statistically indistinguishable from zero at every level (t between −0.40 and −0.19), so its sign carries no information.
Equal-weighted bias is roughly double value-weighted, which is what the mechanism predicts: backfilled firms are disproportionately small, and equal weighting is where small firms count.
Phase 2 intersected firm-months across its four timing rules so that only the availability date differed. A backfilled observation has no announcement date in the month a calendar rule would first reach for it — so the intersection silently deleted exactly the observations this phase is about.
That was the right call for Phase 2 and it is why those look-ahead numbers are clean timing estimates. But it means the backfill effect was discarded rather than controlled for, and it had to be measured on its own.
The two are not additive as reported. Phase 2's look-ahead compares lag0
against rdq on an intersected universe; Phase 3 compares seasoning levels under
lag6 on the full one. They are separate measurements of similar size pointing
the same way, not two terms of a decomposition that sum. A genuinely additive
figure needs one further run — naive (lag0, no seasoning) against honest
(rdq, 24-month seasoning) on a common sample — and until that is run, the
honest statement is:
For gross profitability equal-weighted, timing look-ahead is worth ~2.8pp a year and backfill ~2.0pp, against a naively-reported alpha of ~12%. Both are material. Their combined effect is not simply 4.8pp and has not yet been measured.
-
Seasoning is a proxy, not the truth. The correct test uses archived database vintages, which this subscription does not include. Seasoning assumes a firm's first two years of coverage are the backfilled ones — true on average, wrong for any firm added in real time at its IPO. It biases the estimate toward zero rather than inflating it, so 2pp is closer to a floor than a ceiling.
Amended 2026-08-18 after the third review. The sign is not established. Seasoning removes genuine real-time entrants along with backfilled ones, and newly listed firms have distinct return behaviour, so the estimate captures the two channels jointly. §7 of the paper now states that the direction of the net error is unknown; this sentence claimed a floor it cannot support.
-
Entry date is inferred from
rdq. For firms whose earliest quarters carry no announcement date,entrylands at the first quarter that has one, which conflates backfilling with missing data. Phase 1 showedrdqcoverage on the study universe is 99%+ from 2000 but weaker before, so this is a pre-2000 concern specifically. -
Nothing here is about survivorship in returns. The Phase 1 panel already handles delisting properly and reconciles to Fama-French at 1.4bp/month. This is bias in the fundamentals, which is a separate channel.
2026-08-17. Findings ordered by severity. Each is stated as a defect with the evidence that establishes it; where a check was run to settle the question, the output is reproduced rather than summarised.
The reviewing standard applied is: would a quantitative researcher reading this cold find a reason to stop trusting it?
The study reports a look-ahead effect of "2.8pp per year" and a backfill effect of "2.0pp" without testing whether either differs from zero. Two alphas with individual t-statistics of 4.4 and 3.3 do not license a claim about their difference.
This is also the cheapest possible fix, because the two legs share a universe and are therefore almost perfectly correlated — the paired difference is far less noisy than either leg. Computed on the saved long-short series:
| Signal | Weight | Difference (%/yr) | NW t | ρ(legs) |
|---|---|---|---|---|
| gross profitability | EW | 2.91 | 8.72 | 0.991 |
| gross profitability | VW | 1.78 | 3.66 | 0.984 |
| book-to-market | EW | 4.04 | 8.60 | 0.991 |
| book-to-market | VW | 2.37 | 2.73 | 0.979 |
| asset growth | EW | −7.20 | −8.44 | 0.945 |
Every difference is significant at the 1% level. The correct estimator is a regression of the difference series on a constant, not a comparison of two separately-estimated alphas.
This strengthens the paper. Omitting it is the single most conspicuous gap.
Asset growth is currently excluded from the headline and logged as an open question on the grounds that a negative look-ahead is a defect signature. Given A1, that classification is wrong: the effect is −7.20%/yr with t = −8.44, stable across Newey-West lags 0 through 18 (t between −8.19 and −9.54).
A robust, highly significant, counterintuitive result is not noise to be set aside. It is the most interesting thing in the study, and three named hypotheses have already been tested and rejected against it. Reframe from "excluded because unexplained" to "a significant negative timing effect that survives three candidate explanations", and state the remaining candidate — a short-horizon drift effect of opposite sign to the long-horizon reversal — as a hypothesis for future work rather than an answer.
A reader who notices that the strongest t-statistic in the study sits in the discarded row will draw an unflattering conclusion about the selection.
The headline exhibit plots country of incorporation and labels the axis "country". Five entries are not markets:
| Code | Firms | Firm-halves | Median lag | % failing at 182d |
|---|---|---|---|---|
| BMU | 771 | 13,117 | 66 | 1.4 |
| JEY | 130 | 1,642 | 74 | 6.8 |
| CYM | 2,674 | 34,034 | 78 | 5.6 |
| GGY | 122 | 1,601 | 87 | 7.3 |
| VGB | 232 | 2,356 | 93 | 10.8 |
That is 9% of chart entries and 6.7% of observations. VGB appears in the ">10% of firm-halves not public at 182 days" list that the README leads with. "The British Virgin Islands has a slow reporting culture" is not a defensible sentence, and a reader from the industry will spot it immediately.
Resolve by reassigning to listing venue via exchg, or by excluding the five and
stating the exclusion. The caveat currently buried in §5 of the Phase 0 memo does
not cover a claim made in the first paragraph of the README.
A book-to-market long-short is regressed on a factor model containing HML, which is itself a book-to-market sort. The reported "alpha" is the residual after controlling for the construct being measured.
The magnitude turns out to be modest — raw mean return 18.23%/yr (t = 4.20) against a reported FF4 alpha of 16.3% for EW, so HML absorbs about 2pp — but the construction invites the objection regardless. Report raw and risk-adjusted side by side.
The mirror-image point should be stated explicitly as a design choice rather than left implicit: FF4 was the correct model here precisely because FF5's RMW would absorb gross profitability the same way HML absorbs book-to-market. Saying so demonstrates the awareness; leaving it silent looks like luck.
Phase 2 was corrected so that all timing rules trade identical firm-months. Phase 3 compares alphas across seasoning levels while the sample falls from 1,685,496 to 1,455,755 firm-months — the same confound, uncorrected.
Unlike Phase 2 this is inherent: seasoning must remove observations, so intersecting would delete the treatment. The comparison is therefore legitimate, but the reader is entitled to see the inconsistency acknowledged rather than discover it. State that the removed observations are the treatment, and add the calendar-composition check — confirm the month count stays at 359 (it does) and report cross-sectional size by seasoning level.
NW_LAGS = 6 appears without explanation. Sensitivity, run on the difference
series:
| Series | L=0 | L=3 | L=6 | L=12 | L=18 |
|---|---|---|---|---|---|
| gp EW | 8.47 | 8.56 | 8.72 | 8.49 | 8.33 |
| bm EW | 7.60 | 8.27 | 8.60 | 9.14 | 9.32 |
| ag EW | −8.19 | −8.55 | −8.44 | −8.94 | −9.54 |
Entirely robust. A single footnote converts an arbitrary constant into a demonstrated non-issue — worth more than the two lines it costs.
The US is measured with rdq; every other country with pdateq. The two were
validated as agreeing to within a day at the median, but the p90 gap runs 5–11
days and drifts over the sample. In an exhibit where the US-to-Germany difference
is 43 days, an instrument inconsistency worth up to 11 days on the reference bar
is material.
Add a robustness row measuring the US with pdateq as well. Coverage is thinner
(24–48%), so it is a check rather than a replacement — but an unchecked
instrument change on the one bar every reader will look at first is a soft spot.
The country exhibit uses 2015–2024; L1 and L3 use 1995–2024. Both choices are
defensible — Global pdateq only reaches full coverage around 2015 — but the
reasoning is not stated, so it reads as arbitrary.
Phase 2 restricts to firm-quarters carrying an announcement date, retaining 69.6%
of the sample. Firms lacking rdq are plausibly smaller and less covered, so the
restriction is unlikely to be random. Report the size and count differences
between retained and dropped firm-quarters; if they are large, the timing
estimates apply to a better-covered subsample and should say so.
Four phase memos plus a README is the right structure for the process and the wrong structure for the product. Restructure as:
PAPER.md— abstract, introduction, data, method, results, robustness, limitations, references. Third person, no phase numbering, results ordered by importance rather than by when they were computed.APPENDIX_process.md— the phase memos and the defect log, unchanged.
The process log is a genuine asset for this audience — a documented record of finding and fixing one's own errors is rarer than a clean result. It reads as a strength when presented as an appendix and as disorganisation when presented as the main text.
The memos use first person and phrasings such as "my bug" and "I was wrong". Correct and valuable in a defect log; wrong in the body of a paper. Move verbatim, do not rewrite — sanitising the defect log would remove the thing that makes it worth reading.
PROPOSAL.mddescribes a US-only three-layer study that the evidence overturned. Mark superseded or remove.- Two files named
PHASE0_MEMO.mdexist — the amended one at the root and the auto-generated one insidephase0_out/. Rename the generated one. phase0/contains every script across four phases. Rename tosrc/.
The signals and corrections are all from the literature and none is cited. A reader from the industry will expect at minimum: Novy-Marx (2013) for gross profitability, Fama and French (1992) for book-to-market, Cooper, Gulen and Schill (2008) for asset growth, Sloan (1996) for accruals, Shumway (1997) and Shumway and Warther (1999) for delisting returns, Newey and West (1987), and Kothari, Shanken and Sloan (1995) on Compustat selection bias. (The Chan, Jegadeesh and Lakonishok paper originally suggested here, and cited in the first version of §5, is the 1996 momentum paper and has nothing to do with backfill. The second review caught it; Banz and Breen (1986) and the two-year exclusion in Fama and French (1993) replaced it.) Absent citations, the work reads as unaware of its own antecedents rather than as building on them.
No LICENSE, no abstract at the top of the README, no statement of what a reader
should take away in one line. The README opens with a question rather than a
result.
Recorded so the list above is not read as a verdict on the whole.
- The Fama-French reconciliation is the strongest thing here. An independently rebuilt CRSP value-weighted market return matching a published series at 1.4bp/month over 419 months is a claim few portfolio pieces can make, and it licenses everything built on the panel.
- The planted-bias tests are the right instinct. Requiring the harness to detect a deliberately inserted look-ahead (t = 182.7) and a deliberately inserted backfill effect before trusting it on real data is a discipline most submitted code does not show.
- The defect log is an asset, not an embarrassment. Five documented self-caught errors, including two hypotheses of the author's own killed by their own tests, demonstrates more than a clean result would.
- Kill criteria were set before results were seen, and one of them fired: L2 was dropped on evidence rather than quietly abandoned.
- The Phase 0 amendment is handled correctly — a visible correction where later work contradicted earlier, not a silent edit.
Recorded so a reader can tell which of the findings in §E changed the paper and which did not. Dated 18 August 2026, after a second review run against the result tables and the scripts rather than against the prose.
| Finding | Disposition |
|---|---|
| A1 headline number carries no standard error | Fixed. robustness.py added; §4.1 now tests the difference series with Newey-West errors and reports ρ between legs. |
| A2 the surprising result is buried | Fixed, then revisited. §4.2 now carries the asset-growth reversal explicitly, and the second review found two of its three rejections did not hold — see below. |
| A3 incorporation havens presented as markets | Fixed. Five domiciles excluded from the chart and from the 53-market count, retained and flagged in the data table. The §E text describing the pre-fix README is left as written. |
| A4 book-to-market alpha is partly circular | Fixed. §6 reports raw against risk-adjusted and quantifies what HML absorbs. |
| B1 Phase 3 does not meet the Phase 2 standard | Fixed in the second pass, and it changed the result. phase3_l3.py now reports the paired Newey-West test on the seasoning difference series. Only one of the four contrasts clears |t| = 2; the backfill layer is downgraded from "worth 2pp" to "worth up to 2pp, weakest of the three results" in the abstract, §5, §8 and the README. |
| B2 Newey-West lag asserted, not justified | Fixed. §6 reports five truncations, and after the second review reports the value-weighted rows as well as the equal-weighted ones. |
| B3 one bar measured with a different instrument | Fixed. §7 carries it, and §2.2 now states what the pdateq validation does and does not license. |
| B4 sample windows differ without explanation | Fixed. §4 states the return sample period; the availability window is stated in §3. |
| B5 the 30% sample restriction is untested | Partly. §6 now distinguishes the raw-file figure from the study-universe figure. Selectivity itself is still not tested. |
| C1 reads as a lab notebook | Fixed. PAPER.md and this appendix were separated. |
| C2 register inconsistent | Fixed. Third person throughout the paper; first person retained here. |
| C3 stale artefacts | Partly. src/ renamed and the generated memo renamed. The superseded proposal was not removed; it was replaced by a separate proposal document that is not part of this repository. |
| C4 no references | Fixed, and corrected in the second pass: Banz and Breen (1986), Carhart (1997) and Fama and French (1993) added, a mis-attributed citation removed, an uncited entry dropped. |
| C5 missing repository conventions | Fixed. Licence, .gitignore, requirements.txt, run order. |
A second pass checked every number against results/*.csv and every method
statement against the scripts. The arithmetic held — one rounding error in
roughly 120 reported figures. Three classes of problem did not.
Labels that did not match what was computed. The §4.1 differences are raw
long-short spreads; parts of the paper described them as differences in alpha,
and §4.2 quoted the alpha version seven lines under a table of the raw one. Both
are now labelled and both are shown. The §2 data description named a filter
(indfmt in ('INDL','FS')) used for the availability extract but not for the
return tests, which use INDL only.
Two hypotheses rejected on the wrong test. §4.2 rejected fiscal-quarter
composition by comparing lag0 with lag6 — two calendar rules, both flat
across quarters — when the effect under investigation is lag0 against rdq,
where the fourth-quarter share moves 25.1% to 18.8%. phase2_l1.py prints a
warning at exactly this condition and it fired during the run that produced the
paper. The longer-horizon candidate was rejected on a 0.58pp decline carrying no
test statistic, while the tested step in the same ladder is significant at 5% in
the opposite direction. Both are now recorded as not rejected, and
phase4_quarter.py was written to decompose the contrast within fiscal quarter,
and it settled both: the longer-horizon candidate stands, and fiscal-quarter
composition is rejected properly — −1.32pp of a −7.42pp total, t = −0.84, with
the reversal present inside all four fiscal quarters. The original verdict was
right and the original test was not, which is a distinction worth keeping
visible.
A conclusion the body contradicted. §8 claimed no detectable convention cost (§4.1 reports one at t = 3.01), described the timing and backfill effects as additive (§5 states in bold that they are not), and said both were avoidable with two extra columns (§7 states the backfill correction needs archived vintages). All three are corrected.
Also corrected: a claim in the README that every script self-tests, when two did not — both now do; the §5 assertion that the backfill measurement needs no model and no threshold, when entry date is proxied by the first announcement date and sub-quarter gaps are floored; and the absence of any multiple-testing statement across eighteen contrasts.
Run 18 August 2026 after the second review. Written because Phase 2 rejected the
fiscal-quarter explanation using lag0 against lag6 — two calendar rules whose
quarter mix is flat by construction — when the effect under investigation is
lag0 against rdq, where the fourth-quarter share moves from 25.1% to 18.8%.
phase2_l1.py prints a warning at exactly that condition and it fired during the
run that produced the paper. Overriding one's own instrumentation is the specific
failure worth recording here.
The decomposition forms the long-short inside each fiscal-quarter stratum, with breakpoints computed inside the stratum, and splits the difference symmetrically into a timing part (mix held fixed) and a composition part. The identity is exact by construction and the self-test asserts it, along with a planted pure composition effect landing entirely in composition and a planted pure timing effect landing entirely in timing.
Result: composition is insignificant in all six cells (|t| ≤ 1.89), and for gross profitability its sign is opposite to the effect it was supposed to explain. For asset growth equal-weighted the announcement rule beats the naive rule inside every fiscal quarter separately — by 9.49, 4.77, 10.23 and 5.15 percentage points a year in Q1 to Q4 — which is not what a composition artefact looks like.
Two things this does not establish. The stratified portfolios are roughly a quarter the size of the pooled ones, so every t here is smaller than its §4.1 counterpart and the insignificance of composition is partly a power statement. And rejecting composition does not explain the reversal; it removes the cheapest explanation and leaves the finding open, which is where the paper leaves it.
A third pass audited the revised paper, the two new estimators, and the artefacts no earlier pass had looked at. Five findings are worth recording because each is a different kind of failure.
A verdict was flipped and its cross-reference was not. §4.3 opened with "the mechanism behind the surviving candidate in §4.2". When that sentence was written, fiscal-quarter composition was the surviving candidate. The second review's decomposition then rejected it and §4.2's verdict changed — but §4.3 still pointed at it, so a reader following the argument in order was told a candidate survived, shown its mechanism, and then told it accounted for nothing. Editing a verdict without re-reading what depends on it is the mistake.
The composition estimate was one number doing two jobs. A component's mean splits into a persistent part and a time-covariance part. The hypothesis under test — the announcement rule holds less annual-report data every year — is about the persistent part alone. Splitting them changed two claims: the asset-growth composition channel is −0.22pp rather than −1.32pp on the persistent measure, which strengthens the rejection; and the claim that composition "runs against the measured effect" for gross profitability held only for the covariance term, with the persistent part running with it. The paper now reports both columns.
"Rejected" is not what a t of −0.84 supports. The composition estimate carries a standard error near 1.6pp, so its interval spans −24% to +59% of the total. The verdict is now "not supported as the principal channel — bounded rather than excluded", which is what the number says.
Three self-tests passed for the wrong reason. Phase 4's channel tests set one
channel exactly to zero, which makes the other vanish identically under any
decomposition of that shape — an asymmetric split passes them unchanged. Its
timing test used a difference that was exactly constant, giving a standard error
of 3e-20 and a t of 1e16, so the t > 10 assertion tested floating-point
round-off. And Phase 3's null-pair test fed two identical series, giving t = NaN,
where not (abs(nan) > 2) is True — it would have passed on an estimator that
always returned NaN. All three now carry noise, and the phase 4 split is pinned
directly to the symmetric form rather than to a degenerate case. A self-test that
cannot fail is worse than no self-test, because it is cited as assurance.
A table with no script behind it. The §3 country table — 53 markets, the
seven markets where the convention fails, and the entire interactive exhibit —
was produced ad hoc while the Phase 0 extracts were open, and could not be
regenerated by anything in the repository. country_lag.py now does it offline
and reproduces every median and firm count exactly against the published table;
observation counts differ by at most 13 rows on duplicate handling. The
regenerated table is the one now shipped, which moved the headline count from
739,511 to 739,470. That is the price of having it reproducible, and it is worth
paying.
Also corrected in this pass: the abstract still said "public availability" and
"unpublished" where §2.2 had been corrected to first appearance in the vendor
database, as did the §3 column header and the whole text layer of the interactive
exhibit; §4.2 asserted that additional information cannot reduce performance,
which holds for an optimal use of information and not for a fixed decile sort;
§6's multiple-testing count predated the estimates §4.4 added and claimed the
truncation grid covered all eighteen contrasts when it covers six; §6 described
two marginal results as "not relied on above" when the abstract and §8 both rely
on them; §8 called the convention "close to costless" on a precision argument
while §4.1 calls the same qualification load-bearing; four sentences narrating
the paper's own revision history were moved here, where they belong; .gitignore
carried two negation rules that excluded nothing because no rule excluded *.csv
in the first place; and numpy was unbounded above a pandas pin that predates
numpy 2.
Two blocking errors, both introduced by earlier corrections rather than present in the original work. That pattern is the finding: each editing pass is itself a source of defects, and a review that only re-reads prose will not catch them.
The paper claimed to exclude financials. It does not, and the filter credited
with excluding them removes nothing. The second review found that §2 described
indfmt in ('INDL','FS') while the return tests used INDL only, and corrected
the description by asserting that financial firms therefore did not enter any
portfolio. That was reasoning from the SQL rather than from the data. Compustat
North America returns industrial presentation for 1,681,465 of 1,681,518 rows —
so the restriction is a no-op there — and 32.7% of firm-quarters, 16,123 of
41,414 firms, carry SIC 6000–6999. There is no industry screen anywhere in the
pipeline. Every source paper for the three signals excludes financials, and "did
you drop financials" is the first question this audience asks. §2 now states
plainly that they are retained, why the alpha levels are consequently not
comparable to published premia, and why the timing differences — computed on a
universe held identical across rules by exact intersection — are not obviously
damaged by it. Adding the screen and re-running §4 to §6 remains open.
A factual error inside the Limitations section. §7 said the United States was
the only market measured on rdq. The instrument is chosen by source file, not
by country, and comp.fundq covers Canada in full and carries
foreign-incorporated SEC filers — so Canada is entirely on rdq, Ireland 43% of
firm-halves, the Netherlands 24%, Israel 18%, France and the United Kingdom 5%,
Japan 2%. The interactive exhibit had it right ("in North America"), and
country_lag.py's docstring repeated the paper's wrong version directly above
the function implementing the correct one. Being wrong in the section whose
purpose is to show you know where the weaknesses are is the worst place to be
wrong.
And a de-duplication that was not de-duplicating. country_lag.py removed
duplicate firm-quarters within each source file. Firms carried in both files —
every Canadian, and every foreign-incorporated SEC filer — therefore appeared
twice, once measured on rdq and once on pdateq, against §2's stated design of
one record per firm-quarter. It affected 1.3% of screened rows, concentrated in
exactly the firms whose two measurements disagree most. De-duplicating across
files, with the North America row winning because rdq is an announcement date
rather than an ingestion stamp, moves the headline from 53 markets and 739,470
firm-halves to 52 and 729,924, moves four quoted medians (United Kingdom 78→79,
Germany 84→86, France 89→91, India 58→59), drops Argentina below the
observation floor, and widens the failure range to 11.7–17.4%. The seven failing
markets are the same seven. country_lag.py --compare flagged the change and
exited non-zero, which is what it is for.
Also corrected: §8 counted four null convention-cost contrasts where the abstract
and §4.1 say five, having silently switched from the 1% threshold to 5%, and then
named three exceptions to a count of four; §3's operative sentence still asserted
non-publication where §2.2 and §8 both say vendor-database availability; §6 put
the asset-growth truncation crossing between six lags and twelve when the file
shows it between three and six, so the six-lag figure used throughout is already
on the significant side; the §4.4 quarter-neutral column was never defined; the
exhibit counted markets above the ten-per-cent line with >=, sweeping in
Germany at 9.98% and captioning eight where the paper says seven; and the
README's bold lede generalised a result that reverses, larger, on the third
signal.
What changed structurally. verify_paper.py now re-derives every tabulated
figure in the paper from results/ and exits non-zero on any mismatch — 191
values, checked at the precision the paper displays, with a self-test that plants
a wrong digit and requires the check to catch it. Three passes of hand-checking
numbers found real errors and cost hours; this makes that class of error a
one-command check for any reader, and removes the need to trust that the author
did it carefully.
The fourth review established that no industry screen existed and that §2 claimed one did. The screen was added and §4 to §6 re-run. Three things about how, and three about what changed.
How. The code is CRSP's siccd, carried on the same interval join as share
code and exchange, so a 1995 portfolio is screened on the classification CRSP
held in 1995. Using Compustat's current sic would have been simpler and would
have put a look-ahead into the one paper that should not contain one. The screen
is applied in Phases 2 to 4 rather than in the panel build, because the market
return that §2.1 reconciles against Fama-French must contain financials; the
reconciliation came back at ρ = 0.999987 and 1.37 bp, unchanged to the digit,
which is the check that the screen landed where it was meant to. And
drop_financials refuses outright on a panel with no siccd column rather than
screening nothing quietly — an older panel would otherwise reproduce the
unscreened numbers while appearing to work.
What changed in the levels. 18.7% of firm-months removed. Value-weighted
gross profitability rose from 7.07% to 7.52% under lag0 and from 5.03% to 5.96%
under rdq, crossing from insignificant to significant at the 5% level — banks
were dragging it down, which is what one would expect of a signal built on cost
of goods sold. The abstract's caveat that the value-weighted contrasts separate
two null alphas now applies to book-to-market alone.
What did not change. The six look-ahead contrasts moved by between 0.07 and 0.48 percentage points, none changed sign, and the same five of six cleared 1%. That is now reported in §6 as a robustness result rather than left implicit: a universe change of nearly a fifth enters both arms of a paired contrast and differences out of it, which is the property the whole design rests on.
Two things got cleaner. The asset-growth value-weighted estimate used to run −1.83 to −2.47 across Newey-West truncations and cross the 5% line inside that grid; it now runs −1.38 to −1.83 and crosses nothing, so §6 no longer has an exception to report and §4.2 no longer has to hedge the small-firm reading on a truncation choice. And the fiscal-quarter composition channel fell from 17.7% of the asset-growth reversal to under 2%, with a 95% interval of −21% to +24% — a rejection rather than a bound.
One more defect, in code written the previous day. Phase 4's
persistent-versus-covariance split was computed in monthly decimals while the
caller compared it against annualised percentages, so the persistent column
printed as 0.000 for every row and the covariance term absorbed each component
whole. It survived because the self-test multiplied by 1200 before asserting: the
test knew the units and the caller did not. The assertion now checks that the
value is in the range an annualised percentage occupies, which is the check that
would have caught it. verify_paper.py's self-test had the same shape of problem
— it anchored on the literal figure +2.91pp, which this re-run moved — and now
finds its anchor by pattern instead. A self-test that breaks whenever the
analysis is re-run is a self-test that gets switched off.
Note on the verifier's value count. Earlier sections quote 191 and 199 values;
the count rises as checks are added and as the paper gains figures. It is 201 as
of the financials screen. The number to trust is whatever verify_paper.py
prints, not any figure written down here.
The instruction that produced this section was, in substance, stop finding issues. It is the right instruction, and the reason it was needed is visible in the record above: the third review's findings were mostly in code written during the second, the fourth's in prose written during the third, and the fifth's in code written during the fourth. Each pass of careful reading repaired real defects and introduced the next pass's. Hand-checking a project of this size does not converge, because the checker and the author are the same person and the error rate on new text is roughly constant.
So this pass added almost no prose. It converted judgement into three mechanical checks and one alignment fix.
The alignment fix. Forward returns were computed after the universe filters
— the CCM-link screen and the new industry screen — so a firm-month adjacent to a
dropped row was matched to the next surviving month's return rather than to the
next calendar month's. With a screen that removes 18.7% of firm-months, that is
not a rare event. forward_return() now runs before every filter and carries an
explicit calendar-adjacency guard: a return is used only if the next observation
is the next calendar month, and the count of voided pairs is printed. The tables
in results/ were produced before this change and have not yet been regenerated;
that is stated in NEXT_STEPS.md as the one open item, and from the next run
onward it is a failing check rather than a note (see the manifests below).
[Amended 19 August 2026: the pipeline was re-run in full on 19 August and the
committed tables are from that run; the manifests carry its timestamps.]
check_repo.py — the repository against its own claims. Several hundred
checks, every one of them a category that has been wrong here at least once
and every one previously caught by a person reading carefully: files the README
links, script counts, section cross-references, bibliography closure in both
directions, the interactive exhibit against its source table cell by cell,
retired phrasing surviving in the exhibit, file counts and byte totals quoted in
the upload instructions, and third-party imports absent from requirements.txt.
Its first run produced two false positives — an exempting qualifier that wrapped
across a line in the rendered HTML, and GitHub's "100 files per upload" limit
read as a claim about this repository — both fixed in the checker rather than
worked around in the text. Its second run failed four checks caused by shipping
the checker itself, which changed the file counts the instructions quote. That is
the tool working.
mutation_test.py — the self-tests against deliberately broken code. Three
of the defects recorded above were self-tests that could not fail. A passing
self-test is weak evidence; one that fails when you damage what it guards is not.
Forty-five targeted mutations — invert the industry screen, delete the year term
from the forward-return alignment, zero the Newey-West lag weights, make value
weighting equal weighting, disable the staleness cutoff, break the symmetry of
the Phase 4 split — each applied to a copy, each required to make the script's own
--selftest exit non-zero. Three survived on the first run: value weighting,
the Newey-West lag weights, and the staleness cutoff were all unguarded, in
phase2_l1.py and again in robustness.py. Three tests were added and all
forty-three are now caught. A survivor is a hole in the test, not a bug in the
code — the code was correct in every one of those three cases, and nothing was
watching it.
Amended 19 August 2026. The two counts in this entry contradict each other and neither belongs to its date. The catalogue stood at thirty-one when this pass ran, reached thirty-seven at the corrected run and forty-three at the second independent audit, and stands at forty-five now. "Forty-five" and "forty-three" were both written into this paragraph later, at different times, which is precisely the silent editing the preamble says this document does not do. The current figure is forty-five applied and forty-five killed; the progression is 31 → 37 → 43 → 45.
The provenance manifests — results against the code that made them.
verify_paper.py checks the paper against results/; check_repo.py checks the
repository against itself. Nothing checked results/ against the code that
produced it, which is exactly how an edited script came to sit beside tables from
the previous version of itself. It is the worst-behaved class of error available
here, because every figure stays internally consistent and every check passes.
Phases 2 to 4 and robustness.py now write a manifest_*.csv recording the
SHA-256 of their own source, and check_repo.py compares those hashes against
src/.
One more defect, found by writing that check down. The documented gather step
was copy phase*_out\*.csv ..\results, which would have copied the twenty-four
long-short return series and database_entry.csv — firm-level and derived from
licensed data — into a public repository. It had never been run as written; the
committed results/ was assembled by hand, correctly, which is why nothing
showed. The step is now an explicit list, and check_repo.py requires that list
and the folder to agree in both directions.
The re-run, and what it moved. Phases 2 to 4 were re-run on the corrected
alignment the same evening. voided 783 of 2,063,785 forward returns — 0.04% of
the panel, which is the honest size of the defect. Every figure in the paper
moved in the third digit and nothing moved in substance: the six look-ahead
contrasts shifted by 0.01 to 0.03 percentage points, none changed sign, the same
five of six cleared 1%, the same five of six convention-cost contrasts stayed
insignificant, the composition channel stayed at about 2% of the asset-growth
reversal, and the same one of four seasoning contrasts cleared |t| = 2. The
paper's tables were then rewritten from the new results/ mechanically rather
than by hand, and verify_paper.py was the acceptance test: 201 values, zero
mismatches. The four manifest_*.csv files from that run carry SHA-256 hashes
that match src/ exactly, so the chain from code to table to paper is now
checkable end to end in two commands.
What this pass did not do. It did not re-read the paper for correctness. The
three checkers plus the eleven self-tests are now the argument that the numbers
are right, and they are reproducible by anyone in four commands. Where a claim in
the paper is not mechanically checkable — the fiscal-quarter shares in §4.3, the
firm-level quantiles in §5 — verify_paper.py names it as unchecked rather than
passing over it.
After the checkers were in place and everything passed, two independent audits
were run against the finished repository: one comparing every prose claim in the
paper against results/, one reading the analysis code for correctness. Between
them they found four real defects and eight stale numbers. That is the honest
value of an outside reading: the suite was measuring what it had been told to
measure, and none of these were in scope.
The factor alignment. decile_ls stamped each row with the formation
month while filling it with fwd_ret, the return earned the month after. The
factor merge in alpha_table then joined on that label, so every risk-adjusted
alpha in the study regressed month t+1 returns on month t factors. The
consequence is not a small bias: the regression explains nothing, the estimated
betas collapse toward zero, and the reported "four-factor alpha" is the raw mean
in disguise. A long-short constructed to be the market factor — beta one, alpha
zero by construction — priced at +15.1%/yr under the shipped code, which is the
market's own mean return over the sample. Rows are now stamped with the month the
return was earned, and the self-test plants exactly that portfolio and requires
it to price at zero.
Nothing in the existing suite could have caught it. The planted-look-ahead test asks whether a cheating signal shows a large alpha and an honest one does not, and a broken risk model answers that correctly, because the raw return already carries the answer. Every other test was about the level of a return, and none about whether the factors lined up with it. This is the second time in this project that a test passed for a reason unrelated to the thing it was guarding.
The headline results survive: the abstract's numbers come from robustness.py's
paired test on raw returns, which never touched the factor merge. What moves
is every alpha level, the Δ four-factor α column in §4.1, §4.2's staleness
ladder, all of §5, and §6's raw-against-adjusted discussion. §4.1's remark that
raw and risk-adjusted differences agreed everywhere was, under the defect,
close to tautological.
Trailing windows counted rows, not quarters. rolling(4) and shift(4) ran
after the sample was restricted to quarters carrying an announcement date, so a
dropped quarter silently widened the window: a fifteen-month asset change
reported as a year. Coverage is about 59% in the 1990s, so the exposure is
concentrated exactly where the sample starts, and asset growth — the signal
behind the paper's open puzzle — is the one most affected. This is the same
defect as the forward-return alignment, one layer up: the code that was fixed to
stop counting rows on the return side was still counting rows on the fundamentals
side. Windows are now computed before the restriction and required to span the
right number of calendar months.
An unstable sort decided which quarter a delinquent filer reported.
sort_values(col) is not stable, so when a firm announced two fiscal quarters on
the same day — a late filer catching up — the as-of join resolved the tie
arbitrarily and non-deterministically between runs. Only the rdq arm can tie,
because every other availability date derives from datadate, which is unique
per firm-quarter; so the effect was one-sided, and in the direction of making the
honest arm hold staler data than it should. Fixed by sorting on datadate too.
Breakpoints were conditioned on next month's return existing. The eligible
set was filtered on fwd_ret.notna() before the NYSE cut points were computed,
so whether a stock would still be trading next month helped decide this month's
deciles. Small, but it is a look-ahead in a paper about look-ahead. Membership
and breakpoints now use only formation-date information.
Two lesser findings were fixed at the same time: country_lag.py inferred which
Compustat file it was reading from the substring "na" anywhere in the filename,
so international.csv.gz would have been read with the wrong availability
instrument and silently dropped every row; and check_repo.py failed on the
__pycache__ that running its own documented command sequence creates.
phase3_l3.py carried a second copy of decile_ls, so the factor-alignment
defect existed twice and had to be found twice. It now imports Phase 2's.
Six new self-tests and six new mutations were added — the catalogue is
thirty-seven — and phases 2 to 4 were re-run. The manifest check earned its
place immediately: with the code fixed and the tables not yet regenerated,
check_repo.py failed on all four manifest_*.csv files by name.
Phases 2 to 4 were re-run on the corrected code. The manifests match src/
exactly, so every figure below is traceable to the file that produced it. The
trailing-window guard voided 7,175 windows — about 0.4% of the extract — and
recovered rather more than it removed, because the trailing sums are now computed
before the announcement-date restriction rather than after it: the common
firm-month count for gross profitability rose from 1.17M to 1.22M.
What moved, and it is not cosmetic. With the risk model actually pricing risk, the alpha levels changed materially and one conclusion changed with them. The value-weighted gross-profitability look-ahead contrast fell from +1.51pp (t = 2.85) to +0.93pp (t = 1.94) and no longer clears 5%, so the paper now reports four of six look-ahead contrasts significant at 1% where it previously reported five. The abstract, §4.1, §6 and §8 were rewritten around that rather than around the old count, and §6's claim that no contrast changes its verdict anywhere in the truncation grid was retired: value-weighted gross profitability sits on the 5% boundary across the whole grid.
The equal-weighted results strengthened slightly and the story is unchanged: gross profitability +3.20pp (t = 9.30), book-to-market +4.15pp (t = 6.32), asset growth −7.80pp (t = −8.03). The fiscal-quarter composition channel fell further, to −0.02pp of a −8.60pp total — a fifth of a percent — with the persistent part now +0.01pp, the wrong sign for the hypothesis altogether, and the reversal significant at 5% inside all four fiscal quarters rather than three. The backfill layer is unchanged in character: one of four paired contrasts clears |t| = 2, now at 3.06.
One result is now stated as a design argument rather than a measurement. §6's industry-screen robustness check compared two runs, both of which the corrections superseded, and re-running the unscreened build was not worth a further WRDS pull. The section says so rather than restating superseded figures.
A cosmetic artefact worth knowing about. Because portfolio rows are now stamped with the month the return was earned, a portfolio is formed in the last month of the panel whose return month lies outside it. The long-short series therefore carry one trailing row with a null return, and the phase logs read "360 months" where the estimates use 359. Every consumer drops it — the full-sample tables all report 359 months, and the stratified ones drop it on the thin-stratum floor — but the line in the log is confusing on first reading, and it is left in place rather than fixed, because changing the code again would invalidate the manifests and require a further re-run for no change in any number.
Three audits were run against the finished repository: one on the paper's prose
against results/, one on the analysis code, one reading the whole thing as a
hiring manager would. Every code finding was reproduced before anything was
touched.
The rdq field had no sanity bound. avail_lag0, avail_lag6 and
avail_lag12 are all functions of datadate, so none of them can be absurd.
avail_rdq was the raw vendor stamp. Phase 0 establishes — and §3 quantifies —
that gaps below zero and above roughly a year are artefacts rather than reporting
delays, and country_lag.py had applied that window to the days measure since
the beginning. The return pipeline had not. The consequence is one-sided by
construction, because only the rdq arm reads the field: a negative gap makes
the leg the design calls the truth hold a figure from before its own fiscal
period closed, and a load stamp makes it hold a figure staler than what lag6
holds in the same month. Demonstrated in both directions before fixing. The
window now runs in build_signals, before the common-sample filter so the
affected firm-quarters leave every arm together; it voids 3,247 announcement
dates, 0.29% of those present.
merge_asof returns the latest announcement, not the latest fiscal quarter.
A firm that files a late Q1 after an on-time Q2 had its Q2 figure replaced by the
older Q1, and the signal walked backwards in fiscal time until the next filing.
The previous audit had fixed the exact-tie case; this is the general one. Again
one-sided, and again in the direction of inflating the look-ahead contrast. The
join now keeps only rows that advance the fiscal frontier.
Phase 3 rebuilt the trailing-four-quarter signal without the calendar guard.
The same defect the previous audit fixed in Phase 2, still present one file over,
because the fix had de-duplicated decile_ls and not the signal construction.
Worse than random here: firms with holes in their fundamentals are
disproportionately the young, recently-added ones, which are precisely §5's
treatment group. Phase 3 now calls Phase 2's builder.
A zero CRSP price gave an infinite book-to-market. mktcap is |price| times
shares, and CRSP reports a price of exactly zero when it has neither a close nor
a bid-ask average. inf passes every notna() guard downstream and sorts
deterministically into the top decile, which is the long leg for that signal.
There is now one book_to_market function, shared by the three files that used
to compute it inline.
Also fixed: seasoning of "12 months" removed thirteen, because it counted
365.28 days; market_return lagged the market-cap weight by one row rather
than one calendar month; phase1_link.py sorted indfmt as a raw string and so
preferred FS over INDL, the opposite of what the two other scripts document;
drop_duplicates resolved ties by SQL row order, which PostgreSQL does not
guarantee; and Phase 2 now refuses to publish if the availability rules do not
share a month set, which the thin-decile guard could have broken silently. Nine
self-tests and six mutations were added — the catalogue is forty-three.
The finding that changed a headline is not a bug. §3 named a 400-day cutoff
for load stamps and showed no sensitivity to it, and that cutoff truncates the
right tail of the very distribution the failure column measures. Rebuilding the
table at 300, 400, 500 and 730 days moves the count of markets above the 10% line
from two to seven to fifteen to twenty-two. The United States figures barely move
and the ranking is stable, so the paper's first result survives intact and its
second is now stated as a ranking rather than a count.
[Amended 20 August 2026: the ranking is not stable — the leading seven appear in
three different orders across the four bounds. What is stable is which markets
lead. §3 and the abstract now say that, and check_repo.py fails any document
that says otherwise, this one included.] --window-scan commits
that table. The lesson generalises: the number was never robust, and nothing in
the suite could have said so, because the suite checks that the paper matches the
data and not that the specification is the only defensible one.
Two additions came from the third audit rather than from a defect: robustness.py
now splits the look-ahead contrast across halves of the sample — which shows the
value-weighted gross-profitability result to be entirely pre-2010, a third reason
not to lean on it — and §5's table leads with the paired difference the
t-statistic actually tests rather than with the alpha gap it sits beside.
With the numbers verified and the code twice audited, three reviews were run against the finished repository asking a different question: not is it right but is it finished. One refereed it as a working paper, one tested whether a stranger could reproduce it from the repository alone, one read it as a hiring manager with forty minutes.
They converged. All three named the same missing test first or second: there is no size conditioning anywhere in the study, and the headline lives in equal-weighted decile spreads whose base portfolios reach Sharpe ratios of 1.20 — the signature of a micro-cap portfolio. The paper cites Hou, Xue and Zhang (2020) in its own introduction for the finding that microcap weighting drives replication outcomes, and then reported results in exactly the specification that reference warns about. Knowing an objection well enough to cite it and not applying it to yourself is a worse failure than not knowing it.
phase2_l1.py now reports the look-ahead contrast on five samples: all names,
names above the NYSE 20th percentile of market capitalisation, and each size
tercile, with the decile breakpoints recomputed inside every subset so the
figures are what a manager who cannot hold micro-caps would actually measure.
The result is in §6 and in results/size_split.csv, whichever way it came out.
The reproduction audit found something worse than an omission. robustness.py
defaulted its output directory to its input directory and validated nothing, so
python robustness.py --dir results produced four empty estimators, wrote all
four over the committed tables, and only then crashed — leaving
manifest_robustness.csv intact and still certifying files that no longer held
anything. The README's own wording invited that command. The reviewer triggered
it and restored the tables; the script now refuses a directory that holds no
long-short series, and a mutation checks that the guard stays.
The same audit found that mutation_test.py carried a mutation whose description
did not match what it did — "value weighting uses contemporaneous rather than
lagged cap" against a mutation that replaces the weights with ones. The
catalogue is the documentation of what is guarded, so a wrong description is a
wrong claim. Fixed.
Smaller items from the same reviews: the ignore file is now named in the
repository map rather than shipped unexplained; the gather step has a POSIX
equivalent beside the PowerShell one; the offline claim is restated accurately
(two scripts need nothing at all, four need earlier phases' files); a Python
floor of 3.10 is stated; and check_repo.py no longer exempts missing manifests
from its gather-step check, which was the one place its own mechanism was
one-sided.
The three completeness reviews converged on one demand and it was met. The look-ahead contrast now runs on five samples: all names, names above the NYSE 20th percentile of market capitalisation, and each size tercile, with the decile breakpoints recomputed inside every subset so the figures are what a manager restricted to that universe would actually measure rather than the full-sample contrast reweighted.
The headline survives at between 38% and 51% of its size. Excluding everything below the NYSE 20th percentile — 55% of names — gross profitability equal-weighted falls from +3.21pp to +1.65pp (51%) and book-to-market from +4.13pp to +1.58pp (38%), both still significant at 1% (t = 5.12 and 3.77). So the effect is not a micro-cap artefact — and it is also not the magnitude the full-universe number advertises. The abstract now carries both figures.
The gradient runs the right way, equal-weighted. Gross profitability is 3.29 / 1.65 / 0.49 across small, mid and large terciles; book-to-market is 5.92 / 1.52 / 1.03. Value-weighted it is not monotone, which the wider standard errors on those portfolios would produce with or without a gradient. The terciles are cuts of the NYSE size distribution rather than equal-count thirds, so "small" holds about two-thirds of the traded names; the paper says so.
Amended 19 August 2026. The restriction to equal weighting is wrong. Gross profitability value-weighted runs 2.70 / 1.78 / 0.69 and book-to-market value-weighted 6.76 / 1.20 / 1.16 — both monotone. The only non-monotone signal is asset growth, and it is non-monotone in both weightings, in the point estimates rather than only in the significance. §6 now says that; the standard-error explanation offered here does not fit a disorder that sits in the point estimates, and has been dropped.
The open puzzle is narrowed, not closed. §4.2 had said a small-firm reading of the asset-growth reversal "is consistent with that but is not established here." The split supports one: equal-weighted the reversal is −10.90pp in the smallest tercile and −0.66pp and −0.76pp in the mid and large, neither distinguishable from zero. It does not vanish under the micro-cap screen (−2.04pp, t = −2.61), so it is not a bottom-decile artefact either. The result stays open; what the split does is constrain the candidate mechanisms to ones that operate on small firms.
The test cost roughly twenty lines and one run. It had not been run because nothing in the suite could tell the author it was missing: every checker asks whether the paper matches the data, and none asks whether the right thing was measured. That is the standing limitation of the whole apparatus, and it is why the outside reviews were worth more than a sixth pass of self-checking.
One last reading, aimed only at the passages written in the previous hour — the size section, the abstract sentences around it, §4.2's rewritten close and §5's reordered table. The project's own pattern held: every number in the new text was correct and every word around it was not.
The blocking defect was a caption. The size table said "Bold marks significance at 1%" and bolded 8 of the 18 cells that qualify, so within the Small column a t of 7.24 was plain while a t of 5.55 was bold. Two sentences written beside it also stated figures no committed table supports: "roughly a third the size" where its own next clause said 51% and 38%, and a §8 range of "1.6 to 1.7 points" where the screened contrasts are 1.06 to 1.65. Both were written while looking at the correct numbers.
Ten smaller things went with them. "In the largest tercile only book-to-market survives" was contradicted by a bolded cell two rows below it — true equal-weighted, and the sentence did not say so. §4.2 and §6 had been left pointing at each other, §6 quoting a sentence §4.2 no longer contained. §6 called the split a resolution while §4.2, §8 and the abstract all still called the result open; the split narrows the candidates and does not close them, and the paper now says that consistently in all four places. The stratum month range was 294 to 320 where the data says 322. The §4.4 power caveat claimed standard errors "one to three times" §4.1's when they run 0.9 to 3.0 — and the exception is the one cell the caveat is used on, where the error is smaller, so power is not the excuse there. §6's subperiod paragraph said five of six contrasts kept their verdict when value-weighted book-to-market loses significance and value-weighted asset growth grows. The multiple-testing inventory had not been updated for the twenty-four new contrasts, which lifts the family bar from 3.45 to 3.68 and puts two figures §6 states plainly below it; both are now named.
Amended 19 August 2026. Both halves of that last sentence are wrong, and the causal account with them. Six of the twenty-four size-conditioned contrasts are bold at 1% and fall below the narrower 3.45 bar, so they fail it whether or not the split is counted in the family; nothing in the size table falls between 3.45 and 3.67 (a later pass found 3.68 was itself a rounding error — 42 tests give 3.6747), so the lift itself changes no verdict. And one of the two originally named — asset growth above the NYSE 20th percentile — is the figure §6 already qualifies four sentences earlier, so it was not stated plainly. All six are now listed.
Two shorthands were quietly wrong about their own samples. "The bottom fifth" describes a screen that removes 55% of names, and "the smallest third of the universe" describes a tercile holding about two-thirds of it — because these are cuts of the NYSE size distribution, not equal-count thirds of the traded universe, which the paper had not said. It says so now.
Finally the front page. The exhibit and the PNG both closed with a flat "7 markets leave more than one firm-half in ten unavailable", which is the count §3 spends a table disowning, and the PNG is the first thing a visitor sees. Both now carry the window caveat, and the PNG is re-rendered from the current HTML.
Nothing in this pass came from the checkers. verify_paper.py passed 309 values
throughout and check_repo.py 434 claims; both were green while the caption was
false and the ratios were wrong. That is the standing limit of the apparatus,
stated once more because it keeps being the answer: the machines check that the
paper matches the data, and a reader is still required to check that the
sentences match the numbers.
A last audit, run once more against the passages the previous pass had written.
It found no blocker and no wrong number: every figure in the new text checked out
against results/*.csv, all eighteen bolded cells in the §6 size table correctly
marked |t| > 2.576, the Bonferroni bars of 3.4524 and 3.6747 were exact, and
the caption's ratios — Small 66.1% of names, Large 14.7%, the screen removing
54.5% and leaving 1,539.5 of 3,385.5 — were all correct. What it found was
seventeen defects in the words.
Seven of them mattered. The worst was the multiple-testing paragraph, which had been rewritten in the previous pass and got its own subject wrong twice: it said two §6 figures fall below the narrower Bonferroni bar when six do, and it described one of those two as reported "without qualification" when it is the one figure §6 explicitly qualifies. Second: "a monotone reading is supported equal-weighted and not value-weighted" is contradicted by the table printed directly above it — value-weighted gross profitability and book-to-market are both strictly monotone, and asset growth is non-monotone in both weightings. Third: "Bangladesh and Romania head the list at every bound" fails at 500 days, where Vietnam sits between them. Then four smaller ones — a −0.14 for a −0.13 in the README, a subperiod count the README had not picked up from §6, and two cross-references pointing at sections that do not say what they are cited for, one of which is the second time §4.2 and §6 were left describing each other's deleted text.
Ten were cosmetic: a range described as "between a third and a half" when one of its endpoints is 51%, a claim that three negative estimates "average away", a paragraph that introduced and closed the same argument twice, a standard-error range quoted over twelve estimates and attributed to six, an out-of-date count of this document's own review passes, an appendix sentence whose causal account of the Bonferroni lift was backwards, "57 markets" for a screen the paper elsewhere reports as 52, a t of −2.31 for a −2.304, a −2.04pp that denotes two unrelated quantities in two sections, and an exhibit caption mixing numerals and words in one list.
All seventeen are corrected above and in PAPER.md. Two are now checked
mechanically rather than by reading: verify_paper.py gained the four
fiscal-quarter t-statistics quoted inline in §4.4 — the source of the −2.31 —
and an audit of the §6 size table's bolding against the 1% threshold and against
the Bonferroni bars the paper quotes.
The pattern this document has recorded at every pass did not break here: no checker found any of it. The count of defects found by reading is well into double figures; the count found by the suite, in prose, is still zero.
Two further rounds followed this one, because the first round of fixes broke two
sentences it touched and the second broke one more. The last of them found the
oldest defect in the paper: §2 had said Compustat North America "returns
industrial presentation for 1,681,465 of 1,681,518 rows", which is a datafmt
count wearing an indfmt label. The file is 100% INDL; the restriction removes
no rows at all, and the 53-row remainder is PRE_AMENDS. That sentence had been
in the paper since the first draft and had passed eight review passes, two code
audits, three completeness reviews and every mechanical check, because the number
was real, the sentence was fluent, and nothing in results/ carries a
presentation split. It is now reconciled arithmetically against
link_funnel.csv, which is the closest a committed table can come to guarding it.
The general lesson, stated once and then left alone: a fix is an edit, and edits are where defects come from. Three rounds of correction here produced four new false sentences between them, every one of them inside a passage that had just been "fixed". The only reliable control found in this project is to re-read the correction as adversarially as the original, from the data up, and to convert whatever can be converted into a check.
Every pass until now compared the paper to results/. But a dozen figures live in
prose rather than in a committed table — the Compustat row counts, the
rdq-against-pdateq calibration, the per-market instrument shares, §5's
backfill statistics — and no checker can reach them, because the extracts they
come from are licensed and not published. This pass recomputed them from the raw
WRDS pulls.
Two of them were describing the wrong population, and both had been in the paper from the beginning.
§2.2's calibration is not a United States statistic. "On the United States
firm-quarters where both exist, the two agree to within one day at the median and
within five days for 86% of observations" — 86% and the 77–97% by-year range are
the Compustat North America figures, over 372,268 rows of which 22.6% are not
United States. On United States rows alone the agreement is closer: the same day
at the median, 91% within five days, 84–98% by year. The sentence understated its
own result and mislabelled its own sample, and it did so in the paragraph that
licenses pdateq for fifty-one markets. §7 had been insisting for three passes
that "this is not a United States against rest-of-world split"; §2.2 had been
saying it was.
§5 is computed on a longer file than the rest of the paper. phase3_l3.py
pulls comp.fundq with no date bound — Compustat's full history, back past the
existence of rdq itself — while §2's extract starts in 1990. So 16.2%, the
32,914 firms and the pre-loaded-history quantiles are not statistics of the
1,680,197-row file §2.2 tabulates, and the sentence qualifying them cited §2.2's
coverage rate as if they were. On the 1990-onward file the same statistic is
12.0% and the median pre-loaded history 1.36 years rather than 1.93. The paper now
states the population and both figures.
Three more that reading found:
§6 never mentioned the one contrast the micro-cap screen kills. Book-to-market value-weighted is bolded at 1% in §4.1 (+2.86pp, t = 2.99) and falls to +1.29pp with t = 1.92 above the NYSE 20th percentile — the only estimate in the paper that is significant at 1% on the full universe and clears no conventional level after the screen. §6 discussed the two contrasts that move the other way and stopped at two of three.
§5 tests two signals, not three. phase3_l3.py's signal dictionary contains
gross profitability and book-to-market. Asset growth — the signal carrying the
paper's one open result — is not in the backfill layer at all, and nothing in §5,
§7, §8 or the abstract said so. "Three of the four series" was accurate about the
four series tested and silent about the third signal that was not.
§8 said an anomaly "survives correction" on a t of 2.23, two sentences after invoking the 1% threshold that 2.23 fails.
And the README's headline Results table had a row split across two physical lines, so on GitHub the backfill answer rendered truncated mid-clause with a spurious empty row after it. That is the first thing a recruiter sees, and it was introduced by a correction two passes earlier.
What is checkable is now checked: check_repo.py scans APPENDIX_process.md for
retired ordering claims as well as the paper and the README — this file had kept
one alive in the present tense for three passes — and accepts a quoted claim only
where a dated amendment follows it. The upload instructions, which are not part of
the published repository and had therefore never been audited, carried the same
retired claim, an overstated one-sentence pitch, a repository description
conflating the 52-market and United States samples, and a git route that never
renamed the ignore file before the first commit.
The count of defects found by reading is now past sixty. The count found by the suite, in prose, is still zero.
Eight review passes, two independent code audits, three completeness reviews and an extract-level recomputation, and every one of them found something. That pattern has an explanation, and it is not that the paper was unusually bad.
Every pass was a sampling audit: read the document, look for problems. Sampling does not terminate. A fresh reader will always find something, and the rate at which findings arrive measures how hard the reader looked rather than how many defects remain. Worse, the corrections themselves were edits, and edits are where defects come from — three of these rounds introduced a false sentence into the very passage being fixed, and two batches of corrections were silently discarded by a patch script that wrote its file only after the last edit succeeded.
What terminates is a census: enumerate the claims, assign each a verification
status, drive the unaccounted-for count to zero. audit_census.py does that. It
extracts every numeric token in PAPER.md — 947 of them — instruments
verify_paper.py to record which figures it actually parses, and sorts the
result into four tiers:
- 544 tier 1, parsed by a check and compared against the specific source cell they belong to;
- 81 tier 2, restatements whose value occurs in
results/at the precision shown — traceable, and read once against their sentences on 20 August; - 322 tier 3, carrying an explicit class: a significance threshold, a calendar year, a citation, a section reference, a statistic of the licensed extracts, or a table the repository prints rather than commits;
- 0 tier 4, unaccounted for. The script exits non-zero if that is ever not zero.
Building it immediately paid for itself twice. The first run showed 310 numeric
claims that no machine check had ever touched, which is why reading kept finding
things: the checkers had been answering "do the tabulated figures match their
sources?" while a third of the paper's numbers were outside their reach. Closing
that gap added seventeen checks to verify_paper.py, and all seventeen kill a
planted error. And the census caught the first sentence written after it existed
— the one describing the census itself quoted a claim count that nothing
verified — so the count is now checked against the census's own tally.
Two structural repairs went with it. Report.cell had been able to pass silently
when its source cell was missing, because a NaN comparison is false in Python;
one of the new checks did exactly that before it was caught, and a missing source
is now a failure. And staging the published copy is now a script that refuses to
exit successfully unless every file in upload/ is byte-identical to its working
original — a stale staged copy had invalidated one full verification run.
The honest limit is unchanged and worth stating plainly rather than dressing up. The census guarantees that no numeric claim is unaccounted for. It does not guarantee that every sentence around those numbers is true; nothing mechanical can, and every defect this project found by reading lived in exactly that gap. What has changed is that the reading now has a finite, enumerated target instead of an open-ended one. What the suite protects is the correspondence between the tables and the data. What it cannot protect is the correspondence between a sentence and the table beside it, and the most reliable predictor of where the next defect sits remains the paragraph most recently written.
Amended 20 August 2026. The counts above describe the census as first built, and it counted digits only. A reviewer planted the sentence "Nineteen review passes and forty-one independent audits were run." into §8; the census reported 947 claims, tier 4 empty, and exited zero. Every figure in that sentence was spelled as a word and therefore invisible to the scanner — and the one false sentence the adjudication below had found in that range, "seven review passes", was of exactly that form. The census now enumerates words as well as digits: 1,236 claims, of which 965 are written in digits and 271 as words. A spelled count is credited only where
verify_paper.pyparsed that position and compared it with a source, never by its value coinciding with some other figure — the loophole that let the planted sentence through. Closing it took the verification suite from 425 checks to 530, almost all of them counts the paper states in prose: "five of six are insignificant at the 1% level", "three of the four series decline monotonically", "seven of its eight weight-rule cells". Each is now recomputed fromresults/rather than read, andaudit_census.py --mutate-countschanges each of them in turn and requires a check to fail on it.The same pass found the defect that prompted it. §4.4 carried the line "estimated on that same sample. What bounds the", with the next line starting "What bounds any selection effect either way" — an earlier correction had rewritten a wrapped sentence and left its own opening words stranded. It had survived every pass because a reader's eye completes the sentence. It is a two-line pattern, so
check_repo.pynow looks for a line that stops mid-clause on a tail that restarts the next line, across all three documents.
Every pass before this one read the passages the previous pass had written. That
is why each found defects in the last pass's prose and none found anything older:
the method could not, by construction. This pass partitioned the whole corpus —
the abstract and every section of PAPER.md, the README end to end, the exhibit
and its caption source, the cross-document seams, and every script against
the methodology they claim — and read all of it against results/.
It found forty-odd defects, none of them a wrong number in a table. The three that mattered most were all old enough to have survived every earlier pass:
The instrument's scope was wrong in §2.2. "Outside the United States the
figures below measure time to first appearance in the database" — but the split
is by source file, not by country. comp.fundq covers Canada in full, so Canadian
rows carry announcement dates. §7 has said so since the fourth review, the exhibit
says so, §3 says so; §2.2, which is where the caveat is defined and which the
other three cite, said the wrong thing.
The residual coverage gap was claimed as "entirely pre-2000". It is not:
964 of roughly 42,300 study-universe firm-quarters with no rdq fall after 2000.
§6 already said "nearly all". This one had been in the paper since the first
draft of §2.2.
§4.4's sample filter is correlated with the quantity §4.4 estimates. The stratum-size floor binds on the quarter mix, which is the treatment, so the months it drops are the months where the mix gap is widest — the code says this in a comment and calls it attenuation; the paper attributed the drops to the listed universe being small after 2012. The composition estimate is therefore a lower bound on the channel, and the section now says so.
Then: the Hou, Xue and Zhang count was 447, the working paper's figure, where the reference cites the published version's 452. The abstract labelled a paired raw-return difference "alpha" — the exact defect the second review recorded and fixed elsewhere in the paper, surviving in the one place a reader starts. The second Bonferroni bar was printed 3.68 where 42 tests give 3.6747; nothing caught it because the check read the bar out of the sentence instead of computing it. Two §4.2 figures were differences of rounded display values rather than of source cells. §5 called the backfill difference "well identified" thirteen lines above the paragraph explaining that the leg correlation is not identification, and in a paper whose abstract calls that layer the least well identified of the three. §4.3 described the asset-growth portfolio as holding "under a fifth" of its weight in annual figures where the asset-growth sample holds 21.0%; 19.4% is the pooled number the paragraph two lines above exists to warn against using. §7 conceded six limitations and not the one §4 states outright — that every return figure is gross. And the previous pass's own fix to §3's market ordering introduced a fresh false clause about the 730-day bound, the second time in two passes that sentence has been wrong.
Four of these are now machine-checked. verify_paper.py recomputes both
Bonferroni bars from the family sizes instead of parsing them out of the prose;
it checks the ordered market lists at all four window bounds, including that the
same seven lead at every bound of 400 days or more, that Bangladesh and Romania
never leave the top three, and that the leading seven appear in three distinct
orders — so a claim that the ranking is stable now fails; it checks the two
§4.2 alpha steps against unrounded source cells; and it checks §4.3's
asset-growth quarter shares and §2.2's post-2000 residual against their tables;
it reconciles §2's row counts to the funnel's starting row, after a later pass
found that §2 had been describing a datafmt count as an indfmt count since the
first draft — the extract is 100% industrial presentation, so the restriction the
paper called "effectively a no-op" removes exactly zero rows, and the 1,681,465 it
quoted is the standard-data-format count;
and it parses §3's ordering sentence and checks each named market against its
rank at each bound, so the class of defect that has now appeared twice in the same
paragraph fails a check rather than a reading. check_repo.py gained a list of
retired ordering claims. The suite went from 376 checks to 394, and check_repo.py
from 434 to 444.
The standing lesson is unchanged and now has a sharper edge. No checker found any of this either. What changed is the reading: auditing only what was written last guarantees that everything written earlier is never audited again. The count of defects this project found by reading is now well past forty; the count found by the suite, in prose, is still zero.
The census answered "is every number accounted for?". It could not answer "is
every sentence true?", and that gap is where every defect found by reading had
lived. So the paper was partitioned into its sentences and each was given one
verdict against results/ and the extracts: TRUE, FALSE, UNSUPPORTED,
OVERCLAIM, OPINION or STRUCTURAL.
The result was not what the previous passes' hit rate predicted. Three sentences were false:
- §8 said the corrections came from "seven review passes" where the record shows eight. A number spelled as a word, in a sentence about counting.
- §6 said Vietnam "moves ahead of Romania at 730", where the ordering at that bound is ROU BGD VNM — Romania first.
- §6 said the value-weighted gross-profitability t is "below 1.96 only at zero lags"; it is also below it at three lags and at eighteen.
Everything else that failed failed differently. A second group was UNSUPPORTED — true as far as anyone knows, with nothing in the repository establishing it — and a third was OVERCLAIM, stating more than the cited table carries. Those are different defects from a wrong number and they need different repairs: a qualifier, a citation, or deletion. All of them were fixed, and the distinction is worth recording, because "the paper contains no false statements" and "every sentence in the paper is supported" are not the same claim, and only the first was ever close to true.
The verdicts themselves are not committed, and no count of them is quoted here.
An adjudication is a reading, and a reading is not reproducible from results/;
quoting its tallies would put a figure in this record that nothing can check,
which is the failure mode the census exists to stop. What is recorded is the
three false sentences, each of which can be checked against a committed table.
Every pass the paper's process record claims is listed here, with the kind it
was. check_repo.py counts this table and compares it against the sentence in
§8 that describes it, so the paper cannot claim a pass the record does not show.
| # | Date (2026) | Pass | Kind |
|---|---|---|---|
| 1 | 17–18 Aug | Internal review | review pass |
| 2 | 18 Aug | Second review | review pass |
| 3 | 18 Aug | Third review | review pass |
| 4 | 18 Aug | Fourth review | review pass |
| 5 | 18 Aug | Fifth review | review pass |
| 6 | 19 Aug | Pre-publication read | review pass |
| 7 | 19 Aug | Final check | review pass |
| 8 | 19 Aug | Full-corpus audit | review pass |
| 9 | 18 Aug | Independent audit | code audit |
| 10 | 19 Aug | Second independent audit | code audit |
| 11 | 19 Aug | Refereed as a working paper | completeness review |
| 12 | 19 Aug | Reproduced from the repository alone | completeness review |
| 13 | 19 Aug | Read as a hiring manager | completeness review |
| 14 | 20 Aug | Extract-level audit | extract recomputation |
| 15 | 20 Aug | Sentence-by-sentence adjudication | sentence adjudication |
| 16 | 20 Aug | Claim census, digits and words | claim census |