Blind review
Blind review is the program’s procedure for testing a manuscript: a fresh AI referee receives the manuscript, with no project files and no conversation history, and is asked to find what is wrong with it and to grade it A, B or C. Before each round the author writes down and commits the expected grade and the weakest point the referee is expected to find, so that the author’s judgement can be scored afterwards; after it, the author rules on every finding in a written adjudication, re-running disputed numbers in code. Four rounds have run. The first two reviewed the soliton dark-matter manuscript #14 with a referee that could search the web but not run code; the third reviewed #14’s third version with a referee that executed code, and its verdict led to the candidate’s retirement; the fourth reviewed the toy-model note and led to its second version. The referees of rounds 3 and 4 were, as their pre-registrations state, the same model family as the author.
Method and record only: the section holds the review method in force and the record of the four rounds, so it ranks no claims of the program’s own; those are tracked in the registers of claims that stand, the graveyard, open problems and the queue, conjectures and superseded statements. What the reviews killed has its own obituaries in the graveyard: from round 1, the first form of the coupling α_d ≈ 34, whose last form fell with the candidate after round 3; from round 3 superheavy conjugate solitons and the Kibble normalization, among others; from round 4 the naive ρ-preserving potential. The section’s one open item is optional: a round-5 review of the toy-model note, “but only before any external use” (root CLAUDE.md).
Map#
The section runs from the procedure to its record. The first four pages say how a review is run and what the rounds have shown about referees and about the author; the last three are the rounds themselves, in date order.
| Round | Manuscript | Referee | Verdict | Forecast | What followed |
|---|---|---|---|---|---|
| 1 | #14 v1 | console: web search, no code | B / medium | scored with round 2: verdicts 2 for 2 | valid findings folded into v2 |
| 2 | #14 v2 | console: web search, no code | B near B/C / medium | (as round 1) | pending list R1–R7; v3 |
| 3 (2026-10-19) | #14 v3 | subagent: shell, Python, web search | C, medium | verdict missed | candidate retired; two lessons |
| 4 (2026-10-19/20) | toy-model note | subagent: shell, Python, web | B, ~0.75 | verdict hit; weakest link partial | note v2; two lessons |
- The blind-review protocol and verdict scale comes first because every round is an instance of it. It sets out the five steps a referee follows (verify every citation, re-run every number in code from its own constants, re-derive the mathematics, audit the argument, grade it with the single weakest link named) and how each was carried out in rounds 3 and 4. It gives the A/B/C scale with its two glosses of C and where the records draw the B/C line, the labels the author’s adjudication uses (upheld, accepted, overturned), and the protocol’s limits.
- The skeptic review console is the tool of the first two rounds, a browser page (#15) that sends the manuscript to a fresh model with web search but no code execution. Its referee prompt, quoted as record, defines the A/B/C scale. The page embeds #14 version 2, and the page traces what became of each of that version’s claims.
- Pre-registration and its calibration record is the forecast written and committed before each round and scored after it: what a forecast contains, the scoring tables of rounds 3 and 4, and the record across rounds (verdicts 2 for 2 in rounds 1–2, a miss in round 3, a hit in round 4; the weakest link’s location improving, its severity still underestimated). It also records how round 3’s forecast reached the referee through a commit subject, and the rule that followed.
- The measured referee failure mode is what the rounds showed about where review goes wrong. In rounds 1–2 it was the referees’ unaided arithmetic: three confident order-of-magnitude indictments, none surviving. Once referees could run code, the errors that survived were the author’s: in round 3 wrong model inputs that a check of arithmetic passed, in round 4 a stability check made on a reduced model and Monte Carlo values without error bars.
- Review rounds 1 and 2 are the two console reviews of #14, versions 1 and 2. Only the handoff’s summary survives in the repository, and the page says first what is missing. It gives round 1’s valid findings and where each went, round 2’s weakest link and its pending list R1–R7 with the status each had when version 3 was cut, and the referees’ own errors.
- Review round 3 is the code-executing audit of #14 v3, graded C. Its rulings found the Kibble normalization wrong and the benchmark mass overproduced, and its decisions retired the soliton candidate. It has its own pages for the forecast, the rulings, the decisions and lessons, and the arbitration.
- Review round 4 is the review of the toy-model note: the referee’s citation audit and re-derivations, findings F1–F6 (above all, a potential unbounded below once the field H varies), the Monte Carlo control re-run, the forecast scored, the six revisions applied in version 2 and the three places where version 2 departs from them (one a requested comparison it did not make, flag 42 in
review/wiki-build-flags.md), and the two rules the round added to the method.
Boundaries#
- The rules for checking numbers that the reviews produced or sharpened — the iron rule, independent anchors, verification checks arithmetic, not the model, the reduced-model stability rule and the Monte Carlo error-bar rule — and the scripts that now gate the reviewed numbers are in verification.
- Findings that changed the physics are recorded on the pages they changed: the retired candidate in the soliton candidate, the repaired models in toy models of broken reciprocity, and the re-attributed compatibility theorem.
- The objection every referee raised against #14, that nothing numerical depended on Born reciprocity, is the ‘framework does no work’ critique, under the research program. The toy Lagrangian that both console referees named as the gate to verdict A is acceptance criterion 1.
- The roles behind the reviews (Paul’s conjecture, direction and veto; Claude’s drafting and verification) are on the collaboration, and the tags the referees test against are on provenance tags.
- The reviewed documents are described on the soliton-candidate and toy-models section pages and in the documents; the dates of the rounds are in the timeline; the statuses that follow from each round are in status.