Verification on The Broken-Born program

The iron rule: no arithmetic by eyeball

Mon, 01 Jan 0001 00:00:00 +0000

The iron rule is the program’s requirement that no number be accepted on inspection: every number a document asserts, and every number a reviewer disputes, is computed by code that is actually run and prints its result, and the code is kept in the repository’s verify/ folder. It was adopted after the first two blind reviews, in which AI referees working without code execution made three confident accusations of order-of-magnitude errors, all of which came from their own unit mistakes. The rule makes the arithmetic checkable by anyone who runs the scripts, and the gate runs them before any work counts as finished. It does not make the physics right, which round 3 showed.

Verification checks arithmetic, not the model

Mon, 01 Jan 0001 00:00:00 +0000

A verification script confirms that a calculation was carried out correctly for the inputs and formulas written into it; it cannot tell whether those inputs and formulas are the right physics. The program learned this from its soliton dark-matter manuscript: every number in it was computed and checked by a gated script, yet one formula, the normalization of how many monopoles the early universe forms, was wrong, and only an independent review caught it. Checkpoint #11 has carried the caution since version 2.1, the rule on independent anchors is the repair, and every script page in this section states what its checks leave to the derivation.

Independent anchors

Mon, 01 Jan 0001 00:00:00 +0000

An anchor is a check that a verification script must pass against something known independently of the paper being checked, such as a textbook identity or a published value, before its own numbers are trusted. A check that only reproduces the paper’s own figure is circular: if the paper’s formula is wrong, the check repeats the error and passes. Review round 3 found such a case in the soliton manuscript, whose script anchored the monopole count to the previous draft’s own figure and computed a normalization too small by a factor of about 4.6×10³ without failing. The rule has been in the program’s method since, and three of the current scripts open with anchor checks taken from textbooks or from the cited literature.

The reduced-model stability rule

Mon, 01 Jan 0001 00:00:00 +0000

The reduced-model stability rule says that when a calculation simplifies a model by holding some of its fields fixed, the full model’s stability must still be checked with every field free to move, and the text must say that the calculation used a reduction. Review round 4 supplied the reason: the potential first used in the program’s coset toy model had been checked only with the phase-space metric H held fixed, and it is unbounded below once H is allowed to vary, so the model’s intended vacuum was a saddle. The repaired potential was checked with all fields dynamical, and the toy-model note and its lattice script now name the simulation as a frozen-H reduction.

The Monte Carlo error-bar rule

Mon, 01 Jan 0001 00:00:00 +0000

The Monte Carlo error-bar rule says that a number produced by a random simulation must come with an error bar estimated from several independent runs. A simulation started from a fixed random seed gives the same output every time it is run, which makes the number reproducible but says nothing about how far it lies from the average the simulation is estimating. Review round 4 found the case that prompted the rule: the high-temperature control values in the toy-model note came from single fixed-seed runs without error bars, and the smallest lattice’s value was a short-run fluctuation. The lattice script now reports means and standard errors over four independent seeds, and its checks use those error bars.

The gate: scripts/check.sh

Mon, 01 Jan 0001 00:00:00 +0000

The gate is the shell script scripts/check.sh, which runs the repository’s six verification scripts in verify/ and then the wiki’s own consistency check, and exits with a failure if any check fails. The project calls it “the definition of done”: work is not finished until it passes, and a check may not be weakened to make it pass. Each verification script prints one line per check, a pass or a fail together with the measured value where there is one, so the record of a run shows what was tested as well as the verdict. What the gate tests is the arithmetic of the models the scripts encode and the form of the wiki; which claims it covers, and which it leaves to derivation, review or nothing, is listed below.

The verification battery T1–T7

Mon, 01 Jan 0001 00:00:00 +0000

The verification battery is a set of seven numerical and exact-integer tests, T1 to T7, of the matrix algebra at the core of the program: the compatibility theorem, the parametrization of the modular field, and the algebra behind the mass of the momentum graviton and the zero vacuum energy of its potential. It was written and run on claude.ai, carried into the repository as Appendix A of the handoff document, and now runs as verify/battery.py in the gate, with the same computations and random seeds. Checkpoint #11 calls the results it confirms Tier 1 and tags them [V]. Six of the tests evaluate an identity at randomly drawn sample points and one is exact integer arithmetic, so the battery confirms the algebra at sample points rather than proving it, and none of the tests asks whether the algebra describes nature.

The soliton-obituary script

Mon, 01 Jan 0001 00:00:00 +0000

The soliton-obituary script, verify/soliton_obituary.py, recomputes the numbers in the record of why the program’s superheavy-soliton dark-matter candidate failed: the corrected count of monopoles formed in the early universe, the mass at which their abundance would match the observed dark matter, the overproduction at the withdrawn manuscript’s benchmark mass, the corrected relic window of the hidden sector’s light fermion, the dark-radiation bound, and the exclusion of the minimal hidden sector. Before computing any of these it reproduces standard cosmology results that do not depend on the manuscript. It replaced an earlier script, knot_numbers.py, which had checked the manuscript’s own claims and had passed a wrong normalization. The candidate it documents is dead; the script keeps the arithmetic of that death reproducible.

The toy-model scripts

Mon, 01 Jan 0001 00:00:00 +0000

The toy-model scripts are four programs in verify/ that recompute the results of the toy-model note, which studies two explicit field theories in which the exchange of position and momentum is an exact symmetry that the vacuum breaks. toy_coset.py checks the coset sigma model (model I), toy_gw.py the self-dual Grosse–Wulkenhaar model (model II), toy_s4.py runs a lattice Monte Carlo of model I with the phase-space metric held fixed, and toy_s5.py checks how the vacuum’s choice can make matter interactions local in position. They were written with the models on 2026-10-19, extended with the corrections of the round-4 review, and run in the gate. Their checks are exact symbolic identities, numerical tests against anchors taken from the cited literature, and seeded simulations with error bars; each confirms the arithmetic of the model as written, and several of the note’s conclusions rest on arguments that no script checks.