Verification

What it says it does

Verification script for Ejdrup et al. 2026 — Striatal Dopamine Model.
eLife | doi:10.7554/eLife.ejdrup2026

FAST MODE (default, ~18 min):
  Clones GitHub repo, applies matplotlib compatibility fix, and runs
  Figure 1 and Figure 2 scripts with a timeout to check for clean exit.

  It said ~5 min for a long time and does not take 5 min. Each of the two figure
  scripts is allowed 600s here, so the worst case is 1200s before the clone is
  counted, and a measured uncontended run took 1064s. The observer's timeout was
  set against the 5-minute figure, so it killed this script every time and
  reported the paper as a failed run when nothing had failed.
  Requirements: pandas, numpy, matplotlib, tqdm
  Data: https://github.com/Gether-Lab/striatal-dopamine-model (~30 MB)

FULL MODE (--full, ~8 hrs):
  Runs the complete tissue-scale simulations to completion (no timeout).
  Figure 1 simulation: ~4 hrs. Figure 2 simulation: ~4 hrs.
  Additional requirements: Standard Python stack (no MATLAB needed)
  Note: All computation is CPU-bound Python; no special hardware required.

Usage:
  python verify.py           # fast mode
  python verify.py --full    # full pipeline (no timeout)
  python verify.py --claim ds-lacks-pervasive-tonic-da

What it says it does

Run a verification script under observation, and record what it actually did.

Only one of the nine verification scripts was ever audited, and auditing it found two
failures that no amount of reading would have caught:

  * a claim recorded `verified` while the function named in its record raised
    `shapes (4,4) and (5,5) not aligned` -- the verdict had been narrated, not observed;
  * a reproduction record naming `fMRI - Choices_singleTrialData.csv` while the code opened
    `Behav - Choices_singleTrialData.csv`.

Both were found by making the script report every path it opened and every value it computed.
That was done for Gaedeke by editing the script to call `used()` at each open. Doing the same
to eight more scripts would mean eight sets of hand-edits, each an opportunity to annotate a
path the code does not take -- which is the very failure being audited.

So this observes from outside instead. It patches `open` and the common loaders, executes the
script in-process, and reads its `ROWS` list afterwards. Nothing is taken from the script's
own account of itself: the file list comes from the file system calls, the results come from
the list the printed table is built from, and an exception is recorded whether or not the
script caught it.

What it emits, beside the script it ran:

  verification/<paper>/provenance.json    files opened, results produced, exceptions raised

Usage:
  python3 verification/audit_run.py <paper-slug> [-- script args]
  python3 verification/audit_run.py --all
  python3 verification/audit_run.py --all --timeout 1800

`--timeout` now defaults to 1800s rather than to no limit. The documented example used to say
900, which is shorter than the slowest verification script's own internal budget: Ejdrup's fast
mode runs two figure scripts and allows each 600s, so its worst case is 1200s before the clone
is counted, and an uncontended run measured 1064s. A run killed at 900s writes no results, and
`audit_verifications` then reports the paper as a failed run — which it was not. The script had
been working the whole time and the observer was giving up first.

A default that cannot accommodate the slowest thing it observes is a fault in the observer, and
one that reads as a fault in the observed, which is the worst way for it to be wrong.