Verification

What it says it does

Verification script for Gadeke et al. 2026 — guilt and anterior insula.
eLife 105391 | doi:10.7554/eLife.105391

FAST MODE (default, ~3 min):
  Clones GitHub repo, loads pre-computed CSVs, runs logistic regression,
  and checks NIfTI peak coordinate from deposited contrast map.
  Requirements: pandas, scipy, statsmodels, nibabel
  Data: https://github.com/BonnSocialNeuroscienceUnit/ResponsibilityExperiment (~50 MB)

FULL MODE (--full, ~3 hrs):
  Downloads raw fMRI data from OpenNeuro ds005588 (~15 GB).
  Runs SPM12 first-level GLMs for all 28 subjects.
  Runs computational model fitting.
  Reproduces all manuscript figures.
  Additional requirements: MATLAB + SPM12
  Additional data: https://openneuro.org/datasets/ds005588 (~15 GB)
  Note: MATLAB license and SPM12 toolbox required.

Usage:
  python verify.py           # fast mode
  python verify.py --full    # full pipeline
  python verify.py --claim participants-probability-choosing-risky-option

What it says it does

Run a verification script under observation, and record what it actually did.

Only one of the nine verification scripts was ever audited, and auditing it found two
failures that no amount of reading would have caught:

  * a claim recorded `verified` while the function named in its record raised
    `shapes (4,4) and (5,5) not aligned` -- the verdict had been narrated, not observed;
  * a reproduction record naming `fMRI - Choices_singleTrialData.csv` while the code opened
    `Behav - Choices_singleTrialData.csv`.

Both were found by making the script report every path it opened and every value it computed.
That was done for Gaedeke by editing the script to call `used()` at each open. Doing the same
to eight more scripts would mean eight sets of hand-edits, each an opportunity to annotate a
path the code does not take -- which is the very failure being audited.

So this observes from outside instead. It patches `open` and the common loaders, executes the
script in-process, and reads its `ROWS` list afterwards. Nothing is taken from the script's
own account of itself: the file list comes from the file system calls, the results come from
the list the printed table is built from, and an exception is recorded whether or not the
script caught it.

What it emits, beside the script it ran:

  verification/<paper>/provenance.json    files opened, results produced, exceptions raised

Usage:
  python3 verification/audit_run.py <paper-slug> [-- script args]
  python3 verification/audit_run.py --all
  python3 verification/audit_run.py --all --timeout 1800

`--timeout` now defaults to 1800s rather than to no limit. The documented example used to say
900, which is shorter than the slowest verification script's own internal budget: Ejdrup's fast
mode runs two figure scripts and allows each 600s, so its worst case is 1200s before the clone
is counted, and an uncontended run measured 1064s. A run killed at 900s writes no results, and
`audit_verifications` then reports the paper as a failed run — which it was not. The script had
been working the whole time and the observer was giving up first.

A default that cannot accommodate the slowest thing it observes is a fault in the observer, and
one that reads as a fault in the observed, which is the worst way for it to be wrong.