No human checks this corpus
Every claim in this repository was produced by a language model, and no person has
verified any of it. That is true of the whole pipeline, not one step of it:
- The claims are found by three model calls reading the paper independently, and
reconciled by a fourth.
- The relations between claims — the deductive spine — are inferred by a model call.
- The method requires an analyst's review gate. There is no such gate. What stood in
for one was a flag —
--review-mode external, which is another model revising the
draft, or --review-mode auto-approve, which wrote the files unread — and both are gone.
The model's revision is now a declared layer, external-review, so changing the prompt
behind it makes every claim tree built on it stale. Approval is what it should always have
been: an operation on a version that already exists, recorded in
runs/<paper>/approvals.jsonl by scripts/pipeline.py approve. That file is empty for
every cell in this corpus — the same fact as before, now stated where it can be checked
rather than inferred from a default.
- The
agent: field on a verification record names the agent identity that ran the
script, not a person who checked the result. All 225 records read agent: mainen-z;
none of them means Zach Mainen read that claim.
- The coverage verdicts in
mappings/ — covered, gap, no-assertion — are model
judgements, made from the paper and the claim text.
Read the corpus as a draft annotation layer, not as adjudicated output. Where this
document says "curated", it means "assembled by this pipeline", not "checked by a person".
A human-review step is compatible with the format — claim files carry provenance fields,
and the tooling can already write anchored comments and claim assignments into the paper
for a person to disposition. It is not implemented, and it has not been run. Adding it
would be a change to the method, and would be recorded here as one.
Claim induction is the translation of a paper from argument format into claim-graph format. It demands reading comprehension, domain judgment, and decisions about what constitutes a claim — and in this version a model makes all of them.
Induction is not one act but a sequence of them, and each is a declared layer with its own
question, its own inputs and its own record of having run. What each does, and how, is
documented beside its declaration in pipeline/layers/ and gathered in the
Layers section. What follows here is what holds across all of them.
What the readers are actually given
Before any reading happens, the paper has to be turned into text, and the format that text
comes in bounds everything downstream. This step is the prepare layer, and it is stated here
because a method that says "three agents read the paper" without saying what they read has
skipped the decision that constrains the rest.
For an eLife DOI the source is JATS XML, not the PDF and not the web page. JATS — the
Journal Article Tag Suite, the archival XML that eLife and most journals typeset from — is
fetched from the article's CDN URL:
https://cdn.elifesciences.org/articles/<article-id>/elife-<article-id>-v1.xml
The choice is not incidental. JATS carries the structure a claim tree needs and a PDF has
already discarded:
|
JATS XML |
PDF |
| Section boundaries |
tagged (<abstract>, <sec>) |
inferred by regex against heading text |
| Figure captions |
tagged, one element per figure, with its own id |
recovered from body text by position |
| Figure identity |
eLife's own <fig id> — fig2, fig3s1 |
guessed from a label like "Figure 2" |
| Reading order |
explicit |
a column-order guess from pdfplumber |
| Tables |
<table-wrap> with a caption |
text that may or may not still be a table |
The panel-level grounding the whole format rests on — every empirical claim naming the panel it
came from — is only as good as the figure structure at intake. Read from a PDF, a claim's
panel field is an inference about a label in running text; read from JATS, it is the
publisher's own element id.
The PDF path still exists and is still used, but only deliberately: --input-format pdf, or
--pdf-path for a paper that is not on the eLife CDN. There is no silent fallback — with a
DOI and the default --input-format auto, the intake is JATS or it fails. A method that
quietly degraded to PDF when a fetch failed would make every claim below it weaker without
saying so.
What prepare writes is runs/<paper>/prepared.json: the four slices the readers are given —
abstract, results, figure captions, methods — the typed figure captions, and
extraction_path recording which route was taken, with the source URL. That file is the
verbatim text the model saw, which is what makes an extraction re-examinable rather than merely
re-runnable. For Gädeke it records:
extraction_path jats
source https://cdn.elifesciences.org/articles/105391/elife-105391-v1.xml
abstract 1,931 characters
results 33,886 characters
captions 8,496 characters (7 typed figure captions)
methods 32,690 characters
The fetch is cached under ~/.cache/elife-extract/, so every later layer on the same paper
runs offline and costs nothing. The practical consequence for anyone debugging a bad
extraction: check the slice sizes first. A short results means section detection failed, and
everything downstream read the wrong text — which is a fault in intake presenting as a fault in
reading.
Reading the abstract first
Read the abstract and identify two to four top-level claims — the paper's main bets. For each, write a candidate slug (3–5 words, lowercase, hyphenated, verb-phrase form). These will be the synthesis or interpretation nodes at the top of the dependency graph. They typically have no single figure of their own — they are the synthesis of the figures below them. The Headley paper, for example, surfaces pv-gamma-sst-beta-correspondence as a single synthesis node interpreting the simulation results in light of prior interneuron-rhythm associations; this node's panel is "fig10 (synthesis / discussion)" rather than a single quantitative panel.
Why three readers
The three agents are deliberately partitioned along the axes along which extractions most often disagree: framing versus literal numerics versus computational structure. A claim that all three surface independently is high-confidence; a claim that only one surfaces is single-source and may be either real-but-buried or an artefact of the reading strategy. The reconcile layer records both cases distinctly.
The partition is by section, and each reader sees its sections and nothing else. The results reader is given the abstract, the Introduction, the Results and the Discussion, preceded by a panel inventory; the Introduction and Discussion were added on 2026-09-11 (#56), after the first runs showed that cutting them lost the organising hypothesis and every literature-context premise. The caption reader is given the figure and table captions with the inventory. The structure reader is given the Methods, the appendices and the supplementary material. No reader sees the whole paper.
A single reading of the whole paper is kept as a comparison, not as a layer. It runs through the same reader code on the raw paper (reader_from_raw) and is scored by evaluate as the fourth-reading row beside the three-reader chain; on Gädeke it recovered slightly more of the curated tree at higher precision and missed every alternative the paper rules out. Whether the partition earns its place is a scheme question (#85), to be settled against adjudicated trees, and the comparison row exists so that it can be. Until it is settled, the three-reader chain is the accepted scheme and every recorded run uses it.
Which model reads is a profile, not a property of the readers. The standard profile puts the three readers on Sonnet and the reconciler, reviewer and edge inference on Opus; frontier puts everything on Opus with leaner tasks. A run answered by Claude Code subagents rather than an API records, in the ledger's by, the model that actually answered each prompt, which need not match the profile it stands in for; Gädeke's v3 readers, for instance, were answered by Opus under the subagent profile.
Three fields are populated during authoring but are not the primary claim sentence:
-
displayClaim (one to two sentences) is the form rendered when the claim is presented in body text or in a card view. It softens the formal claim sentence into something readable in a paragraph; it preserves the proposition's content but allows shorter constructions, parenthetical units, and contractions where the formal claim field cannot. Authored with the claim files; can be revised without changing the underlying claim.
-
shortClaim (single short clause, ≤90 characters typical) is the headline form: what fits in a tooltip, a hover preview, or a graph-node label. Required for synthesis, interpretation, and literature-context nodes that must be readable at a glance in the synthesis layer; optional but recommended for hypotheses, predictions, and high-traffic empirical claims. Authored with the claim files.
-
number and numberParts are not authored manually. They are computed by the build pipeline from the claim's role and its position in the dependency graph (hypotheses get H#, predictions hang off the hypothesis they derived-from as H#.P#, empirical claims testing those predictions hang off as H#.P#.E#, scope claims become Sc#, methodological become M#, controls become C#, literature-context becomes L#, synthesis becomes S#, interpretation becomes I#, and standalone empirical not under any hypothesis loop become E#). The numbering is regenerated on every build; do not paste numbers into source files.
role is assigned at the review gate and is the most consequential single field in the schema after claim, because it governs how the synthesis layer groups the claim and how the claim is numbered in the build.
↑ Contents