When to use it
The system is built for a specific shape of work: extracting structured claims from eLife papers at a rate higher than manual curation can sustain. This page maps that shape to your operational situation, and names the cases where the system is the wrong tool.
Use cases the system is designed for
Section titled “Use cases the system is designed for”Adding a new paper to a curated corpus
Section titled “Adding a new paper to a curated corpus”You have an eLife paper that should join your claim-graph corpus. You want a draft set of claims that’s high enough quality to review and adjust, not start from scratch.
Recommended: --review-mode interactive. The CLI extracts, reconciles, and the external reviewer produces a draft. Your $EDITOR opens with the YAML; you spend 15-30 minutes per paper adjusting roles, splitting/merging where the agents got too granular, then save and the claim files are written.
Cost: ~$7 in API + 30 min of analyst time per paper. Compare to manual extraction (~4 hours per paper from scratch).
Bulk-extracting a corpus for downstream use
Section titled “Bulk-extracting a corpus for downstream use”You have N papers (10, 100, 1000) you want to populate as claim graphs. Manual review of each is infeasible; you’ll do spot-checking and accept some rough edges.
Recommended: --review-mode external. The Opus reviewer pass substitutes for the curator at the review gate; output approaches curator quality (~96% role agreement on the round-trip benchmark) without requiring per-paper analyst time.
Cost: ~$7 per paper × N. For 100 papers: ~$700 + ~17 hours sequential (or fewer with shell-level parallelism). For 1000 papers: ~$7K + ~3 days sequential. Vertex AI rate quotas are the practical ceiling on parallelism.
Validating a prompt change
Section titled “Validating a prompt change”You modified one of the agent prompts (e.g., added a signal-phrase rule for the control role) and want to know if the change improves overall quality before shipping.
Recommended: evaluate --all against the curated reference corpus. Produces an aggregate scorecard; diff against the prior baseline. If recovery and role agreement both go up (or one goes up and the other holds), ship; if either drops materially, the change is a regression.
Cost: ~$100 per full 10-paper sweep, ~85 min wall time.
Validating a model substitution
Section titled “Validating a model substitution”You want to know whether the cheaper Haiku-4-5 model can handle the Caption-reader’s role at acceptable quality.
Recommended: evaluate --model-caption claude-haiku-4-5 --all and compare aggregate scorecards. The CLI’s --model-{results,caption,structure,reconcile} flags exist precisely for this kind of substitution test.
Augmenting the eLife review process
Section titled “Augmenting the eLife review process”You want claim-level structure surfaced for papers under peer review, to help reviewers and editors trace what the paper actually claims and where the evidence lives.
Recommended: --review-mode external per submission, results piped to your editorial system. The CLI doesn’t (yet) integrate with eLife’s editorial backend, but it produces a stable JSON intermediate (draft-<slug>.json) that any downstream system can ingest.
Use cases the system is not designed for
Section titled “Use cases the system is not designed for”Verifying that a paper’s analysis reproduces
Section titled “Verifying that a paper’s analysis reproduces”The system extracts claims; it does not re-run the paper’s analysis on deposited code/data to check that reproduced numerics match the published figures. That’s Step 8 of the methodology — handled per-paper by verification/<slug>/verify.py in the corpus repo, not by the CLI.
Papers without typical structure
Section titled “Papers without typical structure”The prompts are calibrated against eLife neuroscience papers. The system handles atlas papers (Artiushin), purely observational surveys, and methods-heavy computational papers, but with reduced quality. Use the --prompt-variant flag to develop and test variants for paper types where the default prompts under-perform; see contributing prompt variants.
Cross-paper graph resolution
Section titled “Cross-paper graph resolution”When the corpus grows enough that citations resolve to other claims in the corpus (rather than just to DOIs), graph-resolution logic is needed beyond what verify-refs provides. CrossRef gives you the DOIs; resolving “this claim’s citation matches that paper’s claim X” requires entity-resolution that the CLI doesn’t yet implement.
One-off questions about a single paper
Section titled “One-off questions about a single paper”If you just want to ask “what does Headley conclude?”, a single Claude call with the PDF as context will give you an answer in 30 seconds for $0.50. The CLI is for producing structured, schema-conformant, version-controllable claim files — not for ad-hoc Q&A.
Papers from publishers other than eLife
Section titled “Papers from publishers other than eLife”The PDF fetch path is hardcoded to the eLife CDN URL pattern. For other publishers, you’d need to adapt prepare.py to handle their PDF-fetch endpoint or pass the PDF directly. The rest of the pipeline (extraction, reconciliation, review, write) is publisher-agnostic.
Quick decision matrix
Section titled “Quick decision matrix”| Your situation | Recommended mode | Per-paper cost |
|---|---|---|
| Adding 1-5 papers, you have analyst time | --review-mode interactive | $5 + 30 min analyst |
| Adding 10-1000 papers, no per-paper review | --review-mode external | $7 |
| Adding many papers but you’ll review the .md files manually after | --review-mode auto-approve | $5 + manual review |
| Testing a prompt change | evaluate --all | $100 / sweep |
| Just inspecting an extraction | --review-mode dry-run | $5 (no write) |
| Papers from non-eLife publisher | Adapt prepare.py first | n/a |
Next steps
Section titled “Next steps”- Install and configure — get the CLI running
- First paper walkthrough — concrete steps from DOI to corpus
- Review modes — when to use each — deeper than this page on the four review options