The prompts

The prompts are the most load-bearing artifact in the system. Five files; ~1500 lines of prompt text total. They specify what each agent reads, how each should think about the task, what failure modes to avoid, and what output schema to produce.

This page is the reference for what’s in each prompt and where it lives. For the design rationale, see the three-agent partition and the external reviewer. For authoring variants, see adding a prompt variant.

Prompt files

All prompts live in extract/prompts/ (relative to the package root). The CLI loads them at runtime via Config.prompt_path(). The --prompt-variant flag swaps in extract/prompts/<variant>/<role>.md instead of the default.

Prompt file	Used by	Model	Role
results-reader.md	Step 3a (extraction agent)	claude-sonnet-4-6	Reads abstract + results prose
caption-reader.md	Step 3b (extraction agent)	claude-sonnet-4-6	Reads figure captions
structure-reader.md	Step 3c (extraction agent)	claude-sonnet-4-6	Reads methods + supplements
reconciler.md	Step 4 (reconciliation)	claude-opus-4-6	Folds three agent outputs into a draft
external-reviewer.md	Step 4.5 (review pass)	claude-opus-4-6	Recovers structural roles the prose extraction misses

Prompt structure

Each prompt is a self-contained Markdown document. The CLI loads it as a system message; the user message is the relevant slice of the paper (or, for the reconciler and reviewer, the agent outputs and/or paper context).

The prompts share a common structure:

1. Identity — who the agent is and what step it serves
2. Input contract — what the user message will contain
3. Role — what the agent is positioned to extract that the others can’t
4. Failure modes — the specific errors this agent is most prone to (drawn from docs/method.md § 3.3 “Common errors in claim extraction”)
5. Output schema — JSON shape the agent must produce
6. Field guidance — what each output field should contain
7. Quantity expectation — how many claims the agent typically surfaces
8. What good looks like — a short paragraph on the qualities of correct output
9. Examples (where relevant) — concrete worked examples to anchor abstract guidance

The reconciler and external-reviewer prompts add sections on:

• The confidence taxonomy (high / contested / single-source) for the reconciler
• The 4 (then 7) bias rules for the reviewer
• A worked example showing the operation on Headley-shaped input

Results-reader (results-reader.md)

~150 lines. Loaded as system message; user message is the paper’s abstract + results section text.

Key instructions:

• Read the abstract and results prose only — does not see captions or methods
• Extract claims from the argument as written
• Captures interpretive and synthesis claims (the paper’s conclusions, the reasoning that connects figures into an argument)
• Failure modes most prone to: overstating strength, discussion contamination, quantitative hallucination, missing negative results
• Output: JSON list of candidate claims with claim, panel, claim_type, role, evidence, confidence, notes

Caption-reader (caption-reader.md)

~140 lines. Loaded as system message; user message is the formatted figure caption text.

Key instructions:

• Read the figure captions only, panel by panel
• Extract panel-level empirical claims with exact quantitative values and panel anchoring
• Failure modes most prone to: wrong panel assignment, methodological-panel-as-claim, simulation-vs-experiment conflation, quantitative hallucination
• Required: every claim must have a non-null panel: field

Structure-reader (structure-reader.md)

~130 lines. Loaded as system message; user message is the methods section text (and supplements when available).

Key instructions:

• Read the methods, supplements, and code descriptions
• Extract methodological claims, scope conditions, structural features of the analysis
• Failure modes most prone to: inferring results from mechanism, reversing direction, simulation-vs-experiment conflation, single-source overconfidence
• Most claims are role: methodological or role: scope; few are role: empirical

Reconciler (reconciler.md)

~180 lines. Loaded as system message; user message contains all three agent outputs verbatim.

Key instructions:

• Define what “the same claim” means (same proposition, same direction, same entities — phrasing may differ)
• Define the confidence taxonomy: high (all three agents agreed), contested (two agree, one differs), single-source (only one)
• Preserve role distinctions when agents surface different roles for related claims (predictions and their empirical tests are different claims, not role-collisions to merge)
• Output: a complete DraftClaimTable JSON

External reviewer (external-reviewer.md)

~250 lines — the longest prompt. Loaded as system message; user message contains the paper’s abstract + results + the reconciled draft.

Key instructions:

• Address seven systematic biases:
  • Bias 1: prediction-role under-coverage
  • Bias 2: hypothesis-role under-coverage
  • Bias 3: multi-panel claim collapse
  • Bias 4: synthesis vs interpretation confusion
  • Bias 5: hypothesis-to-prediction over-shifting (added after Phase F kammer iteration)
  • Bias 6: control-role under-recognition (added after kammer iteration)
  • Bias 7: literature-context under-recognition (added after kammer iteration)
• May change a claim’s role/panel; may add new claims; may not delete claims (downgrade to notes instead)
• Output: a complete revised DraftClaimTable JSON

The reviewer prompt also includes a worked Headley-shaped example showing how to surface implicit hypothesis and prediction claims from an empirical sequence.

Authoring a prompt variant

To test a prompt change without modifying the defaults:

mkdir prompts/<variant-name>
cp prompts/results-reader.md prompts/<variant-name>/
# Edit prompts/<variant-name>/results-reader.md
# (other prompts in the variant are inherited from default if not present)

elife-extract extract --doi <DOI> --prompt-variant <variant-name>

Validate via evaluate --prompt-variant <variant-name> before deployment. See adding a prompt variant for the full workflow.

Implementation references

• extract/prompts/ — all prompt files
• elife_extract/agents.py:load_prompt() — prompt loading at runtime
• elife_extract/config.py:Config.prompt_path() — variant-aware path resolution

Next steps

• The three-agent partition — design rationale for the three extraction prompts
• The external reviewer — design rationale for the reviewer prompt
• Adding a prompt variant — workflow for authoring and testing variants