Cost and performance

This page is the operational counterpart to batch operation. It breaks down where the cost goes per paper, what knobs are available to reduce it, and what the measured numbers look like for the default configuration.

All costs are at Anthropic’s published Vertex AI pricing for claude-sonnet-4-6 and claude-opus-4-6 as of mid-2026. Direct API pricing is comparable. CrossRef API is free.

Per-paper cost breakdown (default configuration)

Default models: Sonnet 4.6 for the three extraction agents, Opus 4.6 for reconciliation and external review. Typical eLife paper (~30 pages, ~30-50 reference claims).

Step	Model	Input tokens	Output tokens	Cost
Step 1 (prepare)	none	0	0	$0
Step 3a (Results-reader)	Sonnet 4.6	~25k (abstract + results prose + prompt)	~6-10k (claim list)	~$1.20
Step 3b (Caption-reader)	Sonnet 4.6	~25k (captions + prompt)	~6-10k (claim list)	~$1.20
Step 3c (Structure-reader)	Sonnet 4.6	~10k (methods + prompt)	~3-5k (claim list)	~$0.60
Step 4 (reconcile)	Opus 4.6	~30k (3 agent outputs + prompt)	~10-15k (draft)	~$1.50
Step 4.5 (external review, optional)	Opus 4.6	~50k (paper context + draft + prompt)	~12-20k (revised draft)	~$2.00
Steps 6-7 (write)	none	0	0	$0
verify-refs (CrossRef)	none	n/a	n/a	$0
Total (auto-approve)		~85k	~25-40k	~$4.50-5
Total (external review)		~135k	~37-60k	~$6.50-7

Round trip scoring (evaluate) adds one more Opus call (~$3) for the matcher, bringing the per-paper validation cost to ~$10 with external review.

Wall time breakdown (sequential, network-good)

Step	Time
Step 1 (PDF fetch + slice; cached: under 1s)	~5-10s
Step 3a-c (three Sonnet calls, sequential)	~3-5 min
Step 4 (Opus reconcile)	~1-2 min
Step 4.5 (Opus reviewer)	~2-5 min
Steps 6-7 (write)	under 1s
verify-refs	~1-3s per cited paper
Total per paper, with external review	~10-15 min
Total per paper, without external review	~5-7 min

Streaming is enabled for all LLM calls (the Anthropic SDK requires it for outputs that may exceed 10 minutes). Wall time is dominated by the slowest LLM call in the sequence; the three Sonnet calls in Step 3 run sequentially in the current implementation, but could be parallelized for a ~2x time savings on the extraction phase.

Cost-reduction knobs

The CLI exposes several flags for trading cost against quality. Each lever’s empirical effect depends on the paper; use evaluate to measure before committing.

--max-claims

Hard cap on per-paper claim count. The agents stop emitting candidates once they hit the cap. Useful for batch operation where you want a per-paper cost ceiling.

elife-extract extract --doi <DOI> --max-claims 20

Effect: limits the agents’ output budget. If a paper would naturally produce 50 claims, capping at 20 forces the agents to prioritize. Quality drops are typical — the agents may keep panel-level empirical claims and drop synthesis or methodological claims. Use sparingly.

Per-agent model overrides

The four model flags let you swap individual agents to cheaper or more expensive models:

elife-extract extract --doi <DOI> \
  --model-results claude-haiku-4-5 \
  --model-caption claude-haiku-4-5 \
  --model-structure claude-haiku-4-5 \
  --model-reconcile claude-sonnet-4-6

Effect: dropping all three extraction agents to Haiku and reconciliation to Sonnet cuts the per-paper cost roughly in half (~$2.50 instead of ~$5). Quality drops measurably — Haiku is less reliable at the methodology’s role-classification task.

Use this for cost-sensitive bulk extraction where the analyst will review the output anyway.

--prompt-variant

Named directory under prompts/<variant>/. Lets you A/B test prompt revisions:

mkdir -p prompts/lean
# Author shorter prompts in prompts/lean/{results,caption,structure}-reader.md
elife-extract extract --doi <DOI> --prompt-variant lean

Shorter prompts reduce input token count (~5-10% savings). Output budget is unchanged. Quality effect depends on what you trim; measure with evaluate.

Skipping the external reviewer

The largest single saving: drop --review-mode external (~$2 per paper) for auto-approve. Quality drops from ~78% to ~65% mean role agreement. Right tradeoff for pure throughput; wrong tradeoff for journal-quality output.

Cost projections at corpus scale

Default configuration (Sonnet × 3 + Opus × 2), with external reviewer:

Corpus size	Cost	Notes
1 paper	$7	The walkthrough cost
10 papers	$70	The 10-paper validation sweep
100 papers	$700	A reasonable corpus build
1000 papers	$7K	A journal-scale extraction
3000 papers	$21K	The panel-claim-unification.md scaling horizon

Without external reviewer (auto-approve): subtract ~$2 per paper. Quality is lower; right tradeoff for some operational shapes.

With Haiku-tier extraction agents: roughly half the cost (~$3.50 / paper with reviewer). Quality drops; measure with evaluate before committing to a Haiku-based pipeline at scale.

Performance optimizations not yet implemented

The current implementation is conservative — sequential execution, no batching, retry-on-failure rather than retry-with-backoff-then-batch. Several optimizations are feasible:

• Concurrent extraction agents. The three Sonnet calls in Step 3 are independent; running them concurrently would cut Step 3 wall time by ~2x with no cost change. Implementation: asyncio.gather(...) in agents.run_all_agents().
• Anthropic Batch API. For corpus-scale extraction without latency requirements, the Batch API offers ~50% cost reduction at the price of 24-hour turnaround. Not currently wired in; a separate batch-extract subcommand would be the natural shape.
• Caching the reconciler. When iterating on the external reviewer, the reconciliation output is identical across reviewer-prompt iterations. Caching the reconciled draft and only re-running the reviewer would save ~$1.50 per iteration.

These are roadmap items, not blockers. The current cost profile is acceptable for journal-scale operation as proposed.

Implementation references

• Per-step pricing assumptions: Anthropic published rates for Sonnet 4.6 and Opus 4.6 on Vertex
• elife_extract/agents.py — token usage per extraction agent
• elife_extract/external_review.py — token usage for the reviewer pass

Next steps

• Validation results — what the cost-quality tradeoff looks like in practice
• Iteration discipline — how to measure cost-quality tradeoffs before shipping a change