Reconciliation

Reconciliation is Step 4 of the methodology. Three agents have read the paper independently and produced three lists of candidate claims. Reconciliation folds them into a single draft claim table, with each unique proposition tagged by how many agents surfaced it (high, contested, single-source).

The hard problem reconciliation solves is semantic matching across phrasings. When the Results-reader emits “Doubling distal inhibition reduces firing from 5.5 to 0.2 Hz” and the Caption-reader emits “Figure 4A shows that 2× distal inhibitory conductance drives somatic firing rate to 0.20 ± 0.15 Hz from a baseline of 5.5 ± 0.86 Hz,” these are the same claim. Literal string match misses it. Embedding similarity is fuzzy and noisy. Asking Opus to do the alignment, given the full lists and the methodology’s confidence taxonomy, works.

What reconciliation produces

A DraftClaimTable with these fields:

paper_slug: headley-2024-spatially-targeted-inhibitory
paper_doi: 10.7554/eLife.95562
paper_title: Spatially targeted inhibitory rhythms ...
extraction_path: pdf
per_agent_counts:
  results: 48
  caption: 47
  structure: 17
claims:
  - claim: "Doubling the strength of distal dendritic inhibition reduces somatic firing rate from approximately 5.5 Hz to approximately 0.2 Hz."
    panel: "fig4a"
    claim_type: empirical
    role: empirical
    confidence: high                                 # all three agents agreed
    sources: [results, caption, structure]
    evidence_by_agent:
      results: "Doubling distal dendritic inhibition reduced firing from ~5.5 to ~0.2 Hz..."
      caption: "Figure 4A shows that 2× distal inhibitory conductance drives somatic firing rate to 0.20 ± 0.15 Hz from a baseline of 5.5 ± 0.86 Hz..."
      structure: "Inhibitory conductance modulation: 1× and 2× baseline; baseline calibrated to ~5.5 Hz somatic firing"
    notes: null
  - claim: "All results derive from a single-cell compartmental model of one L5 pyramidal tract neuron..."
    panel: null
    claim_type: assessment
    role: scope
    confidence: single-source
    sources: [structure]
    evidence_by_agent:
      structure: "We adapted a previously published model of an L5 pyramidal neuron..."
    notes: "[reconciler] only Structure-reader surfaced this scope condition; expected — methods is the right slice for global scope claims"
  ...
config_snapshot:
  model_results: claude-sonnet-4-6
  model_caption: claude-sonnet-4-6
  model_structure: claude-sonnet-4-6
  model_reconcile: claude-opus-4-6
  prompt_variant: default
  reconcile_strategy: confidence-tagged

The draft is JSON in <output-dir>/draft-<slug>.json. The extract subcommand writes it; the write subcommand consumes it.

The confidence taxonomy

Per docs/method.md § 3.3 Step 4:

Confidence	Meaning	Typical example
high	All three agents surfaced this claim independently	A panel-level empirical finding the prose, the caption, and the methods all describe
contested	Two agents agree, one differs	An empirical claim where the Results-reader’s interpretation differs from the Caption-reader’s literal reading
single-source	Only one agent surfaced this claim	A scope claim only Structure-reader saw; a synthesis claim only Results-reader could form; a panel-level detail only Caption-reader extracted

Single-source is the largest bucket on most papers (37 of 76 on Headley). This is not a quality concern — the partition is designed so each agent surfaces claims the others don’t have a path to. Single-source means “only this agent’s slice surfaced this claim,” not “this claim is unreliable.”

The exception is the rare case where a single-source claim should have been multi-source. If the Caption-reader produces a panel-level empirical claim that the Results-reader didn’t capture in its prose summary, that may indicate the Results-reader missed a load-bearing finding. The reconciler’s notes field flags these for analyst attention.

How reconciliation works

The reconciler is a single Opus call. The prompt is at prompts/reconciler.md; the implementation is at reconcile.py.

The user message to Opus contains:

1. The paper’s slug, DOI, title (for context)
2. All three agent outputs verbatim — formatted as nested markdown sections, each candidate claim listed with its claim text, panel, claim_type, role, evidence quote, agent confidence, and notes.

The system prompt instructs Opus on:

• What “the same claim” means (same proposition, same direction, same entities — phrasing may differ)
• When to keep claims separate even if related (a hypothesis and its empirical test are different claims)
• How to assign confidence (high if all three agents surfaced; contested if two agree and one differs; single-source otherwise)
• How to resolve role-class collisions when agents disagree on role
• The output schema (a complete DraftClaimTable JSON)

Opus reads the three lists, identifies which candidate claims describe the same proposition, picks the most precise phrasing for the canonical claim field, preserves the per-agent evidence quotes for audit, assigns confidence, and emits the unified draft.

Why Opus and not Sonnet

The reconciliation task is harder than the extraction task. The extraction agents are reading text and surfacing propositions — a relatively constrained operation that Sonnet handles well. The reconciler is doing semantic alignment across three lists, with judgment calls about claim equivalence and role-class arbitration. Opus is meaningfully better at this kind of holistic synthesis.

Empirically, Sonnet on the reconciliation step produces draft tables with ~30% more spurious “single-source” assignments — claims that should have been recognized as the same proposition across agents but the matcher missed. Opus’s stronger semantic alignment closes this gap.

The cost trade is small: ~$1 for one Opus call vs ~$0.50 for one Sonnet call. For a step that produces the input to every downstream operation (review, write, evaluate), the quality differential dominates.

Reconciliation strategies

The CLI exposes --reconcile-strategy for empirical comparison of alternatives:

Strategy	What it does	When to use
confidence-tagged (default)	The methodology’s prescription. Each unique claim tagged high / contested / single-source.	Standard operation.
union	Keep all claims from all agents, no merging.	Debugging extraction quality without reconciliation interference.
intersection-only	Keep only claims surfaced by all three agents.	Maximum-precision use cases where missing claims is acceptable.
majority-vote	Keep only claims surfaced by ≥2 agents.	Mid-precision option between intersection and union.

union and intersection-only are diagnostic; not recommended for production extraction. Only confidence-tagged is currently fully implemented; the others raise NotImplementedError if invoked. The hooks exist so future development can wire them up if a use case emerges.

What reconciliation does not do

• Does not modify claim text. The canonical claim sentence chosen comes from one of the agents’ verbatim phrasings (or a minor merge); the reconciler is constrained against rewriting claims into prose Opus would prefer.
• Does not surface new claims. If none of the three agents extracted a claim, reconciliation can’t add it. The structural claims the reviewer adds at Step 4.5 are a separate operation.
• Does not classify roles definitively. Role assignments are provisional — the analyst (or the external reviewer) makes the final call at Step 5.

Implementation references

• prompts/reconciler.md — the full reconciler prompt
• elife_extract/reconcile.py — the implementation (~150 lines)
• elife_extract/schema.py — the DraftClaimTable and ReconciledClaim Pydantic models

Next steps

• The external reviewer (Step 4.5) — what runs after reconciliation in external mode
• The review gate — Step 5 and the four review modes
• The prompts reference — the full reconciler prompt text