Pipeline

A layer is a question asked of a paper — what does it claim, which claims does its abstract carry, what did re-running its code produce. Each answer is a version, and each run records the content hash of everything it read, so an answer that has fallen behind its inputs says so.

10 papers, 31 layers each. 59 of the 310 cells hold an answer. 2 have been read by a person.

How a paper is taken apart, and why the pipeline is shaped this way.

The layers

Every layer rests on the ones to its left, and a change travels the other way: revise the claim format and every tree beneath it is out of date. Dashed borders mark the decisions that belong to no single paper — what a relation asserts, what counts as a claim — which is why altering one unsettles the whole corpus at once.

depth 0
depth 1
depth 2
depth 3
depth 4
depth 5
depth 6
depth 7
depth 8
how many papers have been through itcorpus-scope

Which papers have which layer

A blank is a layer that has not been run for that paper, not a layer that ran and found nothing. No two papers have been through the same set.

Source Induction Tree Measures Measures Interchange Tree Measures Tree
Paper prepare article results-reader caption-reader structure-reader reconcile external-review edge-inference claim-tree stance questions parts reference-check abstract-map synthesis summaries plain-claim verification coverage adjudication marks gap-claim edge-review replication mira-export oxa-export dg-export formats-report warrant verification-check modules Fill
Artiushin 14/31
Bouyeure 15/31
Ejdrup 17/31
Gadeke 30/31
Headley 16/31
Kämmer 16/31
Kolb 16/31
Rozak 15/31
Scheller 15/31
Wengert 16/31
not run 140 blocked upstream 8 stale 103 run not observed 59
approved — a person read and approved this version partial — a reading is under way no mark — awaiting approval

Undecided 2

Questions the corpus has not answered. Each is a decision about what the schema means, and every claim tree built under one is provisional until it is taken.

Epistemic vocabulary Does epistemic record support strength, or restate the role? zmainen/claim-graphs#21 Evaluation How well does each model profile recover a paper's claim tree? zmainen/claim-graphs#24

How far each tree’s argument reaches

The warrant layer grades each claim from the tree’s argument. A claim it reaches nothing for is unassessed — counted apart from weak, which is a claim the argument reaches and finds weakly supported. Only papers whose warrant layer has run appear.

Gadeke the argument reaches 6 of 6 claims

How much of this is checkable

A cell goes stale when a file it read has changed since. That comparison needs a recorded run, and most of this corpus predates the ledger.

0 ran with their inputs recorded, and every one still matches the files it read
59 produced an output before the ledger existed. Whether they are current is not known and cannot be worked out from what was kept
140 have never been run

111 cells have fallen behind an input:

Recently run

The last answers the corpus produced, newest first.