RISE Humanities Data Benchmark

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
Model configuration – provider, model version, temperature, and other generation parameters.
Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
Usage and cost data – token counts and calculated API costs.
Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Result 121 of 208

Test T1050 at 2026-05-22

book-page information-extraction printed 20 en list bibliographic

genai gemini-3.5-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score33.7%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"abridged by D. C. Somervell","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":"Abridgement of vols. I-VI of A Study of History","edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":["146","147","148"],"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":null,"author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":null,"author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":null,"author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":null,"author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"other","title":"Philosophy of History","container_title":null,"author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":null,"editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":null,"author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":null,"author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":null,"author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":null,"author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":null,"author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":null,"author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":null,"author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":null,"author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":null,"author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":null,"author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score63.0%

Speed

Model time, all inputs118.5 s

Mean per input23.70 s

Slowest input28.93 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens5.8K

Output tokens10.7K

Total tokens16.5K

Input cost$0.0087

Output cost$0.096

Total cost$0.2635

Reasoning cost usd$0.1588

Total reasoning tokens17.6K

Priced from 2026-05-22 · Input $/M $1.50 · Output $/M $9.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 122 of 208

Test T1037 at 2026-04-27

book-page information-extraction printed 20 en list bibliographic

openai gpt-5.5-2026-04-23 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score44.4%

{"metadata":{"title":"Bibliography","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History.","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":"Abridged by D. C. Somervell","incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht; Antwerpen","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia","container_title":null,"author":null,"note":"Relevant articles listed in entries 146-148.","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":"15-20 November, 1946","related":["146","147","148"],"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d’histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"Reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959.","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":null}]}

Scoring

Fuzzy score71.6%

Speed

Model time, all inputs177.8 s

Mean per input35.57 s

Slowest input43.11 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens18.1K

Output tokens15.2K

Total tokens33.3K

Input cost$0.0906

Output cost$0.4561

Total cost$0.5466

Priced from 2026-04-27 · Input $/M $5.00 · Output $/M $30.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 123 of 208

Test T0983 at 2026-04-22

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-9b-20260310 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score0.0%

{
  "$defs": {
    "Author": {
      "properties": {
        "family": {
          "title": "Family",
          "type": "string"
        },
        "given": {
          "title": "Given",
          "type": "string"
        }
      },
      "required": [
        "family",
        "given"
      ],
      "title": "Author",
      "type": "object"
    },
    "Entry": {
      "properties": {
        "id": {
          "title": "Id",
          "type": "string"
        },
        "type": {
          "$ref": "#/$defs/EntryType"
        },
        "title": {
          "title": "Title",
          "type": "string"
        },
        "container_title": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Container Title"
        },
        "author": {
          "anyOf": [
            {
              "items": {
                "$ref": "#/$defs/Author"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Author"
        },
        "note": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Note"
        },
        "publisher": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Publisher"
        },
        "editor": {
          "anyOf": [
            {
              "items": {
                "type": "string"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Editor"
        },
        "publisher_place": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Publisher Place"
        },
        "issued": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Issued"
        },
        "event_date": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Event Date"
        },
        "related": {
          "anyOf": [
            {
              "items": {
                "type": "string"
              },
              "type": "array"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Related"
        },
        "relation": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Relation"
        },
        "volume": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Volume"
        },
        "page": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Page"
        },
        "fascicle": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Fascicle"
        },
        "reprint": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Reprint"
        },
        "edition": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Edition"
        },
        "incomplete": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Incomplete"
        }
      },
      "required": [
        "id",
        "type",
        "title"
      ],
      "title": "Entry",
      "type": "object"
    },
    "EntryType": {
      "enum": [
        "book",
        "journal-article",
        "other"
      ],
      "title": "EntryType",
      "type": "string"
    },
    "Metadata": {
      "properties": {
        "title": {
          "title": "Title",
          "type": "string"
        },
        "year": {
          "title": "Year",
          "type": "string"
        },
        "page_number": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "title": "Page Number"
        }
      },
      "required": [
        "title",
        "year"
      ],
      "title": "Metadata",
      "type": "object"
    }
  },
  "properties": {
    "metadata": {
      "$ref": "#/$defs/Metadata"
    },
    "entries": {
      "items": {
        "$ref": "#/$defs/Entry"
      },
      "title": "Entries",
      "type": "array"
    }
  },
  "required": [
    "metadata",
    "entries"
  ],
  "title": "Document",
  "type": "object"
}

Scoring

Fuzzy score0.0%

Speed

Model time, all inputs8692.0 s

Mean per input1738.41 s

Slowest input2270.71 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens16.5K

Output tokens71.9K

Total tokens88.4K

Input cost$0.0017

Output cost$0.0108

Total cost$0.0124

Priced from 2026-04-21 · Input $/M $0.10 · Output $/M $0.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 124 of 208

Test T0996 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

openrouter google/gemma-4-26b-a4b-it-20260403 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score17.8%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":9},"entries":[{"id":"140","type":"book","title":"Smellie, Why We Read History","container_title":"P. Elek","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"Toynbee, A Study of History, abridged by D. C. Somerwell","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"Oxford","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"142","type":"book","title":"Verberne, In den spiegel van het verleden: historische opstellen","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Vevir, Esquemas de metodologia eclesiástica","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Walz, Hans H., Sinn und Ziel der Geschichte","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":"Universitäts-Verlag","incomplete":false},{"id":"145","type":"journal-article","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":"Milano: Castellani","author":null,"note":"Relevant articles:","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"journal-article","title":"Pietro C. Drago, \"Storia e materialismo storico\"","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"vol. I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"journal-article","title":"Vito Fazzi-Allmyer, \"Critica del giudizio storico\"","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"vol. I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"journal-article","title":"A. Nyman, \"Forza e spirito nella storia\"","container_title":null,"author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"vol. I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"book","title":"Benjamin, Walter, \"Sur le concept d'histoire\"","container_title":"Les modernes II","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"book","title":"Boodin, John E., \"Philosophy of History\"","container_title":"Dagobert D. Runes, ed., 20th Century Philosophy","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":"Philosophical Library","incomplete":false},{"id":"151","type":"book","title":"Caillois, R. P., \"De la perception à l'histoire\"","container_title":"La philosophie de Merleau-Ponty","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"57-85","fascicle":null,"reprint":null,"edition":"Decaulion II","incomplete":false},{"id":"152","type":"book","title":"Cantimori, Delio, \"Commemorazione di Adolfo Omodeo\"","container_title":"Annali della Scuola Normale di Pisa","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":"Torino","issued":null,"event_date":null,"related":null,"relation":null,"volume":"fasc. 3-4","page":null,"fascicle":null,"reprint":"reprinted in D. Cantimori, Studi di storia","edition":"Einaudi, 1959","incomplete":false},{"id":"153","type":"book","title":"Cowan, Thomas, \"The Historian and the Philosophy of Science\"","container_title":"XXXVIII","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"book","title":"Crawford, R. M., \"History as Science\"","container_title":"Historical Studies: Australia and New Zealand III","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"book","title":"Danielou, J., \"Christianisme et histoire\"","container_title":"Etudes CCLIV","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"book","title":"Dell, Stanley, \"Wahrheit der Geschichte\"","container_title":"Geschichte der Wahrheit","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"book","title":"Fay, Sidney B., \"The Idea of Progress\"","container_title":"American Historical Review II","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"book","title":"Fessard, Gaston, \"Théologie et histoire\"","container_title":"Dieu vivant VIII","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"book","title":"Fish, Andrew, \"Acton, Creighton, and Lea: a Study in History and Ethics\"","container_title":"Pacific Historical Review XVI","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"book","title":"Garosci, A., \"Le conscience historique\"","container_title":"Les modernes III","author":null,"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score41.7%

Speed

Model time, all inputs1859.1 s

Mean per input371.82 s

Slowest input1588.84 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens2.8K

Output tokens14.6K

Total tokens17.4K

Input cost$0.0002

Output cost$0.0051

Total cost$0.0053

Priced from 2026-04-21 · Input $/M $0.08 · Output $/M $0.35

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 125 of 208

Test T0905 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-122b-a10b-20260224 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score33.9%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgegement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score67.7%

Speed

Model time, all inputs138.6 s

Mean per input27.72 s

Slowest input36.54 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens17.5K

Output tokens16K

Total tokens33.6K

Input cost$0.0046

Output cost$0.0334

Total cost$0.0379

Priced from 2026-04-21 · Input $/M $0.26 · Output $/M $2.08

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 126 of 208

Test T1009 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

openrouter google/gemma-4-31b-it-20260402 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score32.2%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":9},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":"London: P. Elek.","publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"abridged by D. C. Somerville, New York: Oxford (Abridgement of vols. I-VI of A Study of History).","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":"Utrecht, Antwerpen: Bruna.","publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":"Barcelona: Instituto.","publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":"Bonn: Universitäts-Verlag.","publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":[],"note":"Milano: Castellani.","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"1946-11-15","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"book","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"book","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Vito Fazzo-Alimayer","given":""}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"book","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"book","title":"Sur le concept d'histoire","container_title":"Les temps modernes II","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"book","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"boodin","given":"E."}],"note":"New York: Philosophical Library","publisher":"Philosophical Library","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"151","type":"book","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion II","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"152","type":"book","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":false},{"id":"153","type":"book","title":"The Historian and the Philosophy of Science","container_title":"Historical Studies","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"book","title":"History as a Science","container_title":"Historical Studies","author":[{"family":"Crawford","given":"R. M."}],"note":"Australia and New Zealand III","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"book","title":"Christianisme et histoire","container_title":"Etudes CCLIV","author":[{"family":"Danielou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"book","title":"Wahrheit der Geschichte","container_title":"Geschichte der Wahrheit","author":[{"family":"Dell","given":"Stanley"}],"note":"Amerikanische Rundschau III","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"book","title":"The Idea of Progress","container_title":"American Historical Review LII","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"book","title":"Théologie et histoire","container_title":"Dieu vivant VIII","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"book","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review XVI","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"book","title":"Le conscience historique en Italie","container_title":"Les temps modernes III","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score61.4%

Speed

Model time, all inputs4021.6 s

Mean per input804.32 s

Slowest input1841.58 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens2.8K

Output tokens13.5K

Total tokens16.2K

Input cost$0.0004

Output cost$0.0051

Total cost$0.0055

Priced from 2026-04-21 · Input $/M $0.13 · Output $/M $0.38

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 127 of 208

Test T0983 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-9b-20260310 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score34.1%

{"metadata":{"title":"Bibliography","year":"1947","page_number":9},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":"","author":[{"family":"Smellie","given":"K. B."}],"note":"","publisher":"P. Elek","editor":[],"publisher_place":"London","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":"","author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgegement of vols. I-VI of A Study of History","publisher":"Oxford","editor":[],"publisher_place":"New York","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":"","author":[{"family":"Verberne","given":"L. G. J."}],"note":"","publisher":"Bruna","editor":[],"publisher_place":"Utrecht, Antwerpen","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":"","author":[{"family":"Vevir","given":"José"}],"note":"","publisher":"Instituto","editor":[],"publisher_place":"Barcelona","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":"","author":[{"family":"Walz","given":"Hans H."}],"note":"","publisher":"Universitäts-Verlag","editor":[],"publisher_place":"Bonn","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":"","author":[],"note":"Relevant articles:","publisher":"Castellani","editor":[],"publisher_place":"Milano","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"","author":[{"family":"Drago","given":"Pietro C."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"vol. I","page":"109-117","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"vol. I","page":"281-290","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"","author":[{"family":"Nyman","given":"A."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"vol. I","page":"191-200","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"","author":[{"family":"Benjamin","given":"Walter"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"II","page":"623-634","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"150","type":"book","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":"","publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"89-106","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"","author":[{"family":"Caillois","given":"R. P."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"II","page":"57-85","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"fasc. 3-4","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"","author":[{"family":"Cowan","given":"Thomas"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"XXXVIII","page":"11-18","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"III","page":"153-175","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"","author":[{"family":"Daniélou","given":"J."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"CCLIV","page":"166-184","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"III","page":"44-53","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"LII","page":"231-246","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"VIII","page":"39-65","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"XVI","page":"59-69","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"III","page":"23-24","fascicle":"","reprint":"","edition":"","incomplete":false}]}

Scoring

Fuzzy score46.1%

Speed

Model time, all inputs2042.3 s

Mean per input408.46 s

Slowest input1087.51 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens15.3K

Output tokens36.2K

Total tokens51.4K

Input cost$0.0015

Output cost$0.0054

Total cost$0.007

Priced from 2026-04-21 · Input $/M $0.10 · Output $/M $0.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 128 of 208

Test T0944 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-397b-a17b-20260216 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score27.7%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":9},"entries":[{"id":"}]}```jsonl 140, ","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 141, ","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 142, ","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 143, ","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 144, ","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 145, ","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":[],"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 146, ","type":"book","title":"Storia e materialismo storico","container_title":null,"author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 147, ","type":"book","title":"Critica del giudizio storico","container_title":null,"author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 148, ","type":"book","title":"Forza e spirito nella storia","container_title":null,"author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 149, ","type":"book","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 150, ","type":"book","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 151, ","type":"book","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 152, ","type":"book","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 153, ","type":"book","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 154, ","type":"book","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 155, ","type":"book","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 156, ","type":"book","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 157, ","type":"book","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 158, ","type":"book","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 159, ","type":"book","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"}]}```jsonl 160, ","type":"book","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score59.8%

Speed

Model time, all inputs1446.2 s

Mean per input289.24 s

Slowest input811.46 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens15.1K

Output tokens33.7K

Total tokens48.9K

Input cost$0.0059

Output cost$0.0789

Total cost$0.0848

Priced from 2026-04-21 · Input $/M $0.39 · Output $/M $2.34

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 129 of 208

Test T1022 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

anthropic claude-opus-4-7 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score44.3%

{"metadata":{"title":"Bibliography","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridegement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":null}]}

Scoring

Fuzzy score69.9%

Speed

Model time, all inputs125.7 s

Mean per input25.15 s

Slowest input30.04 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens28.6K

Output tokens9.5K

Total tokens38K

Input cost$0.1428

Output cost$0.2368

Total cost$0.3796

Priced from 2026-04-21 · Input $/M $5.00 · Output $/M $25.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 130 of 208

Test T0931 at 2026-04-21

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-35b-a3b-20260224 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score34.5%

{"metadata":{"title":"Bibliography","year":"1947","page_number":9},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"book","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":null,"edition":null,"incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score64.3%

Speed

Model time, all inputs328.6 s

Mean per input65.71 s

Slowest input118.31 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens14.2K

Output tokens34.6K

Total tokens48.8K

Input cost$0.0023

Output cost$0.0449

Total cost$0.0472

Priced from 2026-04-21 · Input $/M $0.1625 · Output $/M $1.30

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Search Test Runs

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV

Test T1050 at 2026-05-22

Test T1037 at 2026-04-27

Test T0983 at 2026-04-22

Test T0996 at 2026-04-21

Test T0905 at 2026-04-21

Test T1009 at 2026-04-21

Test T0983 at 2026-04-21

Test T0944 at 2026-04-21

Test T1022 at 2026-04-21

Test T0931 at 2026-04-21

Search Test Runs

Search Results Show compact results Refine Search New Search Download Download JSON Download CSV

Test T1050 at 2026-05-22

Test T1037 at 2026-04-27

Test T0983 at 2026-04-22

Test T0996 at 2026-04-21

Test T0905 at 2026-04-21

Test T1009 at 2026-04-21

Test T0983 at 2026-04-21

Test T0944 at 2026-04-21

Test T1022 at 2026-04-21

Test T0931 at 2026-04-21

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV