RISE Humanities Data Benchmark

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
Model configuration – provider, model version, temperature, and other generation parameters.
Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
Usage and cost data – token counts and calculated API costs.
Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Result 41 of 208

Test T0918 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-27b · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score32.5%

{"metadata":{"title":"Bibliography","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":null,"author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":null,"author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":null,"author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes II","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion II","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"(reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959)","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":null,"edition":null,"incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis XXXVIII","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand III","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes CCLIV","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau III","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review LII","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant VIII","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review XVI","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes III","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score59.7%

Speed

Model time, all inputs1044.2 s

Mean per input208.85 s

Slowest input778.20 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens17.6K

Output tokens23.4K

Total tokens41K

Input cost$0.0038

Output cost$0.0434

Total cost$0.0472

Priced from 2026-09-16 · Input $/M $0.195 · Output $/M $1.56

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 42 of 208

Test T1141 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

cohere command-a-vision-07-2025 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

{
  "metadata": {
    "title": "BIBLIOGRAPHY",
    "year": "1947",
    "page_number": 9
  },
  "entries": [
    {
      "id": "140",
      "type": "book",
      "title": "Why We Read History",
      "author": [
        {
          "family": "Smellie",
          "given": "K. B."
        }
      ],
      "publisher": "P. Elek",
      "publisher_place": "London"
    },
    {
      "id": "141",
      "type": "book",
      "title": "A Study of History",
      "author": [
        {
          "family": "Toynbee",
          "given": "Arnold J."
        }
      ],
      "editor": [
        {
          "family": "Somerville",
          "given": "D. C."
        }
      ],
      "publisher": "Oxford",
      "publisher_place": "New York",
      "note": "Abridgement of vols. I-VI of A Study of History"
    },
    {
      "id": "142",
      "type": "journal-article",
      "title": "In den spiegel van het verleden: historische opstellen",
      "author": [
        {
          "family": "Verberne",
          "given": "L. G. J."
        }
      ],
      "publisher": "Bruna",
      "publisher_place": "Utrecht",
      "page": "Antwerpen"
    },
    {
      "id": "143",
      "type": "journal-article",
      "title": "Esquemas de metodologia eclesiaste",
      "author": [
        {
          "family": "Vevir",
          "given": "José"
        }
      ],
      "publisher": "Institutito",
      "publisher_place": "Barcelona"
    },
    {
      "id": "144",
      "type": "book",
      "title": "Sinn und Ziel der Geschichte",
      "author": [
        {
          "family": "Walz",
          "given": "Hans H."
        }
      ],
      "publisher": "Universitätsverlag",
      "publisher_place": "Bonn"
    },
    {
      "id": "145",
      "type": "journal-article",
      "title": "Atti del congresso internazionale di filosofia 15-20 November, 1946",
      "publisher": "Castellani",
      "publisher_place": "Milano",
      "note": "Relevant articles"
    },
    {
      "id": "146",
      "type": "journal-article",
      "title": "Storia e materialismo storico",
      "author": [
        {
          "family": "Drago",
          "given": "Pietro C."
        }
      ],
      "publisher": "vol. I, 109-117"
    },
    {
      "id": "147",
      "type": "journal-article",
      "title": "Critica del giudizio storico",
      "author": [
        {
          "family": "Fazio-Allmayer",
          "given": "Vito"
        }
      ],
      "publisher": "vol. I, 281-290"
    },
    {
      "id": "148",
      "type": "journal-article",
      "title": "Forza e spirito nella storia",
      "author": [
        {
          "family": "Nymann",
          "given": "A."
        }
      ],
      "publisher": "vol. I, 191-200"
    },
    {
      "id": "149",
      "type": "journal-article",
      "title": "Sur le concept d'histoire",
      "author": [
        {
          "family": "Benjamin",
          "given": "Walter"
        }
      ],
      "publisher": "Les temps modernes II, 623-634"
    },
    {
      "id": "150",
      "type": "book",
      "title": "Philosophy of History",
      "author": [
        {
          "family": "Boodin",
          "given": "John E."
        }
      ],
      "editor": [
        {
          "family": "Runes",
          "given": "Dagobert D."
        }
      ],
      "publisher": "Philosophical Library',
      "publisher_place": "New York",
      "page": "89-106"
    },
    {
      "id": "151",
      "type": "journal-article",
      "title": "De la perception à l'histoire. La philosophie de M. Merleau-Ponty",
      "author": [
        {
          "family": "Caillois",
          "given": "R. P."
        }
      ],
      "publisher": "Deucalion II, 57-85"
    },
    {
      "id": "152",
      "type": "journal-article",
      "title": "Commemorazione di Adolfo Omodeo",
      "author": [
        {
          "family": "Cantimori",
          "given": "Delio"
        }
      ],
      "publisher": "Annali della Scuola Normale di Pisa, fasc. 3-4; (reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959)'
    },
    {
      "id": "153",
      "type": "journal-article",
      "title": "The Historian and the Philosophy of Science",
      "author": [
        {
          "family": "Cowan",
          "given": "Thomas"
        }
      ],
      "publisher": "Isis XXXVIII, 11-18"
    },
    {
      "id": "154",
      "type": "journal-article",
      "title": "History as a Science",
      "author": [
        {
          "family": "Crawford",
          "given": "R. M."
        }
      ],
      "publisher": "Historical Studies: Australia and New Zealand III, 153-175"
    },
    {
      "id": "155",
      "type": "journal-article",
      "title": "Christianisme et histoire",
      "author": [
        {
          "family": "Daniélou",
          "given": "J."
        }
      ],
      "publisher": "Etudes CCLIV, 166-184"
    },
    {
      "id": "156",
      "type": "journal-article",
      "title": "Wahrheit der Geschichte. Geschichte der Wahrheit",
      "author": [
        {
          "family": "Dell",
          "given": "Stanley"
        }
      ],
      "publisher": "Amerikanische Rundschau III, 44-53"
    },
    {
      "id": "157",
      "type": "journal-article",
      "title": "The Idea of Progress",
      "author": [
        {
          "family": "Fay",
          "given": "Sidney B."
        }
      ],
      "publisher": "American Historical Review LIII, 231-246"
    },
    {
      "id": "158",
      "type": "journal-article",
      "title": "Théologie et histoire",
      "author": [
        {
          "family": "Fessard",
          "given": "Gaston"
        }
      ],
      "publisher": "Dieu vivant VIII, 39-65"
    },
    {
      "id": "159",
      "type": "journal-article",
      "title": "Acton, Creighton, and Lea: a Study in History and Ethics",
      "author": [
        {
          "family": "Fish",
          "given": "Andrew"
        }
      ],
      "publisher": "Pacific Historical Review XVI, 59-69"
    },
    {
      "id": "160",
      "type": "journal-article",
      "title": "Le conscience historique en Italie",
      "author": [
        {
          "family": "Garosci",
          "given": "A."
        }
      ],
      "publisher": "Les temps modernes III, 23-24"
    }
  ]
}

Scoring

Fuzzy score42.7%

Speed

Model time, all inputs2008.8 s

Mean per input401.76 s

Slowest input943.78 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens7.6K

Output tokens9K

Total tokens16.6K

Input cost$0.00

Output cost$0.00

Total cost$0.00

Priced from 2026-09-16 · Input $/M $0.00 · Output $/M $0.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 43 of 208

Test T0970 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-flash-02-23 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score33.7%

{"metadata":{"title":"Bibliography","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History, abridged by D. C. Somervell","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"(Abridgegement of vols. I-VI of A Study of History).","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":["146","147","148","149"],"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score60.0%

Speed

Model time, all inputs510.9 s

Mean per input127.72 s

Slowest input231.36 s

Inputs timed4

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens14K

Output tokens37.1K

Total tokens51.1K

Input cost$0.0009

Output cost$0.0096

Total cost$0.0105

Priced from 2026-09-16 · Input $/M $0.065 · Output $/M $0.26

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 44 of 208

Test T0234 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter meta-llama/llama-4-maverick · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score33.3%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":"","author":[{"family":"Smellie","given":"K. B."}],"note":"","publisher":"P. Elek","editor":[],"publisher_place":"London","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":"","author":[{"family":"Toynbee","given":"Arnold J."}],"note":"abridged by D. C. Somervell","publisher":"Oxford","editor":[],"publisher_place":"New York","issued":"","event_date":"","related":["Abridgegement of vols. I-VI of A Study of History"],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":"","author":[{"family":"Verberne","given":"L. G. J."}],"note":"","publisher":"Bruna","editor":[],"publisher_place":"Utrecht, Antwerpen","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":"","author":[{"family":"Vevir","given":"José"}],"note":"","publisher":"Instituto","editor":[],"publisher_place":"Barcelona","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":"","author":[{"family":"Walz","given":"Hans H."}],"note":"","publisher":"Universitäts-Verlag","editor":[],"publisher_place":"Bonn","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":"","author":[],"note":"Relevant articles:","publisher":"Castellani","editor":[],"publisher_place":"Milano","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Drago","given":"Pietro C."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"145","volume":"I","page":"109-117","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"145","volume":"I","page":"281-290","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Nyman","given":"A."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"145","volume":"I","page":"191-200","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes II","author":[{"family":"Benjamin","given":"Walter"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"623-634","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"150","type":"book","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":"","publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"89-106","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion II","author":[{"family":"Caillois","given":"R. P."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"57-85","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa, fasc. 3-4","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"","fascicle":"3-4","reprint":"D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":"","incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis XXXVIII","author":[{"family":"Cowan","given":"Thomas"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"XXXVIII","page":"11-18","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand III","author":[{"family":"Crawford","given":"R. M."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"153-175","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes CCXLIV","author":[{"family":"Danielou","given":"J."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"166-184","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau III","author":[{"family":"Dell","given":"Stanley"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"44-53","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review LII","author":[{"family":"Fay","given":"Sidney B."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"LII","page":"231-246","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant VIII","author":[{"family":"Fessard","given":"Gaston"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"39-65","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review XVI","author":[{"family":"Fish","given":"Andrew"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"XVI","page":"59-69","fascicle":"","reprint":"","edition":"","incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes III","author":[{"family":"Garosci","given":"A."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"","event_date":"","related":[],"relation":"","volume":"","page":"23-24","fascicle":"","reprint":"","edition":"","incomplete":true}]}

Scoring

Fuzzy score60.4%

Speed

Model time, all inputs257.4 s

Mean per input51.49 s

Slowest input64.79 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens12.1K

Output tokens13.6K

Total tokens25.7K

Input cost$0.0023

Output cost$0.0089

Total cost$0.0112

Priced from 2026-09-16 · Input $/M $0.1875 · Output $/M $0.6525

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 45 of 208

Test T1390 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter moonshotai/kimi-k3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score44.4%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":"abridged","incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score70.0%

Speed

Model time, all inputs562.5 s

Mean per input112.51 s

Slowest input153.67 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens21.8K

Output tokens44.8K

Total tokens66.5K

Input cost$0.0576

Output cost$0.5949

Total cost$0.6525

Priced from 2026-09-16 · Input $/M $2.65 · Output $/M $13.28

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 46 of 208

Test T0931 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-35b-a3b · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score44.0%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgegement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d’histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":null}]}

Scoring

Fuzzy score63.0%

Speed

Model time, all inputs979.8 s

Mean per input195.97 s

Slowest input613.17 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens12K

Output tokens45.1K

Total tokens57.1K

Input cost$0.0022

Output cost$0.0643

Total cost$0.0665

Priced from 2026-09-16 · Input $/M $0.1625 · Output $/M $1.30

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 47 of 208

Test T0957 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3.5-plus-02-15 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score34.5%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek.","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"abridged by D. C. Somervell (Abridgement of vols. I-VI of A Study of History)","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna.","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto.","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag.","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani.","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"“Storia e materialismo storico”","container_title":null,"author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"“Critica del giudizio storico”","container_title":null,"author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"“Forza e spirito nella storia”","container_title":null,"author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"“Sur le concept d’histoire”","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"other","title":"“Philosophy of History”","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"“De la perception à l’histoire. La philosophie de M. Merleau-Ponty”","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"“Commemorazione di Adolfo Omodeo”","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"“The Historian and the Philosophy of Science”","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"“History as a Science”","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"“Christianisme et histoire”","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"“Wahrheit der Geschichte. Geschichte der Wahrheit”","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"“The Idea of Progress”","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"“Théologie et histoire”","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"“Acton, Creighton, and Lea: a Study in History and Ethics”","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"“Le conscience historique en Italie”","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":null}]}

Scoring

Fuzzy score67.9%

Speed

Model time, all inputs767.9 s

Mean per input153.57 s

Slowest input228.68 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens17.5K

Output tokens44K

Total tokens61.5K

Input cost$0.0046

Output cost$0.0686

Total cost$0.0731

Priced from 2026-09-16 · Input $/M $0.26 · Output $/M $1.56

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 48 of 208

Test T1630 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter z-ai/glm-5v-turbo · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score34.3%

{"metadata":{"title":"Bibliography","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":null,"author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":null,"author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":null,"author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":"145","volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":null}]}

Scoring

Fuzzy score66.9%

Speed

Model time, all inputs676.4 s

Mean per input135.27 s

Slowest input158.40 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens22.1K

Output tokens16.7K

Total tokens38.9K

Input cost$0.0093

Output cost$0.0669

Total cost$0.0763

Priced from 2026-08-18 · Input $/M $1.20 · Output $/M $4.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 49 of 208

Test T1128 at 2026-09-17

book-page information-extraction printed 20 en list bibliographic

openrouter stepfun/step-3.7-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score33.0%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":"London: P. Elek.","author":[{"family":"Smellie","given":"K. B."}],"note":"Abridgement of vols. I-VI of A Study of History.","publisher":"P. Elek.","editor":["D. C. Somervell"],"publisher_place":"London","issued":"P. Elek.","event_date":"1947","related":["141"],"relation":"141","volume":"I-VI","page":"9","fascicle":"I-VI","reprint":"A Study of History","edition":"abridged by D. C. Somervell, New York: Oxford (Abridgement of vols. I-VI of A Study of History).","incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":"New York: Oxford","author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I-VI of A Study of History.","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":"Oxford","event_date":"1947","related":["140"],"relation":"140","volume":"I-VI","page":"9","fascicle":"I-VI","reprint":"A Study of History","edition":"abridged by D. C. Somervell","incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":"Utrecht, Antwerpen: Bruna.","author":[{"family":"Verberne","given":"L. G. J."}],"note":"","publisher":"Bruna.","editor":[],"publisher_place":"Utrecht, Antwerpen","issued":"Bruna.","event_date":"1947","related":[],"relation":"","volume":"","page":"9","fascicle":"","reprint":"In den spiegel van het verleden: historische opstellen","edition":"In den spiegel van het verleden: historische opstellen","incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":"Barcelona: Instituto.","author":[{"family":"Vevir","given":"José"}],"note":"","publisher":"Instituto.","editor":[],"publisher_place":"Barcelona","issued":"Instituto.","event_date":"1947","related":[],"relation":"","volume":"","page":"9","fascicle":"","reprint":"Esquemas de metodología eclesiástica","edition":"Esquemas de metodología eclesiástica","incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":"Bonn: Universitäts-Verlag.","author":[{"family":"Walz","given":"Hans H."}],"note":"","publisher":"Universitäts-Verlag.","editor":[],"publisher_place":"Bonn","issued":"Universitäts-Verlag.","event_date":"1947","related":[],"relation":"","volume":"","page":"9","fascicle":"","reprint":"Sinn und Ziel der Geschichte","edition":"Sinn und Ziel der Geschichte","incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":"Milano: Castellani.","author":[],"note":"Relevant articles:","publisher":"Castellani.","editor":[],"publisher_place":"Milano","issued":"Castellani.","event_date":"1946-11-15/1946-11-20","related":["146","147","148","149"],"relation":"","volume":"","page":"9","fascicle":"","reprint":"Atti del congresso internazionale di filosofia 15-20 November, 1946","edition":"Atti del congresso internazionale di filosofia 15-20 November, 1946","incomplete":false},{"id":"146","type":"book","title":"Storia e materialismo storico","container_title":"vol. I, 109-117.","author":[{"family":"Drago","given":"Pietro C."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"109-117.","event_date":"1947","related":["145"],"relation":"145","volume":"I","page":"9","fascicle":"I","reprint":"Storia e materialismo storico","edition":"Storia e materialismo storico","incomplete":false},{"id":"147","type":"book","title":"Critica del giudizio storico","container_title":"vol. I, 281-290.","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"281-290.","event_date":"1947","related":["145"],"relation":"145","volume":"I","page":"9","fascicle":"I","reprint":"Critica del giudizio storico","edition":"Critica del giudizio storico","incomplete":false},{"id":"148","type":"book","title":"Forza e spirito nella storia","container_title":"vol. I, 191-200.","author":[{"family":"Nyman","given":"A."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"191-200.","event_date":"1947","related":["145"],"relation":"145","volume":"I","page":"9","fascicle":"I","reprint":"Forza e spirito nella storia","edition":"Forza e spirito nella storia","incomplete":false},{"id":"149","type":"book","title":"Sur le concept d’histoire","container_title":"Les temps modernes II, 623-634.","author":[{"family":"Benjamin","given":"Walter"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"623-634.","event_date":"1947","related":["145"],"relation":"145","volume":"II","page":"9","fascicle":"II","reprint":"Sur le concept d’histoire","edition":"Sur le concept d’histoire","incomplete":false},{"id":"150","type":"book","title":"Philosophy of History","container_title":"in Dagobert D. Runes, ed., 20th Century Philosophy, New York: Philosophical Library, 89-106.","author":[{"family":"Boodin","given":"John E."}],"note":"","publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"89-106.","event_date":"1947","related":[],"relation":"","volume":"","page":"9","fascicle":"","reprint":"Philosophy of History","edition":"Philosophy of History","incomplete":false},{"id":"151","type":"book","title":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion II, 57-85.","author":[{"family":"Caillois","given":"R. P."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"57-85.","event_date":"1947","related":[],"relation":"","volume":"II","page":"9","fascicle":"II","reprint":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","edition":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","incomplete":false},{"id":"152","type":"book","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa, fasc. 3-4; (reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959).","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":"Einaudi","editor":[],"publisher_place":"Torino","issued":"1959","event_date":"1947","related":[],"relation":"","volume":"","page":"9","fascicle":"3-4","reprint":"Commemorazione di Adolfo Omodeo","edition":"Commemorazione di Adolfo Omodeo","incomplete":false},{"id":"153","type":"book","title":"The Historian and the Philosophy of Science","container_title":"Isis XXXVIII, 11-18.","author":[{"family":"Cowan","given":"Thomas"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"11-18.","event_date":"1947","related":[],"relation":"","volume":"XXXVIII","page":"9","fascicle":"XXXVIII","reprint":"The Historian and the Philosophy of Science","edition":"The Historian and the Philosophy of Science","incomplete":false},{"id":"154","type":"book","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand III, 153-175.","author":[{"family":"Crawford","given":"R. M."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"153-175.","event_date":"1947","related":[],"relation":"","volume":"III","page":"9","fascicle":"III","reprint":"History as a Science","edition":"History as a Science","incomplete":false},{"id":"155","type":"book","title":"Christianisme et histoire","container_title":"Etudes CCLIV, 166-184.","author":[{"family":"Daniélou","given":"J."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"166-184.","event_date":"1947","related":[],"relation":"","volume":"CCLIV","page":"9","fascicle":"CCLIV","reprint":"Christianisme et histoire","edition":"Christianisme et histoire","incomplete":false},{"id":"156","type":"book","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau III, 44-53.","author":[{"family":"Dell","given":"Stanley"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"44-53.","event_date":"1947","related":[],"relation":"","volume":"III","page":"9","fascicle":"III","reprint":"Wahrheit der Geschichte. Geschichte der Wahrheit","edition":"Wahrheit der Geschichte. Geschichte der Wahrheit","incomplete":false},{"id":"157","type":"book","title":"The Idea of Progress","container_title":"American Historical Review LII, 231-246.","author":[{"family":"Fay","given":"Sidney B."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"231-246.","event_date":"1947","related":[],"relation":"","volume":"LII","page":"9","fascicle":"LII","reprint":"The Idea of Progress","edition":"The Idea of Progress","incomplete":false},{"id":"158","type":"book","title":"Théologie et histoire","container_title":"Dieu vivant VIII, 39-65.","author":[{"family":"Fessard","given":"Gaston"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"39-65.","event_date":"1947","related":[],"relation":"","volume":"VIII","page":"9","fascicle":"VIII","reprint":"Théologie et histoire","edition":"Théologie et histoire","incomplete":false},{"id":"159","type":"book","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review XVI, 59-69.","author":[{"family":"Fish","given":"Andrew"}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"59-69.","event_date":"1947","related":[],"relation":"","volume":"XVI","page":"9","fascicle":"XVI","reprint":"Acton, Creighton, and Lea: a Study in History and Ethics","edition":"Acton, Creighton, and Lea: a Study in History and Ethics","incomplete":false},{"id":"160","type":"book","title":"Le conscience historique en Italie","container_title":"Les temps modernes III, 23-24.","author":[{"family":"Garosci","given":"A."}],"note":"","publisher":"","editor":[],"publisher_place":"","issued":"23-24.","event_date":"1947","related":[],"relation":"","volume":"III","page":"9","fascicle":"III","reprint":"Le conscience historique en Italie","edition":"Le conscience historique en Italie","incomplete":false}]}

Scoring

Fuzzy score61.5%

Speed

Model time, all inputs384.7 s

Mean per input76.93 s

Slowest input87.39 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens6.3K

Output tokens64.2K

Total tokens70.6K

Input cost$0.0013

Output cost$0.0739

Total cost$0.0751

Priced from 2026-09-16 · Input $/M $0.20 · Output $/M $1.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Result 50 of 208

Test T1360 at 2026-09-16

book-page information-extraction printed 20 en list bibliographic

openai gpt-5.6-terra · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score35.5%

{"metadata":{"title":"Bibliography","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridgement of vols. I–VI of A Study of History.","publisher":"Oxford","editor":["D. C. Somervell"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":"Abridged","incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht; Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia","container_title":null,"author":null,"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1946","event_date":"1946-11-15/1946-11-20","related":["146","147","148"],"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1946","event_date":null,"related":null,"relation":"145","volume":"I","page":"109–117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1946","event_date":null,"related":null,"relation":"145","volume":"I","page":"281–290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1946","event_date":null,"related":null,"relation":"145","volume":"I","page":"191–200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d’histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623–634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"other","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89–106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57–85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3–4","reprint":"Reprinted in D. Cantimori, Studi di storia (Torino: Einaudi, 1959).","edition":null,"incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11–18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153–175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166–184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte, Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44–53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231–246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39–65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59–69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23–24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score68.2%

Speed

Model time, all inputs276.8 s

Mean per input55.35 s

Slowest input108.15 s

Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens19.3K

Output tokens11.8K

Total tokens31.1K

Input cost$0.0385

Output cost$0.1422

Total cost$0.1807

Priced from 2026-09-16 · Input $/M $2.00 · Output $/M $12.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827

Search Test Runs

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV

Test T0918 at 2026-09-17

Test T1141 at 2026-09-17

Test T0970 at 2026-09-17

Test T0234 at 2026-09-17

Test T1390 at 2026-09-17

Test T0931 at 2026-09-17

Test T0957 at 2026-09-17

Test T1630 at 2026-09-17

Test T1128 at 2026-09-17

Test T1360 at 2026-09-16

Search Test Runs

Search Results Show compact results Refine Search New Search Download Download JSON Download CSV

Test T0918 at 2026-09-17

Test T1141 at 2026-09-17

Test T0970 at 2026-09-17

Test T0234 at 2026-09-17

Test T1390 at 2026-09-17

Test T0931 at 2026-09-17

Test T0957 at 2026-09-17

Test T1630 at 2026-09-17

Test T1128 at 2026-09-17

Test T1360 at 2026-09-16

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV