RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Loading module...

Search Results

Your search for Benchmark 'bibliographic_data__true' with Search Hidden 'False' returned 208 results, showing page 18 of 21.
Result 171 of 208

Test T0139 at 2026-01-24

book-page information-extraction printed 20 en list bibliographic

openai gpt-4.1-2025-04-14 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score18.6%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":"Milano: Castellani. Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1946-11","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":null,"author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":["145"],"relation":"145","volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":null,"author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":["145"],"relation":"145","volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":null,"author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":["145"],"relation":"145","volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d’histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":null,"author":[{"family":"Boodin","given":"John E."}],"note":"in Dagobert D. Runes, ed., 20th Century Philosophy, New York: Philosophical Library, 89-106.","publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"fasc. 3-4; (reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959).","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":"Studi di storia, Torino: Einaudi, 1959","edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score46.2%

Speed

Model time, all inputs262.6 s
Mean per input52.51 s
Slowest input63.82 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens8.9K
Output tokens7.7K
Total tokens16.6K
Input cost$0.0178
Output cost$0.0616
Total cost$0.0794

Priced from 2026-01-23 · Input $/M $2.00 · Output $/M $8.00

Result 172 of 208

Test T0532 at 2026-01-24

book-page information-extraction printed 20 en list bibliographic

mistral mistral-large-2512 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score3.0%

{"metadata":{"title":"Historical Journal of Film, Radio and Television","year":"1995","page_number":10},"entries":[{"id":"hjfrt-1995-10-01","type":"journal-article","title":"The British Film Industry in the 1930s: A Critical Reassessment","container_title":"Historical Journal of Film, Radio and Television","author":[{"family":"Richards","given":"Jeffrey"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1995","event_date":null,"related":null,"relation":null,"volume":"15","page":"5-22","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"hjfrt-1995-10-02","type":"journal-article","title":"Hollywood and the Home Front: The American Film Industry During World War II","container_title":"Historical Journal of Film, Radio and Television","author":[{"family":"Koppes","given":"Clayton R."},{"family":"Black","given":"Gregory D."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1995","event_date":null,"related":null,"relation":null,"volume":"15","page":"23-44","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"hjfrt-1995-10-03","type":"journal-article","title":"Review of 'The Last Empire: De Beers, Diamonds, and the World' by Stefan Kanfer","container_title":"Historical Journal of Film, Radio and Television","author":[{"family":"Chapman","given":"James"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1995","event_date":null,"related":null,"relation":"kanfer-1993","volume":"15","page":"45-47","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"hjfrt-1995-10-04","type":"journal-article","title":"Television and the Cold War: The Case of BBC External Services","container_title":"Historical Journal of Film, Radio and Television","author":[{"family":"Smith","given":"Anthony"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1995","event_date":null,"related":null,"relation":null,"volume":"15","page":"48-6","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score4.4%

Speed

Model time, all inputs44.3 s
Mean per input8.86 s
Slowest input11.06 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens4.9K
Output tokens2.9K
Total tokens7.9K
Input cost$0.0025
Output cost$0.0044
Total cost$0.0069

Priced from 2026-01-23 · Input $/M $0.50 · Output $/M $1.50

Result 173 of 208

Test T0403 at 2025-11-25

book-page information-extraction printed 20 en list bibliographic

openai gpt-5.1-2025-11-13 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score44.5%

{"metadata":{"title":"Bibliography 1947","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"Abridged by D. C. Somervell (Abridgement of vols. I–VI of A Study of History).","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"143","type":"book","title":"Esquemas de metodología eclesiástica","container_title":null,"author":[{"family":"Vevír","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"145","type":"book","title":"Atti del congresso internazionale di filosofia 15–20 November, 1946","container_title":null,"author":null,"note":"Relevant articles:","publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":"1947","event_date":"1946-11-15/1946-11-20","related":["146","147","148","149"],"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"[Atti del congresso internazionale di filosofia]","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"[Atti del congresso internazionale di filosofia]","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"[Atti del congresso internazionale di filosofia]","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":["145"],"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"149","type":"journal-article","title":"Sur le concept d’histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":"Philosophical Library","editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"151","type":"journal-article","title":"De la perception à l’histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"Reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959.","publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":null,"page":"fasc. 3-4","fascicle":"3-4","reprint":null,"edition":null,"incomplete":false},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":false},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1947","event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":false}]}

Scoring

Fuzzy score52.2%

Speed

Model time, all inputs92.8 s
Mean per input18.55 s
Slowest input25.99 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens7.9K
Output tokens8.3K
Total tokens16.2K
Input cost$0.0099
Output cost$0.0829
Total cost$0.0928

Priced from 2025-11-24 · Input $/M $1.25 · Output $/M $10.00

Result 174 of 208

Test T0195 at 2025-11-24

book-page information-extraction printed 20 en list bibliographic

genai gemini-2.5-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score35.2%

{"metadata":{"title":"BIBLIOGRAPHY","year":"1947","page_number":10},"entries":[{"id":"140","type":"book","title":"Why We Read History","container_title":null,"author":[{"family":"Smellie","given":"K. B."}],"note":null,"publisher":"P. Elek","editor":null,"publisher_place":"London","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"141","type":"book","title":"A Study of History","container_title":null,"author":[{"family":"Toynbee","given":"Arnold J."}],"note":"abridged by D. C. Somervell, New York: Oxford (Abridgement of vols. I-VI of A Study of History).","publisher":"Oxford","editor":null,"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"142","type":"book","title":"In den spiegel van het verleden: historische opstellen","container_title":null,"author":[{"family":"Verberne","given":"L. G. J."}],"note":null,"publisher":"Bruna","editor":null,"publisher_place":"Utrecht, Antwerpen","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"143","type":"book","title":"Esquemas de metodologia eclesiástica","container_title":null,"author":[{"family":"Vevir","given":"José"}],"note":null,"publisher":"Instituto","editor":null,"publisher_place":"Barcelona","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"144","type":"book","title":"Sinn und Ziel der Geschichte","container_title":null,"author":[{"family":"Walz","given":"Hans H."}],"note":null,"publisher":"Universitäts-Verlag","editor":null,"publisher_place":"Bonn","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"145","type":"other","title":"Atti del congresso internazionale di filosofia 15-20 November, 1946","container_title":null,"author":null,"note":null,"publisher":"Castellani","editor":null,"publisher_place":"Milano","issued":null,"event_date":"15-20 November, 1946","related":null,"relation":null,"volume":null,"page":null,"fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"146","type":"journal-article","title":"Storia e materialismo storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Drago","given":"Pietro C."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1946","event_date":null,"related":null,"relation":null,"volume":"I","page":"109-117","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"147","type":"journal-article","title":"Critica del giudizio storico","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Fazio-Allmayer","given":"Vito"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1946","event_date":null,"related":null,"relation":null,"volume":"I","page":"281-290","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"148","type":"journal-article","title":"Forza e spirito nella storia","container_title":"Atti del congresso internazionale di filosofia","author":[{"family":"Nyman","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":"1946","event_date":null,"related":null,"relation":null,"volume":"I","page":"191-200","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"149","type":"journal-article","title":"Sur le concept d'histoire","container_title":"Les temps modernes","author":[{"family":"Benjamin","given":"Walter"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"623-634","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"150","type":"journal-article","title":"Philosophy of History","container_title":"20th Century Philosophy","author":[{"family":"Boodin","given":"John E."}],"note":null,"publisher":null,"editor":["Dagobert D. Runes"],"publisher_place":"New York","issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":"89-106","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"151","type":"journal-article","title":"De la perception à l'histoire. La philosophie de M. Merleau-Ponty","container_title":"Deucalion","author":[{"family":"Caillois","given":"R. P."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"II","page":"57-85","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"152","type":"journal-article","title":"Commemorazione di Adolfo Omodeo","container_title":"Annali della Scuola Normale di Pisa","author":[{"family":"Cantimori","given":"Delio"}],"note":"reprinted in D. Cantimori, Studi di storia, Torino: Einaudi, 1959","publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":null,"page":null,"fascicle":"3-4","reprint":null,"edition":null,"incomplete":null},{"id":"153","type":"journal-article","title":"The Historian and the Philosophy of Science","container_title":"Isis","author":[{"family":"Cowan","given":"Thomas"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XXXVIII","page":"11-18","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"154","type":"journal-article","title":"History as a Science","container_title":"Historical Studies: Australia and New Zealand","author":[{"family":"Crawford","given":"R. M."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"153-175","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"155","type":"journal-article","title":"Christianisme et histoire","container_title":"Etudes","author":[{"family":"Daniélou","given":"J."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"CCLIV","page":"166-184","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"156","type":"journal-article","title":"Wahrheit der Geschichte. Geschichte der Wahrheit","container_title":"Amerikanische Rundschau","author":[{"family":"Dell","given":"Stanley"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"44-53","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"157","type":"journal-article","title":"The Idea of Progress","container_title":"American Historical Review","author":[{"family":"Fay","given":"Sidney B."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"LII","page":"231-246","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"158","type":"journal-article","title":"Théologie et histoire","container_title":"Dieu vivant","author":[{"family":"Fessard","given":"Gaston"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"VIII","page":"39-65","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"159","type":"journal-article","title":"Acton, Creighton, and Lea: a Study in History and Ethics","container_title":"Pacific Historical Review","author":[{"family":"Fish","given":"Andrew"}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"XVI","page":"59-69","fascicle":null,"reprint":null,"edition":null,"incomplete":null},{"id":"160","type":"journal-article","title":"Le conscience historique en Italie","container_title":"Les temps modernes","author":[{"family":"Garosci","given":"A."}],"note":null,"publisher":null,"editor":null,"publisher_place":null,"issued":null,"event_date":null,"related":null,"relation":null,"volume":"III","page":"23-24","fascicle":null,"reprint":null,"edition":null,"incomplete":true}]}

Scoring

Fuzzy score66.9%

Speed

Model time, all inputs134.2 s
Mean per input26.84 s
Slowest input32.57 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens1.7K
Output tokens13.2K
Total tokens15K
Input cost$0.0005
Output cost$0.0331
Total cost$0.0783
Reasoning cost usd$0.0447
Total reasoning tokens17.9K

Priced from 2025-11-24 · Input $/M $0.30 · Output $/M $2.50

Result 175 of 208

Test T0233 at 2025-10-17

book-page information-extraction printed 20 en list bibliographic

openrouter qwen/qwen3-vl-8b-thinking · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Scoring

Fuzzy score65.7%

Speed

Model time, all inputs923.1 s
Mean per input184.62 s
Slowest input225.19 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens12.1K
Output tokens59.3K
Total tokens71.4K
Input cost$0.0022
Output cost$0.1246
Total cost$0.1268

Priced from 2025-10-17 · Input $/M $0.18 · Output $/M $2.10

Result 176 of 208

Test T0234 at 2025-10-17

book-page information-extraction printed 20 en list bibliographic

openrouter meta-llama/llama-4-maverick · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Scoring

Fuzzy score62.6%

Speed

Model time, all inputs151.0 s
Mean per input30.20 s
Slowest input59.36 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens7.7K
Output tokens8.4K
Total tokens16.2K
Input cost$0.0012
Output cost$0.0051
Total cost$0.0062

Priced from 2025-10-17 · Input $/M $0.15 · Output $/M $0.60

Result 177 of 208

Test T0133 at 2025-10-01

book-page information-extraction printed 20 en list bibliographic

openai o3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Scoring

Fuzzy score66.7%

Speed

Model time, all inputs391.0 s
Mean per input78.21 s
Slowest input108.61 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens6.7K
Output tokens21.9K
Total tokens28.6K
Input cost$0.0135
Output cost$0.175
Total cost$0.1885

Priced from 2025-10-01 · Input $/M $2.00 · Output $/M $8.00

Result 178 of 208

Test T0141 at 2025-10-01

book-page information-extraction printed 20 en list bibliographic

openai gpt-4.1-nano · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Scoring

Fuzzy score32.0%

Speed

Model time, all inputs105.8 s
Mean per input21.17 s
Slowest input40.09 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens11.7K
Output tokens8.2K
Total tokens19.9K
Input cost$0.0012
Output cost$0.0033
Total cost$0.0044

Priced from 2025-10-01 · Input $/M $0.10 · Output $/M $0.40

Result 179 of 208

Test T0131 at 2025-10-01

book-page information-extraction printed 20 en list bibliographic

openai gpt-5-nano · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Scoring

Fuzzy score59.0%

Speed

Model time, all inputs401.6 s
Mean per input80.32 s
Slowest input121.89 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens8.4K
Output tokens69.1K
Total tokens77.5K
Input cost$0.0004
Output cost$0.0276
Total cost$0.0281

Priced from 2025-10-01 · Input $/M $0.05 · Output $/M $0.40

Result 180 of 208

Test T0225 at 2025-10-01

book-page information-extraction printed 20 en list bibliographic

anthropic claude-sonnet-4-5-20250929 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Scoring

Fuzzy score0.0%

Speed

Model time, all inputs121.6 s
Mean per input24.33 s
Slowest input29.66 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens12.8K
Output tokens8.3K
Total tokens21.1K
Input cost$0.0384
Output cost$0.1252
Total cost$0.1636

Priced from 2025-10-01 · Input $/M $3.00 · Output $/M $15.00