RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV (Downloads are limited to 250 results.)

Your search for Benchmark 'company_lists__true' with Search Hidden 'False' returned 358 results, showing page 2 of 36.
Result 11 of 358

Test T1848 at 2026-09-27

book-page information-extraction printed 20 en, de list company

openai gpt-6-luna · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision47.0%
Recall47.7%
True positives31
False positives35
False negatives34
F1 score47.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":null},{"entry_id":"24-2","company_name":"Brunnenverwaltung","location":"Eptingen"},{"entry_id":"24-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24-4","company_name":"Alf. Dürr, J. H. Neukomm’s Nachf.","location":null},{"entry_id":"24-5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-7","company_name":"Adolf Bürgin","location":null},{"entry_id":"24-8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24-10","company_name":"H. Hartmann","location":null},{"entry_id":"24-11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24-12","company_name":"M. Russ","location":null},{"entry_id":"24-13","company_name":"Alf. Schray","location":null},{"entry_id":"24-14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24-15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24-16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24-18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24-20","company_name":"Hug & Co.","location":null},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24-23","company_name":"Henri Grunauer","location":null},{"entry_id":"24-24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24-30","company_name":"G. P. Richter","location":null},{"entry_id":"24-31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro51.5%
F1 macro52.0%
Micro precision51.6%
Micro recall51.5%
Instances15
True positives511
False positives480
False negatives481

Speed

Model time, all inputs137.4 s
Mean per input9.16 s
Slowest input13.42 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens57.4K
Output tokens18.3K
Total tokens75.7K
Input cost$0.0057
Output cost$0.0092
Total cost$0.0149

Priced from 2026-09-27 · Input $/M $0.10 · Output $/M $0.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 12 of 358

Test T1864 at 2026-09-27

book-page information-extraction printed 20 en, de list company

x-ai grok-4.7 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision49.2%
Recall49.2%
True positives32
False positives33
False negatives33
F1 score49.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":null},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"24-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24-4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"24-5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-7","company_name":"Adolf Bürgin","location":null},{"entry_id":"24-8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24-10","company_name":"H. Hartmann","location":null},{"entry_id":"24-11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24-12","company_name":"M. Russ","location":null},{"entry_id":"24-13","company_name":"Alf. Schray","location":null},{"entry_id":"24-14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24-15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24-16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24-18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24-20","company_name":"Hug & Co.","location":null},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24-23","company_name":"Henri Grunauer","location":null},{"entry_id":"24-24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24-30","company_name":"G. P. Richter","location":null},{"entry_id":"24-31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro47.4%
F1 macro47.5%
Micro precision47.5%
Micro recall47.3%
Instances15
True positives469
False positives518
False negatives523

Speed

Model time, all inputs1020.8 s
Mean per input68.05 s
Slowest input147.63 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens58.5K
Output tokens11.4K
Total tokens69.9K
Input cost$0.117
Output cost$0.0683
Total cost$0.6315
Reasoning cost usd$0.4462
Total reasoning tokens74.4K

Priced from 2026-09-27 · Input $/M $2.00 · Output $/M $6.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 13 of 358

Test T1834 at 2026-09-27

book-page information-extraction printed 20 en, de list company

openai gpt-6-sol · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision47.0%
Recall47.7%
True positives31
False positives35
False negatives34
F1 score47.0%
Total fields94

{"page_id":"{page_id}","entries":[{"entry_id":"1","company_name":"A. Boreux","location":null},{"entry_id":"2","company_name":"Brunnenverwaltung","location":"Eptingen"},{"entry_id":"3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm’s Nachf.","location":null},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"7","company_name":"Adolf Bürgin","location":null},{"entry_id":"8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"10","company_name":"H. Hartmann","location":null},{"entry_id":"11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"12","company_name":"M. Russ","location":null},{"entry_id":"13","company_name":"Alf. Schray","location":null},{"entry_id":"14","company_name":"Wolf & Glatz","location":null},{"entry_id":"15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"20","company_name":"Hug & Co.","location":null},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"23","company_name":"Henri Grunauer","location":null},{"entry_id":"24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"30","company_name":"G. P. Richter","location":null},{"entry_id":"31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro48.3%
F1 macro46.7%
Micro precision48.3%
Micro recall48.3%
Instances15
True positives479
False positives512
False negatives513

Speed

Model time, all inputs151.6 s
Mean per input10.11 s
Slowest input15.42 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens52.3K
Output tokens13.5K
Total tokens65.8K
Input cost$0.1046
Output cost$0.1348
Total cost$0.2394

Priced from 2026-09-27 · Input $/M $2.00 · Output $/M $10.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 14 of 358

Test T1833 at 2026-09-27

book-page information-extraction printed 20 en, de list company

openai gpt-6-sol · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision49.2%
Recall49.2%
True positives32
False positives33
False negatives33
F1 score49.0%
Total fields94

{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"A. Boreux","location":null},{"entry_id":"{page_id}-2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"{page_id}-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"{page_id}-4","company_name":"Alf. Dürr, J. H. Neukomm’s Nachf.","location":null},{"entry_id":"{page_id}-5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"{page_id}-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"{page_id}-7","company_name":"Adolf Bürgin","location":null},{"entry_id":"{page_id}-8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"{page_id}-9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"{page_id}-10","company_name":"H. Hartmann","location":null},{"entry_id":"{page_id}-11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"{page_id}-12","company_name":"M. Russ","location":null},{"entry_id":"{page_id}-13","company_name":"Alf. Schray","location":null},{"entry_id":"{page_id}-14","company_name":"Wolf & Glatz","location":null},{"entry_id":"{page_id}-15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"{page_id}-16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"{page_id}-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"{page_id}-18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"{page_id}-19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"{page_id}-20","company_name":"Hug & Co.","location":null},{"entry_id":"{page_id}-21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"{page_id}-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"{page_id}-23","company_name":"Henri Grunauer","location":null},{"entry_id":"{page_id}-24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"{page_id}-25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"{page_id}-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"{page_id}-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"{page_id}-28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"{page_id}-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"{page_id}-30","company_name":"G. P. Richter","location":null},{"entry_id":"{page_id}-31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro61.2%
F1 macro61.1%
Micro precision61.2%
Micro recall61.1%
Instances15
True positives606
False positives384
False negatives386

Speed

Model time, all inputs159.2 s
Mean per input10.61 s
Slowest input15.62 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens57.4K
Output tokens14.4K
Total tokens71.8K
Input cost$0.1148
Output cost$0.1436
Total cost$0.2584

Priced from 2026-09-27 · Input $/M $2.00 · Output $/M $10.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 15 of 358

Test T1818 at 2026-09-27

book-page information-extraction printed 20 en, de list company

anthropic claude-opus-5-5 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.6%
Recall1.5%
True positives1
False positives60
False negatives64
F1 score2.0%
Total fields94

```json
[
  {"entry_id": "{page_id}-1", "company_name": "A. Boreux", "location": null},
  {"entry_id": "{page_id}-2", "company_name": "Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G.", "location": null},
  {"entry_id": "{page_id}-3", "company_name": "Alf. Dürr, J. H. Neukomm's Nachf.", "location": null},
  {"entry_id": "{page_id}-4", "company_name": "Ausstattungsmagazine Ed. Senft", "location": null},
  {"entry_id": "{page_id}-5", "company_name": "Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.", "location": null},
  {"entry_id": "{page_id}-6", "company_name": "Adolf Bürgin", "location": null},
  {"entry_id": "{page_id}-7", "company_name": "Bussinger & Cie.", "location": null},
  {"entry_id": "{page_id}-8", "company_name": "Fränkel & Voellmy", "location": null},
  {"entry_id": "{page_id}-9", "company_name": "H. Hartmann", "location": null},
  {"entry_id": "{page_id}-10", "company_name": "Kunsthandlung Rath", "location": null},
  {"entry_id": "{page_id}-11", "company_name": "M. Russ", "location": null},
  {"entry_id": "{page_id}-12", "company_name": "Alf. Schray", "location": null},
  {"entry_id": "{page_id}-13", "company_name": "Wolf & Glatz", "location": null},
  {"entry_id": "{page_id}-14", "company_name": "Gebr. Antonietti", "location": null},
  {"entry_id": "{page_id}-15", "company_name": "A.-G. Bächtold & Cie.", "location": null},
  {"entry_id": "{page_id}-16", "company_name": "Maschinenfabrik Burckhardt A.-G.", "location": null},
  {"entry_id": "{page_id}-17", "company_name": "G. Meidinger & Cie.", "location": null},
  {"entry_id": "{page_id}-18", "company_name": "A.-G. für Motorlastwagen Soller", "location": null},
  {"entry_id": "{page_id}-19", "company_name": "Hug & Co.", "location": null},
  {"entry_id": "{page_id}-20", "company_name": "Schweiz. Geigenbaugesellschaft Liestal", "location": null},
  {"entry_id": "{page_id}-21", "company_name": "Schweizerisches Musikhaus Bannwart & Cie.", "location": null},
  {"entry_id": "{page_id}-22", "company_name": "Henri Grunauer", "location": null},
  {"entry_id": "{page_id}-23", "company_name": "Hans Ritter-Soder", "location": null},
  {"entry_id": "{page_id}-24", "company_name": "Alb. Singeisen-Halder", "location": null},
  {"entry_id": "{page_id}-25", "company_name": "Ch. H. Pfister & Co.", "location": null},
  {"entry_id": "{page_id}-26", "company_name": "E. Müry & Cie.", "location": null},
  {"entry_id": "{page_id}-27", "company_name": "Affentranger, Haas & Plattner", "location": "Niederdorf"},
  {"entry_id": "{page_id}-28", "company_name": "Schweizerische Nähmaschinenfabrik A.-G.", "location": null},
  {"entry_id": "{page_id}-29", "company_name": "G. P. Richter", "location": null},
  {"entry_id": "{page_id}-30", "company_name": "E. H. Schacke", "location": null}
]
```

Scoring

F1 micro48.9%
F1 macro49.6%
Micro precision49.5%
Micro recall48.4%
Instances15
True positives480
False positives490
False negatives512

Speed

Model time, all inputs209.0 s
Mean per input13.94 s
Slowest input19.93 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens62.7K
Output tokens24.1K
Total tokens86.8K
Input cost$0.2508
Output cost$0.4821
Total cost$0.733

Priced from 2026-09-27 · Input $/M $4.00 · Output $/M $20.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 16 of 358

Test T1849 at 2026-09-27

book-page information-extraction printed 20 en, de list company

openai gpt-6-luna · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision48.5%
Recall49.2%
True positives32
False positives34
False negatives33
F1 score49.0%
Total fields94

{"page_id":"{page_id}","entries":[{"entry_id":"24-01","company_name":"A. Boreux","location":null},{"entry_id":"24-02","company_name":"Brunnenverwaltung Eptingen","location":"Eptingen"},{"entry_id":"24-03","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24-04","company_name":"Alf. Dürr, J. H. Neukomm’s Nachf.","location":null},{"entry_id":"24-05","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24-06","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-07","company_name":"Adolf Bürgin","location":null},{"entry_id":"24-08","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-09","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24-10","company_name":"H. Hartmann","location":null},{"entry_id":"24-11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24-12","company_name":"M. Russ","location":null},{"entry_id":"24-13","company_name":"Alf. Schray","location":null},{"entry_id":"24-14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24-15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24-16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24-18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24-20","company_name":"Hug & Co.","location":null},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24-23","company_name":"Henri Grunauer","location":null},{"entry_id":"24-24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24-30","company_name":"G. P. Richter","location":null},{"entry_id":"24-31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro47.5%
F1 macro46.6%
Micro precision47.5%
Micro recall47.5%
Instances15
True positives471
False positives520
False negatives521

Speed

Model time, all inputs131.3 s
Mean per input8.75 s
Slowest input12.44 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens52.3K
Output tokens17K
Total tokens69.3K
Input cost$0.0052
Output cost$0.0085
Total cost$0.0137

Priced from 2026-09-27 · Input $/M $0.10 · Output $/M $0.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 17 of 358

Test T1819 at 2026-09-27

book-page information-extraction printed 20 en, de list company

anthropic claude-opus-5-5 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision0.0%
Recall0.0%
True positives0
False positives70
False negatives65
F1 score0.0%
Total fields180

```json
{
  "page_id": "{page_id}",
  "page_number": 24,
  "language": "German",
  "document_type": "Trade directory (classified index of firms by product category, apparently from Basel, Switzerland)",
  "layout": "Two columns",
  "categories": [
    {
      "heading": "Mikroskopie und Zubehör",
      "qualifier": "Deckgläser, Objektträger, Cartonnage",
      "entries": [
        {"firm": "A. Boreux", "note": null}
      ]
    },
    {
      "heading": "Mineralwasser",
      "entries": [
        {"firm": "Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G.", "note": null},
        {"firm": "Alf. Dürr, J. H. Neukomm's Nachf.", "note": "künstliches"}
      ]
    },
    {
      "heading": "Möbel, hölzerne",
      "entries": [
        {"firm": "Ausstattungsmagazine Ed. Senft", "note": null},
        {"firm": "Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.", "note": null},
        {"firm": "Adolf Bürgin", "note": null},
        {"firm": "Bussinger & Cie.", "note": null},
        {"firm": "Fränkel & Voellmy", "note": null},
        {"firm": "H. Hartmann", "note": null},
        {"firm": "Kunsthandlung Rath", "note": null},
        {"firm": "M. Russ", "note": null},
        {"firm": "Alf. Schray", "note": "Kleinmöbel"},
        {"firm": "Wolf & Glatz", "note": null}
      ]
    },
    {
      "heading": "Mosaik- und Terrazzo-Böden",
      "entries": [
        {"firm": "Gebr. Antonietti", "note": null}
      ]
    },
    {
      "heading": "Motoren",
      "entries": [
        {"firm": "A.-G. Bächtold & Cie.", "note": "Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren"},
        {"firm": "Maschinenfabrik Burckhardt A.-G.", "note": "Druckluftmotoren für Laboratorien"},
        {"firm": "G. Meidinger & Cie.", "note": "für Webstuhlantrieb, Centrifugenantrieb, fahrbare Motoren mit Zahnradbetrieb, Automobilmotoren"}
      ],
      "cross_reference": "s. Elektromotoren"
    },
    {
      "heading": "Motorlastwagen",
      "entries": [
        {"firm": "A.-G. für Motorlastwagen Soller", "note": "auch Reparaturstücke"}
      ]
    },
    {
      "heading": "Musikinstrumente",
      "entries": [
        {"firm": "Hug & Co.", "note": "Blechblasinstrumente"},
        {"firm": "Schweiz. Geigenbaugesellschaft Liestal", "note": "Geigen, Celli, Bässe, Bögen, Gitarren, Zithern"},
        {"firm": "Schweizerisches Musikhaus Bannwart & Cie.", "note": "Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente"}
      ]
    },
    {
      "heading": "Musterkarten",
      "qualifier": "für chemische Fabriken, Färbereien etc.",
      "entries": [
        {"firm": "Henri Grunauer", "note": null},
        {"firm": "Hans Ritter-Soder", "note": null},
        {"firm": "Alb. Singeisen-Halder", "note": null}
      ]
    },
    {
      "heading": "Magnesiacement",
      "entries": [
        {"firm": "Ch. H. Pfister & Co.", "note": null}
      ]
    },
    {
      "heading": "Nähfaden",
      "entries": [
        {"firm": "E. Müry & Cie.", "note": null}
      ]
    },
    {
      "heading": "Nähmaschinen",
      "entries": [
        {"firm": "Affentranger, Haas & Plattner, Niederdorf", "note": "Schrauben und kleine Bestandteile für Nähmaschinen"},
        {"firm": "Schweizerische Nähmaschinenfabrik A.-G.", "note": "für Hand-, Fuss- und elektrischen Betrieb, Fabriktische für Kraftbetrieb"}
      ]
    },
    {
      "heading": "Nähseide",
      "entries": [],
      "cross_reference": "s. Floretseide"
    },
    {
      "heading": "Nährsalzpräparate",
      "entries": [
        {"firm": "G. P. Richter", "note": "Nährsalze «Gerstyl», Nährsalz-Bananencacao, Nährsalz-Hafercacao, Nährsalz-Kaffee"},
        {"firm": "E. H. Schacke", "note": "Nährsalz-Cacao, Nährsalz-Kaffee"}
      ]
    }
  ],
  "historical_notes": {
    "estimated_period": "Early 20th century (circa 1910s–1920s), based on typography, product types (Sauggasmotoren, Motorlastwagen, Nährsalz preparations) and firm names",
    "place": "Basel region, Switzerland (Basler Möbelfabrik, Liestal, Eptingen, Maschinenfabrik Burckhardt)",
    "observations": [
      "Categories are alphabetical, though 'Magnesiacement' is placed out of order after 'Musterkarten'.",
      "The directory reflects a mix of traditional crafts (furniture, musical instruments) and modern industry (combustion engines, trucks, electric sewing machines).",
      "Nährsalz (nutrient salt) products reflect the contemporary Lebensreform/health-food movement."
    ]
  }
}
```

Scoring

F1 micro0.9%
F1 macro1.5%
Micro precision0.8%
Micro recall1.2%
Instances15
True positives12
False positives1,557
False negatives980

Speed

Model time, all inputs319.7 s
Mean per input21.31 s
Slowest input32.96 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens55.6K
Output tokens39.9K
Total tokens95.5K
Input cost$0.2223
Output cost$0.7983
Total cost$1.02

Priced from 2026-09-27 · Input $/M $4.00 · Output $/M $20.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 18 of 358

Test T1863 at 2026-09-27

book-page information-extraction printed 20 en, de list company

x-ai grok-4.7 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision50.0%
Recall49.2%
True positives32
False positives32
False negatives33
F1 score50.0%
Total fields94

{"page_id":"a1Gzi","entries":[{"entry_id":"a1Gzi-1","company_name":"A. Boreux","location":null},{"entry_id":"a1Gzi-2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"a1Gzi-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"a1Gzi-4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"a1Gzi-5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"a1Gzi-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"a1Gzi-7","company_name":"Adolf Bürgin","location":null},{"entry_id":"a1Gzi-8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"a1Gzi-9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"a1Gzi-10","company_name":"H. Hartmann","location":null},{"entry_id":"a1Gzi-11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"a1Gzi-12","company_name":"M. Russ","location":null},{"entry_id":"a1Gzi-13","company_name":"Alf. Schray","location":null},{"entry_id":"a1Gzi-14","company_name":"Wolf & Glatz","location":null},{"entry_id":"a1Gzi-15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"a1Gzi-16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"a1Gzi-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"a1Gzi-18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"a1Gzi-19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"a1Gzi-20","company_name":"Hug & Co.","location":null},{"entry_id":"a1Gzi-21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":null},{"entry_id":"a1Gzi-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"a1Gzi-23","company_name":"Henri Grunauer","location":null},{"entry_id":"a1Gzi-24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"a1Gzi-25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"a1Gzi-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"a1Gzi-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"a1Gzi-28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"a1Gzi-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"a1Gzi-30","company_name":"G. P. Richter","location":null},{"entry_id":"a1Gzi-31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro52.1%
F1 macro51.9%
Micro precision52.2%
Micro recall51.9%
Instances15
True positives515
False positives471
False negatives477

Speed

Model time, all inputs1351.2 s
Mean per input90.08 s
Slowest input146.53 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens63.7K
Output tokens12.1K
Total tokens75.8K
Input cost$0.1274
Output cost$0.0727
Total cost$0.8205
Reasoning cost usd$0.6204
Total reasoning tokens103.4K

Priced from 2026-09-27 · Input $/M $2.00 · Output $/M $6.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 19 of 358

Test T1729 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.8-flash · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision28.6%
Recall27.7%
True positives18
False positives45
False negatives47
F1 score28.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24_1","company_name":"A. Boreux","location":null},{"entry_id":"24_2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"24_3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24_4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf. (künstliches)","location":null},{"entry_id":"24_5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24_6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24_7","company_name":"Adolf Bürgin","location":null},{"entry_id":"24_8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24_9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24_10","company_name":"H. Hartmann","location":null},{"entry_id":"24_11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24_12","company_name":"M. Russ","location":null},{"entry_id":"24_13","company_name":"Alf. Schray (Kleirmöbel)","location":null},{"entry_id":"24_14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24_15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24_16","company_name":"A.-G. Bächtold & Cie. (Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren)","location":null},{"entry_id":"24_17","company_name":"Maschinenfabrik Burckhardt A.-G. (Druckluftmotoren für Laboratorien)","location":null},{"entry_id":"24_18","company_name":"G. Meidinger & Cie. (für Webstuhlantrieb, Centrifugenantrieb, fahrbare Motoren mit Zahnradbetrieb, Automobilmotoren)","location":null},{"entry_id":"24_19","company_name":"A.-G. für Motorlastwagen Soller (auch Reparaturstücke)","location":null},{"entry_id":"24_20","company_name":"Hug & Co. (Blechblasinstrumente)","location":null},{"entry_id":"24_21","company_name":"Schweiz. Geigenbaugesellschaft Liestal (Geigen,celli, Bässe, Bögen, Gitarren, Zithern)","location":null},{"entry_id":"24_22","company_name":"Schweizerisches Musikhaus Bannwart & Cie. (Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)","location":null},{"entry_id":"24_23","company_name":"Henri Grunauer","location":null},{"entry_id":"24_24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24_25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24_26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24_27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24_28","company_name":"Affentranger, Haas & Plattner, Niederdorf (Schrauben und kleine Bestandteile für Nähmaschinen)","location":null},{"entry_id":"24_29","company_name":"Schweizerische Nähmaschinenfabrik A.-G. (für Hand-, Fuss- und elektrischen Betrieb, Fabrik tische für Kraftbetrieb)","location":null},{"entry_id":"24_30","company_name":"G. P. Richter (Nährsalze «Gerstyl», Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee)","location":null},{"entry_id":"24_31","company_name":"E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee)","location":null}]}

Scoring

F1 micro39.6%
F1 macro39.6%
Micro precision40.3%
Micro recall38.9%
Instances15
True positives386
False positives572
False negatives606

Speed

Model time, all inputs703.2 s
Mean per input46.88 s
Slowest input302.82 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens41.9K
Output tokens39.2K
Total tokens81.1K
Input cost$0.0063
Output cost$0.0184
Total cost$0.0247

Priced from 2026-09-18 · Input $/M $0.15 · Output $/M $0.47

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 20 of 358

Test T0396 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter meta-llama/llama-4-maverick · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision10.0%
Recall1.5%
True positives1
False positives9
False negatives64
F1 score3.0%
Total fields94

{"page_id":"{page_id}","entries":[{"entry_id":"1","company_name":"A. Boreux","location":"left column, first entry, page 24, {page_id} 24"},{"entry_id":"2","company_name":"Hug & Co.","location":"right column, first entry, page 24, {page_id} 24"},{"entry_id":"3","company_name":"G. P. Richter","location":"right column, last entry, page 24, {page_id} 24"}]}

Scoring

F1 micro37.9%
F1 macro35.3%
Micro precision38.5%
Micro recall37.3%
Instances15
True positives370
False positives590
False negatives622

Speed

Model time, all inputs1250.5 s
Mean per input83.37 s
Slowest input153.93 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens35.4K
Output tokens13.9K
Total tokens49.3K
Input cost$0.0066
Output cost$0.0091
Total cost$0.0157

Priced from 2026-09-16 · Input $/M $0.1875 · Output $/M $0.6525

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827