RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV (Downloads are limited to 250 results.)

Your search for Benchmark 'company_lists__true' with Search Hidden 'False' returned 358 results, showing page 3 of 36.
Result 21 of 358

Test T1017 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter google/gemma-4-31b-it · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision7.4%
Recall7.7%
True positives5
False positives63
False negatives60
F1 score8.0%
Total fields100

{"page_id":"not_provided","entries":[{"entry_id":"page-1","company_name":"A. Boreux","location":null},{"entry_id":"page-2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"page-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"page-4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"page-5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"page-6","company_name":"Basler Möbelfabrik A.-G.","location":null},{"entry_id":"page-7","company_name":"Hermann Wagner & Cie.","location":null},{"entry_id":"page-8","company_name":"Adolruf Bürger","location":null},{"entry_id":"page-9","company_name":"Bussinger & Cie.","location":null},{"entry_id":"page-10","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"page-11","company_name":"H. Hartmann","location":null},{"entry_id":"page-12","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"page-13","company_name":"M. Russ","location":null},{"entry_id":"page-14","company_name":"Alf. Schray (Kleinmöbel)","location":null},{"entry_id":"page-15","company_name":"Wolf & Glatz","location":null},{"entry_id":"page-16","company_name":"Gebr. Antonietti","location":null},{"entry_id":"page-17","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"page-18","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"page-19","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"page-20","company_name":"Elektromotoren","location":null},{"entry_id":"page-21","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"page-22","company_name":"Hug & Co.","location":null},{"entry_id":"page-23","company_name":"Schweiz. Geigenbaugesellschaft Listel","location":null},{"entry_id":"page-24","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"page-25","company_name":"Henri Grunauer","location":null},{"entry_id":"page-26","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"page-27","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"page-28","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"page-29","company_name":"E. Müry & Cie.","location":null},{"entry_id":"page-30","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"page-31","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"page-32","company_name":"G. P. Richter","location":null},{"entry_id":"page-33","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro51.4%
F1 macro50.9%
Micro precision51.5%
Micro recall51.3%
Instances15
True positives509
False positives480
False negatives483

Speed

Model time, all inputs611.1 s
Mean per input40.74 s
Slowest input109.86 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens10.2K
Output tokens15.9K
Total tokens26.1K
Input cost$0.0009
Output cost$0.0054
Total cost$0.0063

Priced from 2026-09-16 · Input $/M $0.09 · Output $/M $0.34

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 22 of 358

Test T1743 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.8-27b · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision30.8%
Recall30.8%
True positives20
False positives45
False negatives45
F1 score31.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":null},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"24-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24-4","company_name":"Alf. Dürri, J. H. Neukomm's Nachf. (künstliches)","location":null},{"entry_id":"24-5","company_name":"Ausstattungsmagazine Ed. Senft.","location":null},{"entry_id":"24-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-7","company_name":"Adolf Bürgin.","location":null},{"entry_id":"24-8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-9","company_name":"Fränkel & Voellmy.","location":null},{"entry_id":"24-10","company_name":"H. Hartmann.","location":null},{"entry_id":"24-11","company_name":"Kunsthandlung Rath.","location":null},{"entry_id":"24-12","company_name":"M. Russ.","location":null},{"entry_id":"24-13","company_name":"Alf. Schray (Kleinmöbel).","location":null},{"entry_id":"24-14","company_name":"Wolf & Glatz.","location":null},{"entry_id":"24-15","company_name":"Gebr. Antonietti.","location":null},{"entry_id":"24-16","company_name":"A.-G. Bächtold & Cie. (Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren).","location":null},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G. (Druckluftmotoren für Laboratorien).","location":null},{"entry_id":"24-18","company_name":"G. Meidinger & Cie. (für Webstuhltrieb, Centrifugentrieb, fahrbare Motoren mit Zahnradbetrieb, Automobilmotoren).","location":null},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller (auch Reparaturstücke).","location":null},{"entry_id":"24-20","company_name":"Hug & Co. (Blechblasinstrumente).","location":null},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft (Geigen, Celli, Bässe, Bögen, Gitarren, Zithern).","location":"Liestal"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie. (Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente).","location":null},{"entry_id":"24-23","company_name":"Henri Grunauer.","location":null},{"entry_id":"24-24","company_name":"Hans Ritter-Soder.","location":null},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder.","location":null},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner (Schrauben und kleine Bestandteile für Nähmaschinen).","location":"Niederdorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G. (für Hand-, Fuss- und elektrischen Betrieb, Fabriktsche für Kraftbetrieb).","location":null},{"entry_id":"24-30","company_name":"G. P. Richter (Näh salze «Gerstyl», Nährsatz-Bananencacao, Nährsatz-Hafencacao, Nährsatz-Kaffee).","location":null},{"entry_id":"24-31","company_name":"E. H. Schacke (Näh salz-Cacao, Nährsatz-Kaffee).","location":null}]}

Scoring

F1 micro52.9%
F1 macro53.6%
Micro precision53.0%
Micro recall52.7%
Instances15
True positives523
False positives464
False negatives469

Speed

Model time, all inputs1537.5 s
Mean per input102.50 s
Slowest input335.35 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens47K
Output tokens32.6K
Total tokens79.6K
Input cost$0.0108
Output cost$0.0824
Total cost$0.0932

Priced from 2026-09-18 · Input $/M $0.214 · Output $/M $2.55

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 23 of 358

Test T0395 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter meta-llama/llama-4-maverick · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.8%
Recall1.5%
True positives1
False positives55
False negatives64
F1 score2.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":null},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G. Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"24-3","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24-4","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-5","company_name":"Adolf Bürgin","location":null},{"entry_id":"24-6","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-7","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24-8","company_name":"H. Hartmann","location":null},{"entry_id":"24-9","company_name":"Kunsthandlung Rath. M. Russ","location":null},{"entry_id":"24-10","company_name":"Alf. Schray (Kleinmöbel). Wolf & Glatz","location":null},{"entry_id":"24-11","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24-12","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24-13","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24-14","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24-15","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24-16","company_name":"Hug & Co.","location":"Schweiz"},{"entry_id":"24-17","company_name":"Geigenbaugesellschaft Liestal","location":null},{"entry_id":"24-18","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24-19","company_name":"Henri Grunauer","location":null},{"entry_id":"24-20","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24-21","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24-22","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-23","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-24","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":null},{"entry_id":"24-25","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24-26","company_name":"G. P. Richter","location":null},{"entry_id":"24-27","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro40.0%
F1 macro39.6%
Micro precision40.4%
Micro recall39.5%
Instances15
True positives392
False positives578
False negatives600

Speed

Model time, all inputs346.9 s
Mean per input23.13 s
Slowest input72.01 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens40.5K
Output tokens13.6K
Total tokens54.1K
Input cost$0.0076
Output cost$0.0089
Total cost$0.0165

Priced from 2026-09-16 · Input $/M $0.1875 · Output $/M $0.6525

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 24 of 358

Test T1789 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.8-max-0902 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision49.3%
Recall50.8%
True positives33
False positives34
False negatives32
F1 score50.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux","location":null},{"entry_id":"2","company_name":"Brunnenverwaltung Eptingen","location":"Eptingen"},{"entry_id":"3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"Basel"},{"entry_id":"7","company_name":"Adolf Bürgin","location":null},{"entry_id":"8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"10","company_name":"H. Hartmann","location":null},{"entry_id":"11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"12","company_name":"M. Russ","location":null},{"entry_id":"13","company_name":"Alf. Schray","location":null},{"entry_id":"14","company_name":"Wolf & Glatz","location":null},{"entry_id":"15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"20","company_name":"Hug & Co.","location":null},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"Liestal"},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"23","company_name":"Henri Grunauer","location":null},{"entry_id":"24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"30","company_name":"G. P. Richter","location":null},{"entry_id":"31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro44.0%
F1 macro46.2%
Micro precision43.9%
Micro recall44.1%
Instances15
True positives437
False positives558
False negatives555

Speed

Model time, all inputs759.6 s
Mean per input50.64 s
Slowest input103.89 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens41.4K
Output tokens42.5K
Total tokens83.8K
Input cost$0.0827
Output cost$0.2548
Total cost$0.3375

Priced from 2026-09-16 · Input $/M $2.00 · Output $/M $6.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 25 of 358

Test T1018 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter google/gemma-4-31b-it · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision7.0%
Recall7.7%
True positives5
False positives66
False negatives60
F1 score7.0%
Total fields100

{"page_id":"{page_id}","entries":[{"entry_id":"mikroskopie_und_zubehor","company_name":"A. Boreux","location":null},{"entry_id":"mineralwasser","company_name":"Brunnenverwaltung Eptingen","location":"Eptingen"},{"entry_id":"mineralwasser_1","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"mineralwasser_2","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"mobel_mobelfabrik","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"mobel_mobelfabrik_1","company_name":"Basler Möbelfabrik A.-G.","location":"Basel"},{"entry_id":"mobel_mobelfabrik_2","company_name":"Hermann Wagner & Cie.","location":null},{"entry_id":"mobel_mobelfabrik_3","company_name":"Adolph Bürgin","location":null},{"entry_id":"mobel_mobelfabrik_4","company_name":"Bussinger & Cie.","location":null},{"entry_id":"mobel_mobelfabrik_5","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"mobel_mobelfabrik_6","company_name":"H. Hartmann","location":null},{"entry_id":"mobel_mobelfabrik_7","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"mobel_mobelfabrik_8","company_name":"M. Russ","location":null},{"entry_id":"mobel_mobelfabrik_9","company_name":"Alf. Schray (Kleinmöbel)","location":null},{"entry_id":"mobel_mobelfabrik_10","company_name":"Wolf & Glatz","location":null},{"entry_id":"mosaik_und_terrazzo_boden","company_name":"Gebr. Antonietti","location":null},{"entry_id":"motoren","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"motoren_1","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"motoren_2","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"motoren_3","company_name":"E. Elektromotoren","location":null},{"entry_id":"motorlastwagen","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"musikinstrumente","company_name":"Hug & Co.","location":null},{"entry_id":"musikinstrumente_1","company_name":"Schweiz. Geigenbugesellschaft Listel","location":"Schweiz"},{"entry_id":"musikinstrumente_2","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"musterkarten","company_name":"Henri Grunauer","location":null},{"entry_id":"musterkarten_1","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"musterkarten_2","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"magnesiacement","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"nahfaden","company_name":"E. Müry & Cie.","location":null},{"entry_id":"nahmaschinen","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"nahmaschinen_1","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"nahrsalzpraparate","company_name":"G. P. Richter","location":null},{"entry_id":"nahrsalzpraparate_1","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro32.8%
F1 macro30.5%
Micro precision35.1%
Micro recall30.8%
Instances15
True positives306
False positives566
False negatives686

Speed

Model time, all inputs578.0 s
Mean per input38.53 s
Slowest input65.98 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens4.6K
Output tokens12.2K
Total tokens16.8K
Input cost$0.0004
Output cost$0.0041
Total cost$0.0046

Priced from 2026-09-16 · Input $/M $0.09 · Output $/M $0.34

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 26 of 358

Test T0992 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.5-9b · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision30.9%
Recall44.6%
True positives29
False positives65
False negatives36
F1 score36.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux.","location":"Mikroskopie und Zubehör (Deckgläser, Objektträger, Cartonnage)."},{"entry_id":"2","company_name":"Brunnenverwaltung Eptingen","location":"Mineralwasser."},{"entry_id":"3","company_name":"Singer & Buchenhorner A.-G.","location":"Mineralwasser."},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"Mineralwasser."},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft","location":"Möbel, hölzerne."},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"Möbel, hölzerne."},{"entry_id":"7","company_name":"Adolf Bürgin","location":"Möbel, hölzerne."},{"entry_id":"8","company_name":"Bussinger & Cie.","location":"Möbel, hölzerne."},{"entry_id":"9","company_name":"Fränkel & Voellmy","location":"Möbel, hölzerne."},{"entry_id":"10","company_name":"H. Hartmann","location":"Möbel, hölzerne."},{"entry_id":"11","company_name":"Kunsthandlung Rath","location":"Möbel, hölzerne."},{"entry_id":"12","company_name":"M. Russ","location":"Möbel, hölzerne."},{"entry_id":"13","company_name":"Alf. Schray (Kleinmöbel)","location":"Möbel, hölzerne."},{"entry_id":"14","company_name":"Wolf & Glatz","location":"Möbel, hölzerne."},{"entry_id":"15","company_name":"Gebr. Antonietti","location":"Mosaik- und Terrazzo-Böden."},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie.","location":"Motoren."},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"Motoren."},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":"Motoren."},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":"Motorlastwagen."},{"entry_id":"20","company_name":"Hug & Co.","location":"Musikinstrumente."},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"Musikinstrumente."},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"Musikinstrumente."},{"entry_id":"23","company_name":"Henri Grunauer","location":"Musterkarten für chemische Fabriken, Färbereien etc."},{"entry_id":"24","company_name":"Hans Ritter-Soder","location":"Musterkarten für chemische Fabriken, Färbereien etc."},{"entry_id":"25","company_name":"Alb. Singeisen-Halder","location":"Musterkarten für chemische Fabriken, Färbereien etc."},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":"Magnesiacement."},{"entry_id":"27","company_name":"E. Müry & Cie.","location":"Nähfaden."},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":"Nähmaschinen."},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"Nähmaschinen."},{"entry_id":"30","company_name":"G. P. Richter","location":"Nährsalzpräparate."},{"entry_id":"31","company_name":"E. H. Schacke","location":"Nährsalzpräparate."}]}

Scoring

F1 micro37.1%
F1 macro38.5%
Micro precision36.6%
Micro recall37.7%
Instances15
True positives374
False positives649
False negatives618

Speed

Model time, all inputs1302.6 s
Mean per input86.84 s
Slowest input141.80 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens41.8K
Output tokens51.2K
Total tokens93K
Input cost$0.0042
Output cost$0.0077
Total cost$0.0119

Priced from 2026-09-16 · Input $/M $0.10 · Output $/M $0.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 27 of 358

Test T1729 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.8-flash · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision28.6%
Recall27.7%
True positives18
False positives45
False negatives47
F1 score28.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24_1","company_name":"A. Boreux","location":null},{"entry_id":"24_2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"24_3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24_4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf. (künstliches)","location":null},{"entry_id":"24_5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24_6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24_7","company_name":"Adolf Bürgin","location":null},{"entry_id":"24_8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24_9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24_10","company_name":"H. Hartmann","location":null},{"entry_id":"24_11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24_12","company_name":"M. Russ","location":null},{"entry_id":"24_13","company_name":"Alf. Schray (Kleirmöbel)","location":null},{"entry_id":"24_14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24_15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24_16","company_name":"A.-G. Bächtold & Cie. (Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren)","location":null},{"entry_id":"24_17","company_name":"Maschinenfabrik Burckhardt A.-G. (Druckluftmotoren für Laboratorien)","location":null},{"entry_id":"24_18","company_name":"G. Meidinger & Cie. (für Webstuhlantrieb, Centrifugenantrieb, fahrbare Motoren mit Zahnradbetrieb, Automobilmotoren)","location":null},{"entry_id":"24_19","company_name":"A.-G. für Motorlastwagen Soller (auch Reparaturstücke)","location":null},{"entry_id":"24_20","company_name":"Hug & Co. (Blechblasinstrumente)","location":null},{"entry_id":"24_21","company_name":"Schweiz. Geigenbaugesellschaft Liestal (Geigen,celli, Bässe, Bögen, Gitarren, Zithern)","location":null},{"entry_id":"24_22","company_name":"Schweizerisches Musikhaus Bannwart & Cie. (Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)","location":null},{"entry_id":"24_23","company_name":"Henri Grunauer","location":null},{"entry_id":"24_24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24_25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24_26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24_27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24_28","company_name":"Affentranger, Haas & Plattner, Niederdorf (Schrauben und kleine Bestandteile für Nähmaschinen)","location":null},{"entry_id":"24_29","company_name":"Schweizerische Nähmaschinenfabrik A.-G. (für Hand-, Fuss- und elektrischen Betrieb, Fabrik tische für Kraftbetrieb)","location":null},{"entry_id":"24_30","company_name":"G. P. Richter (Nährsalze «Gerstyl», Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee)","location":null},{"entry_id":"24_31","company_name":"E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee)","location":null}]}

Scoring

F1 micro39.6%
F1 macro39.6%
Micro precision40.3%
Micro recall38.9%
Instances15
True positives386
False positives572
False negatives606

Speed

Model time, all inputs703.2 s
Mean per input46.88 s
Slowest input302.82 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens41.9K
Output tokens39.2K
Total tokens81.1K
Input cost$0.0063
Output cost$0.0184
Total cost$0.0247

Priced from 2026-09-18 · Input $/M $0.15 · Output $/M $0.47

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 28 of 358

Test T1072 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.7-plus · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision48.5%
Recall49.2%
True positives32
False positives34
False negatives33
F1 score49.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24_1","company_name":"A. Boreux","location":null},{"entry_id":"24_2","company_name":"Brunnenverwaltung Eptingen","location":"Eptingen"},{"entry_id":"24_3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24_4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"24_5","company_name":"Ausstattungsmagazine Ed. Senft.","location":null},{"entry_id":"24_6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24_7","company_name":"Adolf Bürgin","location":null},{"entry_id":"24_8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24_9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24_10","company_name":"H. Hartmann","location":null},{"entry_id":"24_11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24_12","company_name":"M. Russ","location":null},{"entry_id":"24_13","company_name":"Alf. Schray","location":null},{"entry_id":"24_14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24_15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24_16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24_17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24_18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24_19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24_20","company_name":"Hug & Co.","location":null},{"entry_id":"24_21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"24_22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24_23","company_name":"Henri Grunauer","location":null},{"entry_id":"24_24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24_25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24_26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24_27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24_28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24_29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24_30","company_name":"G. P. Richter","location":null},{"entry_id":"24_31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro48.0%
F1 macro47.8%
Micro precision48.1%
Micro recall48.0%
Instances15
True positives476
False positives514
False negatives516

Speed

Model time, all inputs1671.8 s
Mean per input111.45 s
Slowest input175.69 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens41.1K
Output tokens46.9K
Total tokens88.1K
Input cost$0.0132
Output cost$0.0601
Total cost$0.0732

Priced from 2026-09-16 · Input $/M $0.32 · Output $/M $1.28

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 29 of 358

Test T0396 at 2026-09-18

book-page information-extraction printed 20 en, de list company

openrouter meta-llama/llama-4-maverick · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision10.0%
Recall1.5%
True positives1
False positives9
False negatives64
F1 score3.0%
Total fields94

{"page_id":"{page_id}","entries":[{"entry_id":"1","company_name":"A. Boreux","location":"left column, first entry, page 24, {page_id} 24"},{"entry_id":"2","company_name":"Hug & Co.","location":"right column, first entry, page 24, {page_id} 24"},{"entry_id":"3","company_name":"G. P. Richter","location":"right column, last entry, page 24, {page_id} 24"}]}

Scoring

F1 micro37.9%
F1 macro35.3%
Micro precision38.5%
Micro recall37.3%
Instances15
True positives370
False positives590
False negatives622

Speed

Model time, all inputs1250.5 s
Mean per input83.37 s
Slowest input153.93 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens35.4K
Output tokens13.9K
Total tokens49.3K
Input cost$0.0066
Output cost$0.0091
Total cost$0.0157

Priced from 2026-09-16 · Input $/M $0.1875 · Output $/M $0.6525

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 30 of 358

Test T1519 at 2026-09-18

book-page information-extraction printed 20 en, de list company

huggingface Qwen/Qwen3-VL-235B-A22B-Instruct · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision0.0%
Recall0.0%
True positives0
False positives1
False negatives65
F1 score0.0%
Total fields94

{"page_id":"24","entries":[]}

Scoring

F1 micro26.9%
F1 macro25.3%
Micro precision32.4%
Micro recall23.0%
Instances15
True positives228
False positives475
False negatives764

Speed

Model time, all inputs1687.9 s
Mean per input112.52 s
Slowest input365.64 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens42.1K
Output tokens11.5K
Total tokens53.6K
Input cost$0.0084
Output cost$0.0101
Total cost$0.0185

Priced from 2026-09-16 · Input $/M $0.20 · Output $/M $0.88

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827