RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV (Downloads are limited to 250 results.)

Your search for Benchmark 'company_lists__true' with Search Hidden 'False' returned 358 results, showing page 8 of 36.
Result 71 of 358

Test T1399 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter moonshotai/kimi-k3 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision48.5%
Recall49.2%
True positives32
False positives34
False negatives33
F1 score49.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-001","company_name":"A. Boreux","location":null},{"entry_id":"24-002","company_name":"Brunnenverwaltung Eptingen","location":"Eptingen"},{"entry_id":"24-003","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24-004","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"24-005","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24-006","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-007","company_name":"Adolf Bürgin","location":null},{"entry_id":"24-008","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-009","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24-010","company_name":"H. Hartmann","location":null},{"entry_id":"24-011","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24-012","company_name":"M. Russ","location":null},{"entry_id":"24-013","company_name":"Alf. Schray","location":null},{"entry_id":"24-014","company_name":"Wolf & Glatz","location":null},{"entry_id":"24-015","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24-016","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24-017","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24-018","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24-019","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24-020","company_name":"Hug & Co.","location":null},{"entry_id":"24-021","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"24-022","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24-023","company_name":"Henri Grunauer","location":null},{"entry_id":"24-024","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24-025","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24-026","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-027","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-028","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24-029","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24-030","company_name":"G. P. Richter","location":null},{"entry_id":"24-031","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro49.9%
F1 macro51.0%
Micro precision49.9%
Micro recall49.9%
Instances15
True positives495
False positives497
False negatives497

Speed

Model time, all inputs596.3 s
Mean per input39.76 s
Slowest input73.97 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens59.7K
Output tokens48.8K
Total tokens108.6K
Input cost$0.1581
Output cost$0.6487
Total cost$0.8069

Priced from 2026-09-16 · Input $/M $2.65 · Output $/M $13.28

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 72 of 358

Test T1398 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter moonshotai/kimi-k3 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision49.2%
Recall49.2%
True positives32
False positives33
False negatives33
F1 score49.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":null},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":null},{"entry_id":"24-3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"24-4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"24-5","company_name":"Ausstattungsmagazine Ed. Senft","location":null},{"entry_id":"24-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":null},{"entry_id":"24-7","company_name":"Adolf Bürgin","location":null},{"entry_id":"24-8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"24-9","company_name":"Fränkel & Voellmy","location":null},{"entry_id":"24-10","company_name":"H. Hartmann","location":null},{"entry_id":"24-11","company_name":"Kunsthandlung Rath","location":null},{"entry_id":"24-12","company_name":"M. Russ","location":null},{"entry_id":"24-13","company_name":"Alf. Schray","location":null},{"entry_id":"24-14","company_name":"Wolf & Glatz","location":null},{"entry_id":"24-15","company_name":"Gebr. Antonietti","location":null},{"entry_id":"24-16","company_name":"A.-G. Bächtold & Cie.","location":null},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"24-18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"24-20","company_name":"Hug & Co.","location":null},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Liestal"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":null},{"entry_id":"24-23","company_name":"Henri Grunauer","location":null},{"entry_id":"24-24","company_name":"Hans Ritter-Soder","location":null},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder","location":null},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":null},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"24-30","company_name":"G. P. Richter","location":null},{"entry_id":"24-31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro59.8%
F1 macro59.3%
Micro precision59.9%
Micro recall59.7%
Instances15
True positives592
False positives396
False negatives400

Speed

Model time, all inputs725.2 s
Mean per input48.35 s
Slowest input105.82 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens64.8K
Output tokens64.6K
Total tokens129.4K
Input cost$0.1717
Output cost$0.8575
Total cost$1.03

Priced from 2026-09-16 · Input $/M $2.65 · Output $/M $13.28

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 73 of 358

Test T0927 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.5-27b · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision46.0%
Recall44.6%
True positives29
False positives34
False negatives36
F1 score45.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux.","location":""},{"entry_id":"2","company_name":"Brunnenverwaltung Eptingen","location":""},{"entry_id":"3","company_name":"Singer & Buchenhorn A.-G.","location":""},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":""},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft.","location":""},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":""},{"entry_id":"7","company_name":"Adolf Bürgin.","location":""},{"entry_id":"8","company_name":"Bussinger & Cie.","location":""},{"entry_id":"9","company_name":"Fränkel & Voellmy.","location":""},{"entry_id":"10","company_name":"H. Hartmann.","location":""},{"entry_id":"11","company_name":"Kunsthandlung Rath.","location":""},{"entry_id":"12","company_name":"M. Russ.","location":""},{"entry_id":"13","company_name":"Alf. Schray (Kleinmöbel).","location":""},{"entry_id":"14","company_name":"Wolf & Glatz.","location":""},{"entry_id":"15","company_name":"Gebr. Antonietti.","location":""},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie.","location":""},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":""},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":""},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":""},{"entry_id":"20","company_name":"Hug & Co.","location":""},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":""},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":""},{"entry_id":"23","company_name":"Henri Grunauer.","location":""},{"entry_id":"24","company_name":"Hans Ritter-Soder.","location":""},{"entry_id":"25","company_name":"Alb. Singeisen-Halder.","location":""},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":""},{"entry_id":"27","company_name":"E. Müry & Cie.","location":""},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":""},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":""},{"entry_id":"30","company_name":"G. P. Richter","location":""},{"entry_id":"31","company_name":"E. H. Schacke","location":""}]}

Scoring

F1 micro42.3%
F1 macro39.5%
Micro precision44.9%
Micro recall40.0%
Instances15
True positives397
False positives487
False negatives595

Speed

Model time, all inputs505.4 s
Mean per input33.70 s
Slowest input74.33 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.6K
Output tokens24.6K
Total tokens64.2K
Input cost$0.0077
Output cost$0.0384
Total cost$0.0461

Priced from 2026-09-16 · Input $/M $0.195 · Output $/M $1.56

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 74 of 358

Test T0953 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.5-397b-a17b · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision31.9%
Recall46.2%
True positives30
False positives64
False negatives35
F1 score38.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux","location":"Mikroskopie und Zubehör"},{"entry_id":"2","company_name":"Brunnenverwaltung Eptingen","location":"Mineralwasser"},{"entry_id":"3","company_name":"Singer & Buchenhorn A.-G.","location":"Mineralwasser"},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"Mineralwasser"},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft","location":"Möbel, hölzerne"},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"Möbel, hölzerne"},{"entry_id":"7","company_name":"Adolf Bürgin","location":"Möbel, hölzerne"},{"entry_id":"8","company_name":"Bussinger & Cie.","location":"Möbel, hölzerne"},{"entry_id":"9","company_name":"Fränkel & Voellmy","location":"Möbel, hölzerne"},{"entry_id":"10","company_name":"H. Hartmann","location":"Möbel, hölzerne"},{"entry_id":"11","company_name":"Kunsthandlung Rath","location":"Möbel, hölzerne"},{"entry_id":"12","company_name":"M. Russ","location":"Möbel, hölzerne"},{"entry_id":"13","company_name":"Alf. Schray","location":"Möbel, hölzerne"},{"entry_id":"14","company_name":"Wolf & Glatz","location":"Möbel, hölzerne"},{"entry_id":"15","company_name":"Gebr. Antonietti","location":"Mosaik- und Terrazzo-Böden"},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie.","location":"Motoren"},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"Motoren"},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":"Motoren"},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":"Motorlastwagen"},{"entry_id":"20","company_name":"Hug & Co.","location":"Musikinstrumente"},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft","location":"Musikinstrumente"},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"Musikinstrumente"},{"entry_id":"23","company_name":"Henri Grunauer","location":"Musterkarten"},{"entry_id":"24","company_name":"Hans Ritter-Soder","location":"Musterkarten"},{"entry_id":"25","company_name":"Alb. Singeisen-Halder","location":"Musterkarten"},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":"Magnesiacement"},{"entry_id":"27","company_name":"E. Müry & Cie.","location":"Nähfaden"},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner","location":"Nähmaschinen"},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"Nähmaschinen"},{"entry_id":"30","company_name":"G. P. Richter","location":"Nährsalzpräparate"},{"entry_id":"31","company_name":"E. H. Schacke","location":"Nährsalzpräparate"}]}

Scoring

F1 micro40.3%
F1 macro42.5%
Micro precision39.9%
Micro recall40.6%
Instances15
True positives403
False positives607
False negatives589

Speed

Model time, all inputs1008.1 s
Mean per input67.21 s
Slowest input146.31 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens38.1K
Output tokens42.5K
Total tokens80.6K
Input cost$0.0209
Output cost$0.1489
Total cost$0.1698

Priced from 2026-09-16 · Input $/M $0.55 · Output $/M $3.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 75 of 358

Test T1413 at 2026-09-17

book-page information-extraction printed 20 en, de list company

anthropic claude-opus-5 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.6%
Recall1.5%
True positives1
False positives63
False negatives64
F1 score2.0%
Total fields94

{"page_id": "24", "entries": [{"entry_id": "24-1", "company_name": "A. Boreux", "location": null}, {"entry_id": "24-2", "company_name": "Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G.", "location": "Eptingen"}, {"entry_id": "24-3", "company_name": "Alf. D\u00fcrr, J. H. Neukomm's Nachf.", "location": null}, {"entry_id": "24-4", "company_name": "Ausstattungsmagazine Ed. Senft", "location": null}, {"entry_id": "24-5", "company_name": "Basler M\u00f6belfabrik A.-G. vorm. Hermann Wagner & Cie.", "location": null}, {"entry_id": "24-6", "company_name": "Adolf B\u00fcrgin", "location": null}, {"entry_id": "24-7", "company_name": "Bussinger & Cie.", "location": null}, {"entry_id": "24-8", "company_name": "Fr\u00e4nkel & Voellmy", "location": null}, {"entry_id": "24-9", "company_name": "H. Hartmann", "location": null}, {"entry_id": "24-10", "company_name": "Kunsthandlung Rath", "location": null}, {"entry_id": "24-11", "company_name": "M. Russ", "location": null}, {"entry_id": "24-12", "company_name": "Alf. Schray", "location": null}, {"entry_id": "24-13", "company_name": "Wolf & Glatz", "location": null}, {"entry_id": "24-14", "company_name": "Gebr. Antonietti", "location": null}, {"entry_id": "24-15", "company_name": "A.-G. B\u00e4chtold & Cie.", "location": null}, {"entry_id": "24-16", "company_name": "Maschinenfabrik Burckhardt A.-G.", "location": null}, {"entry_id": "24-17", "company_name": "G. Meidinger & Cie.", "location": null}, {"entry_id": "24-18", "company_name": "A.-G. f\u00fcr Motorlastwagen Soller", "location": null}, {"entry_id": "24-19", "company_name": "Hug & Co.", "location": null}, {"entry_id": "24-20", "company_name": "Schweiz. Geigenbaugesellschaft Liestal", "location": "Liestal"}, {"entry_id": "24-21", "company_name": "Schweizerisches Musikhaus Bannwart & Cie.", "location": null}, {"entry_id": "24-22", "company_name": "Henri Grunauer", "location": null}, {"entry_id": "24-23", "company_name": "Hans Ritter-Soder", "location": null}, {"entry_id": "24-24", "company_name": "Alb. Singeisen-Halder", "location": null}, {"entry_id": "24-25", "company_name": "Ch. H. Pfister & Co.", "location": null}, {"entry_id": "24-26", "company_name": "E. M\u00fcry & Cie.", "location": null}, {"entry_id": "24-27", "company_name": "Affentranger, Haas & Plattner", "location": "Niederdorf"}, {"entry_id": "24-28", "company_name": "Schweizerische N\u00e4hmaschinenfabrik A.-G.", "location": null}, {"entry_id": "24-29", "company_name": "G. P. Richter", "location": null}, {"entry_id": "24-30", "company_name": "E. H. Schacke", "location": null}]}

Scoring

F1 micro40.9%
F1 macro41.0%
Micro precision39.8%
Micro recall41.9%
Instances15
True positives416
False positives628
False negatives576

Speed

Model time, all inputs206.5 s
Mean per input13.77 s
Slowest input19.96 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens72.9K
Output tokens18.9K
Total tokens91.8K
Input cost$0.3646
Output cost$0.4714
Total cost$0.836

Priced from 2026-09-16 · Input $/M $5.00 · Output $/M $25.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 76 of 358

Test T0900 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.6-plus · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision49.2%
Recall49.2%
True positives32
False positives33
False negatives33
F1 score49.0%
Total fields94

{
  "page_id": "24",
  "entries": [
    {
      "entry_id": "24-1",
      "company_name": "A. Boreux",
      "location": null
    },
    {
      "entry_id": "24-2",
      "company_name": "Brunnenverwaltung Eptingen",
      "location": null
    },
    {
      "entry_id": "24-3",
      "company_name": "Singer & Buchenhorner A.-G.",
      "location": null
    },
    {
      "entry_id": "24-4",
      "company_name": "Alf. Dürr, J. H. Neukomm's Nachf.",
      "location": null
    },
    {
      "entry_id": "24-5",
      "company_name": "Ausstattungsmagazine Ed. Senft",
      "location": null
    },
    {
      "entry_id": "24-6",
      "company_name": "Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.",
      "location": null
    },
    {
      "entry_id": "24-7",
      "company_name": "Adolf Bürgin",
      "location": null
    },
    {
      "entry_id": "24-8",
      "company_name": "Bussinger & Cie.",
      "location": null
    },
    {
      "entry_id": "24-9",
      "company_name": "Fränkel & Voellmy",
      "location": null
    },
    {
      "entry_id": "24-10",
      "company_name": "H. Hartmann",
      "location": null
    },
    {
      "entry_id": "24-11",
      "company_name": "Kunsthandlung Rath",
      "location": null
    },
    {
      "entry_id": "24-12",
      "company_name": "M. Russ",
      "location": null
    },
    {
      "entry_id": "24-13",
      "company_name": "Alf. Schray",
      "location": null
    },
    {
      "entry_id": "24-14",
      "company_name": "Wolf & Glatz",
      "location": null
    },
    {
      "entry_id": "24-15",
      "company_name": "Gebr. Antonietti",
      "location": null
    },
    {
      "entry_id": "24-16",
      "company_name": "A.-G. Bächtold & Cie.",
      "location": null
    },
    {
      "entry_id": "24-17",
      "company_name": "Maschinenfabrik Burckhardt A.-G.",
      "location": null
    },
    {
      "entry_id": "24-18",
      "company_name": "G. Meidinger & Cie.",
      "location": null
    },
    {
      "entry_id": "24-19",
      "company_name": "A.-G. für Motorlastwagen Soller",
      "location": null
    },
    {
      "entry_id": "24-20",
      "company_name": "Hug & Co.",
      "location": null
    },
    {
      "entry_id": "24-21",
      "company_name": "Schweiz. Geigenbaugesellschaft",
      "location": "Liestal"
    },
    {
      "entry_id": "24-22",
      "company_name": "Schweizerisches Musikhaus Bannwart & Cie.",
      "location": null
    },
    {
      "entry_id": "24-23",
      "company_name": "Henri Grunauer",
      "location": null
    },
    {
      "entry_id": "24-24",
      "company_name": "Hans Ritter-Soder",
      "location": null
    },
    {
      "entry_id": "24-25",
      "company_name": "Alb. Singeisen-Halder",
      "location": null
    },
    {
      "entry_id": "24-26",
      "company_name": "Ch. H. Pfister & Co.",
      "location": null
    },
    {
      "entry_id": "24-27",
      "company_name": "E. Müry & Cie.",
      "location": null
    },
    {
      "entry_id": "24-28",
      "company_name": "Affentranger, Haas & Plattner",
      "location": "Niederdorf"
    },
    {
      "entry_id": "24-29",
      "company_name": "Schweizerische Nähmaschinenfabrik A.-G.",
      "location": null
    },
    {
      "entry_id": "24-30",
      "company_name": "G. P. Richter",
      "location": null
    },
    {
      "entry_id": "24-31",
      "company_name": "E. H. Schacke",
      "location": null
    }
  ]
}

Scoring

F1 micro50.5%
F1 macro51.7%
Micro precision50.7%
Micro recall50.4%
Instances15
True positives500
False positives487
False negatives492

Speed

Model time, all inputs2066.7 s
Mean per input137.78 s
Slowest input280.15 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens46.4K
Output tokens50.5K
Total tokens96.9K
Input cost$0.0151
Output cost$0.0984
Total cost$0.1135

Priced from 2026-09-16 · Input $/M $0.325 · Output $/M $1.95

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 77 of 358

Test T1549 at 2026-09-17

book-page information-extraction printed 20 en, de list company

huggingface thinkingmachines/Inkling-Small · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision8.3%
Recall3.1%
True positives2
False positives22
False negatives63
F1 score4.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"Mikroskopie und Zubehör","company_name":"A. Boreux","location":"Deck- (Deckgläser, Objektträger, Cartonnage)"},{"entry_id":"Mineralwasser","company_name":"Brunnenverwaltung Eptingen","location":"Singer & Buchenhorner A.-G. Alf. Dürr, J. H. Neukomm's Nachf. (künstliches)"},{"entry_id":"Möbel, hölzerne","company_name":"Ausstattungsmagazine Ed. Senft, Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie., Adolf Bürgin, Bussinger & Cie., Fränkel & Voellmy, H. Hartmann, Kunsthandlung Rath, M. Russ, Alf. Schray (Kleinmöbel), Wolf & Glatz","location":"Mosaik- und Terrazzo-Böden: Gebr. Antonietti"},{"entry_id":"Motoren","company_name":"A.-G. Bächtold & Cie. (Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren), Maschinenfabrik Burckhardt A.-G. (Druckluftmotoren für Laboratorien), G. Meidinger & Cie. (für Webstuhlantrieb, Centrifugenantrieb, fahrbare Motoren mit Zahnradbetrieb, Automobilmotoren), s. Elektromotoren","location":"Motorlastwagen: A.-G. für Motorlastwagen Soller (auch Reparaturstücke)"},{"entry_id":"Musikinstrumente","company_name":"Hug & Co. (Blechblasinstrumente), Schweiz. Geigenbaugesellschaft Liestal (Geigen, Celli, Bässe, Bögen, Gitarren, Zithern), Schweizerisches Musikhaus Bannwart & Cie. (Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)","location":"Musterkarten für chemische Fabriken, Färbereien etc.: Henri Grunauer, Hans Ritter-Soder, Alb. Singeisen-Halder"},{"entry_id":"Magnesiacement","company_name":"Ch. H. Pfister & Co.","location":"Nähfaden: E. Müry & Cie."},{"entry_id":"Nähmaschinen","company_name":"Affentranger, Haas & Plattner, Niederdorf (Schrauben und kleine Bestandteile für Nähmaschinen), Schweizerische Nähmaschinenfabrik A.-G. (für Hand-, Fuss- und elektrischen Betrieb, Fabrikfische für Kraftbetrieb)","location":"Nähseide, s. Floretseide"},{"entry_id":"Nährsalzpräparate","company_name":"G. P. Richter (Nährsalze «Gerstyl», Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee), E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee)","location":""}]}

Scoring

F1 micro25.7%
F1 macro24.7%
Micro precision31.9%
Micro recall21.6%
Instances15
True positives214
False positives457
False negatives778

Speed

Model time, all inputs465.9 s
Mean per input31.06 s
Slowest input267.08 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens27.8K
Output tokens21K
Total tokens48.7K
Input cost$0.0125
Output cost$0.0252
Total cost$0.0377

Priced from 2026-09-16 · Input $/M $0.45 · Output $/M $1.20

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 78 of 358

Test T1683 at 2026-09-17

book-page information-extraction printed 20 en, de list company

anthropic claude-fable-5-1 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.6%
Recall1.5%
True positives1
False positives61
False negatives64
F1 score2.0%
Total fields94

```json
[
  {"entry_id": "{page_id}-1", "company_name": "A. Boreux", "location": null},
  {"entry_id": "{page_id}-2", "company_name": "Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G.", "location": null},
  {"entry_id": "{page_id}-3", "company_name": "Alf. Dürr, J. H. Neukomm's Nachf.", "location": null},
  {"entry_id": "{page_id}-4", "company_name": "Ausstattungsmagazine Ed. Senft", "location": null},
  {"entry_id": "{page_id}-5", "company_name": "Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.", "location": null},
  {"entry_id": "{page_id}-6", "company_name": "Adolf Bürgin", "location": null},
  {"entry_id": "{page_id}-7", "company_name": "Bussinger & Cie.", "location": null},
  {"entry_id": "{page_id}-8", "company_name": "Fränkel & Voellmy", "location": null},
  {"entry_id": "{page_id}-9", "company_name": "H. Hartmann", "location": null},
  {"entry_id": "{page_id}-10", "company_name": "Kunsthandlung Rath", "location": null},
  {"entry_id": "{page_id}-11", "company_name": "M. Russ", "location": null},
  {"entry_id": "{page_id}-12", "company_name": "Alf. Schray", "location": null},
  {"entry_id": "{page_id}-13", "company_name": "Wolf & Glatz", "location": null},
  {"entry_id": "{page_id}-14", "company_name": "Gebr. Antonietti", "location": null},
  {"entry_id": "{page_id}-15", "company_name": "A.-G. Bächtold & Cie.", "location": null},
  {"entry_id": "{page_id}-16", "company_name": "Maschinenfabrik Burckhardt A.-G.", "location": null},
  {"entry_id": "{page_id}-17", "company_name": "G. Meidinger & Cie.", "location": null},
  {"entry_id": "{page_id}-18", "company_name": "A.-G. für Motorlastwagen Soller", "location": null},
  {"entry_id": "{page_id}-19", "company_name": "Hug & Co.", "location": null},
  {"entry_id": "{page_id}-20", "company_name": "Schweiz. Geigenbaugesellschaft", "location": "Liestal"},
  {"entry_id": "{page_id}-21", "company_name": "Schweizerisches Musikhaus Bannwart & Cie.", "location": null},
  {"entry_id": "{page_id}-22", "company_name": "Henri Grunauer", "location": null},
  {"entry_id": "{page_id}-23", "company_name": "Hans Ritter-Soder", "location": null},
  {"entry_id": "{page_id}-24", "company_name": "Alb. Singeisen-Halder", "location": null},
  {"entry_id": "{page_id}-25", "company_name": "Ch. H. Pfister & Co.", "location": null},
  {"entry_id": "{page_id}-26", "company_name": "E. Müry & Cie.", "location": null},
  {"entry_id": "{page_id}-27", "company_name": "Affentranger, Haas & Plattner", "location": "Niederdorf"},
  {"entry_id": "{page_id}-28", "company_name": "Schweizerische Nähmaschinenfabrik A.-G.", "location": null},
  {"entry_id": "{page_id}-29", "company_name": "G. P. Richter", "location": null},
  {"entry_id": "{page_id}-30", "company_name": "E. H. Schacke", "location": null}
]
```

Scoring

F1 micro54.9%
F1 macro53.9%
Micro precision55.6%
Micro recall54.3%
Instances15
True positives539
False positives431
False negatives453

Speed

Model time, all inputs283.1 s
Mean per input18.87 s
Slowest input33.48 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens62.7K
Output tokens22.6K
Total tokens85.3K
Input cost$0.6271
Output cost$1.13
Total cost$1.76

Priced from 2026-09-08 · Input $/M $10.00 · Output $/M $50.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 79 of 358

Test T0400 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3-vl-8b-instruct · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision17.5%
Recall26.2%
True positives17
False positives80
False negatives48
F1 score21.0%
Total fields97

{"page_id":"page_24","entries":[{"entry_id":"page_24_mikroskopie_zubehor","company_name":"A. Boreux","location":"page_24"},{"entry_id":"page_24_mineralwasser","company_name":"Brunnenverwaltung Eptingen","location":"page_24"},{"entry_id":"page_24_mineralwasser","company_name":"Singer & Buchenhorner A.-G.","location":"page_24"},{"entry_id":"page_24_mineralwasser","company_name":"Alf. Dürr, J. H. Neukomm’s Nachf.","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Ausstattungsmagazine Ed. Senft","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Adolf Bürgin","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Bussinger & Cie.","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Fränkel & Voellmy","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"H. Hartmann","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Kunsthandlung Rath","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"M. Russ","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Alf. Schray (Kleinmöbel)","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Wolf & Glatz","location":"page_24"},{"entry_id":"page_24_mosaik_und_terrazzo_boden","company_name":"Gebr. Antonietti","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"A.-G. Bächttold & Cie.","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"G. Meidinger & Cie.","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"s. Elektromotoren","location":"page_24"},{"entry_id":"page_24_motorlastwagen","company_name":"A.-G. für Motorlastwagen Soller","location":"page_24"},{"entry_id":"page_24_musikinstrumente","company_name":"Hug & Co.","location":"page_24"},{"entry_id":"page_24_musikinstrumente","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"page_24"},{"entry_id":"page_24_musikinstrumente","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"page_24"},{"entry_id":"page_24_musterkarten_fuer_chemische_fabriken","company_name":"Henri Grunauer","location":"page_24"},{"entry_id":"page_24_musterkarten_fuer_chemische_fabriken","company_name":"Hans Ritter-Soder","location":"page_24"},{"entry_id":"page_24_musterkarten_fuer_chemische_fabriken","company_name":"Alb. Singeisen-Halder","location":"page_24"},{"entry_id":"page_24_magnesiacement","company_name":"Ch. H. Pfister & Co.","location":"page_24"},{"entry_id":"page_24_naehfaden","company_name":"E. Müry & Cie.","location":"page_24"},{"entry_id":"page_24_naehmaschinen","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":"page_24"},{"entry_id":"page_24_naehmaschinen","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"page_24"},{"entry_id":"page_24_naehrsalzpraeparate","company_name":"G. P. Richter","location":"page_24"},{"entry_id":"page_24_naehrsalzpraeparate","company_name":"E. H. Schacke","location":"page_24"}]}

Scoring

F1 micro36.0%
F1 macro37.0%
Micro precision35.2%
Micro recall36.9%
Instances15
True positives366
False positives674
False negatives626

Speed

Model time, all inputs325.2 s
Mean per input21.68 s
Slowest input179.76 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.3K
Output tokens16K
Total tokens55.3K
Input cost$0.0046
Output cost$0.0074
Total cost$0.012

Priced from 2026-09-16 · Input $/M $0.117 · Output $/M $0.455

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 80 of 358

Test T1533 at 2026-09-17

book-page information-extraction printed 20 en, de list company

huggingface MiniMaxAI/MiniMax-M3 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.5%
Recall1.5%
True positives1
False positives66
False negatives64
F1 score2.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Borex.","location":"Basel"},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":"Singer & Buchenhorner A.-G."},{"entry_id":"24-3","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"(künstliches)"},{"entry_id":"24-4","company_name":"Ausstellungsmagazine Ed. Senft.","location":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie."},{"entry_id":"24-5","company_name":"Adolf Bürgin.","location":"Bussinger & Cie."},{"entry_id":"24-6","company_name":"Fränkel & Voellmy.","location":"H. Hartmann."},{"entry_id":"24-7","company_name":"Kunsthandlung Rath.","location":"M. Russ."},{"entry_id":"24-8","company_name":"Alf. Schray","location":"(Kleinmöbel)"},{"entry_id":"24-9","company_name":"Wolf & Glatz.","location":"Gebr. Antonietti."},{"entry_id":"24-10","company_name":"A.-G. Bächtold & Cie.","location":"(Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasnotoren, Schiffsmotoren)."},{"entry_id":"24-11","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"(Druckluftmotoren für Laboratorien)."},{"entry_id":"24-12","company_name":"G. Meidinger & Cie.","location":"(für Webstuhlantrieb, Centrifugenantrieb, fabrbare Motoren mit Zahnradbetrieb, Automobilmotoren)."},{"entry_id":"24-13","company_name":"s. Elektromotoren.","location":"A.-G. für Motorlastwagen Soller (auch Reparaturstücke)."},{"entry_id":"24-14","company_name":"Hug & Co.","location":"(Blechblasinstrumente)"},{"entry_id":"24-15","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"(Geigen, Celli, Bässe, Bögen, Gitarren, Zithern)."},{"entry_id":"24-16","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"(Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)."},{"entry_id":"24-17","company_name":"Henri Grunauer.","location":"Hans Bitter-Soder."},{"entry_id":"24-18","company_name":"Alb. Singeisen-Halder.","location":"Ch. H. Pfister & Co."},{"entry_id":"24-19","company_name":"E. Müry & Cie.","location":"Affoltranger, Haas & Plattner, Niederdorf (Schrauben und kleine Bestandteile für Nähmaschinen)."},{"entry_id":"24-20","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"(für Hand-, Fuss- und elektrischen Betrieb, Fabrikstische für Kraftbetrieb)."},{"entry_id":"24-21","company_name":"s. Floretseide.","location":"G. P. Richter (Nährsalz -Gerstyls, Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee)."},{"entry_id":"24-22","company_name":"E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee).","location":"Basel"}]}

Scoring

F1 micro34.2%
F1 macro34.2%
Micro precision33.8%
Micro recall34.6%
Instances15
True positives343
False positives671
False negatives649

Speed

Model time, all inputs807.4 s
Mean per input53.82 s
Slowest input86.47 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens16.3K
Output tokens14.1K
Total tokens30.4K
Input cost$0.0046
Output cost$0.0155
Total cost$0.0201

Priced from 2026-09-16 · Input $/M $0.28 · Output $/M $1.10

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827