RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV (Downloads are limited to 250 results.)

Your search for Benchmark 'company_lists__true' with Search Hidden 'False' returned 358 results, showing page 6 of 36.
Result 51 of 358

Test T0394 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3-vl-8b-thinking · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision43.5%
Recall46.2%
True positives30
False positives39
False negatives35
F1 score45.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux","location":null},{"entry_id":"2","company_name":"Brunnenvverwaltung Eptingen","location":"Eptingen"},{"entry_id":"3","company_name":"Singer & Buchenhorner A.-G.","location":null},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":null},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft.","location":null},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"Basel"},{"entry_id":"7","company_name":"Adolf Bürgin.","location":null},{"entry_id":"8","company_name":"Bussinger & Cie.","location":null},{"entry_id":"9","company_name":"Fränkel & Voellmy.","location":null},{"entry_id":"10","company_name":"H. Hartmann.","location":null},{"entry_id":"11","company_name":"Kunsthändlung Rath.","location":"Rath"},{"entry_id":"12","company_name":"M. Russ.","location":null},{"entry_id":"13","company_name":"Alf. Schray (Kleinmöbel).","location":null},{"entry_id":"14","company_name":"Wolf & Glatz.","location":null},{"entry_id":"15","company_name":"Gebr. Antonietti.","location":null},{"entry_id":"16","company_name":"A.-G. Bächttold & Cie.","location":null},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":null},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":null},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":null},{"entry_id":"20","company_name":"Hug & Co.","location":null},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"Liestal"},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"Bannwart"},{"entry_id":"23","company_name":"Henri Grunauer.","location":null},{"entry_id":"24","company_name":"Hans Ritter-Soder.","location":null},{"entry_id":"25","company_name":"Alb. Singeisen-Halder.","location":null},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":null},{"entry_id":"27","company_name":"E. Mür y & Cie.","location":null},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":"Niederdorf"},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":null},{"entry_id":"30","company_name":"G. P. Richter","location":null},{"entry_id":"31","company_name":"E. H. Schacke","location":null}]}

Scoring

F1 micro33.2%
F1 macro35.2%
Micro precision33.2%
Micro recall33.2%
Instances13
True positives261
False positives525
False negatives524

Speed

Model time, all inputs836.1 s
Mean per input55.74 s
Slowest input127.11 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens41.4K
Output tokens96.6K
Total tokens137.9K
Input cost$0.0074
Output cost$0.2028
Total cost$0.2102

Priced from 2026-09-16 · Input $/M $0.18 · Output $/M $2.10

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 52 of 358

Test T1199 at 2026-09-17

book-page information-extraction printed 20 en, de list company

anthropic claude-fable-5 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.1%
Recall1.5%
True positives1
False positives90
False negatives64
F1 score1.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-01","company_name":"A. Boreux","location":"Basel"},{"entry_id":"24-02","company_name":"Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G.","location":"Eptingen"},{"entry_id":"24-03","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"Basel"},{"entry_id":"24-04","company_name":"Ausstattungsmagazine Ed. Senft","location":"Basel"},{"entry_id":"24-05","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"Basel"},{"entry_id":"24-06","company_name":"Adolf Bürgin","location":"Basel"},{"entry_id":"24-07","company_name":"Bussinger & Cie.","location":"Basel"},{"entry_id":"24-08","company_name":"Fränkel & Voellmy","location":"Basel"},{"entry_id":"24-09","company_name":"H. Hartmann","location":"Basel"},{"entry_id":"24-10","company_name":"Kunsthandlung Rath","location":"Basel"},{"entry_id":"24-11","company_name":"M. Russ","location":"Basel"},{"entry_id":"24-12","company_name":"Alf. Schray (Kleinmöbel)","location":"Basel"},{"entry_id":"24-13","company_name":"Wolf & Glatz","location":"Basel"},{"entry_id":"24-14","company_name":"Gebr. Antonietti","location":"Basel"},{"entry_id":"24-15","company_name":"A.-G. Bächtold & Cie.","location":"Basel"},{"entry_id":"24-16","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"Basel"},{"entry_id":"24-17","company_name":"G. Meidinger & Cie.","location":"Basel"},{"entry_id":"24-18","company_name":"A.-G. für Motorlastwagen Soller","location":"Basel"},{"entry_id":"24-19","company_name":"Hug & Co.","location":"Basel"},{"entry_id":"24-20","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"Liestal"},{"entry_id":"24-21","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"Basel"},{"entry_id":"24-22","company_name":"Henri Grunauer","location":"Basel"},{"entry_id":"24-23","company_name":"Hans Ritter-Soder","location":"Basel"},{"entry_id":"24-24","company_name":"Alb. Singeisen-Halder","location":"Basel"},{"entry_id":"24-25","company_name":"Ch. H. Pfister & Co.","location":"Basel"},{"entry_id":"24-26","company_name":"E. Müry & Cie.","location":"Basel"},{"entry_id":"24-27","company_name":"Affentranger, Haas & Plattner","location":"Niederdorf"},{"entry_id":"24-28","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"Basel"},{"entry_id":"24-29","company_name":"G. P. Richter","location":"Basel"},{"entry_id":"24-30","company_name":"E. H. Schacke","location":"Basel"}]}

Scoring

F1 micro45.1%
F1 macro46.3%
Micro precision43.4%
Micro recall46.9%
Instances15
True positives465
False positives606
False negatives527

Speed

Model time, all inputs210.1 s
Mean per input14.01 s
Slowest input19.68 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens65.8K
Output tokens18.4K
Total tokens84.2K
Input cost$0.6584
Output cost$0.9175
Total cost$1.58

Priced from 2026-09-16 · Input $/M $10.00 · Output $/M $50.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 53 of 358

Test T1198 at 2026-09-17

book-page information-extraction printed 20 en, de list company

anthropic claude-fable-5 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.6%
Recall1.5%
True positives1
False positives62
False negatives64
F1 score2.0%
Total fields94

{"entries": [{"entry_id": "24-1", "company_name": "A. Boreux", "location": null}, {"entry_id": "24-2", "company_name": "Brunnenverwaltung Eptingen Singer & Buchenhorner A.-G.", "location": null}, {"entry_id": "24-3", "company_name": "Alf. D\u00fcrr, J. H. Neukomm's Nachf.", "location": null}, {"entry_id": "24-4", "company_name": "Ausstattungsmagazine Ed. Senft", "location": null}, {"entry_id": "24-5", "company_name": "Basler M\u00f6belfabrik A.-G. vorm. Hermann Wagner & Cie.", "location": null}, {"entry_id": "24-6", "company_name": "Adolf B\u00fcrgin", "location": null}, {"entry_id": "24-7", "company_name": "Bussinger & Cie.", "location": null}, {"entry_id": "24-8", "company_name": "Fr\u00e4nkel & Voellmy", "location": null}, {"entry_id": "24-9", "company_name": "H. Hartmann", "location": null}, {"entry_id": "24-10", "company_name": "Kunsthandlung Rath", "location": null}, {"entry_id": "24-11", "company_name": "M. Russ", "location": null}, {"entry_id": "24-12", "company_name": "Alf. Schray", "location": null}, {"entry_id": "24-13", "company_name": "Wolf & Glatz", "location": null}, {"entry_id": "24-14", "company_name": "Gebr. Antonietti", "location": null}, {"entry_id": "24-15", "company_name": "A.-G. B\u00e4chtold & Cie.", "location": null}, {"entry_id": "24-16", "company_name": "Maschinenfabrik Burckhardt A.-G.", "location": null}, {"entry_id": "24-17", "company_name": "G. Meidinger & Cie.", "location": null}, {"entry_id": "24-18", "company_name": "A.-G. f\u00fcr Motorlastwagen Soller", "location": null}, {"entry_id": "24-19", "company_name": "Hug & Co.", "location": null}, {"entry_id": "24-20", "company_name": "Schweiz. Geigenbaugesellschaft", "location": "Liestal"}, {"entry_id": "24-21", "company_name": "Schweizerisches Musikhaus Bannwart & Cie.", "location": null}, {"entry_id": "24-22", "company_name": "Henri Grunauer", "location": null}, {"entry_id": "24-23", "company_name": "Hans Ritter-Soder", "location": null}, {"entry_id": "24-24", "company_name": "Alb. Singeisen-Halder", "location": null}, {"entry_id": "24-25", "company_name": "Ch. H. Pfister & Co.", "location": null}, {"entry_id": "24-26", "company_name": "E. M\u00fcry & Cie.", "location": null}, {"entry_id": "24-27", "company_name": "Affentranger, Haas & Plattner", "location": "Niederdorf"}, {"entry_id": "24-28", "company_name": "Schweizerische N\u00e4hmaschinenfabrik A.-G.", "location": null}, {"entry_id": "24-29", "company_name": "G. P. Richter", "location": null}, {"entry_id": "24-30", "company_name": "E. H. Schacke", "location": null}], "page_id": "24"}

Scoring

F1 micro52.1%
F1 macro53.1%
Micro precision52.3%
Micro recall51.9%
Instances15
True positives515
False positives470
False negatives477

Speed

Model time, all inputs198.8 s
Mean per input13.26 s
Slowest input20.44 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens73K
Output tokens16.7K
Total tokens89.7K
Input cost$0.7298
Output cost$0.834
Total cost$1.56

Priced from 2026-09-16 · Input $/M $10.00 · Output $/M $50.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 54 of 358

Test T0952 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.5-397b-a17b · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision47.0%
Recall47.7%
True positives31
False positives35
False negatives34
F1 score47.0%
Total fields94

{
  "page_id": "24",
  "entries": [
    {
      "entry_id": "24-1",
      "company_name": "A. Boreux",
      "location": null
    },
    {
      "entry_id": "24-2",
      "company_name": "Brunnenverwaltung",
      "location": "Eptingen"
    },
    {
      "entry_id": "24-3",
      "company_name": "Singer & Buchenhorner A.-G.",
      "location": null
    },
    {
      "entry_id": "24-4",
      "company_name": "Alf. Dürr, J. H. Neukomm's Nachf.",
      "location": null
    },
    {
      "entry_id": "24-5",
      "company_name": "Ausstattungsmagazine Ed. Senft.",
      "location": null
    },
    {
      "entry_id": "24-6",
      "company_name": "Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.",
      "location": null
    },
    {
      "entry_id": "24-7",
      "company_name": "Adolf Bürgin",
      "location": null
    },
    {
      "entry_id": "24-8",
      "company_name": "Bussinger & Cie.",
      "location": null
    },
    {
      "entry_id": "24-9",
      "company_name": "Fränkel & Voellmy",
      "location": null
    },
    {
      "entry_id": "24-10",
      "company_name": "H. Hartmann",
      "location": null
    },
    {
      "entry_id": "24-11",
      "company_name": "Kunsthandlung Rath",
      "location": null
    },
    {
      "entry_id": "24-12",
      "company_name": "M. Russ",
      "location": null
    },
    {
      "entry_id": "24-13",
      "company_name": "Alf. Schray",
      "location": null
    },
    {
      "entry_id": "24-14",
      "company_name": "Wolf & Glatz",
      "location": null
    },
    {
      "entry_id": "24-15",
      "company_name": "Gebr. Antonietti",
      "location": null
    },
    {
      "entry_id": "24-16",
      "company_name": "A.-G. Bächtold & Cie.",
      "location": null
    },
    {
      "entry_id": "24-17",
      "company_name": "Maschinenfabrik Burckhardt A.-G.",
      "location": null
    },
    {
      "entry_id": "24-18",
      "company_name": "G. Meidinger & Cie.",
      "location": null
    },
    {
      "entry_id": "24-19",
      "company_name": "A.-G. für Motorlastwagen Soller",
      "location": null
    },
    {
      "entry_id": "24-20",
      "company_name": "Hug & Co.",
      "location": null
    },
    {
      "entry_id": "24-21",
      "company_name": "Schweiz. Geigenbaugesellschaft",
      "location": "Liestal"
    },
    {
      "entry_id": "24-22",
      "company_name": "Schweizerisches Musikhaus Bannwart & Cie.",
      "location": null
    },
    {
      "entry_id": "24-23",
      "company_name": "Henri Grunauer",
      "location": null
    },
    {
      "entry_id": "24-24",
      "company_name": "Hans Ritter-Soder",
      "location": null
    },
    {
      "entry_id": "24-25",
      "company_name": "Alb. Singeisen-Halder",
      "location": null
    },
    {
      "entry_id": "24-26",
      "company_name": "Ch. H. Pfister & Co.",
      "location": null
    },
    {
      "entry_id": "24-27",
      "company_name": "E. Müry & Cie.",
      "location": null
    },
    {
      "entry_id": "24-28",
      "company_name": "Affentranger, Haas & Plattner",
      "location": "Niederdorf"
    },
    {
      "entry_id": "24-29",
      "company_name": "Schweizerische Nähmaschinenfabrik A.-G.",
      "location": null
    },
    {
      "entry_id": "24-30",
      "company_name": "G. P. Richter",
      "location": null
    },
    {
      "entry_id": "24-31",
      "company_name": "E. H. Schacke",
      "location": null
    }
  ]
}

Scoring

F1 micro50.0%
F1 macro51.2%
Micro precision50.0%
Micro recall50.0%
Instances15
True positives496
False positives496
False negatives496

Speed

Model time, all inputs1759.5 s
Mean per input117.30 s
Slowest input360.80 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens43.7K
Output tokens69.5K
Total tokens113.2K
Input cost$0.0234
Output cost$0.2333
Total cost$0.2567

Priced from 2026-09-16 · Input $/M $0.55 · Output $/M $3.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 55 of 358

Test T0400 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3-vl-8b-instruct · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision17.5%
Recall26.2%
True positives17
False positives80
False negatives48
F1 score21.0%
Total fields97

{"page_id":"page_24","entries":[{"entry_id":"page_24_mikroskopie_zubehor","company_name":"A. Boreux","location":"page_24"},{"entry_id":"page_24_mineralwasser","company_name":"Brunnenverwaltung Eptingen","location":"page_24"},{"entry_id":"page_24_mineralwasser","company_name":"Singer & Buchenhorner A.-G.","location":"page_24"},{"entry_id":"page_24_mineralwasser","company_name":"Alf. Dürr, J. H. Neukomm’s Nachf.","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Ausstattungsmagazine Ed. Senft","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Adolf Bürgin","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Bussinger & Cie.","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Fränkel & Voellmy","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"H. Hartmann","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Kunsthandlung Rath","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"M. Russ","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Alf. Schray (Kleinmöbel)","location":"page_24"},{"entry_id":"page_24_mobel_holzerne","company_name":"Wolf & Glatz","location":"page_24"},{"entry_id":"page_24_mosaik_und_terrazzo_boden","company_name":"Gebr. Antonietti","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"A.-G. Bächttold & Cie.","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"G. Meidinger & Cie.","location":"page_24"},{"entry_id":"page_24_motoren","company_name":"s. Elektromotoren","location":"page_24"},{"entry_id":"page_24_motorlastwagen","company_name":"A.-G. für Motorlastwagen Soller","location":"page_24"},{"entry_id":"page_24_musikinstrumente","company_name":"Hug & Co.","location":"page_24"},{"entry_id":"page_24_musikinstrumente","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"page_24"},{"entry_id":"page_24_musikinstrumente","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"page_24"},{"entry_id":"page_24_musterkarten_fuer_chemische_fabriken","company_name":"Henri Grunauer","location":"page_24"},{"entry_id":"page_24_musterkarten_fuer_chemische_fabriken","company_name":"Hans Ritter-Soder","location":"page_24"},{"entry_id":"page_24_musterkarten_fuer_chemische_fabriken","company_name":"Alb. Singeisen-Halder","location":"page_24"},{"entry_id":"page_24_magnesiacement","company_name":"Ch. H. Pfister & Co.","location":"page_24"},{"entry_id":"page_24_naehfaden","company_name":"E. Müry & Cie.","location":"page_24"},{"entry_id":"page_24_naehmaschinen","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":"page_24"},{"entry_id":"page_24_naehmaschinen","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"page_24"},{"entry_id":"page_24_naehrsalzpraeparate","company_name":"G. P. Richter","location":"page_24"},{"entry_id":"page_24_naehrsalzpraeparate","company_name":"E. H. Schacke","location":"page_24"}]}

Scoring

F1 micro36.0%
F1 macro37.0%
Micro precision35.2%
Micro recall36.9%
Instances15
True positives366
False positives674
False negatives626

Speed

Model time, all inputs325.2 s
Mean per input21.68 s
Slowest input179.76 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.3K
Output tokens16K
Total tokens55.3K
Input cost$0.0046
Output cost$0.0074
Total cost$0.012

Priced from 2026-09-16 · Input $/M $0.117 · Output $/M $0.455

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 56 of 358

Test T1533 at 2026-09-17

book-page information-extraction printed 20 en, de list company

huggingface MiniMaxAI/MiniMax-M3 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision1.5%
Recall1.5%
True positives1
False positives66
False negatives64
F1 score2.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Borex.","location":"Basel"},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":"Singer & Buchenhorner A.-G."},{"entry_id":"24-3","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"(künstliches)"},{"entry_id":"24-4","company_name":"Ausstellungsmagazine Ed. Senft.","location":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie."},{"entry_id":"24-5","company_name":"Adolf Bürgin.","location":"Bussinger & Cie."},{"entry_id":"24-6","company_name":"Fränkel & Voellmy.","location":"H. Hartmann."},{"entry_id":"24-7","company_name":"Kunsthandlung Rath.","location":"M. Russ."},{"entry_id":"24-8","company_name":"Alf. Schray","location":"(Kleinmöbel)"},{"entry_id":"24-9","company_name":"Wolf & Glatz.","location":"Gebr. Antonietti."},{"entry_id":"24-10","company_name":"A.-G. Bächtold & Cie.","location":"(Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasnotoren, Schiffsmotoren)."},{"entry_id":"24-11","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"(Druckluftmotoren für Laboratorien)."},{"entry_id":"24-12","company_name":"G. Meidinger & Cie.","location":"(für Webstuhlantrieb, Centrifugenantrieb, fabrbare Motoren mit Zahnradbetrieb, Automobilmotoren)."},{"entry_id":"24-13","company_name":"s. Elektromotoren.","location":"A.-G. für Motorlastwagen Soller (auch Reparaturstücke)."},{"entry_id":"24-14","company_name":"Hug & Co.","location":"(Blechblasinstrumente)"},{"entry_id":"24-15","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"(Geigen, Celli, Bässe, Bögen, Gitarren, Zithern)."},{"entry_id":"24-16","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"(Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)."},{"entry_id":"24-17","company_name":"Henri Grunauer.","location":"Hans Bitter-Soder."},{"entry_id":"24-18","company_name":"Alb. Singeisen-Halder.","location":"Ch. H. Pfister & Co."},{"entry_id":"24-19","company_name":"E. Müry & Cie.","location":"Affoltranger, Haas & Plattner, Niederdorf (Schrauben und kleine Bestandteile für Nähmaschinen)."},{"entry_id":"24-20","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"(für Hand-, Fuss- und elektrischen Betrieb, Fabrikstische für Kraftbetrieb)."},{"entry_id":"24-21","company_name":"s. Floretseide.","location":"G. P. Richter (Nährsalz -Gerstyls, Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee)."},{"entry_id":"24-22","company_name":"E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee).","location":"Basel"}]}

Scoring

F1 micro34.2%
F1 macro34.2%
Micro precision33.8%
Micro recall34.6%
Instances15
True positives343
False positives671
False negatives649

Speed

Model time, all inputs807.4 s
Mean per input53.82 s
Slowest input86.47 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens16.3K
Output tokens14.1K
Total tokens30.4K
Input cost$0.0046
Output cost$0.0155
Total cost$0.0201

Priced from 2026-09-16 · Input $/M $0.28 · Output $/M $1.10

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 57 of 358

Test T0927 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.5-27b · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision46.0%
Recall44.6%
True positives29
False positives34
False negatives36
F1 score45.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux.","location":""},{"entry_id":"2","company_name":"Brunnenverwaltung Eptingen","location":""},{"entry_id":"3","company_name":"Singer & Buchenhorn A.-G.","location":""},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":""},{"entry_id":"5","company_name":"Ausstattungsmagazine Ed. Senft.","location":""},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":""},{"entry_id":"7","company_name":"Adolf Bürgin.","location":""},{"entry_id":"8","company_name":"Bussinger & Cie.","location":""},{"entry_id":"9","company_name":"Fränkel & Voellmy.","location":""},{"entry_id":"10","company_name":"H. Hartmann.","location":""},{"entry_id":"11","company_name":"Kunsthandlung Rath.","location":""},{"entry_id":"12","company_name":"M. Russ.","location":""},{"entry_id":"13","company_name":"Alf. Schray (Kleinmöbel).","location":""},{"entry_id":"14","company_name":"Wolf & Glatz.","location":""},{"entry_id":"15","company_name":"Gebr. Antonietti.","location":""},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie.","location":""},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":""},{"entry_id":"18","company_name":"G. Meidinger & Cie.","location":""},{"entry_id":"19","company_name":"A.-G. für Motorlastwagen Soller","location":""},{"entry_id":"20","company_name":"Hug & Co.","location":""},{"entry_id":"21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":""},{"entry_id":"22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":""},{"entry_id":"23","company_name":"Henri Grunauer.","location":""},{"entry_id":"24","company_name":"Hans Ritter-Soder.","location":""},{"entry_id":"25","company_name":"Alb. Singeisen-Halder.","location":""},{"entry_id":"26","company_name":"Ch. H. Pfister & Co.","location":""},{"entry_id":"27","company_name":"E. Müry & Cie.","location":""},{"entry_id":"28","company_name":"Affentranger, Haas & Plattner, Niederdorf","location":""},{"entry_id":"29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":""},{"entry_id":"30","company_name":"G. P. Richter","location":""},{"entry_id":"31","company_name":"E. H. Schacke","location":""}]}

Scoring

F1 micro42.3%
F1 macro39.5%
Micro precision44.9%
Micro recall40.0%
Instances15
True positives397
False positives487
False negatives595

Speed

Model time, all inputs505.4 s
Mean per input33.70 s
Slowest input74.33 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.6K
Output tokens24.6K
Total tokens64.2K
Input cost$0.0077
Output cost$0.0384
Total cost$0.0461

Priced from 2026-09-16 · Input $/M $0.195 · Output $/M $1.56

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 58 of 358

Test T0900 at 2026-09-17

book-page information-extraction printed 20 en, de list company

openrouter qwen/qwen3.6-plus · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision49.2%
Recall49.2%
True positives32
False positives33
False negatives33
F1 score49.0%
Total fields94

{
  "page_id": "24",
  "entries": [
    {
      "entry_id": "24-1",
      "company_name": "A. Boreux",
      "location": null
    },
    {
      "entry_id": "24-2",
      "company_name": "Brunnenverwaltung Eptingen",
      "location": null
    },
    {
      "entry_id": "24-3",
      "company_name": "Singer & Buchenhorner A.-G.",
      "location": null
    },
    {
      "entry_id": "24-4",
      "company_name": "Alf. Dürr, J. H. Neukomm's Nachf.",
      "location": null
    },
    {
      "entry_id": "24-5",
      "company_name": "Ausstattungsmagazine Ed. Senft",
      "location": null
    },
    {
      "entry_id": "24-6",
      "company_name": "Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.",
      "location": null
    },
    {
      "entry_id": "24-7",
      "company_name": "Adolf Bürgin",
      "location": null
    },
    {
      "entry_id": "24-8",
      "company_name": "Bussinger & Cie.",
      "location": null
    },
    {
      "entry_id": "24-9",
      "company_name": "Fränkel & Voellmy",
      "location": null
    },
    {
      "entry_id": "24-10",
      "company_name": "H. Hartmann",
      "location": null
    },
    {
      "entry_id": "24-11",
      "company_name": "Kunsthandlung Rath",
      "location": null
    },
    {
      "entry_id": "24-12",
      "company_name": "M. Russ",
      "location": null
    },
    {
      "entry_id": "24-13",
      "company_name": "Alf. Schray",
      "location": null
    },
    {
      "entry_id": "24-14",
      "company_name": "Wolf & Glatz",
      "location": null
    },
    {
      "entry_id": "24-15",
      "company_name": "Gebr. Antonietti",
      "location": null
    },
    {
      "entry_id": "24-16",
      "company_name": "A.-G. Bächtold & Cie.",
      "location": null
    },
    {
      "entry_id": "24-17",
      "company_name": "Maschinenfabrik Burckhardt A.-G.",
      "location": null
    },
    {
      "entry_id": "24-18",
      "company_name": "G. Meidinger & Cie.",
      "location": null
    },
    {
      "entry_id": "24-19",
      "company_name": "A.-G. für Motorlastwagen Soller",
      "location": null
    },
    {
      "entry_id": "24-20",
      "company_name": "Hug & Co.",
      "location": null
    },
    {
      "entry_id": "24-21",
      "company_name": "Schweiz. Geigenbaugesellschaft",
      "location": "Liestal"
    },
    {
      "entry_id": "24-22",
      "company_name": "Schweizerisches Musikhaus Bannwart & Cie.",
      "location": null
    },
    {
      "entry_id": "24-23",
      "company_name": "Henri Grunauer",
      "location": null
    },
    {
      "entry_id": "24-24",
      "company_name": "Hans Ritter-Soder",
      "location": null
    },
    {
      "entry_id": "24-25",
      "company_name": "Alb. Singeisen-Halder",
      "location": null
    },
    {
      "entry_id": "24-26",
      "company_name": "Ch. H. Pfister & Co.",
      "location": null
    },
    {
      "entry_id": "24-27",
      "company_name": "E. Müry & Cie.",
      "location": null
    },
    {
      "entry_id": "24-28",
      "company_name": "Affentranger, Haas & Plattner",
      "location": "Niederdorf"
    },
    {
      "entry_id": "24-29",
      "company_name": "Schweizerische Nähmaschinenfabrik A.-G.",
      "location": null
    },
    {
      "entry_id": "24-30",
      "company_name": "G. P. Richter",
      "location": null
    },
    {
      "entry_id": "24-31",
      "company_name": "E. H. Schacke",
      "location": null
    }
  ]
}

Scoring

F1 micro50.5%
F1 macro51.7%
Micro precision50.7%
Micro recall50.4%
Instances15
True positives500
False positives487
False negatives492

Speed

Model time, all inputs2066.7 s
Mean per input137.78 s
Slowest input280.15 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens46.4K
Output tokens50.5K
Total tokens96.9K
Input cost$0.0151
Output cost$0.0984
Total cost$0.1135

Priced from 2026-09-16 · Input $/M $0.325 · Output $/M $1.95

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 59 of 358

Test T1549 at 2026-09-17

book-page information-extraction printed 20 en, de list company

huggingface thinkingmachines/Inkling-Small · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision8.3%
Recall3.1%
True positives2
False positives22
False negatives63
F1 score4.0%
Total fields94

{"page_id":"24","entries":[{"entry_id":"Mikroskopie und Zubehör","company_name":"A. Boreux","location":"Deck- (Deckgläser, Objektträger, Cartonnage)"},{"entry_id":"Mineralwasser","company_name":"Brunnenverwaltung Eptingen","location":"Singer & Buchenhorner A.-G. Alf. Dürr, J. H. Neukomm's Nachf. (künstliches)"},{"entry_id":"Möbel, hölzerne","company_name":"Ausstattungsmagazine Ed. Senft, Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie., Adolf Bürgin, Bussinger & Cie., Fränkel & Voellmy, H. Hartmann, Kunsthandlung Rath, M. Russ, Alf. Schray (Kleinmöbel), Wolf & Glatz","location":"Mosaik- und Terrazzo-Böden: Gebr. Antonietti"},{"entry_id":"Motoren","company_name":"A.-G. Bächtold & Cie. (Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren), Maschinenfabrik Burckhardt A.-G. (Druckluftmotoren für Laboratorien), G. Meidinger & Cie. (für Webstuhlantrieb, Centrifugenantrieb, fahrbare Motoren mit Zahnradbetrieb, Automobilmotoren), s. Elektromotoren","location":"Motorlastwagen: A.-G. für Motorlastwagen Soller (auch Reparaturstücke)"},{"entry_id":"Musikinstrumente","company_name":"Hug & Co. (Blechblasinstrumente), Schweiz. Geigenbaugesellschaft Liestal (Geigen, Celli, Bässe, Bögen, Gitarren, Zithern), Schweizerisches Musikhaus Bannwart & Cie. (Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)","location":"Musterkarten für chemische Fabriken, Färbereien etc.: Henri Grunauer, Hans Ritter-Soder, Alb. Singeisen-Halder"},{"entry_id":"Magnesiacement","company_name":"Ch. H. Pfister & Co.","location":"Nähfaden: E. Müry & Cie."},{"entry_id":"Nähmaschinen","company_name":"Affentranger, Haas & Plattner, Niederdorf (Schrauben und kleine Bestandteile für Nähmaschinen), Schweizerische Nähmaschinenfabrik A.-G. (für Hand-, Fuss- und elektrischen Betrieb, Fabrikfische für Kraftbetrieb)","location":"Nähseide, s. Floretseide"},{"entry_id":"Nährsalzpräparate","company_name":"G. P. Richter (Nährsalze «Gerstyl», Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee), E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee)","location":""}]}

Scoring

F1 micro25.7%
F1 macro24.7%
Micro precision31.9%
Micro recall21.6%
Instances15
True positives214
False positives457
False negatives778

Speed

Model time, all inputs465.9 s
Mean per input31.06 s
Slowest input267.08 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens27.8K
Output tokens21K
Total tokens48.7K
Input cost$0.0125
Output cost$0.0252
Total cost$0.0377

Priced from 2026-09-16 · Input $/M $0.45 · Output $/M $1.20

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 60 of 358

Test T1534 at 2026-09-17

book-page information-extraction printed 20 en, de list company

huggingface MiniMaxAI/MiniMax-M3 · temp 0.5 · dataclass ListPage

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Precision12.0%
Recall18.5%
True positives12
False positives88
False negatives53
F1 score15.0%
Total fields100

{"page_id":"24","entries":[{"entry_id":"1","company_name":"A. Boreux","location":"Mikroskopie und Zubehör (Deckgläser, Objektträger, Cartonnage)"},{"entry_id":"2","company_name":"Brunnenverwaltung Eptingen","location":"Mineralwasser"},{"entry_id":"3","company_name":"Singer & Buchenhorner A.-G.","location":"Mineralwasser"},{"entry_id":"4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf. (künstliches)","location":"Mineralwasser"},{"entry_id":"5","company_name":"Ausstellungsmagazine Ed. Senft","location":"Möbel, hölzerne"},{"entry_id":"6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"Möbel, hölzerne"},{"entry_id":"7","company_name":"Adolf Bürgin","location":"Möbel, hölzerne"},{"entry_id":"8","company_name":"Bussinger & Cie.","location":"Möbel, hölzerne"},{"entry_id":"9","company_name":"Fränkel & Voellmy","location":"Möbel, hölzerne"},{"entry_id":"10","company_name":"H. Hartmann","location":"Möbel, hölzerne"},{"entry_id":"11","company_name":"Kunsthandlung Rath","location":"Möbel, hölzerne"},{"entry_id":"12","company_name":"M. Russ","location":"Möbel, hölzerne"},{"entry_id":"13","company_name":"Alf. Schray (Kleinmöbel)","location":"Möbel, hölzerne"},{"entry_id":"14","company_name":"Wolf & Glatz","location":"Möbel, hölzerne"},{"entry_id":"15","company_name":"Gebr. Antonietti","location":"Mosaik- und Terrazzo-Böden"},{"entry_id":"16","company_name":"A.-G. Bächtold & Cie. (Benzinmotoren, Dieselmotoren, Gasmotoren, Rohölmotoren, Sauggasmotoren, Schiffsmotoren)","location":"Motoren"},{"entry_id":"17","company_name":"Maschinenfabrik Burckhardt A.-G. (Druckluftmotoren für Laboratorien)","location":"Motoren"},{"entry_id":"18","company_name":"G. Meidinger & Cie. (für Wechselstrombetrieb, Centrifugenantrieb, fabrbare Motoren mit Zahnradbetrieb, Automobilmotoren)","location":"Motoren"},{"entry_id":"19","company_name":"s. Elektromotoren","location":"Motoren"},{"entry_id":"20","company_name":"A.-G. für Motorlastwagen Soller (auch Reparaturstücke)","location":"Motorlastwagen"},{"entry_id":"21","company_name":"Hug & Co. (Blechblasinstrumente)","location":"Musikinstrumente"},{"entry_id":"22","company_name":"Schweiz. Geigenbaugesellschaft Liestal (Geigen, Celli, Bässe, Bögen, Gitarren, Zithern)","location":"Musikinstrumente"},{"entry_id":"23","company_name":"Schweizerisches Musikhaus Bannwart & Cie. (Mundharmonikas, Ziehharmonikas, Zungenmusikinstrumente, Blechblasinstrumente)","location":"Musikinstrumente"},{"entry_id":"24","company_name":"Henri Grunauer","location":"Musterkarten für chemische Fabriken, Färbereien etc."},{"entry_id":"25","company_name":"Hans Bitter-Soder","location":"Musterkarten für chemische Fabriken, Färbereien etc."},{"entry_id":"26","company_name":"Alb. Singeisen-Halder","location":"Musterkarten für chemische Fabriken, Färbereien etc."},{"entry_id":"27","company_name":"Ch. H. Pfister & Co.","location":"Magnesiazement"},{"entry_id":"28","company_name":"E. Müry & Cie.","location":"Nähfaden"},{"entry_id":"29","company_name":"Affentranger, Haas & Plattner, Nieder dorf (Schrauben und kleine Bestandteile für Nähmaschinen)","location":"Nähmaschinen"},{"entry_id":"30","company_name":"Schweizerische Nähmaschinenfabrik A.-G. (für Hand-, Fuss- und elektrischen Betrieb, Fabrikstische für Kraftbetrieb)","location":"Nähmaschinen"},{"entry_id":"31","company_name":"s. Floretseide","location":"Nähseide"},{"entry_id":"32","company_name":"G. P. Richter (Nährsalz «Gerstyl», Nährsalz - Bananencacao, Nährsalz - Hafercacao, Nährsalz - Kaffee)","location":"Nährsalzpräparate"},{"entry_id":"33","company_name":"E. H. Schacke (Nährsalz-Cacao, Nährsalz-Kaffee)","location":"Nährsalzpräparate"}]}

Scoring

F1 micro35.8%
F1 macro35.2%
Micro precision35.3%
Micro recall36.4%
Instances15
True positives361
False positives661
False negatives631

Speed

Model time, all inputs1055.6 s
Mean per input70.37 s
Slowest input208.92 s
Inputs timed15

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens11.2K
Output tokens15.5K
Total tokens26.6K
Input cost$0.0031
Output cost$0.017
Total cost$0.0201

Priced from 2026-09-16 · Input $/M $0.28 · Output $/M $1.10

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827