RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV (Downloads are limited to 250 results.)

Your search for Benchmark 'business_letters__true' with Search Hidden 'False' returned 613 results, showing page 5 of 62.
Result 41 of 613

Test T1723 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

openrouter qwen/qwen3.8-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":["null"],"sender_persons":["Basler Rheinschifffahrt-Aktiengesellschaft"],"receiver_persons":["Herrn Christ"]}}

Scoring

F1 micro51.0%
F1 macro56.0%

Speed

Model time, all inputs2105.8 s
Mean per input65.81 s
Slowest input296.76 s
Inputs timed32

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens179.3K
Output tokens112.1K
Total tokens291.4K
Input cost$0.0267
Output cost$0.0527
Total cost$0.0793

Priced from 2026-09-18 · Input $/M $0.15 · Output $/M $0.47

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 42 of 613

Test T1572 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

huggingface meta-models/Muse-Glimmer-30B · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":null,"sender_persons":null,"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro87.0%
F1 macro76.0%

Speed

Model time, all inputs949.8 s
Mean per input37.99 s
Slowest input85.51 s
Inputs timed25

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens143.2K
Output tokens31.6K
Total tokens174.7K
Input cost$0.0501
Output cost$0.0474
Total cost$0.0975

Priced from 2026-09-16 · Input $/M $0.35 · Output $/M $1.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 43 of 613

Test T1573 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

huggingface meta-models/Muse-Glimmer-30B · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":null,"sender_persons":null,"receiver_persons":["Christ"]}}

Scoring

F1 micro65.0%
F1 macro65.0%

Speed

Model time, all inputs2722.0 s
Mean per input85.06 s
Slowest input910.01 s
Inputs timed32

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens253.3K
Output tokens43.9K
Total tokens297.2K
Input cost$0.0887
Output cost$0.0659
Total cost$0.1545

Priced from 2026-09-16 · Input $/M $0.35 · Output $/M $1.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 44 of 613

Test T1722 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

openrouter qwen/qwen3.8-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":["null"],"sender_persons":["null"],"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro63.0%
F1 macro54.0%

Speed

Model time, all inputs983.5 s
Mean per input40.98 s
Slowest input146.77 s
Inputs timed24

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens98.6K
Output tokens53.6K
Total tokens152.2K
Input cost$0.0148
Output cost$0.0252
Total cost$0.0399

Priced from 2026-09-18 · Input $/M $0.15 · Output $/M $0.47

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 45 of 613

Test T1571 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

huggingface meta-models/Muse-Glimmer-30B · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":null,"sender_persons":null,"receiver_persons":["Christ"]}}

Scoring

F1 micro71.0%
F1 macro65.0%

Speed

Model time, all inputs5040.0 s
Mean per input88.42 s
Slowest input1736.56 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens396.9K
Output tokens75.7K
Total tokens472.6K
Input cost$0.1389
Output cost$0.1136
Total cost$0.2525

Priced from 2026-09-16 · Input $/M $0.35 · Output $/M $1.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 46 of 613

Test T1782 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

openrouter qwen/qwen3.8-max-0902 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":["Bestätigung des Schreibenhalts und Weiterführung der Besprechungen wegen einer Betriebsgemeinschaft"],"sender_persons":null,"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro63.0%
F1 macro53.0%

Speed

Model time, all inputs1516.5 s
Mean per input63.19 s
Slowest input176.58 s
Inputs timed24

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens97.7K
Output tokens74.9K
Total tokens172.7K
Input cost$0.1955
Output cost$0.4495
Total cost$0.645

Priced from 2026-09-16 · Input $/M $2.00 · Output $/M $6.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 47 of 613

Test T0984 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

openrouter qwen/qwen3.5-9b · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp1
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":["null"],"sender_persons":["Basler Rheinschifffahrt-Aktiengesellschaft"],"receiver_persons":["Herr Christ","Paravicini, Christ & Co."]}}

Scoring

F1 micro52.0%
F1 macro53.0%

Speed

Model time, all inputs7162.2 s
Mean per input125.65 s
Slowest input1522.23 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens280.6K
Output tokens219.1K
Total tokens499.7K
Input cost$0.0284
Output cost$0.0333
Total cost$0.0617

Priced from 2026-09-16 · Input $/M $0.10 · Output $/M $0.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 48 of 613

Test T1736 at 2026-09-18

letter information-extraction typed, handwritten 20 de prose person, location

openrouter qwen/qwen3.8-27b · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":["null"],"sender_persons":["null"],"receiver_persons":["Christ"]}}

Scoring

F1 micro57.0%
F1 macro56.0%

Speed

Model time, all inputs6335.7 s
Mean per input111.15 s
Slowest input606.27 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens299.7K
Output tokens321.7K
Total tokens621.4K
Input cost$0.1259
Output cost$0.9652
Total cost$1.09

Priced from 2026-09-18 · Input $/M $0.214 · Output $/M $2.55

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 49 of 613

Test T1010 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter google/gemma-4-31b-it · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":["null"],"sender_persons":["Basler Rheinschifffahrt-Aktiengesellschaft"],"receiver_persons":["Herr Christ"]}}

Scoring

F1 micro54.0%
F1 macro54.0%

Speed

Model time, all inputs303.1 s
Mean per input5.32 s
Slowest input11.55 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens46.3K
Output tokens4.5K
Total tokens50.8K
Input cost$0.0042
Output cost$0.0015
Total cost$0.0057

Priced from 2026-09-16 · Input $/M $0.09 · Output $/M $0.34

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 50 of 613

Test T0248 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter meta-llama/llama-4-maverick · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":["/m"],"sender_persons":["Basler Rheinschiffahrt-Aktiengesellschaft"],"receiver_persons":["Christ"]}}

Scoring

F1 micro53.0%
F1 macro52.0%

Speed

Model time, all inputs196.0 s
Mean per input3.44 s
Slowest input6.10 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens246K
Output tokens3.7K
Total tokens249.6K
Input cost$0.0461
Output cost$0.0024
Total cost$0.0485

Priced from 2026-09-16 · Input $/M $0.1875 · Output $/M $0.6525

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827