RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV (Downloads are limited to 250 results.)

Your search for Benchmark 'business_letters__true' with Search Hidden 'False' returned 613 results, showing page 7 of 62.
Result 61 of 613

Test T0523 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

anthropic claude-haiku-4-5-20251001 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":null,"sender_persons":["<UNKNOWN>"],"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro49.0%
F1 macro43.0%

Speed

Model time, all inputs50.6 s
Mean per input2.02 s
Slowest input3.13 s
Inputs timed25

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens85.2K
Output tokens2.6K
Total tokens87.8K
Input cost$0.0852
Output cost$0.013
Total cost$0.0982

Priced from 2026-09-16 · Input $/M $1.00 · Output $/M $5.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 62 of 613

Test T1693 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter meta/muse-spark-1.3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":null,"sender_persons":null,"receiver_persons":["Christ"]}}

Scoring

F1 micro61.0%
F1 macro63.0%

Speed

Model time, all inputs637.3 s
Mean per input19.92 s
Slowest input43.06 s
Inputs timed32

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens258.5K
Output tokens53.8K
Total tokens312.3K
Input cost$0.3231
Output cost$0.2288
Total cost$0.5519

Priced from 2026-09-08 · Input $/M $1.25 · Output $/M $4.25

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 63 of 613

Test T1528 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

huggingface MiniMaxAI/MiniMax-M3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":[""],"sender_persons":["Basler Rheinschiffahrt-Aktiengesellschaft"],"receiver_persons":["Herr Christ"]}}

Scoring

F1 micro43.0%
F1 macro47.0%

Speed

Model time, all inputs212.3 s
Mean per input6.64 s
Slowest input14.02 s
Inputs timed32

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens50.6K
Output tokens2.3K
Total tokens52.9K
Input cost$0.0142
Output cost$0.0025
Total cost$0.0167

Priced from 2026-09-16 · Input $/M $0.28 · Output $/M $1.10

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 64 of 613

Test T0998 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter google/gemma-4-26b-a4b-it · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":null,"sender_persons":null,"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro56.0%
F1 macro49.0%

Speed

Model time, all inputs77.6 s
Mean per input3.10 s
Slowest input4.92 s
Inputs timed25

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens18.2K
Output tokens2K
Total tokens20.2K
Input cost$0.0016
Output cost$0.0006
Total cost$0.0022

Priced from 2026-09-16 · Input $/M $0.09 · Output $/M $0.30

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 65 of 613

Test T1130 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter stepfun/step-3.7-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":null,"sender_persons":null,"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro69.0%
F1 macro57.0%

Speed

Model time, all inputs588.0 s
Mean per input23.52 s
Slowest input56.73 s
Inputs timed25

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens49.3K
Output tokens64.6K
Total tokens113.9K
Input cost$0.0099
Output cost$0.0743
Total cost$0.0841

Priced from 2026-09-16 · Input $/M $0.20 · Output $/M $1.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 66 of 613

Test T1616 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter meta/muse-spark-1.2 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn1
Receiver persons fp0
Receiver persons tp0
Send date fn1
Send date fp0
Send date tp0
Sender persons fn2
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":null,"letter_title":null,"sender_persons":null,"receiver_persons":null}}

Scoring

F1 micro62.0%
F1 macro59.0%

Speed

Model time, all inputs651.0 s
Mean per input11.42 s
Slowest input20.36 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens396.4K
Output tokens134.1K
Total tokens530.4K
Input cost$0.4952
Output cost$0.5697
Total cost$1.06

Priced from 2026-08-18 · Input $/M $1.25 · Output $/M $4.25

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 67 of 613

Test T1131 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter stepfun/step-3.7-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":[""],"sender_persons":["Basler Rheinschiffahrt-Aktiengesellschaft"],"receiver_persons":["Herrn Christ"]}}

Scoring

F1 micro50.0%
F1 macro56.0%

Speed

Model time, all inputs926.2 s
Mean per input28.94 s
Slowest input76.63 s
Inputs timed32

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens84.4K
Output tokens102.8K
Total tokens187.2K
Input cost$0.0169
Output cost$0.1183
Total cost$0.1351

Priced from 2026-09-16 · Input $/M $0.20 · Output $/M $1.15

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 68 of 613

Test T1407 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

anthropic claude-opus-5 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn0
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-03-29"],"letter_title":["Betriebsgemeinschaft - Internationale Ausstellung für Binnenschiffahrt und Wasserkraftnutzung Basel 1926"],"sender_persons":null,"receiver_persons":["Dr. W. Krasting"]}}

Scoring

F1 micro87.0%
F1 macro74.0%

Speed

Model time, all inputs106.9 s
Mean per input4.27 s
Slowest input6.34 s
Inputs timed25

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens166.9K
Output tokens3.4K
Total tokens170.3K
Input cost$0.8346
Output cost$0.0838
Total cost$0.9184

Priced from 2026-09-16 · Input $/M $5.00 · Output $/M $25.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 69 of 613

Test T0260 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter qwen/qwen3-vl-8b-instruct · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn1
Receiver persons fp1
Receiver persons tp0
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp1
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":["null"],"sender_persons":["Basler Rheinschiffahrt-Aktiengesellschaft"],"receiver_persons":["Herrn Christ, i/Fa. Paravicini, Christ & Co."]}}

Scoring

F1 micro52.0%
F1 macro51.0%

Speed

Model time, all inputs236.2 s
Mean per input4.14 s
Slowest input8.71 s
Inputs timed57

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens280.6K
Output tokens4.4K
Total tokens285K
Input cost$0.0328
Output cost$0.002
Total cost$0.0348

Priced from 2026-09-16 · Input $/M $0.117 · Output $/M $0.455

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 70 of 613

Test T1393 at 2026-09-17

letter information-extraction typed, handwritten 20 de prose person, location

openrouter moonshotai/kimi-k3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Receiver persons fn0
Receiver persons fp0
Receiver persons tp1
Send date fn0
Send date fp0
Send date tp1
Sender persons fn2
Sender persons fp0
Sender persons tp0

{"metadata":{"send_date":["1926-02-16"],"letter_title":null,"sender_persons":null,"receiver_persons":["Christ"]}}

Scoring

F1 micro56.0%
F1 macro61.0%

Speed

Model time, all inputs2573.0 s
Mean per input80.41 s
Slowest input601.36 s
Inputs timed32

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens263.8K
Output tokens102.1K
Total tokens365.8K
Input cost$0.6892
Output cost$1.37
Total cost$2.06

Priced from 2026-09-16 · Input $/M $2.65 · Output $/M $13.28

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827