RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'blacklist_cards__true' with Search Hidden 'False' returned 180 results, showing page 3 of 18.
Result 21 of 180

Test T1727 at 2026-09-18

index-card information-extraction typed, handwritten 20 de company

openrouter qwen/qwen3.8-flash · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score80.0%

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score92.1%

Speed

Model time, all inputs1077.2 s
Mean per input32.64 s
Slowest input106.12 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens98.7K
Output tokens62K
Total tokens160.7K
Input cost$0.0147
Output cost$0.0292
Total cost$0.0438

Priced from 2026-09-18 · Input $/M $0.15 · Output $/M $0.47

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 22 of 180

Test T1562 at 2026-09-18

index-card information-extraction typed, handwritten 20 de company

huggingface thinkingmachines/Inkling · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score80.0%

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score87.5%

Speed

Model time, all inputs790.1 s
Mean per input23.94 s
Slowest input90.51 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens72.8K
Output tokens30.5K
Total tokens103.2K
Input cost$0.0728
Output cost$0.1233
Total cost$0.1961

Priced from 2026-09-16 · Input $/M $1.00 · Output $/M $4.05

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 23 of 180

Test T1637 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

openrouter z-ai/glm-5v-turbo · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score77.1%

{"company":{"transcription":"A b e g g & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score86.1%

Speed

Model time, all inputs1540.1 s
Mean per input46.67 s
Slowest input90.44 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens121.1K
Output tokens26K
Total tokens147.1K
Input cost$0.1387
Output cost$0.1038
Total cost$0.2425

Priced from 2026-08-18 · Input $/M $1.20 · Output $/M $4.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 24 of 180

Test T0951 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

openrouter qwen/qwen3.5-397b-a17b · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score76.5%

{"company":{"transcription":"A b e g g & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.26"},"date":"","information":[]}

Scoring

Fuzzy score91.2%

Speed

Model time, all inputs4789.9 s
Mean per input145.15 s
Slowest input738.39 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens81.3K
Output tokens60.1K
Total tokens141.4K
Input cost$0.0447
Output cost$0.2104
Total cost$0.2551

Priced from 2026-09-16 · Input $/M $0.55 · Output $/M $3.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 25 of 180

Test T0925 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

openrouter qwen/qwen3.5-27b · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score80.0%

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score95.4%

Speed

Model time, all inputs1049.4 s
Mean per input31.80 s
Slowest input132.51 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens78.8K
Output tokens31.3K
Total tokens110.1K
Input cost$0.0155
Output cost$0.0517
Total cost$0.0672

Priced from 2026-09-16 · Input $/M $0.195 · Output $/M $1.56

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 26 of 180

Test T0912 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

openrouter qwen/qwen3.5-122b-a10b · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score80.0%

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score84.6%

Speed

Model time, all inputs215.8 s
Mean per input6.54 s
Slowest input13.55 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens91.1K
Output tokens9.9K
Total tokens101K
Input cost$0.0237
Output cost$0.0206
Total cost$0.0443

Priced from 2026-09-16 · Input $/M $0.26 · Output $/M $2.08

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 27 of 180

Test T1697 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

openrouter meta/muse-spark-1.3 · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score80.0%

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score95.6%

Speed

Model time, all inputs345.9 s
Mean per input10.48 s
Slowest input17.97 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens115.5K
Output tokens43.5K
Total tokens159.1K
Input cost$0.1444
Output cost$0.185
Total cost$0.3294

Priced from 2026-09-08 · Input $/M $1.25 · Output $/M $4.25

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 28 of 180

Test T1622 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

openrouter meta/muse-spark-1.2 · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score80.0%

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[]}

Scoring

Fuzzy score94.4%

Speed

Model time, all inputs184.9 s
Mean per input5.60 s
Slowest input10.77 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens108.5K
Output tokens41.7K
Total tokens150.2K
Input cost$0.1356
Output cost$0.1772
Total cost$0.3129

Priced from 2026-08-18 · Input $/M $1.25 · Output $/M $4.25

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 29 of 180

Test T1412 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

anthropic claude-opus-5 · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score56.6%

{"company":{"transcription":"A b e g g  & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"\"\"","information":[{"transcription":""}]}

Scoring

Fuzzy score81.2%

Speed

Model time, all inputs158.8 s
Mean per input4.81 s
Slowest input6.18 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens143.3K
Output tokens8.4K
Total tokens151.7K
Input cost$0.7165
Output cost$0.2104
Total cost$0.9268

Priced from 2026-09-16 · Input $/M $5.00 · Output $/M $25.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 30 of 180

Test T1682 at 2026-09-17

index-card information-extraction typed, handwritten 20 de company

anthropic claude-fable-5-1 · temp 0.5 · dataclass Card

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score77.1%

{
  "company": {"transcription": "A b e g g & Cie."},
  "location": {"transcription": "Zürich"},
  "b_id": {"transcription": "B.51.322.GB.266"},
  "date": "",
  "information": []
}

Scoring

Fuzzy score94.8%

Speed

Model time, all inputs252.8 s
Mean per input7.66 s
Slowest input13.69 s
Inputs timed33

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens113.2K
Output tokens5.6K
Total tokens118.8K
Input cost$1.13
Output cost$0.2812
Total cost$1.41

Priced from 2026-09-08 · Input $/M $10.00 · Output $/M $50.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827