RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Loading module...

Search Results

Your search for Benchmark 'medieval_manuscripts__true' with Search Hidden 'False' returned 177 results, showing page 11 of 18.
Result 101 of 177

Test T1082 at 2026-06-04

manuscript transcription handwritten 15 de prose

anthropic claude-opus-4-8 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score97.4%
CER ↓1.7%

{"folios":[{"folio":"3","text":"vnd ein pferit die mir vnd
minen knechten vber hulffend
den do was menen kein weg
denne den wir machtend
vnd viehend die knecht dick
vnd vil in vntz an den ars
vnd die pferit vntz an die
settel vnd was ze mol ein grosser
nebel dz wir kum gesahend
vnd als mit grosser arbeit kome
wir ze mittem tag zu sant
kristoffel vff den berg do
do sach ich die bucher do gar
vil herren wopen zu stone
die ir stur do hyn geben hand
do stund mines vatters seligen
wopen och in dem einen","addition1":null,"addition2":null,"addition3":null}]}

Scoring

Fuzzy score83.4%
CER ↓17.0%

Speed

Model time, all inputs146.6 s
Mean per input12.21 s
Slowest input43.12 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens61.9K
Output tokens3.9K
Total tokens65.7K
Input cost$0.3093
Output cost$0.0968
Total cost$0.4061

Priced from 2026-06-04 · Input $/M $5.00 · Output $/M $25.00

Result 102 of 177

Test T0538 at 2026-06-04

manuscript transcription handwritten 15 de prose

mistral mistral-large-2512 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score89.7%
CER ↓13.4%

{"folios":[{"folio":"3","text":"Vnd ein pferit die wir mit vnſe-
ren knechten wider rulſtend
 do was nieman ſin weg
 denne ſi wir machtend
 vnd viehent die knecht vnd
 vnd vil in vnſern dörffern
 vnd die pferit vnd ſwaz ſi in die
 ſettel vnd was ſi mochtend in groſſe
 nebel ſi wir kum gefangen
 vnd als mit groſſer arbeit kome
 wir ſe mitten tag zu ſant
 Erhart vff den berg do
 do ſachend die burger ſo gar
 vil derren wopſen in ſtone
 die in iren edeln ſtetten gand
 ſo ſtund mines vatterſ lehen
 wopſen ſach in dem einſy","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score57.8%
CER ↓46.8%

Speed

Model time, all inputs437.4 s
Mean per input54.67 s
Slowest input196.54 s
Inputs timed8

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens22.8K
Output tokens2K
Total tokens24.8K
Input cost$0.0114
Output cost$0.0031
Total cost$0.0144

Priced from 2026-03-02 · Input $/M $0.50 · Output $/M $1.50

Result 103 of 177

Test T1069 at 2026-06-04

manuscript transcription handwritten 15 de prose

openrouter qwen/qwen3.7-plus-20260602 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score95.0%
CER ↓4.4%

{"folios":[{"folio":"3","text":"Vnd ein pferit die mir vnd
minen knechten vber hulfend
sen do was menen fain weg
deme das wir machtend
vnd viehend die knecht die
vnd vil in vntz an den aub
vnd die pferit vntz an die
pettel vnd was ze mol ein grosser
nebel dz wir kum gepassend
vnd als mit grosser arbeit kame
wir ze mittem tag zu sant
kristoffel vff den berg do
do sach ich die buecher so gar
vil herren wopen in stone
die ir stur och im geley hand
so fand mincß vatters pflugen
wopen och in dem einen","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score71.4%
CER ↓31.1%

Speed

Model time, all inputs148.1 s
Mean per input12.34 s
Slowest input18.58 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.6K
Output tokens5.7K
Total tokens45.3K
Input cost$0.0158
Output cost$0.0091
Total cost$0.0249

Priced from 2026-06-04 · Input $/M $0.40 · Output $/M $1.60

Result 104 of 177

Test T0549 at 2026-06-04

manuscript transcription handwritten 15 de prose

mistral ministral-14b-2512 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score90.1%
CER ↓10.9%

{"folios":[{"folio":"3","text":"Und ein pferit sie mir mit
minen knechten woher gulfend
sen so was minney lang weg
deme der mir machtend
und vielent sie knecht mit
vil im rutz an sie an
und sie pferit von an sie
petel und was ze mol ein grosser
nebel & wir kum oʃʃaffend
und also mit grosser arbeit kam
wir ze mitten tag zu sant
Kyrstoffel uff sen berg so
do ach ich die bucher so gar
vil gerney wapen in ston
die in stin edlin gelich hand
o stin miness watters felgen
wapen soʃ im sen sinney","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score64.1%
CER ↓38.7%

Speed

Model time, all inputs54.7 s
Mean per input4.97 s
Slowest input6.49 s
Inputs timed11

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens31.3K
Output tokens2.6K
Total tokens33.8K
Input cost$0.0063
Output cost$0.0005
Total cost$0.0068

Priced from 2026-03-02 · Input $/M $0.20 · Output $/M $0.20

Result 105 of 177

Test T1095 at 2026-06-04

manuscript transcription handwritten 15 de prose

mistral mistral-medium-3.5 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score92.7%
CER ↓7.4%

{"folios":[{"folio":"3","text":"Vnd ein pferd die mir vnd
miner knechten wer hulfend
So was menig lang weg
Semlich das wir machtend
vnd vielend die knecht dut
vnd vil in vntz an den arb
vnd die pferd vntz an die
pettel vnd was ze mol ein grosser
nebel das wir kum ge sahend
vnd als mit grosser arbeit kome
wir ze mitten tag zu sant
kristoffel off den berg do
So fach ich die bürger do gar
vil gerren wopen yn ston
Die ir stür dolgen gebend
So stund minen watters plegen
wopen sat in dem snee","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score63.4%
CER ↓38.0%

Speed

Model time, all inputs695.9 s
Mean per input57.99 s
Slowest input77.17 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens34.1K
Output tokens3K
Total tokens37.1K
Input cost$0.0512
Output cost$0.0226
Total cost$0.0738

Priced from 2026-06-04 · Input $/M $1.50 · Output $/M $7.50

Result 106 of 177

Test T0560 at 2026-06-04

manuscript transcription handwritten 15 de prose

mistral ministral-8b-2512 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score89.7%
CER ↓11.6%

{"folios":[{"folio":"3","text":"Und ein pfirit die wir mit
minen Enckeyten woz gulffend
sen. So was menich ley weg
denne der wir machtend
und vielend sie Enckeyte stet
und vil im wntz an sie an
und sie pfirit wntz an die
petel und was ze mol ein groser
nebel & wir kum gefangen
und also mit groser arbeit kam
wir ze mittem tag zu sant
kristoffel wff den berg so
do auch ich die bücher so gar
vil genrey woppey in ston
die in stür edlin gelicy gand
so stürn mines watteres pflegen
woppey soch im sem enckay","addition1":null,"addition2":null,"addition3":null}]}

Scoring

Fuzzy score55.2%
CER ↓47.9%

Speed

Model time, all inputs55.0 s
Mean per input5.00 s
Slowest input7.64 s
Inputs timed11

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens31.3K
Output tokens4.4K
Total tokens35.7K
Input cost$0.0047
Output cost$0.0007
Total cost$0.0053

Priced from 2026-03-02 · Input $/M $0.15 · Output $/M $0.15

Result 107 of 177

Test T1056 at 2026-05-22

manuscript transcription handwritten 15 de prose

genai gemini-3.5-flash · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score96.8%
CER ↓2.5%

{"folios":[{"folio":"3","text":"Vnd ein pferit die mir vnd
minen knechten vber hulfend
den do was minen kein weg
denne den wir machend
vnd vichend die knecht vns
vnd vil in vns an den ars
vnd die pferit vns an die
sattel vnd was ze mol ein grosser
nebel dz wir kum gesachend
vnd als mit grosser arbeit kome
wir ze mittem tag zu sant
kristoffel vff den berg do
do sach ich die bucher do gar
vil herren wopen jn stond
die ir stur do hyn geben hand
do stund mines vatters seligen
wopen och in dem einon","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score82.7%
CER ↓17.3%

Speed

Model time, all inputs2125.6 s
Mean per input177.13 s
Slowest input239.85 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens19.1K
Output tokens2.8K
Total tokens21.9K
Input cost$0.0287
Output cost$0.0253
Total cost$5.22
Reasoning cost usd$5.16
Total reasoning tokens573.8K

Priced from 2026-05-22 · Input $/M $1.50 · Output $/M $9.00

Result 108 of 177

Test T1043 at 2026-04-27

manuscript transcription handwritten 15 de prose

openai gpt-5.5-2026-04-23 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score93.9%
CER ↓6.4%

{"folios":[{"folio":"3","text":"vnd ein pfferit die mir mit
minen knechten wider hulffend
dan do was mench lang weg
denne den wir machtend
vnd ziehend die knechte druf
vnd vil in vns an den ayß
vnd die pfferit vns an die
kelte vnd vns ze mol ein grosse
nebel dz wir kum gesachend
vnd als mit grosser arbeit kome
wir ze mittem tag zuo sant
kristoffel uff den berg do
Do sach ich die schilter do gar
vil herren wopen zuo stane
die ir stuer dahin geleit hand
do stuond mines vaters seligen
wopen och in dem eingang","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score73.0%
CER ↓28.9%

Speed

Model time, all inputs1404.6 s
Mean per input117.05 s
Slowest input190.24 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens44.5K
Output tokens70.1K
Total tokens114.6K
Input cost$0.2223
Output cost$2.10
Total cost$2.33

Priced from 2026-04-27 · Input $/M $5.00 · Output $/M $30.00

Result 109 of 177

Test T0989 at 2026-04-22

manuscript transcription handwritten 15 de prose

openrouter qwen/qwen3.5-9b-20260310 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score90.6%
CER ↓9.1%

{"folios":[{"folio":"73","text":"Itf zu dem ersten Min herrew graff albrecht vo
mhalt Itf her Ludwig langgraf von huchtenberg
Itf graff ott von hemmenberg
Itf graff Berthart von Vegenstein
Itf graff gottfried von hochenloch
Itf graff gottfried von Mansfeld
Itf graff Ludwig von Rruburg
Itf her Gotthalt von Blesz bedpanchyn
Itf her Hans von tungern
Itf Ierf Bapenhein
Itf her Suppchin vom Stein vnz vorhin Rutter
Itf tug vom Rotenhan
Itf Ierf von Waldenfelf
Itf Gylpand von Hungen
Itf Ierf von Eichen
Itf Henijn quast","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score62.7%
CER ↓37.7%

Speed

Model time, all inputs3708.2 s
Mean per input741.64 s
Slowest input1677.66 s
Inputs timed5

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens17.2K
Output tokens23.5K
Total tokens40.7K
Input cost$0.0017
Output cost$0.0035
Total cost$0.0052

Priced from 2026-04-21 · Input $/M $0.10 · Output $/M $0.15

Result 110 of 177

Test T0924 at 2026-04-21

manuscript transcription handwritten 15 de prose

openrouter qwen/qwen3.5-27b-20260224 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score93.1%
CER ↓7.0%

{"folios":[{"folio":"3","text":"vnd ein pferit die mir vnd
minen knechten wider gultend
san so was menen lang weg
keme dan wir machtemd
vnd vichtend die knechte durb
vnd vil in vntz an den arb
vnd die pferit vntz an die
fettel vnd was ze mol ein grosser
nebel dz wir kum gepastend
vnd als mit grosser arbeit kame
wir ze mitttem tag zu Sant
Christoffel vff den berg so
so sach ich die burger so gar
vil derren wopen im steine
die ir stur dz hnn geben hand
so stand minneb vatter v plegen
wopen och in dem steine","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score76.9%
CER ↓29.2%

Speed

Model time, all inputs132.0 s
Mean per input11.00 s
Slowest input28.53 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.6K
Output tokens3K
Total tokens42.7K
Input cost$0.0077
Output cost$0.0047
Total cost$0.0125

Priced from 2026-04-21 · Input $/M $0.195 · Output $/M $1.56