RISE Humanities Data Benchmark, 0.6.1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Loading module...

Search Results

Your search for Benchmark 'medieval_manuscripts__true' with Search Hidden 'False' returned 178 results, showing page 9 of 18.
Result 81 of 178

Test T1546 at 2026-08-16

manuscript transcription handwritten 15 de prose

huggingface thinkingmachines/Inkling-Small · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score23.1%
CER ↓77.7%

{"folios":[{"folio":"j","text":"Und ein praut Die mir und
minen freunden oder guisfend
Ich waß menny lang weg
Dame ich mir machend
und vilbant sie bracht gut
mir vil in vng an Sen ang
und Sie plerut vng in die
part und waß ze mol im grossen
nebel ich bin freudend
und als mir grossen arbeit kam
mir ze mittem tag zu sant
Enstoff uff. Sey bexey So
So frech ich die buche So far
ich frey wogen in ston
Die in stur Schwy gelben hand
So stund mich wacker Schwy
wogen doch in dem weg","addition1":"a","addition2":"b","addition3":""}]}

Scoring

Fuzzy score42.4%
CER ↓63.4%

Speed

Model time, all inputs1062.9 s
Mean per input88.58 s
Slowest input185.46 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens28K
Output tokens104K
Total tokens132.1K
Input cost$0.0126
Output cost$0.1249
Total cost$0.1375

Priced from 2026-08-15 · Input $/M $0.45 · Output $/M $1.20

Result 82 of 178

Test T1561 at 2026-08-16

manuscript transcription handwritten 15 de prose

huggingface thinkingmachines/Inkling · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

We need parse image. Need identify folio number, main text, marginal additions. Let's examine image: page with handwritten text in black ink, likely one column. Top right corner has number "3". So folio "3". Left margin maybe some small text? Let's inspect. There's a small vertical text on left edge? Maybe "w" or something? Let's read main text carefully.

We need extract exactly as appears, preserve line breaks, abbreviations, special chars.

Let's transcribe line by line from image.

Starting near top:
"vns ein pferit die vns vnd"
Wait need examine precisely.

Let's read visible lines:
1. vns ein pferit die vns vnd
2. innen knechten vder helfend
3. den So was menen ein weg
4. darne den wir machfend
5. vnd viehend die knecht Jnt
6. vnt vil in vng an den arb
7. vnd die pferit vns an die
8. sotel vnd was ze mol ein grosser
9. nebel dz wir kum gesachend
10. vnd als mit grosser arbeit kome
11. wir ze mittem tag zuͦ sant
12. Cristoffel vff den berg So
13. do sach ich die buͦcher So gar
14. vil herren wopen in stont
15. die in siner da hin gelich hand
16. do stund mines vatters seligen
17. wopen och in dem eimch

Need check spelling and characters. Let's refine.

Line 1: "vns ein pferit die vns vnd" maybe "vns ein pferit die vns vnd" but could be "vns ein pferdt die vns vnd". Let's keep as seen.

Line 2: "innen knechten vder helfend" maybe "innen knechten vder helfend" or "jnnnen knechten vder helfend". The first letter looks like "j" with long s? Actually "Jnnen knechten vder helfend". Let's write "Jnnen knechten vder helfend" but need be precise. Let's examine: The word starts with capital I? In image, "Jnnen" maybe "Jnnen". We'll use "Jnnen knechten vder helfend".

Line 3: "den So was menen ein weg" maybe "den So was menen ein weg". Or "den So was menen ein weg". We'll keep.

Line 4: "darne den wir machfend" maybe "darne den wir machfend". Or "darne den wir machfend". Yes.

Line 5: "vnd viehend die knecht Jnt" maybe "vnd viehend die knecht Jnt". Actually "vnd viehend die knecht Jnt" but "Jnt" might be "Jnt" with abbreviation? We'll keep.

Line 6: "vnt vil in vng an den arb" maybe "vnt vil in vng an den arb". Or "vnd vil in vng an den arb". The first word might be "vnd". Let's use "vnd vil in vng an den arb".

Line 7: "vnd die pferit vns an die" -> "vnd die pferit vns an die"

Line 8: "sotel vnd was ze mol ein grosser" -> "sotel vnd was ze mol ein grosser" maybe "sottel"? We'll keep "sotel".

Line 9: "nebel dz wir kum gesachend" -> "nebel dz wir kum gesachend"

Line 10: "vnd als mit grosser arbeit kome" -> "vnd als mit grosser arbeit kome"

Line 11: "wir ze mittem tag zuͦ sant" -> "wir ze mittem tag zuͦ sant". There's special char "zuͦ" with superscript o? Actually "zuo" with o superscript? The instruction: if superscribed letter, normalize by writing superscribed letter after main letter, e.g., "uo". So "zuo" but they said write "uo" after main? Wait example: "u with superscribed o" is spelled as "uo". So "z" then "u" then "o" => "zuo". But in text it's "zuͦ" maybe with ring above? We'll write "zuo". But need preserve? They said normalize by writing superscribed letter after main letter. So "zuo". Then "sant".

Line 12: "Cristoffel vff den berg So" -> "Cristoffel vff den berg So"

Line 13: "do sach ich die buͦcher So gar" -> "do sach ich die buocher So gar"? Wait "buͦcher" has u with superscript o? So "buocher". We'll write "buocher". Then "So gar".

Line 14: "vil herren wopen in stont" -> "vil herren wopen in stont" maybe "vil herren wopen jn stont"? Let's check image: "vil herren wopen in stont" yes.

Line 15: "die in siner da hin gelich hand" -> "die in siner da hin gelich hand" maybe "die in siner da hin gelich hand". Or "gelichen hand"? We'll keep "gelich hand".

Line 16: "do stund mines vatters seligen" -> "do stund mines vatters seligen"

Line 17: "wopen och in dem eimch" -> "wopen och in dem eimch" maybe "eimch" is abbreviation? We'll keep.

Now check for marginal additions. On left margin, there might be small writing near bottom left? Actually left edge has some faint vertical marks, maybe not readable. On right margin? Maybe none. At top right is folio number "3". So addition fields empty.

But need check if there's addition on left side: I can see near left middle edge some small letters? Let's examine image edges. There's a dark border. The left margin of page has some faint writing? Maybe "w" near bottom left? Actually in the bottom left corner outside text? Not clear. Given instructions, if present label addition1 etc. But likely none visible clearly. We'll set addition1, addition2, addition3 to ""? Wait schema allows null or string, but instructions say if field has no content use empty string "" (not null). So we'll use "".

But need check if there is a marginal note at top? No.

Thus JSON:
{
  "folios": [
    {
      "folio": "3",
      "text": "vns ein pferit die vns vnd
Jnnen knechten vder helfend
den So was menen ein weg
darne den wir machfend
vnd viehend die knecht Jnt
vnd vil in vng an den arb
vnd die pferit vns an die
sotel vnd was ze mol ein grosser
nebel dz wir kum gesachend
vnd als mit grosser arbeit kome
wir ze mittem tag zuo sant
Cristoffel vff den berg So
do sach ich die buocher So gar
vil herren wopen in stont
die in siner da hin gelich hand
do stund mines vatters seligen
wopen och in dem eimch",
      "addition1": "",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Need verify line breaks match image lines. Let's count lines in image: There are about 17 lines of text. Yes.

But need be precise with characters: "pferit" or "pferdt"? The image shows "pferit" with an "i" maybe "pferdt"? Actually in medieval German "Pferd" is horse. The writing might be "pferdt" with a "d" that looks like "t"? Let's examine: The word is "pferit" maybe "pferit" but likely "pferdt". However we must transcribe exactly as seen. The letters: p-f-e-r-i-t? Or p-f-e-r-d-t? The last letter might be "t" with a cross? Let's not overthink. We'll write "pferit" if that's what it looks like. But maybe it's "pferdt" with "d" and "t" ligature? Actually in the image, line 1: "vns ein pferit die vns vnd". The word after "ein" looks like "pferit" with an "i" dot? Hmm. Let's keep "pferit".

Line 2: "Jnnen knechten vder helfend" - maybe "vder" is "vder" (oder?). Yes.

Line 3: "den So was menen ein weg" - maybe "menen" is "menen". Yes.

Line 4: "darne den wir machfend" - "machfend" maybe "machfend". Yes.

Line 5: "vnd viehend die knecht Jnt" - "viehend" maybe "viehend". Yes.

Line 6: "vnd vil in vng an den arb" - "vng" maybe "vng". Yes.

Line 7: "vnd die pferit vns an die" - yes.

Line 8: "sotel vnd was ze mol ein grosser" - "

Scoring

Fuzzy score61.9%
CER ↓38.1%

Speed

Model time, all inputs510.2 s
Mean per input42.52 s
Slowest input45.87 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens30.8K
Output tokens24.5K
Total tokens55.3K
Input cost$0.0308
Output cost$0.099
Total cost$0.1299

Priced from 2026-08-15 · Input $/M $1.00 · Output $/M $4.05

Result 83 of 178

Test T1576 at 2026-08-16

manuscript transcription handwritten 15 de prose

huggingface meta-models/Muse-Glimmer-30B · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score89.3%
CER ↓10.8%

{"folios":[{"folio":"3","text":"Und ein pferit Sie wir und
minen Enckten vnder Aufsand
Den So was menen Eyn weg
Semme Den wir machten
und vichent Sie Enckte Suck
und vil in vntz an Sen anb
und Sie pferit vntz an Sie
petel und was ze mol ein groffer
nebel Es wir kum gespand
und als mit groffer arbait kome
wir ze mittrem tag zu Sant
Eripoffel vff Sen Berg So
So fach ich die Bücher So gar
vil Gerrin wopen in Rome
Sie in fir Ed Gyn gelben Gand
So pünt mines vatters Plagen
wopen aof in Sen emen","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score64.9%
CER ↓38.9%

Speed

Model time, all inputs824.7 s
Mean per input68.73 s
Slowest input300.47 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens49.7K
Output tokens57K
Total tokens106.7K
Input cost$0.0174
Output cost$0.0856
Total cost$0.1029

Priced from 2026-08-15 · Input $/M $0.35 · Output $/M $1.50

Result 84 of 178

Test T1531 at 2026-08-15

manuscript transcription handwritten 15 de prose

huggingface MiniMaxAI/MiniMax-M3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score91.5%
CER ↓8.7%

{"folios":[{"folio":"3","text":"vnd ein pꝛeit die wir vnd
vnsern knechten oder hulffend
sey So was nemen kün weg
denne das wir machtend
vnd richtend die knecht jnck
mit vil in vng an sey vnd
vnd die pꝛeit vng an die
ketel vnd was ze mol in grosser
not dz wir kum ge sadtend
vnd als mit grosser arbeite kame
wir ze mittem tag zu sant
fridpffel vff den berg do
do sach ich die brucken do gar
vil herren wopen in sone
die in feilr schin geberg gand
do stund minen vatterd klagend
wopen och in dem einen","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score70.4%
CER ↓33.2%

Speed

Model time, all inputs192.6 s
Mean per input16.05 s
Slowest input32.78 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens14.5K
Output tokens2.7K
Total tokens17.2K
Input cost$0.0041
Output cost$0.003
Total cost$0.007

Priced from 2026-08-15 · Input $/M $0.28 · Output $/M $1.10

Result 85 of 178

Test T1516 at 2026-08-15

manuscript transcription handwritten 15 de prose

huggingface Qwen/Qwen3-VL-235B-A22B-Instruct · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score93.1%
CER ↓6.9%

{"folios":[{"folio":"3","text":"Vnd ein pferit die wir vnd
minen knechten vber Rulfend
Gen do was menny ein Weg
Seme den wir machten
vnd vielend die knechte sich
vnd vil in vntz an den arb
vnd die pferit vnz an die
Petrel vnd was ze mot ein großer
nebel & wir kum gepaffen
vnd als mit großer arbeit kame
wir ze mitten tag zu Pant
Einstoffel vff den berg do
do fach ich die bucher do gar
vil herrin wopen in stone
Die ir stiur & gnu geben hand
do fand mineb vatterb flugen
wopen sich in dem smeg","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score68.0%
CER ↓33.3%

Speed

Model time, all inputs109.5 s
Mean per input9.12 s
Slowest input14.81 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens39.3K
Output tokens2.8K
Total tokens42.1K
Input cost$0.0079
Output cost$0.0025
Total cost$0.0103

Priced from 2026-08-15 · Input $/M $0.20 · Output $/M $0.88

Result 86 of 178

Test T1501 at 2026-08-14

manuscript transcription handwritten 15 de prose

huggingface swiss-ai/apertus-v1.5-8b · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score25.8%
CER ↓75.4%

{"folios":[{"folio":"","text":"Und ein pferit Sie mir mit meinen knechten oder fulpend
 Sen so was meinen kam weg
 deme Say mir machend
 und vriend Sie fuerst duf
 und vil in uns an dey and
 und Sie pferit vns an dey der petrel und was ze mot ein groffer
 nebel dz wir kum ire pferd
 und also mit groffer arbeit kenne
 wir ze mieten tag zu lant
 binstapel vff Say berg do
 do pfer ich die bucher So gar
 vil bawrey wogey in tone
 Sie in stier So bin gebey hand
 so find mines vatter v schrey
 wogey so in dem enien
","addition1":"3","addition2":"","addition3":""}]}

Scoring

Fuzzy score35.8%
CER ↓70.4%

Speed

Model time, all inputs150.3 s
Mean per input12.52 s
Slowest input22.37 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens98.9K
Output tokens2.6K
Total tokens101.5K
Input cost$0.0099
Output cost$0.0005
Total cost$0.0104

Priced from 2026-08-10 · Input $/M $0.10 · Output $/M $0.20

Result 87 of 178

Test T1486 at 2026-08-14

manuscript transcription handwritten 15 de prose

huggingface swiss-ai/apertus-v1.5-70b · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score90.0%
CER ↓10.4%

{"folios":[{"folio":"3","text":"Und ein pfert die mir vnd
minen bruderen vber guldend
San So was menen kunig weg
demne San wir machsend
vnd vriehend die fuerst vnd
vil in vng andezen vnd
vnd die pfert vns vng an die
petel vnd was ze mol ein groffer
nebel da wir kunig gesachend
vnd also mit groffer arbeit keme
wir ze mitten tag zu sant
Kunigoffel vff den berg do
So sich ir die bucher So gar
vil hauen wepen in stouze
Die in feur ze hin gebrachend
So stund mines vaters schgen
wagen off in dem einen
","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score69.1%
CER ↓35.8%

Speed

Model time, all inputs226.8 s
Mean per input18.90 s
Slowest input20.61 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens98.9K
Output tokens2.5K
Total tokens101.4K
Input cost$0.0811
Output cost$0.0073
Total cost$0.0884

Priced from 2026-08-10 · Input $/M $0.82 · Output $/M $2.92

Result 88 of 178

Test T1456 at 2026-08-04

manuscript transcription handwritten 15 de prose

x-ai grok-4.5 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score96.8%
CER ↓2.4%

{"folios":[{"folio":"3","text":"Vnd ein pferit die mir vnd
minen knechten vber hulfend
den do was nienen kein weg
denne den wir machtend
vnd ziehend die knechte dick
vnd vil in vntz an den arb
vnd die pferit vntz an die
ketel vnd was ze mol ein grosser
nebel dz wir kum gesehend
vnd als mit grosser arbeit kame
wir ze mittem tag zuo sant
kristoffel vff den berg do
do sach ich die buocher do gar
vil herren wapen jn stont
die ir stir do hin geben hand
do stund mines vatters schiltgen
wapen och in dem einen","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score83.2%
CER ↓16.3%

Speed

Model time, all inputs540.9 s
Mean per input45.08 s
Slowest input100.78 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens40.1K
Output tokens2.8K
Total tokens42.9K
Input cost$0.0802
Output cost$0.0165
Total cost$0.2612
Reasoning cost usd$0.1644
Total reasoning tokens27.4K

Priced from 2026-08-04 · Input $/M $2.00 · Output $/M $6.00

Result 89 of 178

Test T1396 at 2026-07-25

manuscript transcription handwritten 15 de prose

openrouter moonshotai/kimi-k3 · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score95.4%
CER ↓4.3%

{"folios":[{"folio":"3","text":"vnd ein pferit die mir vnd
minen knechten wider luffend
sen do was nimen lang weg
denne den wir machtend
vnd viehend die knecht sich
vnd vil in vns an den arb
vnd die pferit vns an die
stel vnd was ze mol ein grosse
nebel dz wir kum gesahent
vnd also mit grosser arbeit kome
wir ze mittem tag zu sant
kristoffel vff den berg do
do sach ich die buecher so gar
vil gerren wopen in stou
die ir stur dd hin gesyen hand
do stund mines vatters pflagen
wopen och in dem einem","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score79.7%
CER ↓21.2%

Speed

Model time, all inputs6239.7 s
Mean per input519.97 s
Slowest input1729.60 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens53.7K
Output tokens204.8K
Total tokens258.5K
Input cost$0.1496
Output cost$3.07
Total cost$3.22

Priced from 2026-07-25 · Input $/M $3.00 · Output $/M $15.00

Result 90 of 178

Test T1441 at 2026-07-25

manuscript transcription handwritten 15 de prose

genai gemini-3.5-flash-lite · temp 0.0 · dataclass Document

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score94.9%
CER ↓4.8%

{"folios":[{"folio":"3","text":"Vnd ein pferit die mir vnd
minen knechten vber Rulfend
Gen Do was menan kein weg
denne den wir machtend
vnd vihend die knechte dut
vnd vil in vns an den arb
vnd die pferit vns an die
settel vnd was ze mol ein grosser
nebel daz wir kum gesehend
vnd als mit grosser arbeit kame
wir ze mittem tag zu sant
christoffel off den berg do
do sach ich die bücher do gar
vil herren wopen in stoue
die ir für deltyn geben hand
do stünd mines vatters rügen
wopen ad in dem sinen","addition1":"","addition2":"","addition3":""}]}

Scoring

Fuzzy score73.2%
CER ↓28.8%

Speed

Model time, all inputs391.3 s
Mean per input32.60 s
Slowest input55.87 s
Inputs timed12

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens19.1K
Output tokens2.8K
Total tokens22K
Input cost$0.0057
Output cost$0.0071
Total cost$0.0128

Priced from 2026-07-25 · Input $/M $0.30 · Output $/M $2.50