RISE Humanities Data Benchmark, 0.5.5-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results

Your search for Benchmark 'medieval_manuscripts__true' with Search Hidden 'False' returned 106 results, showing page 1 of 11.
Result 1 of 106

Test T1591 at 2026-08-18

manuscript transcription handwritten 15 de prose

Configuration
Providergenai
Modelgemini-3.7-flash
  
Temperature0.0
DataclassDocument
  
Normalized Score85.20 %
Test time7.11 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"18","text":"Der graf von swarczburg
vnd sust all ander ritter
vnd knecht wolten nit
dar den ir was enteil vast
siech vßgenomen her jacob
von haregurt vnd hans
von hertzenberg ritter
die zwen werend gern
mit mir gesein aber der
patron wolt vnser nit
beiten do der merteil nit
dar wolt vnd also můsten
wir mit grossem vnwilleñ
danna scheyden dey wir
mit dem patron vnd
och den brůderen dar
vmb hattend ze zipren
wachset dz zuker","addition1":"Saccharū
in cypro
nascitur","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.84 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 17.5K IT + 2.6K OT = 20.1K TTCost: 0.013$0.010$0.023$
Result 2 of 106

Test T1606 at 2026-08-18

manuscript transcription handwritten 15 de prose

Configuration
Providerx-ai
Modelgrok-4.6
  
Temperature0.0
DataclassDocument
  
Normalized Score81.50 %
Test time65.99 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"73","text":"Itz zu dem ersten Min herre graff albrecht vo
lanhalt
Itz her ludwig lantgraf von lichtenberg
Itz greff ott von kemmenberg
Itz greff Bernhart von Regenstain
Itz greff göttfrid von hashenlach
Itz greff gebhart von Mansfelt
Itz greff ludwig von Isenburg
Itz her Gotthalt von Bleff
Itz her hans von tungöw
Itz Jerg Bappamhein
Itz her Buppelin vom stein wz worhin Ritter
Itz luz vom Rotenhan
Itz Jerg von waldenffss
Itz hilprand von Thüngen
Itz Jerg von Elieben
Itz henin quast","addition1":"bedpanchin","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.81 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 40.1K IT + 2.7K OT = 42.9K TTCost: 0.080$0.016$0.097$
Result 3 of 106

Test T1621 at 2026-08-18

manuscript transcription handwritten 15 de prose

Configuration
Provideropenrouter
Modelmeta/muse-spark-1.2
  
Temperature0.0
DataclassDocument
  
Normalized Score81.40 %
Test time53.77 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"","text":"Selbs im fronaltar die schend
wir gantz vnd ein stuck von
dem heilgen fron critz dz
lit ir vff ir brust vnd ein
bein von sant marien madalene
vinger vot Jst vff samstag noch
dem ostertag der do was der
ander tag des abrellen do
furend wir gon meran do
ligend in sant steffans
kilchen bi hundert der kindlin
die herodes er slug in einem
altar die schend wir vnd
sicht man noch best gedentlich
in hend vnd fuos vnd die
nagel dar an vnndes an
dem sumentag fruog do","addition1":"Muran
glab
Sitten","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.82 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 49.5K IT + 28.4K OT = 77.9K TTCost: 0.062$0.121$0.183$
Result 4 of 106

Test T1636 at 2026-08-18

manuscript transcription handwritten 15 de prose

Configuration
Provideropenrouter
Modelz-ai/glm-5v-turbo
  
Temperature0.0
DataclassDocument
  
Normalized Score0.00 %
Test time82.28 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{
  "folios": [
    {
      "folio": "73",
      "text": "Jff zu dem ersten Herrn graff albrecht vo
ambalt
Jff herr ludwig langgraf von huttenberg
Jff graff ott von henneberg
Jff graff Bernhart von Egenstein
Jff graff güntzle von habsburg
Jff graff yhant von Manffelß
Jff graff ludwig von pfennburg
Jff herr Sotthalt von Steff
Jff herr hans von tungon
Jff Jerg Bappombein
Jff herr Suppelin von pein wz wochin Ritter
Jff hug vom Rotenhan
Jff Jerg von walsserff
Jff güprecht von thüngen
Jff Iexg von elchon
Jff Henin quat",
      "addition1": "bespanckn",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.00 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 51.1K IT + 22.8K OT = 73.9K TTCost: 0.061$0.091$0.152$
Result 5 of 106

Test T1576 at 2026-08-16

manuscript transcription handwritten 15 de prose

Configuration
Providerhuggingface
Modelmeta-models/Muse-Glimmer-30B
  
Temperature0.0
DataclassDocument
  
Normalized Score61.10 %
Test time29.07 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"3","text":"Und ein pferit Sie wir und
minen Enckten vnder Aufsand
Den So was menen Eyn weg
Semme Den wir machten
und vichent Sie Enckte Suck
und vil in vntz an Sen anb
und Sie pferit vntz an Sie
petel und was ze mol ein groffer
nebel Es wir kum gespand
und als mit groffer arbait kome
wir ze mittrem tag zu Sant
Eripoffel vff Sen Berg So
So fach ich die Bücher So gar
vil Gerrin wopen in Rome
Sie in fir Ed Gyn gelben Gand
So pünt mines vatters Plagen
wopen aof in Sen emen","addition1":"","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.65 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 49.7K IT + 57.0K OT = 106.7K TTCost: 0.017$0.086$0.103$
Result 6 of 106

Test T1561 at 2026-08-16

manuscript transcription handwritten 15 de prose

Configuration
Providerhuggingface
Modelthinkingmachines/Inkling
  
Temperature0.0
DataclassDocument
  
Normalized Score61.90 %
Test time44.29 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

We need parse image of manuscript page, 15th c, Basel, late medieval German. Need extract text exactly, preserve line breaks, abbreviations, special chars. Need identify folio number, main text, marginal additions.

We need examine image carefully. Let's analyze layout.

Image shows one page with handwritten text in brown ink, likely one column continuous but with some lines. There's text on left margin? Let's inspect.

We need read all visible text. Let's decode line by line.

Top right? Actually page has text centered? Let's parse visible lines from top to bottom.

Starting near top:
Line 1: "vnd dem erzbischoff do selbe" maybe? Let's examine precisely.

We need be careful with abbreviations and special characters like ꝛ, ꝰ, superscript letters.

Let's read the main block (center/right?). There's also left margin text running vertically? Let's understand layout: The page seems to have main text in larger script occupying right/center, and smaller text in left margin? Actually image shows text spread across page with some lines indented? Let's parse systematically.

Looking at image:
Top line (centered/top): "vnd dem erzbischoff do selbe" maybe "vnd dem erzbischof do selbe"? Let's read characters.

Second line: "vnd zcu frankenfurt vnd dz hertz" maybe? Let's decode.

Third line: "in dem thume do selbe Jch" maybe?

Fourth line: "sant thÿtus houpt der do ge" ?

Fifth line: "koppÿ singen enter wz Jch sat" ?

Sixth line: "Barbare eint künig do züpern" ?

Seventh line: "tochter houpt Jch sant blasiÿ" ?

Then left margin has smaller writing? Let's parse left side text:
"vnd zcu genewa" ?
"vnd an zc k" ?
"die an der" ?
"vnd darvmb" ?
"wer in" ?
"dentail blaut" ?
"nach ent vnd" ?
"tüch die kl" ?
"ging vnd hab" ?
"vnd zcu min" ?
"vnd do vß" ?
"vnd vns vnd" ?
"warten vß" ?
"vnd furcht" ?
"vnd furcht" ?
"vnd furcht" ?

But need be precise.

Also bottom left has more lines:
"geÿn de jmal" ?
"vß dem klain" ?
"bayfasse kloster" ?

This is challenging. Let's try to read the image more concretely using mental zoom.

We need return JSON with folio (maybe empty if not visible), text (main text body?), addition1, addition2, addition3 for marginal notes.

We need determine what is main text and what are additions. The prompt says: "The manuscript may contain: 1. A main text body (often in one column as continuous text) 2. A folio number (if present) 3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)"

Thus we need separate main continuous text and marginal notes. In this image, there might be one main text block on right/center, and left margin annotations. Let's identify.

Looking at the image, there is a large block of text on the right side with about 15 lines. On the left side, there is smaller text running along left edge, maybe 10 lines. At bottom left, there are a few more lines. So likely main text is the larger block, and left margin is addition1, maybe bottom left addition2, etc.

But we need extract all text exactly as appears, maintaining line breaks with 
. For main text, we should combine lines in order? Probably preserve line breaks as they appear in main block.

Let's decode main block line by line precisely.

I'll try to read each line from top to bottom, right to left? Actually text is left-to-right within lines.

Line 1 (top): "vnd dem erzbischoff do selbe"
But there might be superscript. Let's examine: "vnd" then "dem" then "erzbischof" maybe with abbreviation "erzbiscoff"? The word looks like "erzbischof" but with special char? Actually "erzbischof" often abbreviated "erzbischof" with loop? Let's keep "erzbischof" or "erzbiscoff"? Let's read letters: e-r-z-b-i-s-c-h-o-f-f? The image shows "erzbischof" maybe with "ff" ligature? We'll write "erzbischof" but need preserve spelling exactly. Let's hold "vnd dem erzbischof do selbe". Actually "do selbe" might be "do selbe" meaning "daselbst"? But we'll write as seen.

Line 2: "vnd zcu frankenfurt vnd dz hertz" maybe? Let's parse: "vnd" then "zcu" (zu) then "franckenfurt" or "frankfurt"? The letters: f-r-a-n-k-f-u-r-t? Maybe "frankfurt". Then "vnd" then "dz" (das) then "hertz"? Actually "hertzog"? Let's see: "dz hertzog"? The word might be "hertzog" abbreviated? We'll write "vnd zcu frankfurt vnd dz hertzog" but need check ending. Maybe "dz hertz" with superscript? Let's keep "vnd zcu frankfurt vnd dz hertzog" but not sure.

Line 3: "in dem thume do selbe Jch" maybe "in dem thume do selbe Jch"? Actually "thume" is "Thum" (cathedral). "do selbe" again. "Jch" maybe "Ich". So "in dem thume do selbe Jch" but maybe "in dem thume do selbe ich" with capital I? We'll write "in dem thume do selbe Jch".

Line 4: "sant thÿtus houpt der do ge" maybe "sant thytus houpt der do ge"? Let's parse: "sant" (Sankt) "thytus" (Titus) with y? "houpt" (Haupt) "der" "do" "ge" maybe part of "geheilt"? Actually line might continue. Let's write "sant thytus houpt der do ge".

Line 5: "koppÿ singen enter wz Jch sat" Hmm. Let's decode: "koppÿ" maybe "kopff"? Actually "kopff" with y? "singen" "enter" "wz" (was?) "Jch" "sat" (sagte?). So "kopffy singen enter wz Jch sat". Not clear. We'll approximate.

Line 6: "Barbare eint künig do züpern" maybe "Barbare eint künig do zypern"? Actually "Barbar" is Barbara? "eint" "künig" "do" "züpern" (Zypern?). So "Barbare eint künig do zypern". Let's write "Barbare eint künig do zypern".

Line 7: "tochter houpt Jch sant blasiÿ" maybe "tochter houpt Jch sant blasiÿ". So "tochter houpt Jch sant blasiÿ". We'll keep.

Then there might be more lines in main block? Let's check image: After line 7, there is a gap? Actually the left margin text starts around same height. The main block continues downward with more lines on right.

Line 8: "tochter houpt Jch sant blasiÿ" is near middle. Then below:
"mein Jch sant andres ainÿ Jch" maybe?
Let's parse lower right lines.

Looking at bottom right area:
Line around middle-bottom: "mein Jch sant andres ainÿ Jch"
Then: "vnd sant steffanus ainÿ Jch vnd" ?
Then: "zwen zen die do sant cipols" ?
Then: "vnd Jch sant andres ainÿ Jch" ?
Then: "vff Weintag vor sant Josabs" ?
Then: "tag do zc Burgundien füren wir" ?
Then: "vnd fünden ein fürenn an J" ?
Then: "vff feyertag wir füren die Jch" ?
Then: "vnd vns vnd zc vnbekantla" ?
Then: "warten vß harznen feyt do warnete sich die" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?
Then: "vnd furcht vnd" ?

Wait, that seems like repetition of "vnd furcht vnd" in left margin? Actually left margin might have repeated phrases.

Let's try to

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.62 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 30.8K IT + 24.5K OT = 55.3K TTCost: 0.031$0.099$0.130$
Result 7 of 106

Test T1546 at 2026-08-16

manuscript transcription handwritten 15 de prose

Configuration
Providerhuggingface
Modelthinkingmachines/Inkling-Small
  
Temperature0.0
DataclassDocument
  
Normalized Score36.60 %
Test time101.47 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"7","text":"vnd die prufet ist bi mir
mit vnd ich der wenczig
ist von fursung vnd gvn
fudung find al mil vnd
ist och ein fubig fur vnd
ein entzug vnd ist och
zen wenczig ze nid
wacht der vmmann vnd
ich och bi ein freud
so bi mir
der von morgen an gar gut
vnd ich mit vnd Candida
ist ein groß misel vnd mil
hat vnd wunsch vnd
der wenczig
Rody. Ist von morgen vnd vnd vnd
vnd ich mit mir abey","addition1":"Cach
Vmm
Mal
freum
mal
mal","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.42 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 28.0K IT + 104.0K OT = 132.1K TTCost: 0.013$0.125$0.137$
Result 8 of 106

Test T1516 at 2026-08-15

manuscript transcription handwritten 15 de prose

Configuration
Providerhuggingface
ModelQwen/Qwen3-VL-235B-A22B-Instruct
  
Temperature0.0
DataclassDocument
  
Normalized Score66.70 %
Test time8.39 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"","text":"Vnd die pffel ist bi aa mil
wit vnd ist der Venediger
Jpf von Eurfan vnz gon
modun find at mil vnd
ist och ein gůbpf stat vnd
ein extzbiſtum vnd ist och
der Venediger ze modun
wachſet der rumanier vnd
tut och do bi ein heilig geſſet
das bi ij milen
Candia Jpf von modun ſib gon candia
Creta. find at mil vnd candia
Vinum ist ein grob pffel vy mil
Malva wit vnd wachſet der
neum maluafier. do vnd ist och
der Venedier
Rhody. Jpf von candia vnz gon rodib
find at mil wer aber","addition1":"","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.68 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 39.3K IT + 2.8K OT = 42.1K TTCost: 0.008$0.002$0.010$
Result 9 of 106

Test T1531 at 2026-08-15

manuscript transcription handwritten 15 de prose

Configuration
Providerhuggingface
ModelMiniMaxAI/MiniMax-M3
  
Temperature0.0
DataclassDocument
  
Normalized Score66.80 %
Test time10.54 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"18","text":"Der grof von stattburg
vnd syt all ander iitter
vnd knecht wolten nit
von key in wat entuolgen
klich abgenomen hie pfelb
von hartgunt vnd hant
den lutpriester bitten
die zwen weren gern
mit mir gestn aber der
patron welt vnser nit
beiten do der welt nit
der welt vnd also mifzen
wir mit grossen vnwillen
danid scheyden do wir
mit dem patron vnd
och den hnidern dar
vnd hatten ze zyren
waffen dz zilte","addition1":"Sachbani
in Cypern
gestilt","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.70 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 14.5K IT + 2.7K OT = 17.2K TTCost: 0.004$0.003$0.007$
Result 10 of 106

Test T1486 at 2026-08-14

manuscript transcription handwritten 15 de prose

Configuration
Providerhuggingface
Modelswiss-ai/apertus-v1.5-70b
  
Temperature0.0
DataclassDocument
  
Normalized Score64.20 %
Test time17.50 seconds
Prompt

IDENTITY and PURPOSE:

You are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.

The manuscript may contain:
1. A main text body (often in one column as continuous text)
2. A folio number (if present)
3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)

You must:
- Identify and transcribe the main text
- Extract the folio number if visible (use empty string "" if not visible)
- Identify and transcribe any marginal additions separately
- Preserve line breaks with \n
- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. "u with superscribed o" is spelled as "uo". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. ꝛ for "er" and ꝰ for "us" or "em" might be the most common special characters.
- Do not modernize or correct the text
- Do not use OCR or attempt to interpret unclear text - transcribe what you can see
- If a field has no content, use an empty string "" (not null)

Take a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.

EXAMPLE OUTPUT:

{
  "folios": [
    {
      "folio": "15",
      "text": "In disem jare kam der kunig\n mit grossem here in daz lant\n vnd belagerte die stat Basel\n do wertend sich die burger\n mit grosser kraft vnd tugent\n vnd triben den kunig dannen\n mit schanden vnd verlust",
      "addition1": "Anno domini 1444",
      "addition2": "",
      "addition3": ""
    }
  ]
}

Results

{"folios":[{"folio":"90","text":"König von Fusteten
 ein dauß
 Turnadus der schreiber
 vnd den stift von
 mentz
 Hans Egli von merin ze mentz
 vnd Johann vnd
 Heinrich von wiffengrabe
 vnd Heinrich smit in Enecht
 von deren vs
 oefftand
 Ludwig von Dietbach
 Hans von der Gruben
 Hans Vlaming","addition1":"","addition2":"","addition3":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.69 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 98.9K IT + 2.5K OT = 101.4K TTCost: 0.081$0.007$0.088$