{"_id": {"$oid": "6a8456c1ef0a19c343227d69"}, "test_id": "T1561", "benchmark": "medieval_manuscripts", "date": "2026-08-16", "tags": {"document-type": ["manuscript"], "writing": ["handwritten"], "century": [15], "language": ["de"], "layout": ["prose"], "task": ["transcription"]}, "contributors": ["ina_serif", "maximilian_hindermann"], "hidden": false, "config": {"provider": "huggingface", "model": "thinkingmachines/Inkling:together", "dataclass": "Document", "temperature": 0.0, "role_description": "You are a historian with expertise in medieval manuscripts and paleography", "prompt_file": "prompt.txt", "legacy_test": false}, "prompt": "IDENTITY and PURPOSE:\n\nYou are presented with an image from a 15th century medieval manuscript written in Basel in late medieval German. Your task is to extract the text from the manuscript in the specified JSON format. You must extract all text exactly as it appears in the manuscript, maintaining historical spellings, punctuation, and formatting. Do not resolve abbreviations.\n\nThe manuscript may contain:\n1. A main text body (often in one column as continuous text)\n2. A folio number (if present)\n3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)\n\nYou must:\n- Identify and transcribe the main text\n- Extract the folio number if visible (use empty string \"\" if not visible)\n- Identify and transcribe any marginal additions separately\n- Preserve line breaks with \\n\n- Maintain all historical spellings and abbreviations exactly as written. If a letter is superscribed, normalize it by writing the superscribed letter after the main letter, e.g. \"u with superscribed o\" is spelled as \"uo\". If special characters are used for abbreviations, do not resolve them, but try to transcribe the special character according to the Medieval Unicode Font Initiative. \ua75b for \"er\" and \ua770 for \"us\" or \"em\" might be the most common special characters.\n- Do not modernize or correct the text\n- Do not use OCR or attempt to interpret unclear text - transcribe what you can see\n- If a field has no content, use an empty string \"\" (not null)\n\nTake a deep breath and think step by step about the layout of the page. First identify the folio number, then the main text area, then any marginal additions. Return only a JSON file with no additional commentary.\n\nEXAMPLE OUTPUT:\n\n{\n  \"folios\": [\n    {\n      \"folio\": \"15\",\n      \"text\": \"In disem jare kam der kunig\\n mit grossem here in daz lant\\n vnd belagerte die stat Basel\\n do wertend sich die burger\\n mit grosser kraft vnd tugent\\n vnd triben den kunig dannen\\n mit schanden vnd verlust\",\n      \"addition1\": \"Anno domini 1444\",\n      \"addition2\": \"\",\n      \"addition3\": \"\"\n    }\n  ]\n}\n", "results": {"text": "We need parse image of manuscript page, 15th c, Basel, late medieval German. Need extract text exactly, preserve line breaks, abbreviations, special chars. Need identify folio number, main text, marginal additions.\n\nWe need examine image carefully. Let's analyze layout.\n\nImage shows one page with handwritten text in brown ink, likely one column continuous but with some lines. There's text on left margin? Let's inspect.\n\nWe need read all visible text. Let's decode line by line.\n\nTop right? Actually page has text centered? Let's parse visible lines from top to bottom.\n\nStarting near top:\nLine 1: \"vnd dem erzbischoff do selbe\" maybe? Let's examine precisely.\n\nWe need be careful with abbreviations and special characters like \ua75b, \ua770, superscript letters.\n\nLet's read the main block (center/right?). There's also left margin text running vertically? Let's understand layout: The page seems to have main text in larger script occupying right/center, and smaller text in left margin? Actually image shows text spread across page with some lines indented? Let's parse systematically.\n\nLooking at image:\nTop line (centered/top): \"vnd dem erzbischoff do selbe\" maybe \"vnd dem erzbischof do selbe\"? Let's read characters.\n\nSecond line: \"vnd zcu frankenfurt vnd dz hertz\" maybe? Let's decode.\n\nThird line: \"in dem thume do selbe Jch\" maybe?\n\nFourth line: \"sant th\u00fftus houpt der do ge\" ?\n\nFifth line: \"kopp\u00ff singen enter wz Jch sat\" ?\n\nSixth line: \"Barbare eint k\u00fcnig do z\u00fcpern\" ?\n\nSeventh line: \"tochter houpt Jch sant blasi\u00ff\" ?\n\nThen left margin has smaller writing? Let's parse left side text:\n\"vnd zcu genewa\" ?\n\"vnd an zc k\" ?\n\"die an der\" ?\n\"vnd darvmb\" ?\n\"wer in\" ?\n\"dentail blaut\" ?\n\"nach ent vnd\" ?\n\"t\u00fcch die kl\" ?\n\"ging vnd hab\" ?\n\"vnd zcu min\" ?\n\"vnd do v\u00df\" ?\n\"vnd vns vnd\" ?\n\"warten v\u00df\" ?\n\"vnd furcht\" ?\n\"vnd furcht\" ?\n\"vnd furcht\" ?\n\nBut need be precise.\n\nAlso bottom left has more lines:\n\"ge\u00ffn de jmal\" ?\n\"v\u00df dem klain\" ?\n\"bayfasse kloster\" ?\n\nThis is challenging. Let's try to read the image more concretely using mental zoom.\n\nWe need return JSON with folio (maybe empty if not visible), text (main text body?), addition1, addition2, addition3 for marginal notes.\n\nWe need determine what is main text and what are additions. The prompt says: \"The manuscript may contain: 1. A main text body (often in one column as continuous text) 2. A folio number (if present) 3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)\"\n\nThus we need separate main continuous text and marginal notes. In this image, there might be one main text block on right/center, and left margin annotations. Let's identify.\n\nLooking at the image, there is a large block of text on the right side with about 15 lines. On the left side, there is smaller text running along left edge, maybe 10 lines. At bottom left, there are a few more lines. So likely main text is the larger block, and left margin is addition1, maybe bottom left addition2, etc.\n\nBut we need extract all text exactly as appears, maintaining line breaks with \\n. For main text, we should combine lines in order? Probably preserve line breaks as they appear in main block.\n\nLet's decode main block line by line precisely.\n\nI'll try to read each line from top to bottom, right to left? Actually text is left-to-right within lines.\n\nLine 1 (top): \"vnd dem erzbischoff do selbe\"\nBut there might be superscript. Let's examine: \"vnd\" then \"dem\" then \"erzbischof\" maybe with abbreviation \"erzbiscoff\"? The word looks like \"erzbischof\" but with special char? Actually \"erzbischof\" often abbreviated \"erzbischof\" with loop? Let's keep \"erzbischof\" or \"erzbiscoff\"? Let's read letters: e-r-z-b-i-s-c-h-o-f-f? The image shows \"erzbischof\" maybe with \"ff\" ligature? We'll write \"erzbischof\" but need preserve spelling exactly. Let's hold \"vnd dem erzbischof do selbe\". Actually \"do selbe\" might be \"do selbe\" meaning \"daselbst\"? But we'll write as seen.\n\nLine 2: \"vnd zcu frankenfurt vnd dz hertz\" maybe? Let's parse: \"vnd\" then \"zcu\" (zu) then \"franckenfurt\" or \"frankfurt\"? The letters: f-r-a-n-k-f-u-r-t? Maybe \"frankfurt\". Then \"vnd\" then \"dz\" (das) then \"hertz\"? Actually \"hertzog\"? Let's see: \"dz hertzog\"? The word might be \"hertzog\" abbreviated? We'll write \"vnd zcu frankfurt vnd dz hertzog\" but need check ending. Maybe \"dz hertz\" with superscript? Let's keep \"vnd zcu frankfurt vnd dz hertzog\" but not sure.\n\nLine 3: \"in dem thume do selbe Jch\" maybe \"in dem thume do selbe Jch\"? Actually \"thume\" is \"Thum\" (cathedral). \"do selbe\" again. \"Jch\" maybe \"Ich\". So \"in dem thume do selbe Jch\" but maybe \"in dem thume do selbe ich\" with capital I? We'll write \"in dem thume do selbe Jch\".\n\nLine 4: \"sant th\u00fftus houpt der do ge\" maybe \"sant thytus houpt der do ge\"? Let's parse: \"sant\" (Sankt) \"thytus\" (Titus) with y? \"houpt\" (Haupt) \"der\" \"do\" \"ge\" maybe part of \"geheilt\"? Actually line might continue. Let's write \"sant thytus houpt der do ge\".\n\nLine 5: \"kopp\u00ff singen enter wz Jch sat\" Hmm. Let's decode: \"kopp\u00ff\" maybe \"kopff\"? Actually \"kopff\" with y? \"singen\" \"enter\" \"wz\" (was?) \"Jch\" \"sat\" (sagte?). So \"kopffy singen enter wz Jch sat\". Not clear. We'll approximate.\n\nLine 6: \"Barbare eint k\u00fcnig do z\u00fcpern\" maybe \"Barbare eint k\u00fcnig do zypern\"? Actually \"Barbar\" is Barbara? \"eint\" \"k\u00fcnig\" \"do\" \"z\u00fcpern\" (Zypern?). So \"Barbare eint k\u00fcnig do zypern\". Let's write \"Barbare eint k\u00fcnig do zypern\".\n\nLine 7: \"tochter houpt Jch sant blasi\u00ff\" maybe \"tochter houpt Jch sant blasi\u00ff\". So \"tochter houpt Jch sant blasi\u00ff\". We'll keep.\n\nThen there might be more lines in main block? Let's check image: After line 7, there is a gap? Actually the left margin text starts around same height. The main block continues downward with more lines on right.\n\nLine 8: \"tochter houpt Jch sant blasi\u00ff\" is near middle. Then below:\n\"mein Jch sant andres ain\u00ff Jch\" maybe?\nLet's parse lower right lines.\n\nLooking at bottom right area:\nLine around middle-bottom: \"mein Jch sant andres ain\u00ff Jch\"\nThen: \"vnd sant steffanus ain\u00ff Jch vnd\" ?\nThen: \"zwen zen die do sant cipols\" ?\nThen: \"vnd Jch sant andres ain\u00ff Jch\" ?\nThen: \"vff Weintag vor sant Josabs\" ?\nThen: \"tag do zc Burgundien f\u00fcren wir\" ?\nThen: \"vnd f\u00fcnden ein f\u00fcrenn an J\" ?\nThen: \"vff feyertag wir f\u00fcren die Jch\" ?\nThen: \"vnd vns vnd zc vnbekantla\" ?\nThen: \"warten v\u00df harznen feyt do warnete sich die\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\nThen: \"vnd furcht vnd\" ?\n\nWait, that seems like repetition of \"vnd furcht vnd\" in left margin? Actually left margin might have repeated phrases.\n\nLet's try to", "model": "thinkingmachines/Inkling", "provider": "huggingface", "finish_reason": "length", "usage": {"input_tokens": 2568, "output_tokens": 2048, "total_tokens": 4616, "cached_tokens": 2304, "input_cost_usd": 0.002568, "output_cost_usd": 0.008294399999999999, "estimated_cost_usd": 0.010862399999999998}, "duration": 44.291160106658936, "timestamp": "2026-08-16T15:09:18.543541", "conversation_id": "3fefad3a-9cfc-4e69-962b-9f63d56a2f52", "score": null, "raw_response": "{\"id\":\"owRzJhu-31r7CA-a2c0abae1a0ce63b\",\"choices\":[{\"finish_reason\":\"length\",\"index\":0,\"logprobs\":null,\"message\":{\"content\":\"We need parse image of manuscript page, 15th c, Basel, late medieval German. Need extract text exactly, preserve line breaks, abbreviations, special chars. Need identify folio number, main text, marginal additions.\\n\\nWe need examine image carefully. Let's analyze layout.\\n\\nImage shows one page with handwritten text in brown ink, likely one column continuous but with some lines. There's text on left margin? Let's inspect.\\n\\nWe need read all visible text. Let's decode line by line.\\n\\nTop right? Actually page has text centered? Let's parse visible lines from top to bottom.\\n\\nStarting near top:\\nLine 1: \\\"vnd dem erzbischoff do selbe\\\" maybe? Let's examine precisely.\\n\\nWe need be careful with abbreviations and special characters like \ua75b, \ua770, superscript letters.\\n\\nLet's read the main block (center/right?). There's also left margin text running vertically? Let's understand layout: The page seems to have main text in larger script occupying right/center, and smaller text in left margin? Actually image shows text spread across page with some lines indented? Let's parse systematically.\\n\\nLooking at image:\\nTop line (centered/top): \\\"vnd dem erzbischoff do selbe\\\" maybe \\\"vnd dem erzbischof do selbe\\\"? Let's read characters.\\n\\nSecond line: \\\"vnd zcu frankenfurt vnd dz hertz\\\" maybe? Let's decode.\\n\\nThird line: \\\"in dem thume do selbe Jch\\\" maybe?\\n\\nFourth line: \\\"sant th\u00fftus houpt der do ge\\\" ?\\n\\nFifth line: \\\"kopp\u00ff singen enter wz Jch sat\\\" ?\\n\\nSixth line: \\\"Barbare eint k\u00fcnig do z\u00fcpern\\\" ?\\n\\nSeventh line: \\\"tochter houpt Jch sant blasi\u00ff\\\" ?\\n\\nThen left margin has smaller writing? Let's parse left side text:\\n\\\"vnd zcu genewa\\\" ?\\n\\\"vnd an zc k\\\" ?\\n\\\"die an der\\\" ?\\n\\\"vnd darvmb\\\" ?\\n\\\"wer in\\\" ?\\n\\\"dentail blaut\\\" ?\\n\\\"nach ent vnd\\\" ?\\n\\\"t\u00fcch die kl\\\" ?\\n\\\"ging vnd hab\\\" ?\\n\\\"vnd zcu min\\\" ?\\n\\\"vnd do v\u00df\\\" ?\\n\\\"vnd vns vnd\\\" ?\\n\\\"warten v\u00df\\\" ?\\n\\\"vnd furcht\\\" ?\\n\\\"vnd furcht\\\" ?\\n\\\"vnd furcht\\\" ?\\n\\nBut need be precise.\\n\\nAlso bottom left has more lines:\\n\\\"ge\u00ffn de jmal\\\" ?\\n\\\"v\u00df dem klain\\\" ?\\n\\\"bayfasse kloster\\\" ?\\n\\nThis is challenging. Let's try to read the image more concretely using mental zoom.\\n\\nWe need return JSON with folio (maybe empty if not visible), text (main text body?), addition1, addition2, addition3 for marginal notes.\\n\\nWe need determine what is main text and what are additions. The prompt says: \\\"The manuscript may contain: 1. A main text body (often in one column as continuous text) 2. A folio number (if present) 3. Additional notes or text written in the margins (labeled as addition1, addition2, etc.)\\\"\\n\\nThus we need separate main continuous text and marginal notes. In this image, there might be one main text block on right/center, and left margin annotations. Let's identify.\\n\\nLooking at the image, there is a large block of text on the right side with about 15 lines. On the left side, there is smaller text running along left edge, maybe 10 lines. At bottom left, there are a few more lines. So likely main text is the larger block, and left margin is addition1, maybe bottom left addition2, etc.\\n\\nBut we need extract all text exactly as appears, maintaining line breaks with \\\\n. For main text, we should combine lines in order? Probably preserve line breaks as they appear in main block.\\n\\nLet's decode main block line by line precisely.\\n\\nI'll try to read each line from top to bottom, right to left? Actually text is left-to-right within lines.\\n\\nLine 1 (top): \\\"vnd dem erzbischoff do selbe\\\"\\nBut there might be superscript. Let's examine: \\\"vnd\\\" then \\\"dem\\\" then \\\"erzbischof\\\" maybe with abbreviation \\\"erzbiscoff\\\"? The word looks like \\\"erzbischof\\\" but with special char? Actually \\\"erzbischof\\\" often abbreviated \\\"erzbischof\\\" with loop? Let's keep \\\"erzbischof\\\" or \\\"erzbiscoff\\\"? Let's read letters: e-r-z-b-i-s-c-h-o-f-f? The image shows \\\"erzbischof\\\" maybe with \\\"ff\\\" ligature? We'll write \\\"erzbischof\\\" but need preserve spelling exactly. Let's hold \\\"vnd dem erzbischof do selbe\\\". Actually \\\"do selbe\\\" might be \\\"do selbe\\\" meaning \\\"daselbst\\\"? But we'll write as seen.\\n\\nLine 2: \\\"vnd zcu frankenfurt vnd dz hertz\\\" maybe? Let's parse: \\\"vnd\\\" then \\\"zcu\\\" (zu) then \\\"franckenfurt\\\" or \\\"frankfurt\\\"? The letters: f-r-a-n-k-f-u-r-t? Maybe \\\"frankfurt\\\". Then \\\"vnd\\\" then \\\"dz\\\" (das) then \\\"hertz\\\"? Actually \\\"hertzog\\\"? Let's see: \\\"dz hertzog\\\"? The word might be \\\"hertzog\\\" abbreviated? We'll write \\\"vnd zcu frankfurt vnd dz hertzog\\\" but need check ending. Maybe \\\"dz hertz\\\" with superscript? Let's keep \\\"vnd zcu frankfurt vnd dz hertzog\\\" but not sure.\\n\\nLine 3: \\\"in dem thume do selbe Jch\\\" maybe \\\"in dem thume do selbe Jch\\\"? Actually \\\"thume\\\" is \\\"Thum\\\" (cathedral). \\\"do selbe\\\" again. \\\"Jch\\\" maybe \\\"Ich\\\". So \\\"in dem thume do selbe Jch\\\" but maybe \\\"in dem thume do selbe ich\\\" with capital I? We'll write \\\"in dem thume do selbe Jch\\\".\\n\\nLine 4: \\\"sant th\u00fftus houpt der do ge\\\" maybe \\\"sant thytus houpt der do ge\\\"? Let's parse: \\\"sant\\\" (Sankt) \\\"thytus\\\" (Titus) with y? \\\"houpt\\\" (Haupt) \\\"der\\\" \\\"do\\\" \\\"ge\\\" maybe part of \\\"geheilt\\\"? Actually line might continue. Let's write \\\"sant thytus houpt der do ge\\\".\\n\\nLine 5: \\\"kopp\u00ff singen enter wz Jch sat\\\" Hmm. Let's decode: \\\"kopp\u00ff\\\" maybe \\\"kopff\\\"? Actually \\\"kopff\\\" with y? \\\"singen\\\" \\\"enter\\\" \\\"wz\\\" (was?) \\\"Jch\\\" \\\"sat\\\" (sagte?). So \\\"kopffy singen enter wz Jch sat\\\". Not clear. We'll approximate.\\n\\nLine 6: \\\"Barbare eint k\u00fcnig do z\u00fcpern\\\" maybe \\\"Barbare eint k\u00fcnig do zypern\\\"? Actually \\\"Barbar\\\" is Barbara? \\\"eint\\\" \\\"k\u00fcnig\\\" \\\"do\\\" \\\"z\u00fcpern\\\" (Zypern?). So \\\"Barbare eint k\u00fcnig do zypern\\\". Let's write \\\"Barbare eint k\u00fcnig do zypern\\\".\\n\\nLine 7: \\\"tochter houpt Jch sant blasi\u00ff\\\" maybe \\\"tochter houpt Jch sant blasi\u00ff\\\". So \\\"tochter houpt Jch sant blasi\u00ff\\\". We'll keep.\\n\\nThen there might be more lines in main block? Let's check image: After line 7, there is a gap? Actually the left margin text starts around same height. The main block continues downward with more lines on right.\\n\\nLine 8: \\\"tochter houpt Jch sant blasi\u00ff\\\" is near middle. Then below:\\n\\\"mein Jch sant andres ain\u00ff Jch\\\" maybe?\\nLet's parse lower right lines.\\n\\nLooking at bottom right area:\\nLine around middle-bottom: \\\"mein Jch sant andres ain\u00ff Jch\\\"\\nThen: \\\"vnd sant steffanus ain\u00ff Jch vnd\\\" ?\\nThen: \\\"zwen zen die do sant cipols\\\" ?\\nThen: \\\"vnd Jch sant andres ain\u00ff Jch\\\" ?\\nThen: \\\"vff Weintag vor sant Josabs\\\" ?\\nThen: \\\"tag do zc Burgundien f\u00fcren wir\\\" ?\\nThen: \\\"vnd f\u00fcnden ein f\u00fcrenn an J\\\" ?\\nThen: \\\"vff feyertag wir f\u00fcren die Jch\\\" ?\\nThen: \\\"vnd vns vnd zc vnbekantla\\\" ?\\nThen: \\\"warten v\u00df harznen feyt do warnete sich die\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\nThen: \\\"vnd furcht vnd\\\" ?\\n\\nWait, that seems like repetition of \\\"vnd furcht vnd\\\" in left margin? Actually left margin might have repeated phrases.\\n\\nLet's try to\",\"refusal\":null,\"role\":\"assistant\",\"annotations\":null,\"audio\":null,\"function_call\":null,\"tool_calls\":[]},\"seed\":null}],\"created\":1786885891,\"model\":\"thinkingmachines/Inkling\",\"object\":\"chat.completion\",\"service_tier\":null,\"system_fingerprint\":\"default\",\"usage\":{\"completion_tokens\":2048,\"prompt_tokens\":2568,\"total_tokens\":4616,\"completion_tokens_details\":null,\"prompt_tokens_details\":{\"audio_tokens\":null,\"cached_tokens\":2304}},\"prompt\":[]}"}, "scoring": {"fuzzy": 0.619, "cer": 0.381, "cost_summary": {"total_input_tokens": 30816, "total_output_tokens": 24453, "total_tokens": 55269, "input_cost_usd": 0.030816000000000007, "output_cost_usd": 0.09903464999999997, "total_cost_usd": 0.12985064999999996}}, "normalized_score": 61.9}