{"_id": {"$oid": "6a2c86a59ede3a3eff37d434"}, "test_id": "T1057", "benchmark": "blacklist_cards", "date": "2026-05-22", "tags": {"document-type": ["index-card"], "writing": ["typed", "handwritten"], "century": [20], "language": ["de"], "entry-type": ["company"], "task": ["information-extraction"]}, "contributors": ["lea_kasper", "sorin_marti"], "hidden": false, "config": {"provider": "genai", "model": "gemini-3.5-flash", "dataclass": "Card", "temperature": 0.5, "role_description": "You are a Historian", "prompt_file": "prompt.txt", "legacy_test": false}, "prompt": "You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.\n\n- \"company\": The primary company name, usually in the top-left. Exclude the location.\n- \"location\": The city or town, often following the company name.\n- \"b_id\": The identifier code, usually in the top-right, starting with \"B.\".\n- \"date\": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.\n- \"information\": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\\\n.\n\nHere is the required JSON format:\n{\n  \"company\": {\"transcription\": \"\"},\n  \"location\": {\"transcription\": \"\"},\n  \"b_id\": {\"transcription\": \"\"},\n  \"date\": \"\",\n  \"information\": [\n    {\"transcription\": \"\"}\n  ]\n}\n\nIf you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.", "results": {"text": "{\"company\":{\"transcription\":\"Skiff A.G.,\"},\"location\":{\"transcription\":\"L u z e r n .\"},\"b_id\":{\"transcription\":\"B.51.322.GB.976.\"},\"date\":null,\"information\":[{\"transcription\":\"Britische schwarze Liste,\\nAmendment 5, April 1943.\"},{\"transcription\":\"gestrichen:\\nAmendment 3, M\u00e4tz 1944.\"}]}", "model": "gemini-3.5-flash", "provider": "genai", "finish_reason": "STOP", "usage": {"input_tokens": 1363, "output_tokens": 100, "total_tokens": 2185, "input_cost_usd": 0.0020445000000000003, "output_cost_usd": 0.0009000000000000001, "estimated_cost_usd": 0.0029445000000000005}, "duration": 4.967107057571411, "timestamp": "2026-05-22T09:24:04.594639", "parsed": {"company": {"transcription": "Skiff A.G.,"}, "location": {"transcription": "L u z e r n ."}, "b_id": {"transcription": "B.51.322.GB.976."}, "date": null, "information": [{"transcription": "Britische schwarze Liste,\nAmendment 5, April 1943."}, {"transcription": "gestrichen:\nAmendment 3, M\u00e4tz 1944."}]}, "conversation_id": "f4be318d-9fe6-4353-9fe2-15cc1123db12", "score": {"fuzzy": 0.9258980785296574}, "raw_response": "{\"sdk_http_response\":{\"headers\":{\"x-gemini-service-tier\":\"standard\",\"content-type\":\"application/json; charset=UTF-8\",\"vary\":\"Origin, X-Origin, Referer\",\"content-encoding\":\"gzip\",\"date\":\"Fri, 22 May 2026 07:24:04 GMT\",\"server\":\"scaffolding on HTTPServer2\",\"x-xss-protection\":\"0\",\"x-frame-options\":\"SAMEORIGIN\",\"x-content-type-options\":\"nosniff\",\"server-timing\":\"gfet4t7; dur=4939\",\"alt-svc\":\"h3=\\\":443\\\"; ma=2592000,h3-29=\\\":443\\\"; ma=2592000\",\"transfer-encoding\":\"chunked\"},\"body\":null},\"candidates\":[{\"content\":{\"parts\":[{\"media_resolution\":null,\"code_execution_result\":null,\"executable_code\":null,\"file_data\":null,\"function_call\":null,\"function_response\":null,\"inline_data\":null,\"text\":\"{\\\"company\\\":{\\\"transcription\\\":\\\"Skiff A.G.,\\\"},\\\"location\\\":{\\\"transcription\\\":\\\"L u z e r n .\\\"},\\\"b_id\\\":{\\\"transcription\\\":\\\"B.51.322.GB.976.\\\"},\\\"date\\\":null,\\\"information\\\":[{\\\"transcription\\\":\\\"Britische schwarze Liste,\\\\nAmendment 5, April 1943.\\\"},{\\\"transcription\\\":\\\"gestrichen:\\\\nAmendment 3, M\u00e4tz 1944.\\\"}]}\",\"thought\":null,\"thought_signature\":\"Es8QCswQAQw51scUjC2YnTpBe5p-XD68oywKfEbVFA1N6v5d351e9CBYirN4Yjd-zegJLYaO87kIyE4jEGdisAvCixwEx32AvmZ-zCShgIugKGxnpZV_COZw841CzCP6j8BjVWqu0-EKnpGgy7JPt14EzRMAXlHTFfDkFRQpWzk_2AgBKV-9vnfrRfsU27AoHKtFqvzJ9yRm86uLRSuh80ywoYWT1HWvu4GquX_CGJ_xM3Gi40_8D-3gPy438r-LnSxx1iDu9VdUyAun9zDqqaOK6Cdlq3XF571hRmsnjmrHjnhhWUu-j6fksPgD0oUxLveZE_LkcTaI0q1M0G7JRf1fe5bty1D2PcwQuCRx9BLIpBTdVdcpsP_nIVgTaTKoKbOPRbqOK7oYUfMYO4VBJrSfWv4mPi-7Q2ZavSqW-D4beqJhenHKifeH0f__-iYaum319Ui8oFNZoLZkFXTuZjv7OSEmn_s6TIEIDYtYGCHwyNBVH_f3tVk_P8yhZ5DImF8mkD0098bo4SzIf8Z8nCisETaje7arY4VwvlHRJA4v0zE_obqfrzFp6nc745B25rasMv8_Ih8TAeCEfyBzMddyvMLXTczRgcwBQSeRBeE6Rsa-CjwI3nUzF26DNXbs-ElXg5WOy68lhSaQmsh5UHXHlcNrJtb8tNLFzQ3b_qIoNFKMTXH2wTq7peG7h4Qnt-N0vk9aic_lfncZkUYCk7boj_4zqTxBvrw_TmOOZyOCgXViSamB987s2yf_M6FXvCyj1vNr-v2zXckfTUWKIjQU5T6u-qorIrgm2VmIEIdtpjknrhWPZHd7BpaJUHrkVbWxQxoCAjPUk1od2-_Cb0INsA9Ls5swT2CglBpexySwfMxes1nO4ht-ONIWMo4deTtYWxH1vrJj4SI40i-hR57IOPua8uZH9yikQkSQN4iVlZuBS6UEDbzlrONOuxAbZ10pSJ7VcKr0zPre37aPTnyavh7VAqkix7T-zLeEsVfMoX5p5TTG5erDFGp_QZ5LjQRIk2rjBG15rHpK7_3TqmjdcebyENJitZl5SwRksKN61ri908VA8OGIygfbagyhihJ0TRA-AdTGDEZ_B4oWvlECkjtdY_u25dkqKlP6BG-7F0KloiITgmRPJubkSblYUHTmdc7Ay5_CJiO7dkBFYLZu9MfmZf78OMmEU1qSWde1ExnEXjYs0ZPy_eKbvMMH06ksOdcN2mvUo0N8Eg4qUlJQ6dKh0SaoUiYze-Ip47_a3jCg6_TjDtnwtz0rxBv8lKD03cARyKxmG48ZlpW2Ld_DsNBUbrFPRhRzT8jjq6To7523bBgv-iT77gBm_0GPU5fQ9zsJH6eLj4sUVxWrHnBH_RkNdg30Lo1llEWKi5CBM3E-kGOwbtK1snLMxwnycTElHjzH9b6R7xV62OFjbXchXtR9a5ons4cAONr-hLMVRIYm5XATEw8J66vNs3oiZINnuyrfxHM-ljMnLvvOlpQnCLrFTatK5aL-38NcwPIA_oAP6ioZaQdG_HvridV9iVgyCX3uZdfft3XfcdJW7yRuVx-7lbhulprGVbkXokfNuv5ASFW7NBWQ93Jp5Cf-PGSYvYhpvMdWfJPEN3vjBOWR6ZhKdrrTxcrfQiAk5Y9pgMpJRm3epLVM4ea0Ef7CottMaHj62PETB5YvxtgXeAee-DRd6J2dstU9VKtyXNVCpZMV4yRSKhgid6BgfneF-qUVQdAPxcFNRthlathGgZI1b5TTgdG4m8CkfREqvTVORbK2ZCZ8TXlXr5ZoFGwUB-tUhfiDeZsUnftFBg6VZ4Ew-gfZXsX_IXWgjCzxd_P8rm04nCNH7qX8ZHG9vh1FcASKjzOuJitvARfOcT5rqFqlS5b8ZvjMl5CsONGGzy3i4oSpP3cUS5G0ubU9ZLLoeXctsbt4srhjGswc791TdyrqGnF7figdFtSm0LvfHMwpA_9MsZ93eimaVfhG-i9NknHnD8sPAsVz0kIaffxyDUBNSwe3-jaP7NDT-D2TIS-KbC8KR7lMxzjMnMPNFFMMpD1omeGYea6te9HllZ41hCVEBg6xC561GtRmTLlk_gh5sP_T0F_0SSacfruagrHpLzjmkqUC2biFIuhfhQEkO9QJuuE8OoQWGWFTgH3rr4DU8M_ENexievwi5lIWHhMWiKtfTdOMg_9DvRgrvvCzERGzPPnsdG-QVHA1hHRo0-ZMbudxIFVfsI7Cw_E92YbQvpxRJvjJ4IhkoSzlJcjaJDc3ZKj4dhucPzoDgCxfPzMeB3HKnHpCbxCNjBUP3s8WnQyDngsHlyjTyiLKqVA6V3crNouYqwWLisq3PfGAnMX1TWa7D1EccAucoy3WKwff4rook1TUpoMou4Ce51AF2gIt6YCw3VE-NVDK3EpHHey6C5AmSxN3nKhmQIl4rnXrXUfA99y7Nq443HPzYavHNdjq8cbbC8Blhh_bTouvdGZBm68INqUb0_DOTd47uDUt368w_qJjM1JVV6vg5ZfDb4fnMTALnk81bDYNjNXa5IaliMs_HExICBywjylaNS7hSeC1JT1EWjz6EnkC3wWoLp9j6QNCxoD0UtODRlXlfc2wuhMELmJiO-EY1RrMvI8BZ0UIU0ytDOSyCZbYRqs6Y_9o5luGXNswoyf4SGiRBRuZyfrPGz_ibEUChPEeQelf8EmE4zXrYBP8a75XmmAV2imNT1VEOk-2RIyBMcS_WwgDSuq7CSh6nalILY0pk58Ai0J3trw_yjhy4H6fmQXnI0RMdq3SWbolR2mbQbUv6wXFromAtCKR41PxGLNFiYMyJr0gOqQhtGCElMYHq66h64m9\",\"video_metadata\":null}],\"role\":\"model\"},\"citation_metadata\":null,\"finish_message\":null,\"token_count\":null,\"finish_reason\":\"STOP\",\"grounding_metadata\":null,\"avg_logprobs\":null,\"index\":0,\"logprobs_result\":null,\"safety_ratings\":null,\"url_context_metadata\":null}],\"create_time\":null,\"model_version\":\"gemini-3.5-flash\",\"prompt_feedback\":null,\"response_id\":\"jwQQat6_K9GKkdUPucOAuQw\",\"usage_metadata\":{\"cache_tokens_details\":null,\"cached_content_token_count\":null,\"candidates_token_count\":100,\"candidates_tokens_details\":null,\"prompt_token_count\":1363,\"prompt_tokens_details\":[{\"modality\":\"IMAGE\",\"token_count\":1092},{\"modality\":\"TEXT\",\"token_count\":271}],\"thoughts_token_count\":722,\"tool_use_prompt_token_count\":null,\"tool_use_prompt_tokens_details\":null,\"total_token_count\":2185,\"traffic_type\":null},\"automatic_function_calling_history\":[],\"parsed\":{\"company\":{\"transcription\":\"Skiff A.G.,\"},\"location\":{\"transcription\":\"L u z e r n .\"},\"b_id\":{\"transcription\":\"B.51.322.GB.976.\"},\"date\":null,\"information\":[{\"transcription\":\"Britische schwarze Liste,\\nAmendment 5, April 1943.\"},{\"transcription\":\"gestrichen:\\nAmendment 3, M\u00e4tz 1944.\"}]}}"}, "scoring": {"fuzzy": 0.9523885397323252, "cost_summary": {"total_input_tokens": 44967, "total_output_tokens": 5294, "total_tokens": 50261, "input_cost_usd": 0.0674505, "output_cost_usd": 0.04764600000000001, "total_cost_usd": 0.11509650000000005}}, "normalized_score": 95.23885397323252}