{"_id": {"$oid": "6a8456c1ef0a19c343227d75"}, "test_id": "T1547", "benchmark": "blacklist_cards", "date": "2026-08-16", "tags": {"document-type": ["index-card"], "writing": ["typed", "handwritten"], "century": [20], "language": ["de"], "entry-type": ["company"], "task": ["information-extraction"]}, "contributors": ["lea_kasper", "sorin_marti"], "hidden": false, "config": {"provider": "huggingface", "model": "thinkingmachines/Inkling-Small:deepinfra", "dataclass": "Card", "temperature": 0.5, "role_description": "You are a Historian", "prompt_file": "prompt.txt", "legacy_test": false}, "prompt": "You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.\n\n- \"company\": The primary company name, usually in the top-left. Exclude the location.\n- \"location\": The city or town, often following the company name.\n- \"b_id\": The identifier code, usually in the top-right, starting with \"B.\".\n- \"date\": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.\n- \"information\": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\\\n.\n\nHere is the required JSON format:\n{\n  \"company\": {\"transcription\": \"\"},\n  \"location\": {\"transcription\": \"\"},\n  \"b_id\": {\"transcription\": \"\"},\n  \"date\": \"\",\n  \"information\": [\n    {\"transcription\": \"\"}\n  ]\n}\n\nIf you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.", "results": {"text": "{\"company\":{\"transcription\":\"Winterstein & Co., Ernst\"},\"location\":{\"transcription\":\"ZUERICH\"},\"b_id\":{\"transcription\":\"B.51.322.C3.452\"},\"date\":\"\",\"information\":[{\"transcription\":\"Britische schwarze Liste\\nAmendment 21, Dezember 1941.\"},{\"transcription\":\"gestrichen:\\nAmendment 13, 23.11.1945.\"}]}", "model": "thinkingmachines/Inkling-Small", "provider": "huggingface", "finish_reason": "stop", "usage": {"input_tokens": 2187, "output_tokens": 1271, "total_tokens": 3458, "cached_tokens": 256, "input_cost_usd": 0.00098415, "output_cost_usd": 0.0015252, "estimated_cost_usd": 0.00250935}, "duration": 10.945716381072998, "timestamp": "2026-08-16T00:42:07.777740", "parsed": {"company": {"transcription": "Winterstein & Co., Ernst"}, "location": {"transcription": "ZUERICH"}, "b_id": {"transcription": "B.51.322.C3.452"}, "date": "", "information": [{"transcription": "Britische schwarze Liste\nAmendment 21, Dezember 1941."}, {"transcription": "gestrichen:\nAmendment 13, 23.11.1945."}]}, "conversation_id": "a26f3c26-6f04-484d-b4e0-42152aaa2d58", "score": {"fuzzy": 0.9731182795698925}, "raw_response": "{\"id\":\"chatcmpl-RHjQbevR3TnECu46HKoWq6ht\",\"choices\":[{\"finish_reason\":\"stop\",\"index\":0,\"logprobs\":null,\"message\":{\"content\":\"{\\n\\n\\n  \\\"company\\\": {\\\"transcription\\\": \\\"Winterstein & Co., Ernst\\\"},\\n  \\\"location\\\": {\\\"transcription\\\": \\\"ZUERICH\\\"},\\n  \\\"b_id\\\": {\\\"transcription\\\": \\\"B.51.322.C3.452\\\"},\\n  \\\"date\\\": \\\"\\\",\\n  \\\"information\\\": [\\n    {\\\"transcription\\\": \\\"Britische schwarze Liste\\\\nAmendment 21, Dezember 1941.\\\"},\\n    {\\\"transcription\\\": \\\"gestrichen:\\\\nAmendment 13, 23.11.1945.\\\"}\\n  ]\\n}\",\"refusal\":null,\"role\":\"assistant\",\"annotations\":null,\"audio\":null,\"function_call\":null,\"tool_calls\":null,\"parsed\":{\"company\":{\"transcription\":\"Winterstein & Co., Ernst\"},\"location\":{\"transcription\":\"ZUERICH\"},\"b_id\":{\"transcription\":\"B.51.322.C3.452\"},\"date\":\"\",\"information\":[{\"transcription\":\"Britische schwarze Liste\\nAmendment 21, Dezember 1941.\"},{\"transcription\":\"gestrichen:\\nAmendment 13, 23.11.1945.\"}]},\"reasoning_content\":\"We need parse image text.\\nTop left: \\\"Winterstein & Co., Ernst, ZUERICH\\\" So company likely \\\"Winterstein & Co.\\\" location \\\"Ernst, ZUERICH\\\"? Wait location is city/town often following company name. Could be \\\"Ernst\\\" is location? But ZUERICH is also location. The text: \\\"Winterstein & Co., Ernst, ZUERICH\\\". Might be company Winterstein & Co., location Ernst, Zurich? Or company Winterstein & Co. Ernst? Let's think: Usually index cards have company name and location. Here \\\"Winterstein & Co., Ernst, ZUERICH\\\" suggests company Winterstein & Co., location Ernst, Zurich? But Ernst might be a town near Zurich? Actually \\\"Ernst\\\" could be part of company name? Let's examine: \\\"Winterstein & Co., Ernst, ZUERICH\\\". Could be \\\"Winterstein & Co.\\\" company, \\\"Ernst\\\" is maybe a person's name? But location field: city or town, often following company name. So location could be \\\"ZUERICH\\\". But \\\"Ernst\\\" is inserted between. Maybe company is \\\"Winterstein & Co. Ernst\\\"? No.\\nLet's break down: The line is \\\"Winterstein & Co., Ernst, ZUERICH\\\". Could mean company: Winterstein & Co., location: Ernst, ZUERICH? That seems odd. Maybe company: Winterstein & Co., Ernst; location: ZUERICH. That is, \\\"Ernst\\\" is part of company name: \\\"Winterstein & Co. Ernst\\\". But the comma after Co. suggests \\\"Winterstein & Co.\\\" then \\\"Ernst\\\" maybe a branch? Let's look at b_id: B.51.322.C3.452.\\nInformation blocks:\\n\\\"Britische schwarze Liste\\\\nAmendment 21, Dezember 1941.\\\"\\n\\\"gestrichen:\\\\nAmendment 13, 23.11.1945.\\\"\\nThese are two blocks.\\nDate: Any stamped dates on card in YYYY-MM-DD. There is \\\"21, Dezember 1941\\\" and \\\"23.11.1945\\\". But are these stamped dates? They are text content, not stamp. The instruction: \\\"Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.\\\" Likely no stamped date visible; the dates are part of information. So date: \\\"\\\".\\nNow company transcription: \\\"Winterstein & Co.\\\"? Or \\\"Winterstein & Co., Ernst\\\"? Let's decide. The primary company name, usually in top-left. Exclude location. So exclude ZUERICH. What about Ernst? Is Ernst location or part of company? Could be company \\\"Winterstein & Co.\\\" and location \\\"Ernst, ZUERICH\\\". But location should be city or town. Ernst is a town? There is Ernst, Switzerland? Not sure. But ZUERICH is definitely a city. So location: \\\"ZUERICH\\\". Then what is Ernst? Could be part of company name: \\\"Winterstein & Co. Ernst\\\". Actually the text \\\"Winterstein & Co., Ernst, ZUERICH\\\" might be formatted as company name \\\"Winterstein & Co.\\\" and below or same line location \\\"Ernst, ZUERICH\\\". But the instruction says location is city or town, often following company name. So likely location is \\\"Ernst, ZUERICH\\\" or \\\"ZUERICH\\\". Let's choose location: \\\"ZUERICH\\\". But to be safe, maybe \\\"Ernst, ZUERICH\\\". However \\\"Exclude the location\\\" from company means company is \\\"Winterstein & Co.\\\" and location is \\\"Ernst, ZUERICH\\\". But if Ernst is also location, then company is just Winterstein & Co. Let's set company: \\\"Winterstein & Co.\\\", location: \\\"Ernst, ZUERICH\\\". Wait but location field is city or town, often following company name. It could be \\\"Ernst\\\" is town and \\\"ZUERICH\\\" is city? Actually Zurich is city; Ernst might be a district? Let's provide location as \\\"Ernst, ZUERICH\\\" to capture both. But maybe better: location: \\\"ZUERICH\\\". Let's inspect spacing: The text is underlined with dots: \\\"Winterstein & Co., Ernst, ZUERICH\\\". The commas separate. In many card indexes, format is \\\"Company, City\\\". So \\\"Winterstein & Co.\\\" is company, \\\"Ernst\\\" might be a second company? No. Could be \\\"Winterstein & Co., Ernst\\\" is company name (like Winterstein & Co. Ernst), and ZUERICH is location. That makes sense: Company: Winterstein & Co. Ernst; Location: ZUERICH. Because \\\"Ernst\\\" could be a surname of partner. So company includes Ernst. Let's choose company: \\\"Winterstein & Co., Ernst\\\". But the instruction: \\\"The primary company name, usually in the top-left. Exclude the location.\\\" So exclude ZUERICH. That leaves \\\"Winterstein & Co., Ernst\\\". Yes, that seems right: Company name is \\\"Winterstein & Co., Ernst\\\". Location is \\\"ZUERICH\\\". Let's go with that.\\nThus:\\ncompany transcription: \\\"Winterstein & Co., Ernst\\\"\\nlocation transcription: \\\"ZUERICH\\\"\\nb_id: \\\"B.51.322.C3.452\\\"\\ndate: \\\"\\\"\\ninformation: [\\n \\\"Britische schwarze Liste\\\\nAmendment 21, Dezember 1941.\\\",\\n \\\"gestrichen:\\\\nAmendment 13, 23.11.1945.\\\"\\n]\\nMake sure to use \\\\\\\\n in JSON strings.\\nLet's craft final JSON.\",\"name\":null}}],\"created\":1786833727,\"model\":\"thinkingmachines/Inkling-Small\",\"object\":\"chat.completion\",\"service_tier\":\"default\",\"system_fingerprint\":null,\"usage\":{\"completion_tokens\":1271,\"prompt_tokens\":2187,\"total_tokens\":3458,\"completion_tokens_details\":null,\"prompt_tokens_details\":{\"audio_tokens\":null,\"cached_tokens\":256,\"cache_write_tokens\":null},\"estimated_cost\":0.0024197499997440004,\"reasoning_tokens\":1158}}"}, "scoring": {"fuzzy": 0.9225154800045379, "cost_summary": {"total_input_tokens": 72795, "total_output_tokens": 19542, "total_tokens": 92337, "input_cost_usd": 0.032757749999999995, "output_cost_usd": 0.0234504, "total_cost_usd": 0.05620814999999999}}, "normalized_score": 92.25154800045378}