{"_id": {"$oid": "6a8456c1ef0a19c343227d64"}, "test_id": "T1583", "benchmark": "general_meeting_minutes", "date": "2026-08-16", "tags": {"document-type": ["minutes"], "writing": ["typed", "handwritten"], "century": [20], "language": ["it", "fr", "de"], "layout": ["table"], "entry-type": ["person", "location"], "task": ["information-extraction"]}, "contributors": ["alexandra_binnenkade", "sven_lienhard", "sorin_marti"], "hidden": false, "config": {"provider": "huggingface", "model": "meta-models/Muse-Glimmer-30B:together", "dataclass": "MinutesPage", "temperature": 0.0, "role_description": "You are a historian with expertise in analyzing and summarizing historical meeting minutes", "prompt_file": "prompt.txt", "legacy_test": false}, "prompt": "Please extract the metadata according to the given output format.\nName and Address are in the same table field. Your task is to extract them into separate fields.\nLose any dashes between name and address but preserve linebreaks.\nBe aware: Addresses may contain multiple dashes; preserve them, only remove \"visual splitting characters\" between name and address.\nFilename: {filename}\nPage: {page_number}", "results": {"text": "{\"document\":\"Se_18_Assembl\u00e9es_3_Bilanz1968_2_page_9\",\"page_number\":9,\"entries\":[{\"number\":\"1\",\"name\":\"Monsieur BREHAM Alain\",\"address\":\"45 bd Charcot - NEUILLY-S/SEINE (92)\\nFRANCE\",\"actions_o\":\"179\",\"actions_p\":\"520\",\"no_de_voix\":\"699\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"2\",\"name\":\"Monsieur BREHAM Marc\",\"address\":\"la Tuilerie - ST NOM LA BRETECHE\\nFRANCE\",\"actions_o\":\"179\",\"actions_p\":\"520\",\"no_de_voix\":\"699\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"3\",\"name\":\"Monsieur MASSON Daniel\",\"address\":\"Villa la Cava, av. de la Cava 06 -\\nCANNES\\nFRANCE\",\"actions_o\":\"10\",\"actions_p\":\"\",\"no_de_voix\":\"10\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"4\",\"name\":\"COMPAGNIE CENTRALE SICLI\",\"address\":\"2/4 rue Blaise Pascal\\n93 - LE BLANC-MESNIL\\nFRANCE\",\"actions_o\":\"168\",\"actions_p\":\"822\",\"no_de_voix\":\"990\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"5\",\"name\":\"SOCIETE DES MINES DE L'AURIERAS\",\"address\":\"11 bis rue Balzac PARIS 8e\\nFRANCE\",\"actions_o\":\"45\",\"actions_p\":\"355\",\"no_de_voix\":\"400\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"6\",\"name\":\"SOCIETE DE RECHERCHES MINIERES FRANCAISES\",\"address\":\"11 bis rue Balzac PARIS 8e\\nFRANCE\",\"actions_o\":\"10\",\"actions_p\":\"\",\"no_de_voix\":\"10\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"7\",\"name\":\"Monsieur PANI Antonio\",\"address\":\"SESSA (Tessin)\\nSUISSE\",\"actions_o\":\"10\",\"actions_p\":\"\",\"no_de_voix\":\"10\",\"signature_present\":false,\"signature\":\"\"}],\"total_actions\":{\"total_o\":\"\",\"total_p\":\"\",\"total_voix\":\"\"}}", "model": "meta-models/Muse-Glimmer-30B", "provider": "huggingface", "finish_reason": "stop", "usage": {"input_tokens": 4170, "output_tokens": 2460, "total_tokens": 6630, "input_cost_usd": 0.0014594999999999999, "output_cost_usd": 0.0036899999999999997, "estimated_cost_usd": 0.0051494999999999996}, "duration": 26.847087383270264, "timestamp": "2026-08-16T03:40:45.733331", "parsed": {"document": "Se_18_Assembl\u00e9es_3_Bilanz1968_2_page_9", "page_number": 9, "entries": [{"number": "1", "name": "Monsieur BREHAM Alain", "address": "45 bd Charcot - NEUILLY-S/SEINE (92)\nFRANCE", "actions_o": "179", "actions_p": "520", "no_de_voix": "699", "signature_present": false, "signature": ""}, {"number": "2", "name": "Monsieur BREHAM Marc", "address": "la Tuilerie - ST NOM LA BRETECHE\nFRANCE", "actions_o": "179", "actions_p": "520", "no_de_voix": "699", "signature_present": false, "signature": ""}, {"number": "3", "name": "Monsieur MASSON Daniel", "address": "Villa la Cava, av. de la Cava 06 -\nCANNES\nFRANCE", "actions_o": "10", "actions_p": "", "no_de_voix": "10", "signature_present": false, "signature": ""}, {"number": "4", "name": "COMPAGNIE CENTRALE SICLI", "address": "2/4 rue Blaise Pascal\n93 - LE BLANC-MESNIL\nFRANCE", "actions_o": "168", "actions_p": "822", "no_de_voix": "990", "signature_present": false, "signature": ""}, {"number": "5", "name": "SOCIETE DES MINES DE L'AURIERAS", "address": "11 bis rue Balzac PARIS 8e\nFRANCE", "actions_o": "45", "actions_p": "355", "no_de_voix": "400", "signature_present": false, "signature": ""}, {"number": "6", "name": "SOCIETE DE RECHERCHES MINIERES FRANCAISES", "address": "11 bis rue Balzac PARIS 8e\nFRANCE", "actions_o": "10", "actions_p": "", "no_de_voix": "10", "signature_present": false, "signature": ""}, {"number": "7", "name": "Monsieur PANI Antonio", "address": "SESSA (Tessin)\nSUISSE", "actions_o": "10", "actions_p": "", "no_de_voix": "10", "signature_present": false, "signature": ""}], "total_actions": {"total_o": "", "total_p": "", "total_voix": ""}}, "conversation_id": "2e5af94e-cba8-4112-b023-6619722d4fd3", "score": {"fuzzy": 0.9972747295639568}, "raw_response": "{\"id\":\"owNKiHY-fdNmn-a2bcbaf72bf1202a\",\"choices\":[{\"finish_reason\":\"stop\",\"index\":0,\"logprobs\":null,\"message\":{\"content\":\"{\\n  \\\"document\\\": \\\"Se_18_Assembl\u00e9es_3_Bilanz1968_2_page_9\\\",\\n  \\\"page_number\\\": 9,\\n  \\\"entries\\\": [\\n    {\\n      \\\"number\\\": \\\"1\\\",\\n      \\\"name\\\": \\\"Monsieur BREHAM Alain\\\",\\n      \\\"address\\\": \\\"45 bd Charcot - NEUILLY-S/SEINE (92)\\\\nFRANCE\\\",\\n      \\\"actions_o\\\": \\\"179\\\",\\n      \\\"actions_p\\\": \\\"520\\\",\\n      \\\"no_de_voix\\\": \\\"699\\\"\\n     ,\\n      \\\"signature_present\\\": false\\n    ,\\n    \\\"signature\\\": \\\"\\\"\\n    },\\n    {\\n      \\\"number\\\": \\\"2\\\",\\n      \\\"name\\\": \\\"Monsieur BREHAM Marc\\\",\\n      \\\"address\\\": \\\"la Tuilerie - ST NOM LA BRETECHE\\\\nFRANCE\\\",\\n      \\\"actions_o\\\": \\\"179\\\",\\n      \\\"actions_p\\\": \\\"520\\\",\\n      \\\"no_de_voix\\\": \\\"699\\\",\\n      \\\"signature_present\\\": false\\n    ,\\n      \\\"signature\\\": \\\"\\\"\\n    },\\n    {\\n      \\\"number\\\": \\\"3\\\",\\n      \\\"name\\\": \\\"Monsieur MASSON Daniel\\\",\\n      \\\"address\\\": \\\"Villa la Cava, av. de la Cava 06 -\\\\nCANNES\\\\nFRANCE\\\",\\n      \\\"actions_o\\\": \\\"10\\\",\\n      \\\"actions_p\\\": \\\"\\\",\\n      \\\"no_de_voix\\\": \\\"10\\\",\\n      \\\"signature_present\\\": false\\n    ,\\n      \\\"signature\\\": \\\"\\\"\\n    },\\n    {\\n      \\\"number\\\": \\\"4\\\",\\n      \\\"name\\\": \\\"COMPAGNIE CENTRALE SICLI\\\",\\n      \\\"address\\\": \\\"2/4 rue Blaise Pascal\\\\n93 - LE BLANC-MESNIL\\\\nFRANCE\\\",\\n      \\\"actions_o\\\": \\\"168\\\",\\n      \\\"actions_p\\\": \\\"822\\\",\\n      \\\"no_de_voix\\\": \\\"990\\\",\\n      \\\"signature_present\\\": false\\n    ,\\n      \\\"signature\\\": \\\"\\\"\\n    },\\n    {\\n      \\\"number\\\": \\\"5\\\",\\n      \\\"name\\\": \\\"SOCIETE DES MINES DE L'AURIERAS\\\",\\n      \\\"address\\\": \\\"11 bis rue Balzac PARIS 8e\\\\nFRANCE\\\",\\n      \\\"actions_o\\\": \\\"45\\\",\\n      \\\"actions_p\\\": \\\"355\\\",\\n      \\\"no_de_voix\\\": \\\"400\\\",\\n      \\\"signature_present\\\": false\\n    ,\\n      \\\"signature\\\": \\\"\\\"\\n    },\\n    {\\n      \\\"number\\\": \\\"6\\\",\\n      \\\"name\\\": \\\"SOCIETE DE RECHERCHES MINIERES FRANCAISES\\\",\\n      \\\"address\\\": \\\"11 bis rue Balzac PARIS 8e\\\\nFRANCE\\\",\\n      \\\"actions_o\\\": \\\"10\\\",\\n      \\\"actions_p\\\": \\\"\\\",\\n      \\\"no_de_voix\\\": \\\"10\\\",\\n      \\\"signature_present\\\": false\\n    ,\\n      \\\"signature\\\": \\\"\\\"\\n    },\\n    {\\n      \\\"number\\\": \\\"7\\\",\\n      \\\"name\\\": \\\"Monsieur PANI Antonio\\\",\\n      \\\"address\\\": \\\"SESSA (Tessin)\\\\nSUISSE\\\",\\n      \\\"actions_o\\\": \\\"10\\\",\\n      \\\"actions_p\\\": \\\"\\\",\\n      \\\"no_de_voix\\\": \\\"10\\\",\\n      \\\"signature_present\\\": false\\n    ,\\n      \\\"signature\\\": \\\"\\\"\\n    }\\n  ]\\n  ,\\n  \\\"total_actions\\\": {\\n    \\\"total_o\\\": \\\"\\\",\\n    \\\"total_p\\\": \\\"\\\",\\n    \\\"total_voix\\\": \\\"\\\"\\n  }\\n}\",\"refusal\":null,\"role\":\"assistant\",\"annotations\":null,\"audio\":null,\"function_call\":null,\"tool_calls\":null,\"parsed\":{\"document\":\"Se_18_Assembl\u00e9es_3_Bilanz1968_2_page_9\",\"page_number\":9,\"entries\":[{\"number\":\"1\",\"name\":\"Monsieur BREHAM Alain\",\"address\":\"45 bd Charcot - NEUILLY-S/SEINE (92)\\nFRANCE\",\"actions_o\":\"179\",\"actions_p\":\"520\",\"no_de_voix\":\"699\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"2\",\"name\":\"Monsieur BREHAM Marc\",\"address\":\"la Tuilerie - ST NOM LA BRETECHE\\nFRANCE\",\"actions_o\":\"179\",\"actions_p\":\"520\",\"no_de_voix\":\"699\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"3\",\"name\":\"Monsieur MASSON Daniel\",\"address\":\"Villa la Cava, av. de la Cava 06 -\\nCANNES\\nFRANCE\",\"actions_o\":\"10\",\"actions_p\":\"\",\"no_de_voix\":\"10\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"4\",\"name\":\"COMPAGNIE CENTRALE SICLI\",\"address\":\"2/4 rue Blaise Pascal\\n93 - LE BLANC-MESNIL\\nFRANCE\",\"actions_o\":\"168\",\"actions_p\":\"822\",\"no_de_voix\":\"990\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"5\",\"name\":\"SOCIETE DES MINES DE L'AURIERAS\",\"address\":\"11 bis rue Balzac PARIS 8e\\nFRANCE\",\"actions_o\":\"45\",\"actions_p\":\"355\",\"no_de_voix\":\"400\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"6\",\"name\":\"SOCIETE DE RECHERCHES MINIERES FRANCAISES\",\"address\":\"11 bis rue Balzac PARIS 8e\\nFRANCE\",\"actions_o\":\"10\",\"actions_p\":\"\",\"no_de_voix\":\"10\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"7\",\"name\":\"Monsieur PANI Antonio\",\"address\":\"SESSA (Tessin)\\nSUISSE\",\"actions_o\":\"10\",\"actions_p\":\"\",\"no_de_voix\":\"10\",\"signature_present\":false,\"signature\":\"\"}],\"total_actions\":{\"total_o\":\"\",\"total_p\":\"\",\"total_voix\":\"\"}},\"reasoning\":\"Please extract the metadata according to the given output format.\\nName and Address are in the same table field. Your task is to extract them into separate fields.\\nLose any dashes between name and address but preserve linebreaks.\\nBe aware: Addresses may contain multiple dashes; preserve them, only remove \\\"visual splitting characters\\\" between name and address.\\nFilename: Se_18_Assembl\u00e9es_3_Bilanz1968_2_page_9\\nPage: 9\\n\\nWe need extract metadata. Probably list each shareholder entry with N\u00b0 d'ordre, Name, Address, Actions O, Actions P, Nombre de voix, Signature? The output format? Not specified. Probably a table.\\n\\nMaybe output format is JSON? Could be CSV. The instruction: \\\"Please extract the metadata according to the given output format.\\\" But no format given. Might be standard: each row with fields: N\u00b0 d'ordre, Name, Address, Actions O, Actions P, Nombre de voix.\\n\\nWe need separate name and address. Lose any dashes between name and address but preserve linebreaks. Be aware addresses may contain multiple dashes; preserve them, only remove \\\"visual splitting characters\\\" between name and address.\\n\\nSo for entry 1:\\nName: Monsieur BREHAM Alain\\nAddress: 45 bd Charcot - NEUILLY-S/SEINE (92)\\nFRANCE\\n\\nBut note the address lines: \\\"45 bd Charcot - NEUILLY-S/SEINE (92)\\\" then \\\"FRANCE\\\". Should preserve linebreaks. So address is two lines.\\n\\nThe dash between name and address is visual splitting character. In the table, name is on one row, address on next row. So no dash to remove? Actually the name row is \\\"Monsieur BREHAM Alain\\\" and next row is \\\"45 bd Charcot - NEUILLY-S/SEINE (92)\\\". There's no dash between them in text, but maybe the table has a dash? The instruction says lose any dashes between name and address but preserve linebreaks. Might be that in the OCR the name and address are on same field with a dash separator. But here they are on separate rows. Hmm.\\n\\nMaybe the field \\\"NOM, PR\u00c9NOMS ET DOMICILE DES ACTIONNAIRES\\\" spans multiple rows per shareholder. So we need to combine rows into one entry: name is first line, address is subsequent lines until next N\u00b0.\\n\\nSo for 1:\\nName: Monsieur BREHAM Alain\\nAddress:\\n45 bd Charcot - NEUILLY-S/SEINE (92)\\nFRANCE\\n\\nFor 2:\\nName: Monsieur BREHAM Marc\\nAddress:\\nla Tuilerie - ST NOM LA BRETECHE\\nFRANCE\\n\\nFor 3:\\nName: Monsieur MASSON Daniel\\nAddress:\\nVilla la Cava, av. de la Cava 06 -\\nFRANCE\\nCANNES\\nWait the table shows:\\nVilla la Cava, av. de la Cava 06 -\\nFRANCE\\nCANNES\\nActually the row shows \\\"Villa la Cava, av. de la Cava 06 -\\\" on one line, and \\\"FRANCE CANNES\\\" on next? The image shows:\\nVilla la Cava, av. de la Cava 06 -\\nFRANCE CANNES\\nMaybe the address is:\\nVilla la Cava, av. de la Cava 06 -\\nCANNES\\nFRANCE? Hmm layout is weird. The column is split. Might be \\\"Villa la Cava, av. de la Cava 06 -\\\" on first line, then \\\"FRANCE\\\" on left and \\\"CANNES\\\" on right? Could be two columns? No.\\n\\nMaybe the address lines are:\\nVilla la Cava, av. de la Cava 06 -\\nCANNES\\nFRANCE\\n\\nBut the image shows \\\"FRANCE\\\" under the first line and \\\"CANNES\\\" under the same line? Let's assume address lines as printed:\\nVilla la Cava, av. de la Cava 06 -\\nFRANCE\\nCANNES\\n\\nBut that seems odd. Might be \\\"Villa la Cava, av. de la Cava 06 - CANNES\\\" and \\\"FRANCE\\\". The dash is part of address. The line break maybe after dash. Could be:\\nVilla la Cava, av. de la Cava 06 - CANNES\\nFRANCE\\n\\nThe table shows \\\"Villa la Cava, av. de la Cava 06 -\\\" on one row, and next row has \\\"FRANCE\\\" left aligned and \\\"CANNES\\\" right aligned under the same cell? Might be a formatting issue. Probably the address is:\\nVilla la Cava, av. de la Cava 06 - CANNES\\nFRANCE\\n\\nI'll assume the address lines are:\\nVilla la Cava, av. de la Cava 06 - CANNES\\nFRANCE\\n\\nBut the image shows FRANCE on left and CANNES on right on same row. Could be two lines merged. Safer to keep as printed lines: line1 \\\"Villa la Cava, av. de la Cava 06 -\\\", line2 \\\"FRANCE\\\", line3 \\\"CANNES\\\"? Hmm.\\n\\nMaybe better to extract exactly as text appears in the cell, preserving linebreaks. The cell for entry 3 contains three lines: \\\"Monsieur MASSON Daniel\\\", \\\"Villa la Cava, av. de la Cava 06 -\\\", \\\"FRANCE CANNES\\\". Wait the third line shows \\\"FRANCE\\\" left and \\\"CANNES\\\" right. Could be \\\"FRANCE CANNES\\\" on same line.\\n\\nLet's assume the address is:\\nVilla la Cava, av. de la Cava 06 - CANNES\\nFRANCE\\n\\nI'll output with linebreak.\\n\\nEntry 4:\\nName: COMPAGNIE CENTRALE SICLI\\nAddress:\\n2/4 rue Blaise Pascal\\n93 - LE BLANC-MESNIL\\nFRANCE\\n\\nEntry 5:\\nName: SOCIETE DES MINES DE L'AURIERAS\\nAddress:\\n11 bis rue Balzac PARIS 8e\\nFRANCE\\n\\nEntry 6:\\nName: SOCIETE DE RECHERCHES MINIERES FRAN-\\nCAISES\\nThe name is split across two lines: \\\"SOCIETE DE RECHERCHES MINIERES FRAN-\\\" and \\\"CAISES 11 bis rue Balzac PARIS 8e\\\". Wait the second line contains both name continuation and address start. The table shows:\\nSOCIETE DE RECHERCHES MINIERES FRAN-\\nCAISES 11 bis rue Balzac PARIS 8e\\nSo name is \\\"SOCIETE DE RECHERCHES MINIERES FRANCAISES\\\". The dash is a hyphenation. Should we keep? Probably name is SOCIETE DE RECHERCHES MINIERES FRANCAISES. The address is:\\n11 bis rue Balzac PARIS 8e\\nFRANCE\\n\\nEntry 7:\\nName: Monsieur PANI Antonio\\nAddress:\\nSESSA (Tessin)\\nSUISSE\\n\\nActions O, P, Nombre de voix:\\n1: O 179, P 520, Voix 699\\n2: O 179, P 520, Voix 699\\n3: O 10, P (blank), Voix 10\\n4: O 168, P 822, Voix 990\\n5: O 45, P 355, Voix 400\\n6: O 10, P (blank), Voix 10\\n7: O 10, P (blank), Voix 10\\n\\nWe need output format. Maybe a markdown table.\\n\\nI'll produce a list of entries with fields: N\u00b0 d'ordre, Name, Address, Actions O, Actions P, Nombre de voix.\\n\\nFilename and Page as metadata.\\n\\nProbably output as CSV.\\n\\nI'll provide a clear table.\\n\\nLet's produce:\\n\\nFilename: Se_18_Assembl\u00e9es_3_Bilanz1968_2_page_9\\nPage: 9\\n\\nEntries:\\n\\n1 | Monsieur BREHAM Alain | 45 bd Charcot - NEUILLY-S/SEINE (92)\\\\nFRANCE | 179 | 520 | 699\\netc.\\n\\nI'll output in markdown.\\n\\nLet's do that.\\n\\nEnsure dashes between name and address removed. There is no dash between name and address in our extraction, good.\\n\\nPreserve linebreaks in address using \\\\n.\\n\\nI'll present.\\n\\n\"},\"token_ids\":null,\"routed_experts\":null,\"seed\":null}],\"created\":1786844420,\"model\":\"meta-models/Muse-Glimmer-30B\",\"object\":\"chat.completion\",\"service_tier\":null,\"system_fingerprint\":\"vllm-0.1.dev19059+g66826a22d-tp2-46311a71\",\"usage\":{\"completion_tokens\":2460,\"prompt_tokens\":4170,\"total_tokens\":6630,\"completion_tokens_details\":null,\"prompt_tokens_details\":{\"audio_tokens\":null,\"cached_tokens\":0,\"created_cache_tokens\":4160,\"multimodal_tokens\":{\"image\":4034}}},\"prompt_token_ids\":null,\"prompt_text\":null,\"kv_transfer_params\":null,\"ec_transfer_params\":null,\"metrics\":null,\"prompt\":[]}"}, "scoring": {"fuzzy": 0.8586962536223648, "cost_summary": {"total_input_tokens": 36776, "total_output_tokens": 25641, "total_tokens": 62417, "input_cost_usd": 0.012871599999999999, "output_cost_usd": 0.0384615, "total_cost_usd": 0.051333100000000007}}, "normalized_score": 85.86962536223648}