{"_id": {"$oid": "69d4eb12b877594bc5718c8e"}, "test_id": "T0665", "benchmark": "general_meeting_minutes", "date": "2026-03-16", "tags": {"document-type": ["minutes"], "writing": ["typed", "handwritten"], "century": [20], "language": ["it", "fr", "de"], "layout": ["table"], "entry-type": ["person", "location"], "task": ["information-extraction"]}, "contributors": ["alexandra_binnenkade", "sven_lienhard", "sorin_marti"], "hidden": false, "config": {"provider": "openai", "model": "gpt-5.4-2026-03-05", "dataclass": "MinutesPage", "temperature": 1.0, "role_description": "You are a historian with expertise in analyzing and summarizing historical meeting minutes", "prompt_file": "extended_prompt.txt", "legacy_test": false}, "prompt": "Filename: {filename}\nPage: {page_number}\n\nPrompt 2\n\nCRITICAL RULES\n--------------\n\n1) Name and Address are in the same table field.\n   - Extract them into separate fields: \"name\" and \"address\".\n   - Remove only the visual splitting characters (e.g., a dash used only to visually separate name and address).\n   - Preserve line breaks exactly as in the source.\n   - Addresses may contain multiple dashes; preserve those. Only remove the dash(es) that act purely as a separator between name and address.\n\n2) Transcription fidelity (no normalization):\n   - Many entries are in French. The original often omits accents (\u00e9, \u00e8) and shows only \"e\".\n   - Do NOT correct spelling and do NOT add French diacritics unless they are explicitly present in the original image.\n   - Transcribe exactly what is visible, including punctuation and spacing as much as possible.\n\n3) totals_actions must NOT be computed:\n   - Do NOT calculate totals.\n   - Only fill \"total_actions\" if the totals are explicitly written on the page.\n   - You can always find the information for the totals at the end of the page after \u201cA Reporter\u201d\n   - If there is no totals row/section, still output the JSON but set total_actions fields to empty strings OR omit total_actions only if your parser requires it (prefer empty strings if the field is required).\n\n4) actions_o and actions_p content:\n   - These fields may contain digits as well as \"/\" and \"-\" characters. Preserve them exactly as seen.\n\n5) Column order may vary:\n   - The order of the \"actions_o\" and \"actions_p\" columns can change from page to page.\n   - Do NOT assume a fixed column order. Always verify from the header / column labels / layout in the original image which column corresponds to \"O\" and which to \"P\", and map values accordingly.", "results": {"text": "{\"document\":\"Se_23_Assembl\u00e9e_1951_Generalversammlung1_page_3\",\"page_number\":3,\"entries\":[{\"number\":\"21\",\"name\":\"M.de la FONTAINE Lionel\",\"address\":\"Boite postale 41 \u00e0 MARRAKECH-\\nMEDINA\",\"actions_o\":\"\",\"actions_p\":\"2\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"22\",\"name\":\"M.LAUMAIN Robert\",\"address\":\"24 Avenue Gambetta MONTROUGE\",\"actions_o\":\"\",\"actions_p\":\"1\",\"no_de_voix\":\"\",\"signature_present\":true,\"signature\":\"Ant. Paur\"},{\"number\":\"23\",\"name\":\"Suc.LHOEST Paul (Mme)\",\"address\":\"7,rue J.B.Colyns IXELLES-\\nBRUXELLES.\",\"actions_o\":\"150\",\"actions_p\":\"\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"24\",\"name\":\"M.MASSON Daniel\",\"address\":\"3,Av.Gaug\u00e9 VIROFLAY (S-et-O)\",\"actions_o\":\"\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":true,\"signature\":\"Ant. Paur\"},{\"number\":\"25\",\"name\":\"M. PANI Antonio\",\"address\":\"SESSA (Tessin) Suisse\",\"actions_o\":\"1\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":true,\"signature\":\"Ant. Paur\"},{\"number\":\"26\",\"name\":\"M. POUQUET Maurice\",\"address\":\"Essendi\u00e9ras,par St M\u00e9dard\\nd'Excisuil (Dordogne)\",\"actions_o\":\"\",\"actions_p\":\"1\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"27\",\"name\":\"Mme RAOULT Suzanne\",\"address\":\"195 rue de Grenelle PARIS\",\"actions_o\":\"\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"28\",\"name\":\"Suc.RICHOMME de POSSON (Mme)\",\"address\":\"37bis Promenade des Anglais\\nNICE (A-M)\",\"actions_o\":\"\",\"actions_p\":\"32\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"29\",\"name\":\"Mme SAVAEY M.L.\",\"address\":\"5,Avenue Weber GENEVE (Suisse)\",\"actions_o\":\"\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"30\",\"name\":\"M. SCHERESCHEVSKY Philippe\",\"address\":\"34,Bd Marbeau PARIS (16\u00b0)\",\"actions_o\":\"\",\"actions_p\":\"1\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"}],\"total_actions\":{\"total_o\":\"2553\",\"total_p\":\"133\",\"total_voix\":\"\"}}", "model": "gpt-5.4-2026-03-05", "provider": "openai", "finish_reason": "stop", "usage": {"input_tokens": 10546, "output_tokens": 582, "total_tokens": 11128, "cached_tokens": 5888, "input_cost_usd": 0.026365, "output_cost_usd": 0.008730000000000002, "estimated_cost_usd": 0.035095}, "duration": 9.305124998092651, "timestamp": "2026-03-16T17:23:07.469959", "parsed": {"document": "Se_23_Assembl\u00e9e_1951_Generalversammlung1_page_3", "page_number": 3, "entries": [{"number": "21", "name": "M.de la FONTAINE Lionel", "address": "Boite postale 41 \u00e0 MARRAKECH-\nMEDINA", "actions_o": "", "actions_p": "2", "no_de_voix": "", "signature_present": false, "signature": ""}, {"number": "22", "name": "M.LAUMAIN Robert", "address": "24 Avenue Gambetta MONTROUGE", "actions_o": "", "actions_p": "1", "no_de_voix": "", "signature_present": true, "signature": "Ant. Paur"}, {"number": "23", "name": "Suc.LHOEST Paul (Mme)", "address": "7,rue J.B.Colyns IXELLES-\nBRUXELLES.", "actions_o": "150", "actions_p": "", "no_de_voix": "", "signature_present": false, "signature": ""}, {"number": "24", "name": "M.MASSON Daniel", "address": "3,Av.Gaug\u00e9 VIROFLAY (S-et-O)", "actions_o": "", "actions_p": "10", "no_de_voix": "", "signature_present": true, "signature": "Ant. Paur"}, {"number": "25", "name": "M. PANI Antonio", "address": "SESSA (Tessin) Suisse", "actions_o": "1", "actions_p": "10", "no_de_voix": "", "signature_present": true, "signature": "Ant. Paur"}, {"number": "26", "name": "M. POUQUET Maurice", "address": "Essendi\u00e9ras,par St M\u00e9dard\nd'Excisuil (Dordogne)", "actions_o": "", "actions_p": "1", "no_de_voix": "", "signature_present": false, "signature": ""}, {"number": "27", "name": "Mme RAOULT Suzanne", "address": "195 rue de Grenelle PARIS", "actions_o": "", "actions_p": "10", "no_de_voix": "", "signature_present": false, "signature": ""}, {"number": "28", "name": "Suc.RICHOMME de POSSON (Mme)", "address": "37bis Promenade des Anglais\nNICE (A-M)", "actions_o": "", "actions_p": "32", "no_de_voix": "", "signature_present": false, "signature": ""}, {"number": "29", "name": "Mme SAVAEY M.L.", "address": "5,Avenue Weber GENEVE (Suisse)", "actions_o": "", "actions_p": "10", "no_de_voix": "", "signature_present": false, "signature": ""}, {"number": "30", "name": "M. SCHERESCHEVSKY Philippe", "address": "34,Bd Marbeau PARIS (16\u00b0)", "actions_o": "", "actions_p": "1", "no_de_voix": "", "signature_present": false, "signature": ""}], "total_actions": {"total_o": "2553", "total_p": "133", "total_voix": ""}}, "conversation_id": "0bf83ef3-15de-46c3-9580-313d7bfb9ed7", "score": {"fuzzy": 0.766493177078503}, "raw_response": "{\"id\":\"chatcmpl-DK50cQrGU52aW0gY50O2X0C1vF2Wo\",\"choices\":[{\"finish_reason\":\"stop\",\"index\":0,\"logprobs\":null,\"message\":{\"content\":\"{\\\"document\\\":\\\"Se_23_Assembl\u00e9e_1951_Generalversammlung1_page_3\\\",\\\"page_number\\\":3,\\\"entries\\\":[{\\\"number\\\":\\\"21\\\",\\\"name\\\":\\\"M.de la FONTAINE Lionel\\\",\\\"address\\\":\\\"Boite postale 41 \u00e0 MARRAKECH-\\\\nMEDINA\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"2\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"},{\\\"number\\\":\\\"22\\\",\\\"name\\\":\\\"M.LAUMAIN Robert\\\",\\\"address\\\":\\\"24 Avenue Gambetta MONTROUGE\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"1\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":true,\\\"signature\\\":\\\"Ant. Paur\\\"},{\\\"number\\\":\\\"23\\\",\\\"name\\\":\\\"Suc.LHOEST Paul (Mme)\\\",\\\"address\\\":\\\"7,rue J.B.Colyns IXELLES-\\\\nBRUXELLES.\\\",\\\"actions_o\\\":\\\"150\\\",\\\"actions_p\\\":\\\"\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"},{\\\"number\\\":\\\"24\\\",\\\"name\\\":\\\"M.MASSON Daniel\\\",\\\"address\\\":\\\"3,Av.Gaug\u00e9 VIROFLAY (S-et-O)\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"10\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":true,\\\"signature\\\":\\\"Ant. Paur\\\"},{\\\"number\\\":\\\"25\\\",\\\"name\\\":\\\"M. PANI Antonio\\\",\\\"address\\\":\\\"SESSA (Tessin) Suisse\\\",\\\"actions_o\\\":\\\"1\\\",\\\"actions_p\\\":\\\"10\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":true,\\\"signature\\\":\\\"Ant. Paur\\\"},{\\\"number\\\":\\\"26\\\",\\\"name\\\":\\\"M. POUQUET Maurice\\\",\\\"address\\\":\\\"Essendi\u00e9ras,par St M\u00e9dard\\\\nd'Excisuil (Dordogne)\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"1\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"},{\\\"number\\\":\\\"27\\\",\\\"name\\\":\\\"Mme RAOULT Suzanne\\\",\\\"address\\\":\\\"195 rue de Grenelle PARIS\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"10\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"},{\\\"number\\\":\\\"28\\\",\\\"name\\\":\\\"Suc.RICHOMME de POSSON (Mme)\\\",\\\"address\\\":\\\"37bis Promenade des Anglais\\\\nNICE (A-M)\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"32\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"},{\\\"number\\\":\\\"29\\\",\\\"name\\\":\\\"Mme SAVAEY M.L.\\\",\\\"address\\\":\\\"5,Avenue Weber GENEVE (Suisse)\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"10\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"},{\\\"number\\\":\\\"30\\\",\\\"name\\\":\\\"M. SCHERESCHEVSKY Philippe\\\",\\\"address\\\":\\\"34,Bd Marbeau PARIS (16\u00b0)\\\",\\\"actions_o\\\":\\\"\\\",\\\"actions_p\\\":\\\"1\\\",\\\"no_de_voix\\\":\\\"\\\",\\\"signature_present\\\":false,\\\"signature\\\":\\\"\\\"}],\\\"total_actions\\\":{\\\"total_o\\\":\\\"2553\\\",\\\"total_p\\\":\\\"133\\\",\\\"total_voix\\\":\\\"\\\"}}\",\"refusal\":null,\"role\":\"assistant\",\"annotations\":[],\"audio\":null,\"function_call\":null,\"tool_calls\":null,\"parsed\":{\"document\":\"Se_23_Assembl\u00e9e_1951_Generalversammlung1_page_3\",\"page_number\":3,\"entries\":[{\"number\":\"21\",\"name\":\"M.de la FONTAINE Lionel\",\"address\":\"Boite postale 41 \u00e0 MARRAKECH-\\nMEDINA\",\"actions_o\":\"\",\"actions_p\":\"2\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"22\",\"name\":\"M.LAUMAIN Robert\",\"address\":\"24 Avenue Gambetta MONTROUGE\",\"actions_o\":\"\",\"actions_p\":\"1\",\"no_de_voix\":\"\",\"signature_present\":true,\"signature\":\"Ant. Paur\"},{\"number\":\"23\",\"name\":\"Suc.LHOEST Paul (Mme)\",\"address\":\"7,rue J.B.Colyns IXELLES-\\nBRUXELLES.\",\"actions_o\":\"150\",\"actions_p\":\"\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"24\",\"name\":\"M.MASSON Daniel\",\"address\":\"3,Av.Gaug\u00e9 VIROFLAY (S-et-O)\",\"actions_o\":\"\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":true,\"signature\":\"Ant. Paur\"},{\"number\":\"25\",\"name\":\"M. PANI Antonio\",\"address\":\"SESSA (Tessin) Suisse\",\"actions_o\":\"1\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":true,\"signature\":\"Ant. Paur\"},{\"number\":\"26\",\"name\":\"M. POUQUET Maurice\",\"address\":\"Essendi\u00e9ras,par St M\u00e9dard\\nd'Excisuil (Dordogne)\",\"actions_o\":\"\",\"actions_p\":\"1\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"27\",\"name\":\"Mme RAOULT Suzanne\",\"address\":\"195 rue de Grenelle PARIS\",\"actions_o\":\"\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"28\",\"name\":\"Suc.RICHOMME de POSSON (Mme)\",\"address\":\"37bis Promenade des Anglais\\nNICE (A-M)\",\"actions_o\":\"\",\"actions_p\":\"32\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"29\",\"name\":\"Mme SAVAEY M.L.\",\"address\":\"5,Avenue Weber GENEVE (Suisse)\",\"actions_o\":\"\",\"actions_p\":\"10\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"},{\"number\":\"30\",\"name\":\"M. SCHERESCHEVSKY Philippe\",\"address\":\"34,Bd Marbeau PARIS (16\u00b0)\",\"actions_o\":\"\",\"actions_p\":\"1\",\"no_de_voix\":\"\",\"signature_present\":false,\"signature\":\"\"}],\"total_actions\":{\"total_o\":\"2553\",\"total_p\":\"133\",\"total_voix\":\"\"}}}}],\"created\":1773678186,\"model\":\"gpt-5.4-2026-03-05\",\"object\":\"chat.completion\",\"service_tier\":\"default\",\"system_fingerprint\":null,\"usage\":{\"completion_tokens\":582,\"prompt_tokens\":10546,\"total_tokens\":11128,\"completion_tokens_details\":{\"accepted_prediction_tokens\":0,\"audio_tokens\":0,\"reasoning_tokens\":0,\"rejected_prediction_tokens\":0},\"prompt_tokens_details\":{\"audio_tokens\":0,\"cached_tokens\":5888}}}"}, "scoring": {"fuzzy": 0.8863610342510287, "cost_summary": {"total_input_tokens": 65350, "total_output_tokens": 5070, "total_tokens": 70420, "input_cost_usd": 0.163375, "output_cost_usd": 0.07604999999999999, "total_cost_usd": 0.239425}}, "normalized_score": 88.63610342510287}