{"_id": {"$oid": "6ac4c7b5164e0f17c0b23533"}, "test_id": "T1622", "benchmark": "blacklist_cards", "date": "2026-08-18", "tags": {"document-type": ["index-card"], "writing": ["typed", "handwritten"], "century": [20], "language": ["de"], "entry-type": ["company"], "task": ["information-extraction"]}, "contributors": ["lea_kasper", "sorin_marti"], "hidden": false, "config": {"provider": "openrouter", "model": "meta/muse-spark-1.2", "dataclass": "Card", "temperature": 0.5, "role_description": "You are a Historian", "prompt_file": "prompt.txt", "legacy_test": false}, "prompt": "You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.\n\n- \"company\": The primary company name, usually in the top-left. Exclude the location.\n- \"location\": The city or town, often following the company name.\n- \"b_id\": The identifier code, usually in the top-right, starting with \"B.\".\n- \"date\": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.\n- \"information\": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\\\n.\n\nHere is the required JSON format:\n{\n  \"company\": {\"transcription\": \"\"},\n  \"location\": {\"transcription\": \"\"},\n  \"b_id\": {\"transcription\": \"\"},\n  \"date\": \"\",\n  \"information\": [\n    {\"transcription\": \"\"}\n  ]\n}\n\nIf you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.", "results": {"text": "{\"company\":{\"transcription\":\"Abegg & Cie.\"},\"location\":{\"transcription\":\"Z\\u0000rich\"},\"b_id\":{\"transcription\":\"B.51.322.GB.266\"},\"date\":\"\",\"information\":[]}", "model": "meta/muse-spark-1.2", "provider": "openrouter", "finish_reason": "stop", "usage": {"input_tokens": 3202, "output_tokens": 761, "total_tokens": 3963, "cached_tokens": 241, "input_cost_usd": 0.0040025, "output_cost_usd": 0.0032342499999999997, "estimated_cost_usd": 0.00723675}, "duration": 12.960479021072388, "timestamp": "2026-08-18T14:21:25.832105", "parsed": {"company": {"transcription": "Abegg & Cie."}, "location": {"transcription": "Z\u0000rich"}, "b_id": {"transcription": "B.51.322.GB.266"}, "date": "", "information": []}, "conversation_id": "9c2f1aea-2dca-4466-aec1-129d9ac93518", "score": {"fuzzy": 0.7666666666666667}, "raw_response": "{\"id\":\"gen-1787055673-MiK7zMVgPztHZlrXRDez\",\"choices\":[{\"finish_reason\":\"stop\",\"index\":0,\"logprobs\":null,\"message\":{\"content\":\"{\\n  \\\"b_id\\\": {\\n    \\\"transcription\\\": \\\"B.51.322.GB.266\\\"\\n  },\\n  \\\"company\\\": {\\n    \\\"transcription\\\": \\\"Abegg & Cie.\\\"\\n  },\\n  \\\"date\\\": \\\"\\\",\\n  \\\"information\\\": [],\\n  \\\"location\\\": {\\n    \\\"transcription\\\": \\\"Z\\\\u0000rich\\\"\\n  }\\n}\",\"refusal\":null,\"role\":\"assistant\",\"annotations\":null,\"audio\":null,\"function_call\":null,\"tool_calls\":null,\"parsed\":{\"company\":{\"transcription\":\"Abegg & Cie.\"},\"location\":{\"transcription\":\"Z\\u0000rich\"},\"b_id\":{\"transcription\":\"B.51.322.GB.266\"},\"date\":\"\",\"information\":[]},\"reasoning\":null,\"reasoning_details\":[{\"type\":\"reasoning.encrypted\",\"data\":\"Q-PaDgEI6qDCO5091d-j9_yiVNWdRU4xKjJyUTSdMTAnu-qqFnHHApvP3K8C-7JK0ooInnoPFfLiVnd4UIInqWRgfiQncc7CBsfv9AakZbmORwwpJ8_3OU9NbRCuzbnhOCj7ftxRfHlJuzyq9iP7pkqwBkyoGmwJ9k2hHaKMG803OU5iB3o9UOXC7HxPVSiXJZipm32FNfpuil0FRA_t9Nt8TojMF5Vwo1ExEMjZD1YM_c-GQi-l9l_CHYmP5FCdsouSvAZV8DRSTS4risv8Nt5VMLIh4oX6hIkeHonRtDzz0wgjaXh0AhH0NizYPNr991K-kIC8TRtgTLU7myv1Ha1HHOp7Cp5HFC2IKeB34za8X-n2CA0JPRzrlIYVdaNNRJxpxUNAnWjrazC3MLQX0FI9J9LGnkHw5ZyGXzrNWoMaI3_3xB2FdCkHruVTYfCTw6t6L-4oLBcub0eeCamxPjRdHtlqWtP_Coc1SryuwgL0i0spa14EgHic79KFhEzqfPBNl4VFLpaOziXvRggR1LYoeDlJ1coOAybxHi7U2AbuWVeu3VuufQuHLjk7AqmSrzRuqROvFl3utuyaHt-VdyYRGzgLQ7G44pck98LNbPcpw55tHvsn8iTZcLDqkUqN7JENNfsgaMJ0TevDRgbCIGQSRlx6c1UQSNuY10c4Van7y1sfDjCgh_rRgDe70EnlHe3awv58BHJlcuzmJRSTlStoVW1TzPFLTjS_yvEZM2-f5cdPkYr_4W7IVVdXW6gZUy-U9h6qNWDYzk3UWqpAuEgXeyYUD39n33THhOyXK4Jl9MVFluiasEFuOWyA1b8hzycxqXL2iKlMKLDFNGoeDpQADTtdMW3va-FYiZIB_Gf1i9OKzGvHYswQr0-Sz6A8J_rvYb4-9aB9-HqtdMpWgoXRN_S9mNveJR2EHdLhUvyNmpaMH4x_7fAxhTNacMFgG1vueioRPREa2_0_Jz7vyvS7plK4Aue6nnC9wyKYpTHu9k6cjXqYYvC8yXiaV8VtKpoSPVHHkxBj5fAcl3yKyUvZIlj_baGksqETfdHV5kj5BQ_OUQ0tdC2z_eJRTl7RRdXVHnnEqdlRC6yu4MX0crZC53L5gV6ecg-_JNePnZFWZkBCgborQAL_IzBEKAw58hTKYN9D5d-mPVEhHi5S7t4Necq-UpLHhZ8co9zincAlqPYph46zzFAKmzmHwcdDjpBEqEEdhjSu-TFFogdmBsGuE9zGd49nT9Nwe2y_5_4p0gpQaKSFPeOgxcZHp3zL7eU5OuVlpEAeC96H0XRNG7c1WNxyRwMjaKGNBZleJhoHEe2UN2IACNg5OHyV6weWUFLtj-N4FXa2OZiGbZ0ONpwmTroBu6ZGxbDvidlvF9xnCCuUeOZMeRIx-J8tjW23tPl272k3qDBO0kntor26QuqJ8eo6VcZMEK074r8GeamOP-tsl29vVI3zRZ_Et3e7HN5lQ4rnTh6Cu9QxRI9SPeE3A3Q6DZPdGNWW48DLBjFpPhWtIhuHSFN9U3UrnSSIVjQbrvmIpDwvd_SL118awZV5qLni47sU2SwZBBgzPmiWH9N-UhcSDy8kH1dHTOLhOFZ5MntAnZmbZGL5t4JcjeeyBaINlLbU_pX1wMp8ewjWEaJLMFHr2ypLHDVykGsPDv6UNYHYaz5Oxgh6PO0t9wdXc1VA7-450KiwUNK2DrT06xAUbdvYTm_wcUZDSL_Dt2t357TpQauTPQdLCZaQQ9FeXSiSrgZU2nEbjgcDhILL56hF8TAUs-Bxeq2eEhebqTZ4fc5yaDTcAzc1JmwsoYYMRwwBnvSp5KB_FQySHEKevUDvQqnxKjHFOP1rAP6wMHPz9rHDKRKWJrvS_BBH0Zmb16oD2tkrqNd35jVaglICrtK10PeFFQzadMQdQ_eUOC0u726f4QWh3heXyI1CQiQN_bTPzOfuxw2OAubrW-CsOn45wTf8cU0TOtZin6Ni6eDlmp73380Rhwd7sRj3m9e1kMuDy5hbR6MTKYxd4GPfJ8NyNUctJ29Q8yweX4TUe6TOTv4boU9A3eUtTYkteaz7n-1dvTigk0quiwyL7mmEz4jB-DlZKx_w0VHVIRCu8C-U8hU6y0nimSPFk3N7zD2GOSKuIdrea62L-kdUD7jvLXYFbuX-JjlbZ2nPVPoZXc9FIhWj6XFdbbGDsjkWPUYF59OjDqA17HHoW3fVYBcUS4DAqnfS_XHXNl3yg43r6NBOOHvgELKDs89CBkAxMV6ldCRj1gsLGXri3GB7kLyaHqMSqro0encXLyxUIdK12e1RqRETYM_MKQBJvwsSzmiXKT5ny6RFZl6OhMk78QGOJAqpRHZ7QYQIY3agejrHI_ufvVjX3PxEJT9rn6l2ZPQsyYBhF5HESGGOmL76NZMMotNoZ_kRxGFk6mRXhmHkMddLTNNNQsUiXxbAX-6h2iLdukpqs7oJ0GS2XoYBahN6ZS930TOjRNx_VC721ppKUN4JBjBzV8-bqVYUOi2FMSk8brrmPemkKH33TRBEPboBWYeVFQ5Kn9VabkZ-PztDYMQ2zREXmCsWQE2xKn1ZU_6GhTy4WRUHhvIOsp0JU3JN191hjemhz5wODtnX2VkY98s43yd90mVZWWwjszql0-xoiHBtiCMowyGF_RICO1dFvk60BgLM-tQToTJBDHk7gbtAM2NVCkFNEGyGeCntt5_y-3Iyhv2kvBTEibY0DJdViXEPIvrP4pe14XPa5dgcBcTE9DcD413Rieef_F94trbwRv5SKPzlqNuy4ag6jX93ZJomeHEaYT0JnpPXczb1r5oHSpXXYD0OaicULfxaakIKUnzOZ3qtuLDGaXXM8Sy8wDdSCYUH6njMxOrEwQCscXRlPuRjIIa5pMFsxlASYXqMYscEORkkfp7ut5A794m00IY_UgGJhytc-VztB7BMugk_W50IWGFyLylSF3biTEEUrKZuowQF2hSwlkwc4OfhFFMvgwS07NmSgxgfrCcTwU-oTdvFfMN3jGWkxmDpEkk7wItxD-wMBUN-xS9XjnQE1Ybeaze_zGNLVP5LIyGbJQmDA36LjD2nvrRPPkhbpv5wIOUOdw8QzqwZgOLB0SxK3ghK0b3MFxDYeVNsL16ueQhcHeS2m69X3XBhFItEVmBiFZV40SNihtAXwoYzVXD6x1Tooj636APe84c-1v_XshPiydOq9yKDX-7MIMcB93v4NLOMu0yJ7_XQ63zSKabq3Y_8g9ytDg01GI-_RZRI5GywzLuEZ48QZL_DgtRQ5IFAaJXrRRq_BVJKqtPe4G-nxVmuuRSwIk8D7iInXoHZHOmBFrWNCIcR9__DhHtS-Ycac8Zz3XP7VC4iDUAckHXY7v_xM9agkYM_Gsg21K01cD2dVQlTkOaPDn1qgRoyupKJRYmsMTpSe0xBmZEp-CsNEnRJqjdcNTHmJ3jFnycy2ivK3FqBEcQm-Pc5-iH8LHrU50AbSsUSWf2GgNlrOlM2hz5gpvYSUoCMFs-XgA72r92LSF0Cx10BEGvQB6fsdIZUy58-zhVk_dwWCmMoHYIVTLZEYzmELOWcaTPQLM4go8lTnPoo8GLTAu7g_VkE9dqlvG4094qpnd-w3BUw5Pl291iGeDs2bqGWbUUcQvvkEY3iumqnUaKYe3vy05t_8OfnqzwPSgDvQRZcZRNxTGQFEyLrcVnLqdmxmzKkteaQc2QMJcuzgcsGYon-TRYx7qRofuPjdcBs2qsGP873zcsQtg20d1yLp0gX5zuTWerE0BknsU6Y-4qeuPIqLoxfmn0_m2QDeu7Urm90J-5smeeulNlP-jnABjoecT8Kefdzph7lSL_iAUlqZ5BdNHjm7CdYZ0xU6EfQaHKfnnG3oTsMiOsHXrIxCwomSQxnzUXZvquFS3Srsw7CJWMeYUWM1q6aYNJJH4W9NyOQt2qRSP3Bi1yrzerDD6iVm4fcjDcgSPu8vAfEUGhmmKo8-BXK1iM_6EFNT6C9LnKtiXBwfV6P5bRIkkdm2BWTwsAqOwfCHNGA9_Dw1owudff8BclkVcn2lvABnqRMf623CVI3RQxYRq6VbzhCu-QosrEhoXtsN_e41tlMzHjQZE1gmXOPsb_QNIl3td5GJwnU5ezbOq32zSbrMfatr4lTYGHQ0YrcUYfvKAPslGrG_tW-UajEcSRiUhsG9uwVWJM7z9vGAcugsWTss36J8HU0XUgFO4TRfqMoUpM5ZJ5vUeiBbtushBHavg_6_sFO1SyLP1oxgA9JhSAbXkEehu9rOvmicY6AVbxniEQ1SsgvRfWttttjIEH5QmDD2WB4PiFcJuZu5-XNRtY7_XqJn8LNWyBuZ5PdAOSKr5h3x5nIDH_bXw82fCZr7d0Sa60EX2HxwhRuPTEsFdXP4C4y6qhT7jt0wKYxkaX7rrrXznoEk4qrrOn86KTXukT-wCg8DWmXnGQ2oWPtmDj0Yf1TPU1rPDv5QJ96Q7hI3Pn2AUFRsGuCcTOeHlgqAdVeyxoqGz6BBFx7XhkGxiENlXmL4e88yMgHsl_DjWFEQ1GsebspnRBAjbvK67h533T9X3X6chjmDlP60w3HNgQup15t6sIOiI9IJ2aDB4L9VND4jhWMdXTI2hbzYwcR6yRjwPzh-H60BMrBdpFBzLuFkU-PItrUD7raMe7OQAZEGuxW6P6CvVGpnJWGE2Ha6tfPEsvTxcP4zTNzn1or5LiiLk7J-Pe68TqZaFsGtk6Ojdu7T7d8FukKuH6b-FNZfCKYut0LcAAOccVMgQ3b7SjR57J1iQqkx9QfU6NvFHtmjwKYiit8k1LmJvVuUjWgxqeAI37ORif6fbRnjjeJ_f4I89p2cqzevyhCEhbSMr0NVDdZMLU6yUUQM7dMzeWcPCp6Em1qRodjlJdgxfyat0JMRPwWLH-ltuJX4dba8oPEJRtt3c2GgZW8s1Z_EkEB6zU35-TK-ADRMeAizg38z4bHkPct4St8DOfhDOlFOORFblCcAynKObN4v7mXmqHySt8iFJ4EyOmFnBNm70ZQ1k4ldNUYN0ssDPvTl8wehjSc2qNTIYnbW12yajZe25QgLSnJ52BuXolDMupUlEok23DJf8sJGpRt4UyCO2l8Oemce6CFdo88ABizMtFWUTnZ2Q4adUbj0O_vfnGsN6rO7PxXY6zOb-4yL-uK__fbidjZXbvkGBXOyiKqvTzM52VTeWCHFfkHma0CdklXAkvbS9oQWKbgHZjuI6s_3V0uc0hVXpwluR3S2GxL2S8LDQhhxwxfLM79D-fwlQr_RGne8SLYsq8iKbY-q6Y8jQovbh013_A7PFGkPFAsl1pFmXdAm-0WkXrDGIp3ngvDIqmYkusjLtOkMSejlStG2zYLKfgUc2QJUpuWKg9lBQrBipA4mRsrFdHIFelJEl0hd7exiJhT1AvTOy5eQQSSJmSif3WA5nRUmF4FfE5GhV9l0Hes0Mh5L3EAsWe1-HtfzuaNhjYrnOQ2cOi2AJ_VtppGniOQSfrnOwiiYrCrkg.eyJlbmRwb2ludF9zbHVnIjoibWV0YS9tdXNlLXNwYXJrLTEuMi0yMDI2MDgwNXxtZXRhIn0\",\"format\":\"meta-responses-v1\",\"id\":\"rs_6a844e3ccbfc7130fdef4faf:rs_01a014d19cd677b089a0a9e8f3109ab7\",\"index\":0}]},\"native_finish_reason\":\"completed\"}],\"created\":1787055673,\"model\":\"meta/muse-spark-1.2\",\"object\":\"chat.completion\",\"service_tier\":\"auto\",\"system_fingerprint\":null,\"usage\":{\"completion_tokens\":761,\"prompt_tokens\":3202,\"total_tokens\":3963,\"completion_tokens_details\":{\"accepted_prediction_tokens\":null,\"audio_tokens\":0,\"reasoning_tokens\":678,\"rejected_prediction_tokens\":null,\"image_tokens\":0},\"prompt_tokens_details\":{\"audio_tokens\":0,\"cached_tokens\":241,\"cache_write_tokens\":0,\"video_tokens\":0},\"cost\":0.00697165,\"is_byok\":false,\"cost_details\":{\"upstream_inference_cost\":0.00697165,\"upstream_inference_prompt_cost\":0.0037374,\"upstream_inference_completions_cost\":0.00323425}},\"provider\":\"Meta\"}"}, "scoring": {"fuzzy": 0.9373451697440682, "cost_summary": {"total_input_tokens": 108054, "total_output_tokens": 37542, "total_tokens": 145596, "input_cost_usd": 0.1350675, "output_cost_usd": 0.15955350000000001, "total_cost_usd": 0.2946210000000001}}, "pricing": {"bucket_date": "2026-08-18", "age_days": 0, "input_price_per_million": 1.25, "output_price_per_million": 4.25}, "timing": {"total_response_s": 569.04, "mean_response_s": 17.24, "slowest_response_s": 31.26, "inputs_timed": 33, "inputs_stored": 33, "span_s": 38.25}, "normalized_score": 93.73451697440683}