Benchmark Results

General Meeting Minutes

Extract names, locations, signatures from table-like meeting minutes of Mines de Costanzo S.A., 1930s - 1960s

Dataset Description Result Overview Test Runs

This benchmark has been run 77 times. It uses fuzzy metric.

Tested providers: openai, x-ai, anthropic, openrouter, genai, mistral, scicore

Tested models: claude-opus-4-5-20251101, claude-opus-4-7, qwen/qwen3.5-9b, gpt-5.5-2026-04-23, claude-sonnet-4-6, gpt-5.3-codex, qwen/qwen3.5-plus-02-15, gpt-5-nano, qwen/qwen3.5-122b-a10b, qwen/qwen3.5-35b-a3b, gpt-4.1-mini, qwen/qwen3-vl-8b-thinking, qwen/qwen3.7-plus, claude-haiku-4-5-20251001, o3, ministral-14b-2512, gpt-5.2-2025-12-11, magistral-medium-2509, mistral-large-2512, gemini-3.1-pro-preview, claude-sonnet-5, gemini-2.5-pro, gpt-5.4-2026-03-05, gpt-5-mini, qwen/qwen3.5-flash-02-23, qwen/qwen3-vl-8b-instruct, ministral-8b-2512, claude-opus-4-6, gpt-4.1-nano, gemini-2.5-flash, gemini-3.5-flash, grok-4.20-0309-reasoning, mistral-medium-2505, gpt-4o-mini, claude-fable-5, claude-sonnet-4-5-20250929, qwen35-397b-a17b-fp8, gemini-2.5-flash-lite, google/gemma-4-31b-it, qwen/qwen3.6-plus, claude-opus-4-1-20250805, claude-opus-4-8, meta-llama/llama-4-scout, qwen/qwen3-vl-30b-a3b-instruct, grok-4.3, gemini-3.1-flash-lite, qwen/qwen3.5-397b-a17b, gpt-4o, mistral-medium-3.5, magistral-small-2509, gemini-3-flash-preview, mistral-medium-2508, meta-llama/llama-4-maverick, gpt-5, gpt-4.1, google/gemma-4-26b-a4b-it, stepfun/step-3.7-flash, mistral-small-2506, gpt-5.1-2025-11-13, qwen/qwen3.5-27b

• Type(s): minutes
• Benchmark task(s):  information-extraction
• Writing: typed, handwritten
• Source creation (century): 20
• Source Layout: table
• Language(s): it, fr, de