Benchmark Results

Personnel Cards

Evaluates models' ability to transcribe and interpret personnel index cards of Swiss federal employees (1941–1961), containing typed and handwritten entries on job title, work location, pay grade, salary, and related notes in German and French.

Dataset Description Result Overview Test Runs

This benchmark has been run 112 times. It uses f1_micro metric.

Tested providers: huggingface, openrouter, scicore, genai, openai, x-ai, alibaba, mistral, anthropic, cohere

Tested models: magistral-medium-2509, gpt-5.6-sol, claude-fable-5, pixtral-large-2411, gpt-5, mistral-small-2506, gemini-3.1-flash-lite-preview, command-a-vision-07-2025, meta/muse-spark-1.2, o3, gemini-3-pro-preview, qwen3.5-27b, claude-opus-4-6, grok-4.5, qwen/qwen3.5-27b, qwen/qwen3.5-9b, qwen35-397b-a17b-fp8, gemini-2.5-flash-lite, gpt-4o, gpt-5.1-2025-11-13, thinkingmachines/Inkling-Small:deepinfra, gemini-3.5-flash-lite, qwen/qwen3.7-plus, google/gemma-4-31b-it, google/gemma-4-26b-a4b-it, claude-sonnet-5, qwen/qwen3-vl-30b-a3b-instruct, qwen3.5-flash-2026-02-23, gpt-4.1-nano, meta-llama/llama-4-maverick, ministral-14b-2512, claude-opus-4-1-20250805, gpt-5.3-codex, qwen/qwen3.5-plus-02-15, qwen/qwen3.5-397b-a17b, mistral-large-2411, qwen/qwen3.5-122b-a10b, gpt-4o-mini, gpt-5.2-2025-12-11, mistral-medium-2505, thinkingmachines/Inkling:together, qwen/qwen3.5-flash-02-23, Qwen/Qwen3-VL-235B-A22B-Instruct:deepinfra, qwen/qwen3-vl-8b-thinking, claude-opus-4-20250514, claude-opus-4-8, meta-models/Muse-Glimmer-30B:together, claude-haiku-4-5-20251001, qwen3.5-plus-2026-02-15, qwen3.5-35b-a3b, claude-opus-5, swiss-ai/Apertus-v1.5-70B:publicai, claude-sonnet-4-6, z-ai/glm-5v-turbo, gpt-5.6-terra, claude-sonnet-4-20250514, gpt-5-mini, gemini-3.5-flash, mistral-medium-3.5, grok-4.20-0309-reasoning, qwen3.5-397b-a17b, gpt-5.4-2026-03-05, gpt-4.1, gemini-3-flash-preview, gemini-2.5-pro, ministral-8b-2512, gpt-5.5-2026-04-23, qwen/qwen3.6-plus, gpt-5.6-luna, gpt-4.1-mini, claude-opus-4-5-20251101, mistral-large-2512, gemini-3.1-flash-lite, gemini-2.5-flash-lite-preview-09-2025, gpt-5-nano, gemini-2.0-flash-lite, x-ai/grok-4, gemini-2.0-flash, qwen/qwen3-vl-8b-instruct, gemini-3.1-pro-preview, grok-4.6, gemini-3.6-flash, claude-opus-4-7, grok-4.3, mistral-medium-2508, qwen/qwen3.5-35b-a3b, claude-sonnet-4-5-20250929, gemini-2.5-flash, stepfun/step-3.7-flash, qwen/qwen3.8-max, qwen3.5-122b-a10b, magistral-small-2509, gemini-2.5-flash-preview-09-2025, moonshotai/kimi-k3, meta-llama/llama-4-scout, gemini-3.7-flash, MiniMaxAI/MiniMax-M3:deepinfra, swiss-ai/Apertus-v1.5-8B:publicai

• Type(s): index-card
• Benchmark task(s):  transcription, document-understanding, data-correction
• Writing: handwritten, typed, printed
• Source creation (century): 20
• Source Layout: table, form
• Language(s): de, fr