Benchmark Results

Book Advert XML files (malformed) from Avisblatt

Dataset Description Result Overview Test Runs

This benchmark has been run 155 times. It uses fuzzy metric.

Tested providers: huggingface, openrouter, scicore, genai, openai, x-ai, alibaba, deepseek, mistral, anthropic, cohere

Tested models: magistral-medium-2509, gpt-5.6-sol, claude-fable-5, pixtral-large-2411, gpt-5, mistral-small-2506, gemini-3.1-flash-lite-preview, meta/muse-spark-1.2, claude-3-7-sonnet-20250219, deepseek-chat, o3, pixtral-12b, gemini-3-pro-preview, qwen3.5-27b, claude-opus-4-6, grok-4.5, qwen/qwen3.5-27b, qwen/qwen3.5-9b, qwen35-397b-a17b-fp8, gemini-2.5-flash-lite, gpt-4o, gpt-5.1-2025-11-13, qwen3-235b-fp8, thinkingmachines/Inkling-Small:deepinfra, gemini-3.5-flash-lite, qwen/qwen3.7-plus, google/gemma-4-31b-it, google/gemma-4-26b-a4b-it, claude-sonnet-5, qwen/qwen3-vl-30b-a3b-instruct, qwen3.5-flash-2026-02-23, meta-llama/llama-4-maverick, gpt-4.1-nano, ministral-14b-2512, command-a-03-2025, claude-opus-4-1-20250805, gpt-5.3-codex, qwen/qwen3.5-plus-02-15, deepseek-v4-flash, qwen/qwen3.5-397b-a17b, command-r7b-12-2024, mistral-large-2411, qwen/qwen3.5-122b-a10b, gpt-4o-mini, GLM-4.5V-FP8, gpt-5.2-2025-12-11, mistral-medium-2505, thinkingmachines/Inkling:together, claude-3-opus-20240229, qwen/qwen3.5-flash-02-23, Qwen/Qwen3-VL-235B-A22B-Instruct:deepinfra, qwen/qwen3-vl-8b-thinking, MiniMaxAI/MiniMax-M3:deepinfra, claude-opus-4-20250514, claude-opus-4-8, meta-models/Muse-Glimmer-30B:together, claude-haiku-4-5-20251001, qwen3.5-plus-2026-02-15, qwen3.5-35b-a3b, claude-opus-5, swiss-ai/Apertus-v1.5-70B:publicai, claude-sonnet-4-6, z-ai/glm-5v-turbo, gpt-5.6-terra, claude-sonnet-4-20250514, deepseek-v4-pro, gpt-5-mini, gemini-3.5-flash, mistral-medium-3.5, grok-4.20-0309-reasoning, qwen3.5-397b-a17b, gpt-5.4-2026-03-05, gpt-4.1, gemini-3-flash-preview, gemini-2.5-pro, ministral-8b-2512, gpt-5.5-2026-04-23, qwen/qwen3.6-plus, gpt-5.6-luna, gpt-4.1-mini, claude-opus-4-5-20251101, mistral-large-2512, gemini-3.1-flash-lite, gpt-5-nano, gemini-2.5-flash-lite-preview-09-2025, gemini-2.0-flash-lite, x-ai/grok-4, gemini-2.0-flash, qwen/qwen3-vl-8b-instruct, gemini-3.1-pro-preview, grok-4.6, gemini-3.6-flash, claude-opus-4-7, grok-4.3, mistral-medium-2508, qwen/qwen3.5-35b-a3b, claude-sonnet-4-5-20250929, gemini-2.5-flash, stepfun/step-3.7-flash, qwen/qwen3.8-max, qwen3.5-122b-a10b, magistral-small-2509, command-r-08-2024, gemini-2.5-flash-preview-09-2025, moonshotai/kimi-k3, meta-llama/llama-4-scout, command-r-plus-08-2024, gemini-3.7-flash, deepseek-reasoner, swiss-ai/Apertus-v1.5-8B:publicai

• Type(s): newspaper-page
• Benchmark task(s):  data-correction
• Writing: n/a
• Source creation (century): 18
• Source Layout: n/a
• Language(s): en