Benchmark Results

Bibliographic Data

Evaluates models' ability to extract bibliographic information from historical documents such as publication details, authors, dates, and other metadata from digitized sources.

Dataset Description Result Overview Test Runs

This benchmark has been run 185 times. It uses fuzzy metric.

Tested providers: huggingface, openrouter, scicore, genai, openai, x-ai, alibaba, mistral, anthropic

Tested models: magistral-medium-2509, gpt-5.6-sol, claude-fable-5, pixtral-large-2411, gpt-5, mistral-small-2506, gemini-3.1-flash-lite-preview, meta/muse-spark-1.2, claude-3-7-sonnet-20250219, o3, pixtral-12b, gemini-3-pro-preview, qwen3.5-27b, claude-opus-4-6, grok-4.5, qwen/qwen3.5-27b, qwen/qwen3.5-9b, qwen35-397b-a17b-fp8, gemini-2.5-flash-lite, gpt-4o, gpt-5.1-2025-11-13, thinkingmachines/Inkling-Small:deepinfra, gemini-3.5-flash-lite, qwen/qwen3.7-plus, google/gemma-4-31b-it, google/gemma-4-26b-a4b-it, claude-sonnet-5, qwen/qwen3-vl-30b-a3b-instruct, gemini-1.5-flash, qwen3.5-flash-2026-02-23, gpt-4.1-nano, meta-llama/llama-4-maverick, ministral-14b-2512, claude-opus-4-1-20250805, gpt-5.3-codex, qwen/qwen3.5-plus-02-15, qwen/qwen3.5-397b-a17b, mistral-large-2411, qwen/qwen3.5-122b-a10b, gpt-4o-mini, GLM-4.5V-FP8, gpt-5.2-2025-12-11, mistral-medium-2505, thinkingmachines/Inkling:together, claude-3-opus-20240229, qwen/qwen3.5-flash-02-23, gpt-4.5-preview, qwen/qwen3-vl-8b-thinking, Qwen/Qwen3-VL-235B-A22B-Instruct:deepinfra, claude-opus-4-20250514, claude-opus-4-8, meta-models/Muse-Glimmer-30B:together, claude-haiku-4-5-20251001, claude-3-5-sonnet-20241022, qwen3.5-plus-2026-02-15, qwen3.5-35b-a3b, claude-opus-5, swiss-ai/Apertus-v1.5-70B:publicai, claude-sonnet-4-6, z-ai/glm-5v-turbo, gpt-5.6-terra, claude-sonnet-4-20250514, gpt-5-mini, gemini-3.5-flash, mistral-medium-3.5, gemini-1.5-pro, grok-4.20-0309-reasoning, qwen3.5-397b-a17b, gpt-5.4-2026-03-05, gpt-4.1, gemini-3-flash-preview, gemini-2.5-pro, ministral-8b-2512, gpt-5.5-2026-04-23, qwen/qwen3.6-plus, gpt-5.6-luna, gpt-4.1-mini, claude-opus-4-5-20251101, mistral-large-2512, gemini-3.1-flash-lite, gpt-5-nano, gemini-2.5-flash-lite-preview-09-2025, gemini-2.0-flash-lite, x-ai/grok-4, gemini-2.0-flash, qwen/qwen3-vl-8b-instruct, gemini-3.1-pro-preview, grok-4.6, gemini-3.6-flash, claude-opus-4-7, grok-4.3, mistral-medium-2508, qwen/qwen3.5-35b-a3b, claude-sonnet-4-5-20250929, gemini-2.5-flash, stepfun/step-3.7-flash, qwen/qwen3.8-max, qwen3.5-122b-a10b, magistral-small-2509, gemini-2.5-flash-preview-09-2025, moonshotai/kimi-k3, meta-llama/llama-4-scout, gemini-3.7-flash, MiniMaxAI/MiniMax-M3:deepinfra, swiss-ai/Apertus-v1.5-8B:publicai

• Type(s): book-page
• Benchmark task(s):  information-extraction
• Writing: printed
• Source creation (century): 20
• Source Layout: list
• Language(s): en