Benchmark Results

Bibliographic Data

Evaluates models' ability to extract bibliographic information from historical documents such as publication details, authors, dates, and other metadata from digitized sources.

Dataset Description Result Overview Test Runs

This benchmark has been run 165 times. It uses fuzzy metric.

Tested providers: openai, x-ai, anthropic, openrouter, genai, mistral, alibaba, scicore

Tested models: qwen/qwen3.5-9b, qwen3.5-27b, claude-sonnet-4-6, gpt-4.5-preview, gpt-5.3-codex, gpt-5-nano, qwen/qwen3.5-122b-a10b, qwen/qwen3-vl-8b-thinking, claude-3-5-sonnet-20241022, gpt-5.4-2026-03-05, gemini-2.5-flash-lite-preview-09-2025, qwen3.5-plus-2026-02-15, mistral-medium-2505, gpt-4.1-nano, qwen35-397b-a17b-fp8, claude-opus-4-1-20250805, gemini-3-pro-preview, GLM-4.5V-FP8, qwen/qwen3-vl-30b-a3b-instruct, gemini-3.1-flash-lite, gpt-4o, magistral-small-2509, gemini-1.5-pro, gemini-2.5-flash-preview-09-2025, qwen/qwen3.5-plus-02-15, claude-sonnet-4-20250514, claude-opus-4-20250514, qwen/qwen3.7-plus, o3, ministral-14b-2512, gemini-3.5-flash, claude-3-opus-20240229, claude-sonnet-4-5-20250929, google/gemma-4-31b-it, grok-4.3, mistral-medium-3.5, google/gemma-4-26b-a4b-it, claude-haiku-4-5-20251001, mistral-large-2512, gemini-2.5-pro, qwen/qwen3-vl-8b-instruct, gemini-2.5-flash, claude-opus-4-6, grok-4.20-0309-reasoning, gpt-4o-mini, qwen3.5-flash-2026-02-23, gemini-2.5-flash-lite, gemini-2.0-flash-lite, qwen/qwen3.6-plus, qwen3.5-397b-a17b, pixtral-large-2411, meta-llama/llama-4-scout, qwen/qwen3.5-397b-a17b, gemini-3-flash-preview, mistral-medium-2508, stepfun/step-3.7-flash, qwen/qwen3.5-27b, claude-opus-4-5-20251101, claude-opus-4-7, mistral-large-2411, gpt-5.5-2026-04-23, pixtral-12b, claude-3-7-sonnet-20250219, gemini-2.0-flash, qwen/qwen3.5-35b-a3b, gpt-4.1-mini, qwen3.5-122b-a10b, gemini-3.1-pro-preview, gpt-5.2-2025-12-11, magistral-medium-2509, claude-sonnet-5, qwen/qwen3.5-flash-02-23, gpt-5-mini, ministral-8b-2512, claude-fable-5, gemini-1.5-flash, x-ai/grok-4, gemini-3.1-flash-lite-preview, qwen3.5-35b-a3b, claude-opus-4-8, meta-llama/llama-4-maverick, gpt-5, gpt-4.1, mistral-small-2506, gpt-5.1-2025-11-13

• Type(s): book-page
• Benchmark task(s):  information-extraction
• Writing: printed
• Source creation (century): 20
• Source Layout: list
• Language(s): en