v0.8.1 released 3 Nov 2026 latest run 29 Oct 2026 open source, GPL-3.0

Which language model works best with your data?

Open benchmarks for information extraction, document understanding, transcription and data correction, on images and texts from humanities research, from the mid-15th century (Medieval Manuscripts) to April 2025 (Duty Rosters). Every run is scored against ground truth and logged with its cost.

benchmarks
12
current models158 tested since 2025
121
current runs3,805 in total
2,632

Best models per task

Each task's benchmarks, and the models with a run on all of them, ranked by their mean score there. ≈ marks models that cannot be told apart within sampling error.

The full leaderboard, all 121 current models →

What's new

Release · 3 Nov 2026

v0.8.1

  • Added 2
  • Changed 6
  • Removed 1
  • Fixed 3

Dataset release 2026-10-29.5 deposited (10.5281/zenodo.23821533, concept DOI 10.5281/zenodo.23821532) and recorded in scripts/data/dataset_releases.json; README and CITATION.cff link it, and CITATION.cff lists the v0.8.0 DOI.

All releases and news →
News · 30 Oct 2026

The results as a citable dataset

Every result behind this website is now published as a dataset on Zenodo: 10.5281/zenodo.23821532. It holds every stored run and request since 2025, 3,805 runs and 166,380 requests in the first release, as Parquet tables with a metric dictionary, together with the original request and scoring records of every model answer. It is licensed CC BY 4.0, and the files live in their own repository.

Read the post →

How it is scored

  • Real research material

    12 benchmarks built from 8 archives, libraries and projects, each documented as a BenchmarkCard: data, ground truth, metric, limitations.

    Browse benchmarks →
  • Every answer on record

    3,805 runs, each with its prompt, every response, the scores and the cost, compared input by input with the ground truth.

    Inspect test runs →
  • Ground truth from domain experts

    17 people contribute as domain experts, data curators, annotators, analysts and engineers. The method is described in the publications.

    Contribute a benchmark →