v0.8.1released 3 Nov 2026latest run 29 Oct 2026open source, GPL-3.0
Which language model works best with your data?
Open benchmarks for information extraction, document understanding, transcription and data correction, on images and texts from humanities research, from the mid-15th century (Medieval Manuscripts) to April 2025 (Duty Rosters). Every run is scored against ground truth and logged with its cost.
Each task's benchmarks, and the models with a run on all of them, ranked by their mean score there. ≈ marks models that cannot be told apart within sampling error.
Information extraction
Bibliographic Data, Blacklist Cards, Business Letters, Company Lists, Duty Rosters, General Meeting Minutes, Library Cards
Dataset release 2026-10-29.5 deposited (10.5281/zenodo.23821533, concept DOI 10.5281/zenodo.23821532) and recorded in scripts/data/dataset_releases.json; README and CITATION.cff link it, and CITATION.cff lists the v0.8.0 DOI.
Every result behind this website is now published as a dataset on Zenodo:
10.5281/zenodo.23821532. It holds every stored run and
request since 2025, 3,805 runs and 166,380 requests in the first release, as Parquet tables with a
metric dictionary, together with the original request and scoring records of every model answer.
It is licensed CC BY 4.0, and the files live in their own
repository.