University of Basel RISE · Research and Infrastructure SupportHumanities Data Benchmark

Leaderboard

Cost and speed

What a score costs in time and money, on equal work: models with a best run on all 12 benchmarks, their mean score, the mean cost per 1,000 documents and the mean time per input.

Provider

95 models with a priced and timed best run on all 12 benchmarksOnly models with a best run on every benchmark are compared, so all did the same work. 12 more have all 12 but lack a price or a timing for one of them.

Score against time per inputTime per input is the mean response time of the best runs. It depends on the provider's load at the time and on how many requests ran in parallel.Bottom right is fast and accurate. Time on a log scale; drag to zoom, double-click to reset.
    Score against costCosts are as recorded when the runs were made. They are given per 1,000 documents, so that benchmarks of different size weigh the same.Bottom right is cheap and accurate. Mean cost per 1,000 documents over the 12 benchmarks, on a log scale.

      Time per score pointMilliseconds per input, divided by the mean score. Lower is more time-efficient.

      1. gemini-3.1-flash-liteGoogle · score 69.5 51 ms
      2. claude-haiku-5-5Anthropic · score 67.5 62 ms
      3. mistral-small-2603Mistral · score 59.6 85 ms
      4. gemini-3.7-flashGoogle · score 84.0 93 ms
      5. claude-sonnet-5-5Anthropic · score 86.9 105 ms
      6. gpt-5.4-2026-03-05OpenAI · score 80.2 107 ms
      7. gpt-5.3-codexOpenAI · score 81.7 119 ms
      8. claude-opus-5-5Anthropic · score 89.8 144 ms
      9. ministral-3b-2512Mistral · score 51.0 161 ms
      10. claude-fable-5Anthropic · score 88.1 163 ms

      Cost per score pointMean cost per 1,000 documents, divided by the mean score, in dollars. Lower is more cost-efficient.

      1. google/gemma-4-31b-itvia OpenRouter · score 69.9 0.006 $
      2. google/gemma-4-26b-a4b-itvia OpenRouter · score 66.2 0.006 $
      3. ministral-3b-2512Mistral · score 51.0 0.008 $
      4. meta-llama/llama-4-scoutvia OpenRouter · score 56.2 0.009 $
      5. qwen/qwen3-vl-8b-instructvia OpenRouter · score 58.8 0.011 $
      6. ministral-14b-2512Mistral · score 63.7 0.012 $
      7. ministral-8b-2512Mistral · score 49.8 0.012 $
      8. claude-haiku-5-5Anthropic · score 67.5 0.013 $
      9. mistral-small-2603Mistral · score 59.6 0.015 $
      10. qwen/qwen3-vl-30b-a3b-instructvia OpenRouter · score 61.9 0.016 $

      Downloadcost and speed (JSON)complete dataset (Zenodo)