University of Basel RISE · Research and Infrastructure SupportHumanities Data Benchmark

qwen/qwen3.5-397b-a17b

Qwenvia OpenRouter
average±1.9 (95%)
72.2
leaderboard rankof 114
≈54
benchmarksof 12
12
per 1,000 documentsmean over its benchmarks
$16
seconds per inputmean response time
172.8
Compare with other models Its row in the Matrix All its test runs

By task

  • Data correction

    56.9

    Best of 114 models with all its benchmarks: claude-opus-5-5, 82.7.

  • Document understanding

    59.4

    Best of 107 models with all its benchmarks: gpt-6-astra, 97.0.

  • Information extraction

    82.8

    Best of 107 models with all its benchmarks: grok-4.6, 90.8.

  • Transcription

    86.9

    Best of 114 models with all its benchmarks: gemini-4-argon, 96.2.

By benchmark

BenchmarkScoreThe best current run on each benchmark, on the benchmark's metric scaled to 0–100. ± is the 95% sampling margin, from resampling the run's inputs.RankRank among all current models on that benchmark; ≈ for models that cannot be told apart within sampling error.Per 1,000 docss / inputBest run
Bibliographic Data 94.5±5.0 ≈57of 115 $35$0.17 run 477.75 T0944 · 17 Sep 2026prompt.txt
Blacklist Cards 93.0±3.0 ≈51of 114 $7.73$0.26 run 145.15 T0951 · 17 Sep 2026prompt.txt
Book Advert XML Repair 21.9±7.4 ≈64of 119 $23$1.13 run 254.95 T0954 · 17 Sep 2026prompt.txt
Business Letters 60.0±6.5 ≈24of 114 $10$0.57 run 94.66 T0945 · 21 Apr 2026prompt.txt
Company Lists 82.1±10.3 ≈55of 114 $17$0.26 run 117.3 T0952 · 17 Sep 2026prompt.txt
Duty Rosters 81.9±4.2 ≈30of 107 $39$0.83 run 381.92 T1326 · 22 Jul 2026prompt.txt
Fraktur Adverts 94.4±4.3 ≈32of 114 $15$0.076 run 77.03 T0948 · 21 Apr 2026prompt_optimized.txt
General Meeting Minutes 87.7±6.9 ≈25of 108 $20$0.18 run 192.05 T1265 · 18 Sep 2026prompt.txt
Library Cards 80.2±1.4 ≈93of 114 $3.98$1.05 run 103.86 T0949 · 17 Sep 2026prompt.txt
Magazine Pages 4.3±6.0 ≈79of 115 $5.18$0.24 run 69.49 T0956 · 21 Apr 2026prompt.txt
Medieval Manuscripts 74.4±10.9 ≈27of 114 $9.87$0.12 run 82.47 T0950 · 21 Apr 2026prompt.txt
Personnel Cards 91.9±5.7 ≈58of 114 $9.02$0.55 run 77.18 T0955 · 21 Apr 2026prompt.txt

Price

FromInput, $ per million tokensOutput, $ per million tokensSource
16 Sep 20260.553.5archived price page
21 Apr 20260.392.34archived price page

Every run

  1. T1265General Meeting Minutes18 Sep 202687.7$0.18
  2. T0956Magazine Pages17 Sep 20264.3$0.24
  3. T0955Personnel Cards17 Sep 202678.9$0.87
  4. T0954Book Advert XML Repair17 Sep 202621.9$1.13
  5. T0953Company Lists17 Sep 202667.2$0.17
  6. T0952Company Lists17 Sep 202682.1$0.26
  7. T0951Blacklist Cards17 Sep 202693.0$0.26
  8. T0950Medieval Manuscripts17 Sep 202671.8$0.24
  9. T0949Library Cards17 Sep 202680.2$1.05
  10. T0948Fraktur Adverts17 Sep 202673.3$0.17
  11. T0947Business Letters17 Sep 2026 · 32 signed letters43.0$0.49
  12. T0946Business Letters17 Sep 2026 · 25 unsigned letters42.0$0.38
  13. T0945Business Letters17 Sep 2026 · all 57 letters52.0$0.91
  14. T0944Bibliographic Data17 Sep 202694.5$0.17
  15. T1326Duty Rosters22 Jul 202681.9$0.83
  16. T1265General Meeting Minutes3 Jul 202644.8$0.093
  17. T0956Magazine Pages21 Apr 20264.3$0.24
  18. T0955Personnel Cards21 Apr 202691.9$0.55
  19. T0954Book Advert XML Repair21 Apr 202621.0$0.97
  20. T0953Company Lists21 Apr 202670.7$0.13
  21. T0952Company Lists21 Apr 202671.2$0.14
  22. T0951Blacklist Cards21 Apr 202684.7$0.17
  23. T0950Medieval Manuscripts21 Apr 202674.4$0.12
  24. T0949Library Cards21 Apr 202678.7$2.10
  25. T0948Fraktur Adverts21 Apr 202694.4$0.076
  26. T0947Business Letters21 Apr 2026 · 32 signed letters64.0$0.41
  27. T0946Business Letters21 Apr 2026 · 25 unsigned letters54.0$0.22
  28. T0945Business Letters21 Apr 2026 · all 57 letters60.0$0.57
  29. T0944Bibliographic Data21 Apr 202686.1$0.085