RISE Humanities Data Benchmark, 0.6.0-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'book_advert_xml__true' with Search Hidden 'False' returned 179 results, showing page 7 of 18.
Result 61 of 179

Test T0484 at 2026-09-16

newspaper-page data-correction 18 en

cohere command-r-plus-08-2024 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Score0.0%

{
    "description": "Main output structure for Book Advert XML files from Avisblatt.",
    "properties": {
        "fixed_xml": {
            "description": "The corrected XML content as a string.",
            "title": "Fixed Xml",
            "type": "string"
        },
        "number_of_corrections": {
            "description": "The number of corrections made to the original XML.",
            "title": "Number Of Corrections",
            "type": "integer"
        },
        "explanation": {
            "default": null,
            "description": "Optional explanation of the corrections made.",
            "title": "Explanation",
            "type": "string"
        }
    },
    "required": [
        "fixed_xml",
        "number_of_corrections"
    ],
    "title": "CorrectedAdvert",
    "type": "object"
}

Scoring

Fuzzy score13.5%
Items50

Speed

Model time, all inputs1937.8 s
Mean per input38.76 s
Slowest input84.54 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens18.9K
Output tokens7.6K
Total tokens26.5K
Input cost$0.0473
Output cost$0.0763
Total cost$0.1236

Priced from 2026-09-16 · Input $/M $2.50 · Output $/M $10.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 62 of 179

Test T1112 at 2026-09-16

newspaper-page data-correction 18 en

x-ai grok-4.3 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Fixed missing '>' in opening <TITLE> tag."}

Scoring

Fuzzy score98.3%
Items50

Speed

Model time, all inputs605.1 s
Mean per input12.10 s
Slowest input19.15 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens43.8K
Output tokens26.1K
Total tokens69.9K
Input cost$0.0547
Output cost$0.0654
Total cost$0.249
Reasoning cost usd$0.1289
Total reasoning tokens51.6K

Priced from 2026-09-16 · Input $/M $1.25 · Output $/M $2.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 63 of 179

Test T0498 at 2026-09-16

newspaper-page data-correction 18 en

openai gpt-5.2-2025-12-11 · temp 1.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Added missing opening tag delimiter for the TITLE element: changed <TITLEEine Tossanische Bibel</TITLE> to <TITLE>Eine Tossanische Bibel</TITLE>."}

Scoring

Fuzzy score96.6%
Items50

Speed

Model time, all inputs727.5 s
Mean per input14.55 s
Slowest input332.83 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens33.7K
Output tokens29K
Total tokens62.7K
Input cost$0.0589
Output cost$0.4066
Total cost$0.4655

Priced from 2026-09-16 · Input $/M $1.75 · Output $/M $14.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 64 of 179

Test T1595 at 2026-09-16

newspaper-page data-correction 18 en

genai gemini-3.7-flash · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Fixed malformed opening tag '<TITLEEine' to properly formed '<TITLE>Eine'."}

Scoring

Fuzzy score97.9%
Items50

Speed

Model time, all inputs432.8 s
Mean per input8.66 s
Slowest input19.64 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens31.5K
Output tokens30K
Total tokens61.5K
Input cost$0.0237
Output cost$0.1124
Total cost$0.4767
Reasoning cost usd$0.3407
Total reasoning tokens90.8K

Priced from 2026-08-18 · Input $/M $0.75 · Output $/M $3.75

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 65 of 179

Test T1445 at 2026-09-16

newspaper-page data-correction 18 en

genai gemini-3.5-flash-lite · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score88.3%

{"fixed_xml":"<ITEM><NUMBER>37.</NUMBER> <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":2,"explanation":"Added closing tag for TITLE and introduced a NUMBER tag to properly structure the item number."}

Scoring

Fuzzy score95.9%
Items50

Speed

Model time, all inputs101.3 s
Mean per input2.03 s
Slowest input3.40 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens31.5K
Output tokens26.2K
Total tokens57.7K
Input cost$0.0095
Output cost$0.0654
Total cost$0.0749

Priced from 2026-09-16 · Input $/M $0.30 · Output $/M $2.50

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 66 of 179

Test T1745 at 2026-09-09

newspaper-page data-correction 18 en

openrouter qwen/qwen3.8-27b · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening tag was malformed as <TITLEEine...>; I corrected it to <TITLE> so it matches the closing </TITLE> tag."}

Scoring

Fuzzy score98.5%
Items45

Speed

Model time, all inputs12638.1 s
Mean per input252.76 s
Slowest input1276.96 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens36.1K
Output tokens634.6K
Total tokens670.6K
Input cost$0.0151
Output cost$1.90
Total cost$1.87

Priced from 2026-09-08 · Input $/M $0.42 · Output $/M $3.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 67 of 179

Test T1715 at 2026-09-08

newspaper-page data-correction 18 en

openrouter z-ai/glm-5.3-flash · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening tag <TITLE was missing its closing angle bracket ('>'). It was corrected to <TITLE>. All other tags (<ITEM>, <BIBL>, </TITLE>, </BIBL>, </ITEM>) were well-formed, so only this single fix was required."}

Scoring

Fuzzy score97.9%
Items49

Speed

Model time, all inputs3826.2 s
Mean per input76.52 s
Slowest input1904.48 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens29.9K
Output tokens72.3K
Total tokens102.3K
Input cost$0.0022
Output cost$0.0136
Total cost$0.0201

Priced from 2026-09-08 · Input $/M $0.075 · Output $/M $0.25

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 68 of 179

Test T1730 at 2026-09-08

newspaper-page data-correction 18 en

openrouter qwen/qwen3.8-flash · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening tag for the title element was malformed as '<TITLEEine' instead of '<TITLE>'. The closing angle bracket was added to properly define the element."}

Scoring

Fuzzy score97.5%
Items49

Speed

Model time, all inputs10403.6 s
Mean per input208.07 s
Slowest input1197.17 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens33.7K
Output tokens647.9K
Total tokens681.6K
Input cost$0.0048
Output cost$0.294
Total cost$0.3068

Priced from 2026-09-08 · Input $/M $0.15 · Output $/M $0.47

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 69 of 179

Test T1700 at 2026-09-08

newspaper-page data-correction 18 en

openrouter meta/muse-spark-1.3 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Corrected malformed opening TITLE tag: '<TITLEEine' was missing the closing '>' . Changed to '<TITLE>Eine Tossanische Bibel</TITLE>' to make the XML well-formed. No other tags needed."}

Scoring

Fuzzy score94.9%
Items50

Speed

Model time, all inputs2344.3 s
Mean per input46.89 s
Slowest input94.29 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens34.8K
Output tokens274.5K
Total tokens309.3K
Input cost$0.0435
Output cost$1.17
Total cost$1.21

Priced from 2026-09-08 · Input $/M $1.25 · Output $/M $4.25

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827
Result 70 of 179

Test T1670 at 2026-09-04

newspaper-page data-correction 18 en

openai gpt-6-astra · temp 1.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Added the missing '>' to the opening TITLE tag."}

Scoring

Fuzzy score96.0%
Items50

Speed

Model time, all inputs555.1 s
Mean per input11.10 s
Slowest input17.34 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens33.6K
Output tokens35.4K
Total tokens69K
Input cost$0.3362
Output cost$1.77
Total cost$2.11

Priced from 2026-09-03 · Input $/M $10.00 · Output $/M $50.00

Cite: Hindermann, Marti, Alberto, et al., (2026). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.21975827