RISE Humanities Data Benchmark, 0.6.0-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Loading module...

Search Results

Your search for Benchmark 'book_advert_xml__true' with Search Hidden 'False' returned 179 results, showing page 3 of 18.
Result 21 of 179

Test T1006 at 2026-09-17

newspaper-page data-correction 18 en

openrouter google/gemma-4-26b-a4b-it · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score98.4%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Toskanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":2,"explanation":"The XML was missing a space between the title and the closing tag of the title element, and the title content 'Eine Tossanische Bibel' likely contains a typo (should be 'Toskanische'). I have added the necessary space and corrected the spelling to ensure valid and meaningful content."}

Scoring

Fuzzy score93.0%
Items50

Speed

Model time, all inputs2120.6 s
Mean per input42.41 s
Slowest input446.68 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens38.6K
Output tokens78.8K
Total tokens117.4K
Input cost$0.0035
Output cost$0.0236
Total cost$0.0271

Priced from 2026-09-16 · Input $/M $0.09 · Output $/M $0.30

Result 22 of 179

Test T1700 at 2026-09-17

newspaper-page data-correction 18 en

openrouter meta/muse-spark-1.3 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Corrected malformed opening TITLE tag: '<TITLEEine' was missing the closing '>' and was fixed to '<TITLE>'. No other structural changes were needed as ITEM, BIBL and closing TITLE tags were well-formed."}

Scoring

Fuzzy score95.2%
Items50

Speed

Model time, all inputs2227.3 s
Mean per input44.55 s
Slowest input145.83 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens35K
Output tokens289.7K
Total tokens324.7K
Input cost$0.0437
Output cost$1.23
Total cost$1.28

Priced from 2026-09-08 · Input $/M $1.25 · Output $/M $4.25

Result 23 of 179

Test T0954 at 2026-09-17

newspaper-page data-correction 18 en

openrouter qwen/qwen3.5-397b-a17b · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The primary fault identified was a malformed opening tag for the TITLE element. The string '<TITLEEine' lacked the closing angle bracket ('>') required to delimit the tag name from its content. This has been corrected to '<TITLE>'. No additional tags were added to the numbering '37.' as it functions validly as character data within the ITEM element, though in a strict schema it might alternatively be an attribute."}

Scoring

Fuzzy score95.1%
Items50

Speed

Model time, all inputs12747.7 s
Mean per input254.95 s
Slowest input1611.37 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens33.1K
Output tokens323.1K
Total tokens356.2K
Input cost$0.0182
Output cost$1.13
Total cost$1.13

Priced from 2026-09-16 · Input $/M $0.55 · Output $/M $3.50

Result 24 of 179

Test T1415 at 2026-09-17

newspaper-page data-correction 18 en

anthropic claude-opus-5 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"Fixed the malformed opening tag <TITLEEine Tossanische Bibel</TITLE>: the <TITLE> start-tag was missing its closing angle bracket, so the title text was fused into the tag name. It is now properly written as <TITLE>Eine Tossanische Bibel</TITLE>, which yields a well-formed nesting of ITEM > BIBL > TITLE. The item number \"37.\" remains as text directly inside <ITEM>, preserving the original structure."}

Scoring

Fuzzy score96.8%
Items50

Speed

Model time, all inputs647.1 s
Mean per input12.94 s
Slowest input25.99 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens81.2K
Output tokens59.5K
Total tokens140.8K
Input cost$0.4061
Output cost$1.49
Total cost$1.89

Priced from 2026-09-16 · Input $/M $5.00 · Output $/M $25.00

Result 25 of 179

Test T0915 at 2026-09-17

newspaper-page data-correction 18 en

openrouter qwen/qwen3.5-122b-a10b · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening <TITLE> tag was malformed, missing the closing angle bracket '>' before the text content. This caused the parser to interpret 'TITLEEine' as the tag name. The fix inserts the missing delimiter to properly close the tag."}

Scoring

Fuzzy score95.1%
Items45

Speed

Model time, all inputs11180.9 s
Mean per input223.62 s
Slowest input1474.30 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens35.3K
Output tokens679.7K
Total tokens715K
Input cost$0.0038
Output cost$0.7329
Total cost$1.51

Priced from 2026-09-16 · Input $/M $0.26 · Output $/M $2.08

Result 26 of 179

Test T0480 at 2026-09-17

newspaper-page data-correction 18 en

openrouter qwen/qwen3-vl-8b-instruct · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score97.6%

{"fixed_xml":"<ITEM><BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":5,"explanation":"The XML string contains several issues: 1) The opening tag for <ITEM> is not properly closed with a closing tag. 2) The title text 'Eine Tossanische Bibel' has an incorrect encoding — it should be properly escaped or corrected if it's meant to be literal. 3) The <BIBL> tag is missing a closing tag. 4) The <TITLE> tag is incorrectly nested inside the <BIBL> tag without proper closure. 5) There’s no root element enclosing the entire structure. I will fix these by adding appropriate closing tags, escaping special characters, and wrapping the content in a valid root element."}

Scoring

Fuzzy score95.0%
Items50

Speed

Model time, all inputs583.8 s
Mean per input11.68 s
Slowest input284.46 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens32.4K
Output tokens31.9K
Total tokens64.2K
Input cost$0.0037
Output cost$0.0145
Total cost$0.0183

Priced from 2026-09-16 · Input $/M $0.117 · Output $/M $0.455

Result 27 of 179

Test T1550 at 2026-09-17

newspaper-page data-correction 18 en

huggingface thinkingmachines/Inkling-Small · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening <TITLE> tag was malformed because it was missing its closing angle bracket (e.g., <TITLE... instead of <TITLE>...). Added the missing '>' so the tag is properly formed as <TITLE>, allowing the XML to parse correctly. No additional structural tags were needed as the rest of the hierarchy (<ITEM>, <BIBL>, closing tags) was intact."}

Scoring

Fuzzy score93.7%
Items50

Speed

Model time, all inputs10606.9 s
Mean per input212.14 s
Slowest input1050.98 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens29.6K
Output tokens281.1K
Total tokens310.6K
Input cost$0.0133
Output cost$0.3373
Total cost$0.3506

Priced from 2026-09-16 · Input $/M $0.45 · Output $/M $1.20

Result 28 of 179

Test T0487 at 2026-09-17

newspaper-page data-correction 18 en

cohere command-a-vision-07-2025 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score25.7%

{
  "fixed_xml": "<book><title>Book Title</title><author>Author Name</author><price>19.99</price></book>",
  "number_of_fixes": 3,
  "explanation": "Added missing book tags, corrected title and author format, and added price tag."
}

Scoring

Fuzzy score12.0%
Items50

Speed

Model time, all inputs90.7 s
Mean per input1.81 s
Slowest input6.26 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens38.4K
Output tokens3.6K
Total tokens42K
Input cost$0.00
Output cost$0.00
Total cost$0.00

Priced from 2026-09-16 · Input $/M $0.00 · Output $/M $0.00

Result 29 of 179

Test T0928 at 2026-09-17

newspaper-page data-correction 18 en

openrouter qwen/qwen3.5-27b · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening tag for the title element was malformed. The closing angle bracket '>' was missing after the tag name 'TITLE', causing the text content 'Eine' to be merged into the tag name. I inserted the missing bracket to properly close the opening tag."}

Scoring

Fuzzy score95.5%
Items44

Speed

Model time, all inputs22013.2 s
Mean per input440.26 s
Slowest input3730.88 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens35.7K
Output tokens636.8K
Total tokens672.4K
Input cost$0.007
Output cost$0.9934
Total cost$1.27

Priced from 2026-09-16 · Input $/M $0.195 · Output $/M $1.56

Result 30 of 179

Test T1625 at 2026-09-17

newspaper-page data-correction 18 en

openrouter meta/muse-spark-1.2 · temp 0.0 · dataclass CorrectedAdvert

Response vs ground truth

One model response from this run. The per-input comparison needs JavaScript.

Fuzzy score100.0%

{"fixed_xml":"<ITEM>37. <BIBL><TITLE>Eine Tossanische Bibel</TITLE></BIBL></ITEM>","number_of_corrections":1,"explanation":"The opening <TITLE> tag was malformed as '<TITLEEine' - missing the closing '>' bracket. Added '>' to correctly form '<TITLE>Eine Tossanische Bibel</TITLE>'."}

Scoring

Fuzzy score96.6%
Items50

Speed

Model time, all inputs590.4 s
Mean per input11.81 s
Slowest input22.59 s
Inputs timed50

Summed model time, not elapsed: requests run in parallel and the worker count is not recorded, so the run finished sooner than this.

Costs / Pricing

Input tokens28.8K
Output tokens185.2K
Total tokens213.9K
Input cost$0.036
Output cost$0.7869
Total cost$0.8229

Priced from 2026-08-18 · Input $/M $1.25 · Output $/M $4.25