RISE Humanities Data Benchmark, 0.5.3-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'book_advert_xml__true' with Search Hidden 'False' returned 97 results, showing page 2 of 10.
Result 11 of 97

Test T1161 at 2026-06-22

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Providerscicore
Modelqwen35-397b-a17b-fp8
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
95.78 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 31.4K IT + 430.3K OT = 461.7K TTCost: 0.000$ + 0.000$ = 0.000$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 12 of 97

Test T1112 at 2026-06-08

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Providerx-ai
Modelgrok-4.3
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
98.60 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 40.7K IT + 26.3K OT = 66.9K TTCost: 0.051$ + 0.066$ = 0.116$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 13 of 97

Test T1125 at 2026-06-05

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Provideropenrouter
Modelmeta-llama/llama-4-scout-17b-16e-instruct
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
56.61 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 29.1K IT + 17.5K OT = 46.6K TTCost: 0.002$ + 0.005$ = 0.008$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 14 of 97

Test T1138 at 2026-06-05

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Provideropenrouter
Modelstepfun/step-3.7-flash-20260528
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
86.28 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 31.7K IT + 1.1M OT = 1.1M TTCost: 0.006$ + 1.240$ = 1.247$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 15 of 97

Test T0542 at 2026-06-04

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Providermistral
Modelmistral-large-2512
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
56.95 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 24.3K IT + 21.9K OT = 46.2K TTCost: 0.012$ + 0.033$ = 0.045$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 16 of 97

Test T1086 at 2026-06-04

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Provideranthropic
Modelclaude-opus-4-8
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
95.04 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 81.4K IT + 58.0K OT = 139.5K TTCost: 0.407$ + 1.451$ = 1.858$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 17 of 97

Test T1073 at 2026-06-04

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.7-plus-20260602
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
79.93 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 38.6K IT + 323.0K OT = 361.7K TTCost: 0.015$ + 0.517$ = 0.532$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 18 of 97

Test T1099 at 2026-06-04

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Providermistral
Modelmistral-medium-3.5
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
96.09 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 40.3K IT + 31.8K OT = 72.1K TTCost: 0.060$ + 0.239$ = 0.299$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 19 of 97

Test T0564 at 2026-06-04

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Providermistral
Modelministral-8b-2512
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
70.39 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 40.3K IT + 46.0K OT = 86.4K TTCost: 0.006$ + 0.007$ = 0.013$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 20 of 97

Test T0553 at 2026-06-04

{'document-type': ['newspaper-page'], 'century': [18], 'language': ['en'], 'task': ['data-correction']}

Configuration
Providermistral
Modelministral-14b-2512
  
Temperature0.0
DataclassCorrectedAdvert
  
Normalized Score100.00 %
Test timeunknown seconds
Prompt 

Fix this xml. Add xml-tags if faulty where it makes sense.
Format your response as JSON. Use the keys 'fixed_xml', 'number_of_fixes', 'explanation'.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
84.76 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 40.3K IT + 48.2K OT = 88.5K TTCost: 0.008$ + 0.010$ = 0.018$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752