RISE Humanities Data Benchmark, 0.5.3-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'bibliographic_data__true' with Search Hidden 'False' returned 119 results, showing page 9 of 12.
Result 81 of 119

Test T0234 at 2025-10-17

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelmeta-llama/llama-4-maverick
  
Temperature0.0
DataclassDocument
  
Normalized Score62.57 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.63 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 8 months ago, 2025-10-17. Tokens: 7.7K IT + 8.4K OT = 16.2K TTCost: 0.001$ + 0.005$ = 0.006$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 82 of 119

Test T0140 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4.1-mini
  
Temperature0.0
DataclassDocument
  
Normalized Score64.83 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.65 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 8.8K IT + 10.2K OT = 19.1K TTCost: 0.004$ + 0.016$ = 0.020$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 83 of 119

Test T0225 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideranthropic
Modelclaude-sonnet-4-5-20250929
  
Temperature0.0
DataclassDocument
  
Normalized Score0.00 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.00 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 12.8K IT + 8.3K OT = 21.1K TTCost: 0.038$ + 0.125$ = 0.164$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 84 of 119

Test T0203 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-2.5-flash-lite
  
Temperature0.0
DataclassDocument
  
Normalized Score54.45 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.54 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 1.7K IT + 9.4K OT = 11.1K TTCost: 0.000$ + 0.004$ = 0.004$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 85 of 119

Test T0139 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4.1
  
Temperature0.0
DataclassDocument
  
Normalized Score65.69 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.66 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 7.2K IT + 10.1K OT = 17.3K TTCost: 0.014$ + 0.081$ = 0.095$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 86 of 119

Test T0130 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-5-mini
  
Temperature0.0
DataclassDocument
  
Normalized Score67.67 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.68 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 7.4K IT + 28.2K OT = 35.6K TTCost: 0.002$ + 0.056$ = 0.058$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 87 of 119

Test T0129 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-5
  
Temperature0.0
DataclassDocument
  
Normalized Score68.53 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.69 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 6.5K IT + 33.4K OT = 39.9K TTCost: 0.008$ + 0.334$ = 0.342$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 88 of 119

Test T0133 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelo3
  
Temperature0.0
DataclassDocument
  
Normalized Score66.68 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.67 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 6.7K IT + 21.9K OT = 28.6K TTCost: 0.013$ + 0.175$ = 0.189$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 89 of 119

Test T0170 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Providermistral
Modelmistral-medium-2505
  
Temperature0.0
DataclassDocument
  
Normalized Score66.14 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.66 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 9.6K IT + 9.2K OT = 18.8K TTCost: 0.004$ + 0.018$ = 0.022$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 90 of 119

Test T0141 at 2025-10-01

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [20], 'language': ['en'], 'layout': ['list'], 'entry-type': ['bibliographic'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4.1-nano
  
Temperature0.0
DataclassDocument
  
Normalized Score32.02 %
Test timeunknown seconds
Prompt
Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.32 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 9 months ago, 2025-10-01. Tokens: 11.7K IT + 8.2K OT = 19.9K TTCost: 0.001$ + 0.003$ = 0.004$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752