RISE Humanities Data Benchmark, 0.5.3-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'general_meeting_minutes__true' with Search Hidden 'False' returned 79 results, showing page 2 of 8.
Result 11 of 79

Test T1237 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providermistral
Modelministral-14b-2512
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score80.93 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.81 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 26.4K IT + 8.0K OT = 34.4K TTCost: 0.005$ + 0.002$ = 0.007$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 12 of 79

Test T1255 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelgoogle/gemma-4-26b-a4b-it-20260403
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score75.67 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.76 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 7.8K IT + 8.2K OT = 16.0K TTCost: 0.001$ + 0.003$ = 0.003$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 13 of 79

Test T1256 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelgoogle/gemma-4-31b-it-20260402
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.79 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.86 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 4.6K IT + 7.3K OT = 11.9K TTCost: 0.001$ + 0.003$ = 0.003$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 14 of 79

Test T1254 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelo3-2025-04-16
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score75.20 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.75 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.7K IT + 31.4K OT = 42.1K TTCost: 0.021$ + 0.251$ = 0.273$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 15 of 79

Test T1244 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4.1-2025-04-14
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score77.26 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.77 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 11.7K IT + 6.6K OT = 18.2K TTCost: 0.023$ + 0.053$ = 0.076$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 16 of 79

Test T1249 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-5-2025-08-07
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score76.91 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.77 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.2K IT + 51.1K OT = 61.3K TTCost: 0.013$ + 0.511$ = 0.524$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 17 of 79

Test T1265 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.5-397b-a17b-20260216
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score44.77 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.45 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 21.1K IT + 36.1K OT = 57.1K TTCost: 0.008$ + 0.084$ = 0.093$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 18 of 79

Test T1213 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideranthropic
Modelclaude-opus-4-6
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.25 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.85 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 25.1K IT + 9.0K OT = 34.0K TTCost: 0.125$ + 0.224$ = 0.349$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 19 of 79

Test T1218 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideranthropic
Modelclaude-sonnet-5
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score67.55 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.68 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 48.9K IT + 9.0K OT = 57.9K TTCost: 0.098$ + 0.090$ = 0.188$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 20 of 79

Test T1261 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3-vl-8b-thinking
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score73.74 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.74 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 28.1K IT + 94.5K OT = 122.6K TTCost: 0.003$ + 0.129$ = 0.132$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752