RISE Humanities Data Benchmark, 0.5.3-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'general_meeting_minutes__true' with Search Hidden 'False' returned 77 results, showing page 1 of 8.
Result 1 of 77

Test T1230 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-2.5-pro
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score84.09 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.84 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 3.4K IT + 9.2K OT = 12.5K TTCost: 0.004$ + 0.092$ = 0.096$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 2 of 77

Test T1214 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideranthropic
Modelclaude-opus-4-7
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.96 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.86 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 51.9K IT + 9.2K OT = 61.1K TTCost: 0.260$ + 0.230$ = 0.489$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 3 of 77

Test T1213 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideranthropic
Modelclaude-opus-4-6
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.25 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.85 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 25.1K IT + 9.0K OT = 34.0K TTCost: 0.125$ + 0.224$ = 0.349$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 4 of 77

Test T1237 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providermistral
Modelministral-14b-2512
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score80.93 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.81 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 26.4K IT + 8.0K OT = 34.4K TTCost: 0.005$ + 0.002$ = 0.007$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 5 of 77

Test T1254 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelo3-2025-04-16
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score75.20 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.75 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.7K IT + 31.4K OT = 42.1K TTCost: 0.021$ + 0.251$ = 0.273$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 6 of 77

Test T1246 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4.1-nano-2025-04-14
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score47.44 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.47 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 51.5K IT + 3.5K OT = 55.0K TTCost: 0.005$ + 0.001$ = 0.007$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 7 of 77

Test T1261 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3-vl-8b-thinking
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score73.74 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.74 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 28.1K IT + 94.5K OT = 122.6K TTCost: 0.003$ + 0.129$ = 0.132$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 8 of 77

Test T1268 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.5-plus-20260216
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.66 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.86 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 28.0K IT + 17.5K OT = 45.5K TTCost: 0.007$ + 0.027$ = 0.035$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 9 of 77

Test T1265 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.5-397b-a17b-20260216
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score44.77 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.45 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 21.1K IT + 36.1K OT = 57.1K TTCost: 0.008$ + 0.084$ = 0.093$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 10 of 77

Test T1263 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.5-27b-20260224
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score84.62 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.85 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 26.1K IT + 16.7K OT = 42.8K TTCost: 0.005$ + 0.026$ = 0.031$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752