RISE Humanities Data Benchmark, 0.5.3-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'general_meeting_minutes__true' with Search Hidden 'False' returned 79 results, showing page 4 of 8.
Result 31 of 79

Test T1256 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelgoogle/gemma-4-31b-it-20260402
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.79 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.86 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 4.6K IT + 7.3K OT = 11.9K TTCost: 0.001$ + 0.003$ = 0.003$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 32 of 79

Test T1269 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.6-plus-04-02
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score88.33 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.88 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 28.0K IT + 59.9K OT = 87.9K TTCost: 0.009$ + 0.117$ = 0.126$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 33 of 79

Test T1249 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-5-2025-08-07
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score76.91 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.77 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.2K IT + 51.1K OT = 61.3K TTCost: 0.013$ + 0.511$ = 0.524$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 34 of 79

Test T1232 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-3.1-flash-lite
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score72.04 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.72 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.8K IT + 8.4K OT = 19.2K TTCost: 0.003$ + 0.013$ = 0.015$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 35 of 79

Test T1238 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providermistral
Modelministral-8b-2512
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score78.27 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.78 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 26.4K IT + 8.0K OT = 34.3K TTCost: 0.004$ + 0.001$ = 0.005$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 36 of 79

Test T1228 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-2.5-flash
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score83.67 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.84 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 3.4K IT + 9.2K OT = 12.6K TTCost: 0.001$ + 0.023$ = 0.024$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 37 of 79

Test T1254 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelo3-2025-04-16
  
Temperature1.0
DataclassMinutesPage
  
Normalized Score75.20 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.75 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.7K IT + 31.4K OT = 42.1K TTCost: 0.021$ + 0.251$ = 0.273$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 38 of 79

Test T1265 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.5-397b-a17b-20260216
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score44.77 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.45 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 21.1K IT + 36.1K OT = 57.1K TTCost: 0.008$ + 0.084$ = 0.093$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 39 of 79

Test T1234 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-3.5-flash
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score80.06 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.80 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 10.8K IT + 71.2K OT = 82.1K TTCost: 0.016$ + 0.641$ = 0.657$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 40 of 79

Test T1268 at 2026-07-03

{'document-type': ['minutes'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['it', 'fr', 'de'], 'layout': ['table'], 'entry-type': ['person', 'location'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3.5-plus-20260216
  
Temperature0.0
DataclassMinutesPage
  
Normalized Score85.66 %
Test timeunknown seconds
Prompt 

Please extract the metadata according to the given output format.
Name and Address are in the same table field. Your task is to extract them into separate fields.
Lose any dashes between name and address but preserve linebreaks.
Be aware: Addresses may contain multiple dashes; preserve them, only remove "visual splitting characters" between name and address.
Filename: {filename}
Page: {page_number}

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.86 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 28.0K IT + 17.5K OT = 45.5K TTCost: 0.007$ + 0.027$ = 0.035$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752