RISE Humanities Data Benchmark, 0.5.0-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'blacklist_cards__true' with Search Hidden 'False' returned 85 results, showing page 8 of 9.
Result 71 of 85

Test T0313 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-2.0-flash
  
Temperature0.5
DataclassCard
  
Normalized Score90.85 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.91 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 69.8K IT + 13.3K OT = 83.0K TTCost: 0.007$ + 0.005$ = 0.012$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 72 of 85

Test T0232 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4.1
  
Temperature0.5
DataclassCard
  
Normalized Score92.28 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.92 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 41.0K IT + 3.8K OT = 44.8K TTCost: 0.082$ + 0.031$ = 0.112$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 73 of 85

Test T0318 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Providergenai
Modelgemini-2.5-flash-lite-preview-09-2025
  
Temperature0.5
DataclassCard
  
Normalized Score85.40 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.85 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 17.3K IT + 5.7K OT = 22.9K TTCost: 0.002$ + 0.002$ = 0.004$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 74 of 85

Test T0326 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Providermistral
Modelpixtral-large-latest
  
Temperature0.5
DataclassCard
  
Normalized Score77.86 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.78 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 120.6K IT + 5.0K OT = 125.6K TTCost: 0.241$ + 0.030$ = 0.271$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 75 of 85

Test T0324 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Provideranthropic
Modelclaude-opus-4-1-20250805
  
Temperature0.5
DataclassCard
  
Normalized Score88.63 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.89 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 65.3K IT + 6.2K OT = 71.4K TTCost: 0.979$ + 0.462$ = 1.442$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 76 of 85

Test T0334 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelqwen/qwen3-vl-30b-a3b-instruct
  
Temperature0.5
DataclassCard
  
Normalized Score69.50 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.69 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 32.6K IT + 8.1K OT = 40.7K TTCost: 0.007$ + 0.006$ = 0.012$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 77 of 85

Test T0305 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Provideropenai
Modelgpt-4o
  
Temperature0.5
DataclassCard
  
Normalized Score93.20 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.93 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 41.0K IT + 3.1K OT = 44.1K TTCost: 0.102$ + 0.031$ = 0.134$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 78 of 85

Test T0336 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Provideropenrouter
Modelx-ai/grok-4
  
Temperature0.5
DataclassCard
  
Normalized Score90.04 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.90 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 47.6K IT + 53.9K OT = 101.5K TTCost: 0.143$ + 0.809$ = 0.952$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 79 of 85

Test T0331 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Providerscicore
ModelGLM-4.5V-FP8
  
Temperature0.5
DataclassCard
  
Normalized Score15.35 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.15 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 0 IT + 0 OT = 0 TTCost: 0.000$ + 0.000$ = 0.000$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 80 of 85

Test T0330 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration
Providermistral
Modelmistral-large-latest
  
Temperature0.5
DataclassCard
  
Normalized Score86.49 %
Test timeunknown seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.86 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: 5 months ago, 2025-10-24. Tokens: 53.1K IT + 4.7K OT = 57.8K TTCost: 0.106$ + 0.028$ = 0.134$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752