RISE Humanities Data Benchmark, 0.5.5-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'blacklist_cards__true' with Search Hidden 'False' returned 106 results, showing page 1 of 11.
Result 1 of 106

Test T1622 at 2026-08-18

index-card information-extraction typed, handwritten 20 de company

Configuration
Provideropenrouter
Modelmeta/muse-spark-1.2
  
Temperature0.5
DataclassCard
  
Normalized Score93.73 %
Test time12.97 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Vieli, Joseph"},"location":{"transcription":"ZUERICH"},"b_id":{"transcription":"B.51.322.GB.501."},"date":"","information":[{"transcription":"Britische schwarze Liste
Amendment 3, März 1942."},{"transcription":"gestrichen:
Amendment 4, März 1945."},{"transcription":"fig. noch auf franz. Liste Suppl. 6, 31.8.45."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.94 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 108.1K IT + 37.5K OT = 145.6K TTCost: 0.135$ + 0.160$ = 0.295$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 2 of 106

Test T1607 at 2026-08-18

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerx-ai
Modelgrok-4.6
  
Temperature0.5
DataclassCard
  
Normalized Score95.83 %
Test time2.29 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Skiff A.G.,"},"location":{"transcription":"Luzern."},"b_id":{"transcription":"B.51.322.GB.976."},"date":"","information":[{"transcription":"Britische schwarze Liste,
Amendment 5, April 1943."},{"transcription":"gestrichen:
Amendment 3, Mätz 1944."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.96 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 103.7K IT + 3.7K OT = 107.5K TTCost: 0.207$ + 0.022$ = 0.230$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 3 of 106

Test T1637 at 2026-08-18

index-card information-extraction typed, handwritten 20 de company

Configuration
Provideropenrouter
Modelz-ai/glm-5v-turbo
  
Temperature0.5
DataclassCard
  
Normalized Score90.56 %
Test time41.24 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{
  "company": {
    "transcription": "Cusin Alexander, Hotel Seidenhof,"
  },
  "location": {
    "transcription": "Zürich / Genève."
  },
  "b_id": {
    "transcription": "B.51.322.GB.1200."
  },
  "date": "",
  "information": [
    {
      "transcription": "Britische schwarze Liste,
Amendment 16, November 1943."
    },
    {
      "transcription": "gestrichen:
Amendment 7, 18. Juni 1945."
    },
    {
      "transcription": "fig. noch auf franzg. Liste Lyppl. 6, 31.8.45."
    }
  ]
}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.91 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 121.5K IT + 24.3K OT = 145.9K TTCost: 0.146$ + 0.097$ = 0.243$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 4 of 106

Test T1592 at 2026-08-18

index-card information-extraction typed, handwritten 20 de company

Configuration
Providergenai
Modelgemini-3.7-flash
  
Temperature0.5
DataclassCard
  
Normalized Score94.39 %
Test time6.25 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Cusin Alexander, Hotel Seidenhof,"},"location":{"transcription":"Zürich / Genève."},"b_id":{"transcription":"B.51.322.GB.1200."},"date":"","information":[{"transcription":"Britische schwarze Liste,
Amendment 16, November 1943."},{"transcription":"gestrichen:
Amendment 7, 18. Juni 1945."},{"transcription":"fig. noch auf franz. Liste Suppl. 6, 31.8.45."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.94 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 45.0K IT + 5.5K OT = 50.5K TTCost: 0.034$ + 0.021$ = 0.054$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 5 of 106

Test T1577 at 2026-08-16

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerhuggingface
Modelmeta-models/Muse-Glimmer-30B
  
Temperature0.5
DataclassCard
  
Normalized Score94.09 %
Test time12.74 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Schneider Max, Dr.,"},"location":{"transcription":"Zürich."},"b_id":{"transcription":"B.51.322.GB.1305."},"date":"","information":[{"transcription":"Britische schwarze Liste,
Amendment 6, Mai 1944."},{"transcription":"gestrichen:
Von der britischen schwarzen Liste.
Amendment No. 15, vom 10.April 1946."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.94 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 108.6K IT + 30.6K OT = 139.2K TTCost: 0.038$ + 0.046$ = 0.084$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 6 of 106

Test T1547 at 2026-08-16

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerhuggingface
Modelthinkingmachines/Inkling-Small
  
Temperature0.5
DataclassCard
  
Normalized Score92.25 %
Test time10.95 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Winterstein & Co., Ernst"},"location":{"transcription":"ZUERICH"},"b_id":{"transcription":"B.51.322.C3.452"},"date":"","information":[{"transcription":"Britische schwarze Liste
Amendment 21, Dezember 1941."},{"transcription":"gestrichen:
Amendment 13, 23.11.1945."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.92 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 72.8K IT + 19.5K OT = 92.3K TTCost: 0.033$ + 0.023$ = 0.056$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 7 of 106

Test T1487 at 2026-08-15

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerhuggingface
Modelswiss-ai/apertus-v1.5-70b
  
Temperature0.5
DataclassCard
  
Normalized Score83.28 %
Test time4.67 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Beatus A.G."},"location":{"transcription":"Basel."},"b_id":{"transcription":"B.51.322.GB.60"},"date":"","information":[{"transcription":"Britische schwarze Liste\
10. Amendment vom Juli 1942."},{"transcription":"gestrichen:\
Amendment 13, 23. November 1945."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.83 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 264.0K IT + 4.3K OT = 268.3K TTCost: 0.216$ + 0.012$ = 0.229$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 8 of 106

Test T1562 at 2026-08-15

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerhuggingface
Modelthinkingmachines/Inkling
  
Temperature0.5
DataclassCard
  
Normalized Score88.60 %
Test time5.00 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Metallwerke"},"location":{"transcription":"Dornach"},"b_id":{"transcription":"B. 17322. 07"},"date":"","information":[]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.89 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 73.1K IT + 24.0K OT = 97.1K TTCost: 0.073$ + 0.097$ = 0.170$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 9 of 106

Test T1532 at 2026-08-15

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerhuggingface
ModelMiniMaxAI/MiniMax-M3
  
Temperature0.5
DataclassCard
  
Normalized Score87.73 %
Test time9.22 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Sacom. (S.a.)"},"location":{"transcription":"LUGANO"},"b_id":{"transcription":"B.51.322.GB.397."},"date":"","information":[{"transcription":"Sacom. (S.a.), LUGANO.
(ciao in conc. dir.)"},{"transcription":"Britische Schwarze Liste
Amendment 16, vom 2.10.1941."},{"transcription":"gestrichen:
Amendment 13, 23.11.1945."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.88 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 32.3K IT + 3.9K OT = 36.2K TTCost: 0.009$ + 0.004$ = 0.013$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 10 of 106

Test T1517 at 2026-08-15

index-card information-extraction typed, handwritten 20 de company

Configuration
Providerhuggingface
ModelQwen/Qwen3-VL-235B-A22B-Instruct
  
Temperature0.5
DataclassCard
  
Normalized Score83.33 %
Test time4.29 seconds
Prompt 

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results 

{"company":{"transcription":"Abegg & Cie."},"location":{"transcription":"Zürich"},"b_id":{"transcription":"B.51.322.GB.266"},"date":"","information":[{"transcription":""}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.83 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 84.7K IT + 4.6K OT = 89.3K TTCost: 0.017$ + 0.004$ = 0.021$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752