RISE Humanities Data Benchmark

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
Model configuration – provider, model version, temperature, and other generation parameters.
Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
Usage and cost data – token counts and calculated API costs.
Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Result 11 of 93

Test T1158 at 2026-06-22

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	scicore
Model	qwen35-397b-a17b-fp8

Temperature	0.5
Dataclass	Card

Normalized Score	88.80 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.89	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 85.4K IT + 94.9K OT = 180.4K TT

Cost: 0.000$ + 0.000$ = 0.000$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 12 of 93

Test T1109 at 2026-06-08

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	x-ai
Model	grok-4.3

Temperature	0.5
Dataclass	Card

Normalized Score	94.70 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.95	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 98.6K IT + 3.7K OT = 102.4K TT

Cost: 0.123$ + 0.009$ = 0.133$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 13 of 93

Test T1122 at 2026-06-05

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	meta-llama/llama-4-scout-17b-16e-instruct

Temperature	0.5
Dataclass	Card

Normalized Score	78.49 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.78	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 87.3K IT + 4.1K OT = 91.4K TT

Cost: 0.007$ + 0.001$ = 0.008$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 14 of 93

Test T1135 at 2026-06-05

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	stepfun/step-3.7-flash-20260528

Temperature	0.5
Dataclass	Card

Normalized Score	82.72 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.83	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 48.8K IT + 143.0K OT = 191.8K TT

Cost: 0.010$ + 0.164$ = 0.174$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 15 of 93

Test T0539 at 2026-06-04

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	mistral
Model	mistral-large-2512

Temperature	0.5
Dataclass	Card

Normalized Score	41.86 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.42	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 37.0K IT + 1.9K OT = 38.8K TT

Cost: 0.018$ + 0.003$ = 0.021$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 16 of 93

Test T1083 at 2026-06-04

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	anthropic
Model	claude-opus-4-8

Temperature	0.5
Dataclass	Card

Normalized Score	84.99 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.85	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 143.4K IT + 8.3K OT = 151.7K TT

Cost: 0.717$ + 0.208$ = 0.925$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 17 of 93

Test T0550 at 2026-06-04

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	mistral
Model	ministral-14b-2512

Temperature	0.5
Dataclass	Card

Normalized Score	77.31 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.77	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 93.6K IT + 13.6K OT = 107.2K TT

Cost: 0.019$ + 0.003$ = 0.021$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 18 of 93

Test T1096 at 2026-06-04

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	mistral
Model	mistral-medium-3.5

Temperature	0.5
Dataclass	Card

Normalized Score	86.19 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.86	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 90.7K IT + 4.2K OT = 95.0K TT

Cost: 0.136$ + 0.032$ = 0.168$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 19 of 93

Test T1070 at 2026-06-04

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	qwen/qwen3.7-plus-20260602

Temperature	0.5
Dataclass	Card

Normalized Score	77.21 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.77	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 97.7K IT + 10.4K OT = 108.0K TT

Cost: 0.039$ + 0.017$ = 0.056$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 20 of 93

Test T0561 at 2026-06-04

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	mistral
Model	ministral-8b-2512

Temperature	0.5
Dataclass	Card

Normalized Score	22.17 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.22	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 93.6K IT + 13.1K OT = 106.7K TT

Cost: 0.014$ + 0.002$ = 0.016$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Search Test Runs

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV

Test T1158 at 2026-06-22

Test T1109 at 2026-06-08

Test T1122 at 2026-06-05

Test T1135 at 2026-06-05

Test T0539 at 2026-06-04

Test T1083 at 2026-06-04

Test T0550 at 2026-06-04

Test T1096 at 2026-06-04

Test T1070 at 2026-06-04

Test T0561 at 2026-06-04

Search Test Runs

Search Results Show compact results Refine Search New Search Download Download JSON Download CSV

Test T1158 at 2026-06-22

Test T1109 at 2026-06-08

Test T1122 at 2026-06-05

Test T1135 at 2026-06-05

Test T0539 at 2026-06-04

Test T1083 at 2026-06-04

Test T0550 at 2026-06-04

Test T1096 at 2026-06-04

Test T1070 at 2026-06-04

Test T0561 at 2026-06-04

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV