RISE Humanities Data Benchmark

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
Model configuration – provider, model version, temperature, and other generation parameters.
Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
Usage and cost data – token counts and calculated API costs.
Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Result 61 of 93

Test T0333 at 2026-01-25

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	meta-llama/llama-4-maverick

Temperature	0.5
Dataclass	Card

Normalized Score	74.13 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.74	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 88.5K IT + 3.9K OT = 92.4K TT

Cost: 0.013$ + 0.002$ = 0.016$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 62 of 93

Test T0410 at 2026-01-25

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openai
Model	gpt-5.1-2025-11-13

Temperature	0.5
Dataclass	Card

Normalized Score	88.66 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.89	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 45.6K IT + 3.5K OT = 49.1K TT

Cost: 0.057$ + 0.035$ = 0.092$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 63 of 93

Test T0561 at 2026-01-25

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	mistral
Model	ministral-8b-2512

Temperature	0.5
Dataclass	Card

Normalized Score	16.35 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.16	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 19.7K IT + 4.9K OT = 24.6K TT

Cost: 0.003$ + 0.001$ = 0.004$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 64 of 93

Test T0335 at 2026-01-25

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	qwen/qwen3-vl-8b-instruct

Temperature	0.5
Dataclass	Card

Normalized Score	74.49 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.74	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 85.7K IT + 4.5K OT = 90.2K TT

Cost: 0.007$ + 0.002$ = 0.009$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 65 of 93

Test T0334 at 2026-01-25

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	qwen/qwen3-vl-30b-a3b-instruct

Temperature	0.5
Dataclass	Card

Normalized Score	78.69 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.79	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 88.8K IT + 4.6K OT = 93.4K TT

Cost: 0.012$ + 0.002$ = 0.014$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 66 of 93

Test T0528 at 2026-01-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	anthropic
Model	claude-haiku-4-5-20251001

Temperature	0.5
Dataclass	Card

Normalized Score	82.87 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.83	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 93.2K IT + 6.4K OT = 99.7K TT

Cost: 0.093$ + 0.032$ = 0.125$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 67 of 93

Test T0539 at 2026-01-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	mistral
Model	mistral-large-2512

Temperature	0.5
Dataclass	Card

Normalized Score	14.75 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.15	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 19.7K IT + 1.6K OT = 21.4K TT

Cost: 0.010$ + 0.002$ = 0.012$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 68 of 93

Test T0317 at 2026-01-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	genai
Model	gemini-2.5-flash-lite

Temperature	0.5
Dataclass	Card

Normalized Score	87.22 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.87	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 17.5K IT + 5.6K OT = 23.1K TT

Cost: 0.002$ + 0.002$ = 0.004$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 69 of 93

Test T0325 at 2026-01-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	anthropic
Model	claude-sonnet-4-5-20250929

Temperature	0.5
Dataclass	Card

Normalized Score	69.91 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.70	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 93.2K IT + 6.1K OT = 99.4K TT

Cost: 0.280$ + 0.092$ = 0.372$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 70 of 93

Test T0316 at 2026-01-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	genai
Model	gemini-2.5-pro

Temperature	0.5
Dataclass	Card

Normalized Score	96.87 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.97	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: n/a, n/a.

Tokens: 17.5K IT + 3.8K OT = 21.3K TT

Cost: 0.022$ + 0.038$ = 0.060$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Search Test Runs

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV

Test T0333 at 2026-01-25

Test T0410 at 2026-01-25

Test T0561 at 2026-01-25

Test T0335 at 2026-01-25

Test T0334 at 2026-01-25

Test T0528 at 2026-01-24

Test T0539 at 2026-01-24

Test T0317 at 2026-01-24

Test T0325 at 2026-01-24

Test T0316 at 2026-01-24

Search Test Runs

Search Results Show compact results Refine Search New Search Download Download JSON Download CSV

Test T0333 at 2026-01-25

Test T0410 at 2026-01-25

Test T0561 at 2026-01-25

Test T0335 at 2026-01-25

Test T0334 at 2026-01-25

Test T0528 at 2026-01-24

Test T0539 at 2026-01-24

Test T0317 at 2026-01-24

Test T0325 at 2026-01-24

Test T0316 at 2026-01-24

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV