RISE Humanities Data Benchmark

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
Model configuration – provider, model version, temperature, and other generation parameters.
Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
Usage and cost data – token counts and calculated API costs.
Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Result 81 of 93

Test T0315 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	genai
Model	gemini-2.5-flash

Temperature	0.5
Dataclass	Card

Normalized Score	87.83 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.88	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 16.2K IT + 3.6K OT = 19.8K TT

Cost: 0.005$ + 0.009$ = 0.014$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 82 of 93

Test T0325 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	anthropic
Model	claude-sonnet-4-5-20250929

Temperature	0.5
Dataclass	Card

Normalized Score	85.29 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.85	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 74.3K IT + 6.6K OT = 80.8K TT

Cost: 0.223$ + 0.098$ = 0.321$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 83 of 93

Test T0335 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	qwen/qwen3-vl-8b-instruct

Temperature	0.5
Dataclass	Card

Normalized Score	71.50 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.71	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 32.8K IT + 4.8K OT = 37.5K TT

Cost: 0.003$ + 0.002$ = 0.005$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 84 of 93

Test T0307 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openai
Model	gpt-4.1-mini

Temperature	0.5
Dataclass	Card

Normalized Score	95.65 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.96	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 56.2K IT + 3.1K OT = 59.3K TT

Cost: 0.022$ + 0.005$ = 0.027$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 85 of 93

Test T0333 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	meta-llama/llama-4-maverick

Temperature	0.5
Dataclass	Card

Normalized Score	82.06 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.82	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 71.7K IT + 3.8K OT = 75.5K TT

Cost: 0.011$ + 0.002$ = 0.013$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 86 of 93

Test T0316 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	genai
Model	gemini-2.5-pro

Temperature	0.5
Dataclass	Card

Normalized Score	92.08 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.92	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 17.3K IT + 3.7K OT = 20.9K TT

Cost: 0.022$ + 0.037$ = 0.058$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 87 of 93

Test T0332 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openrouter
Model	qwen/qwen3-vl-8b-thinking

Temperature	0.5
Dataclass	Card

Normalized Score	15.35 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.15	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 55.0K IT + 3.4K OT = 58.4K TT

Cost: 0.010$ + 0.007$ = 0.017$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 88 of 93

Test T0309 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openai
Model	gpt-5

Temperature	0.5
Dataclass	Card

Normalized Score	90.60 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.91	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 36.4K IT + 46.1K OT = 82.5K TT

Cost: 0.045$ + 0.461$ = 0.506$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 89 of 93

Test T0308 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openai
Model	gpt-4.1-nano

Temperature	0.5
Dataclass	Card

Normalized Score	78.20 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.78	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 77.2K IT + 3.1K OT = 80.3K TT

Cost: 0.008$ + 0.001$ = 0.009$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Result 90 of 93

Test T0310 at 2025-10-24

{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}

Configuration

Provider	openai
Model	gpt-5-mini

Temperature	0.5
Dataclass	Card

Normalized Score	85.73 %
Test time	unknown seconds

Prompt

You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.

- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.

Here is the required JSON format:
{
  "company": {"transcription": ""},
  "location": {"transcription": ""},
  "b_id": {"transcription": ""},
  "date": "",
  "information": [
    {"transcription": ""}
  ]
}

If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.

Results

no valid result

Scoring

Fuzzy Score	F1 micro / macro		Micro precision/recall		Tue/False Positives
0.86	n/a	n/a	n/a	n/a	n/a	n/a	n/a	n/a
			Micro Precision	Micro Recall	Instances	TP	FP	FN

Costs / Pricing

Pricing Date: 9 months ago, 2025-10-24.

Tokens: 45.6K IT + 36.5K OT = 82.1K TT

Cost: 0.011$ + 0.073$ = 0.084$

Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752

Search Test Runs

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV

Test T0315 at 2025-10-24

Test T0325 at 2025-10-24

Test T0335 at 2025-10-24

Test T0307 at 2025-10-24

Test T0333 at 2025-10-24

Test T0316 at 2025-10-24

Test T0332 at 2025-10-24

Test T0309 at 2025-10-24

Test T0308 at 2025-10-24

Test T0310 at 2025-10-24

Search Test Runs

Search Results Show compact results Refine Search New Search Download Download JSON Download CSV

Test T0315 at 2025-10-24

Test T0325 at 2025-10-24

Test T0335 at 2025-10-24

Test T0307 at 2025-10-24

Test T0333 at 2025-10-24

Test T0316 at 2025-10-24

Test T0332 at 2025-10-24

Test T0309 at 2025-10-24

Test T0308 at 2025-10-24

Test T0310 at 2025-10-24

Search Results
Show compact results Refine Search New Search

Download JSON Download CSV