A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.
A test run includes:
Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3-vl-8b-instruct |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 71.50 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.71 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 32.8K IT + 4.8K OT = 37.5K TT | Cost: 0.003$ + 0.002$ = 0.005$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3-vl-8b-thinking |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 15.35 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.15 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 55.0K IT + 3.4K OT = 58.4K TT | Cost: 0.010$ + 0.007$ = 0.017$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openai |
| Model | gpt-5 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 90.60 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.91 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 36.4K IT + 46.1K OT = 82.5K TT | Cost: 0.045$ + 0.461$ = 0.506$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | mistral-medium-2508 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 89.35 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.89 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 53.1K IT + 4.7K OT = 57.8K TT | Cost: 0.021$ + 0.009$ = 0.031$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openai |
| Model | gpt-4o-mini |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 77.92 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.78 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 857.3K IT + 4.5K OT = 861.8K TT | Cost: 0.129$ + 0.003$ = 0.131$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | mistral-medium-2505 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 87.90 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.88 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 53.1K IT + 4.7K OT = 57.8K TT | Cost: 0.021$ + 0.009$ = 0.031$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openai |
| Model | o3 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 85.04 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.85 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 37.9K IT + 16.9K OT = 54.8K TT | Cost: 0.076$ + 0.135$ = 0.211$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | anthropic |
| Model | claude-sonnet-4-20250514 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 79.00 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.79 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 65.3K IT + 6.4K OT = 71.6K TT | Cost: 0.196$ + 0.095$ = 0.291$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | genai |
| Model | gemini-2.5-flash-lite |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 85.05 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.85 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 17.3K IT + 5.7K OT = 23.0K TT | Cost: 0.002$ + 0.002$ = 0.004$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openai |
| Model | gpt-5-nano |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 76.12 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.76 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: 5 months ago, 2025-10-24. | Tokens: 53.1K IT + 85.8K OT = 138.9K TT | Cost: 0.003$ + 0.034$ = 0.037$ |