A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.
A test run includes:
Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | x-ai |
| Model | grok-4.3 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 94.70 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.95 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 98.6K IT + 3.7K OT = 102.4K TT | Cost: 0.123$ + 0.009$ = 0.133$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | meta-llama/llama-4-scout-17b-16e-instruct |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 78.49 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.78 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 87.3K IT + 4.1K OT = 91.4K TT | Cost: 0.007$ + 0.001$ = 0.008$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | stepfun/step-3.7-flash-20260528 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 82.72 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.83 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 48.8K IT + 143.0K OT = 191.8K TT | Cost: 0.010$ + 0.164$ = 0.174$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | magistral-small-2509 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 17.49 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.17 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 8.5K IT + 454 OT = 9.0K TT | Cost: 0.004$ + 0.001$ = 0.005$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | mistral-large-2512 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 41.86 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.42 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 37.0K IT + 1.9K OT = 38.8K TT | Cost: 0.018$ + 0.003$ = 0.021$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | ministral-14b-2512 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 77.31 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.77 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 93.6K IT + 13.6K OT = 107.2K TT | Cost: 0.019$ + 0.003$ = 0.021$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | mistral-medium-3.5 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 86.19 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.86 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 90.7K IT + 4.2K OT = 95.0K TT | Cost: 0.136$ + 0.032$ = 0.168$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | ministral-8b-2512 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 22.17 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.22 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 93.6K IT + 13.1K OT = 106.7K TT | Cost: 0.014$ + 0.002$ = 0.016$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | magistral-medium-2509 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 57.19 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.57 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 57.0K IT + 2.2K OT = 59.2K TT | Cost: 0.114$ + 0.011$ = 0.125$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | mistral |
| Model | mistral-small-2506 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 27.52 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.28 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 16.9K IT + 841 OT = 17.7K TT | Cost: 0.002$ + 0.000$ = 0.002$ |