A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.
A test run includes:
Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.6-plus-04-02 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 92.48 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.92 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 97.5K IT + 38.3K OT = 135.9K TT | Cost: 0.032$ + 0.075$ = 0.106$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.5-flash-20260224 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 91.99 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.92 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 97.8K IT + 5.5K OT = 103.3K TT | Cost: 0.006$ + 0.001$ = 0.008$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.5-397b-a17b-20260216 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 83.47 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.83 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 89.9K IT + 59.1K OT = 149.0K TT | Cost: 0.035$ + 0.138$ = 0.173$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | anthropic |
| Model | claude-opus-4-7 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 90.58 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.91 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 156.6K IT + 8.2K OT = 164.8K TT | Cost: 0.783$ + 0.204$ = 0.987$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.5-27b-20260224 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 92.58 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.93 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 97.7K IT + 5.1K OT = 102.8K TT | Cost: 0.019$ + 0.008$ = 0.027$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.5-9b-20260310 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 92.80 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.93 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 85.7K IT + 81.6K OT = 167.4K TT | Cost: 0.009$ + 0.012$ = 0.021$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.5-122b-a10b-20260224 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 91.45 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.91 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 97.7K IT + 5.4K OT = 103.1K TT | Cost: 0.025$ + 0.011$ = 0.037$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | google/gemma-4-31b-it-20260402 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 90.53 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.91 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 20.5K IT + 4.8K OT = 25.2K TT | Cost: 0.003$ + 0.002$ = 0.004$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | openrouter |
| Model | qwen/qwen3.5-35b-a3b-20260224 |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 87.57 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.88 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 90.6K IT + 41.2K OT = 131.8K TT | Cost: 0.015$ + 0.054$ = 0.068$ |
{'document-type': ['index-card'], 'writing': ['typed', 'handwritten'], 'century': [20], 'language': ['de'], 'entry-type': ['company'], 'task': ['information-extraction']}
| Provider | alibaba |
| Model | qwen3.5-397b-a17b |
| Temperature | 0.5 |
| Dataclass | Card |
| Normalized Score | 88.34 % |
| Test time | unknown seconds |
You are a meticulous archivist extracting data from an index card image. Analyze the provided image and extract the following information. Return the data ONLY as a valid JSON object.
- "company": The primary company name, usually in the top-left. Exclude the location.
- "location": The city or town, often following the company name.
- "b_id": The identifier code, usually in the top-right, starting with "B.".
- "date": Any stamped dates on the card in YYYY-MM-DD format. If no date is present, use an empty string.
- "information": A list of text blocks from the main body of the card. Each block should be a separate string in the list. Maintain line breaks with \\n.
Here is the required JSON format:
{
"company": {"transcription": ""},
"location": {"transcription": ""},
"b_id": {"transcription": ""},
"date": "",
"information": [
{"transcription": ""}
]
}
If you cannot find a value for a field, leave its transcription value as an empty string. Do not add any explanatory text outside of the JSON object.
no valid result| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| 0.88 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 96.5K IT + 5.6K OT = 102.1K TT | Cost: 0.058$ + 0.020$ = 0.078$ |