A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.
A test run includes:
Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | openrouter |
| Model | z-ai/glm-5v-turbo |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 84.22 % |
| Test time | 40.44 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{
"type": {
"type": "Dissertation or thesis"
},
"author": {
"last_name": "Opitz",
"first_name": "Hermann"
},
"publication": {
"title": "Synthesen in der Imidazolreihe.",
"year": 1912,
"place": "Freiburg i. B.",
"pages": "34",
"publisher": "Wagner",
"format": "8°"
},
"library_reference": {
"shelfmark": "[U 12. 1261",
"subjects": "Naturw.-math. Diss."
}
}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.85 | 0.84 | 0.83 | 0.87 | 261 | 2079 | 428 | 317 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 519.6K IT + 324.4K OT = 843.9K TT | Cost: 0.623$ + 1.298$ = 1.921$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | openrouter |
| Model | meta/muse-spark-1.2 |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 89.23 % |
| Test time | 246.69 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Nave","first_name":"Karl Heinz"},"publication":{"title":"Das Grundschulproblem, betrachtet im Zusammenhang mit der Entstehung der deutschen Grundschule nach der Novemberrevolution von 1918 und aus der Sichtweise der modernen anthropologischen Forschung. Eine schulpolit. Studie.","year":"1959","place":"o.O.","pages":"359","publisher":"","format":"8'","editor":null},"library_reference":{"shelfmark":"Diss.Köln 1959","subjects":""}}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.90 | 0.89 | 0.90 | 0.90 | 262 | 2170 | 229 | 237 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 489.3K IT + 979.0K OT = 1.5M TT | Cost: 0.612$ + 4.161$ = 4.772$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | genai |
| Model | gemini-3.7-flash |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 87.51 % |
| Test time | 3.97 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Schwarz","first_name":"C W G Eduard"},"publication":{"title":"Die Iphigenien-Sage und ihre dramatischen Bearbeitungen.","year":"1869","place":"Leipzig","pages":"35","publisher":null,"format":"8.","editor":null},"library_reference":{"shelfmark":"Rostock. Diss. phil. 1869.","subjects":null}}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.88 | 0.88 | 0.87 | 0.90 | 263 | 2178 | 333 | 237 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 406.2K IT + 49.6K OT = 455.8K TT | Cost: 0.305$ + 0.186$ = 0.491$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | x-ai |
| Model | grok-4.6 |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 89.39 % |
| Test time | 45.24 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Walch","first_name":"Ludwig"},"publication":{"title":"Die rechte Gewere im Vergleich zur Tabularersitzung und dem Eigentumserwerb durch Aufgebotsverfahren nach bürgerlichem Recht","year":"1912","place":"Bühl (Baden)","pages":"VIII, 37","publisher":"Konkordia A.-G.","format":"8°","editor":null},"library_reference":{"shelfmark":"U 12. 2216","subjects":null}}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.90 | 0.89 | 0.89 | 0.91 | 263 | 2205 | 259 | 210 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 523.1K IT + 33.1K OT = 556.3K TT | Cost: 1.046$ + 0.199$ = 1.245$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | huggingface |
| Model | swiss-ai/apertus-v1.5-70b |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 43.52 % |
| Test time | 189.39 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{
"type": {
"type": "Dissertation or thesis"
},
"author": {
"last_name": "Müller",
"first_name": "Johannes"
},
"publication": {
"title": "Kapitalaanschub in ein Handelsgewerbe",
"year": 1921,
"place": "",
"pages": "45 S.",
"publisher": "",
"format": "8°"
},
"library_reference": {
"shelfmark": "Diss. v. 28. Nov. 1921 [1422]",
"subjects": ""
}
}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.44 | 0.44 | 0.45 | 0.43 | 263 | 1046 | 1293 | 1369 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 808.6K IT + 46.0K OT = 854.6K TT | Cost: 0.663$ + 0.134$ = 0.797$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | huggingface |
| Model | Qwen/Qwen3-VL-235B-A22B-Instruct |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 75.49 % |
| Test time | 369.50 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{
"type": {
"type": "Dissertation or thesis"
},
"author": {
"last_name": "Widmer",
"first_name": "Ulrich Rudolf"
},
"publication": {
"title": "Ladungsaccelerierte und ladungskontrollierte aromatische sigmatropische Umlagerungen.",
"year": 1974,
"place": "Freiburg i.Br.",
"pages": "107 Seiten, Tab.",
"publisher": "Zürich, Univ., Phil. Fak. II",
"format": "8°"
},
"library_reference": {
"shelfmark": "Diss.1975,2594",
"subjects": ""
}
}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.76 | 0.75 | 0.72 | 0.81 | 263 | 1957 | 768 | 458 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 346.8K IT + 49.8K OT = 396.6K TT | Cost: 0.069$ + 0.044$ = 0.113$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | huggingface |
| Model | MiniMaxAI/MiniMax-M3 |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 76.06 % |
| Test time | 12.20 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Wehmeyer","first_name":"Waltraut"},"publication":{"title":"Die Produktionsanlagen als Datum und Objekt unternehmerischer Dispositionen","year":"place","place":"München","pages":"250,XVI","publisher":"","format":"4°","editor":"Diss.Staatswirtsch.Fak.Univ.München"},"library_reference":{"shelfmark":"Diss.München Univ.1960 4°","subjects":"Unternehmen.Führung/Investition,betriebswirtschaftliche/Betriebswirtschaftslehre/Produktionsmittel/ S Diss"}}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.76 | 0.76 | 0.73 | 0.81 | 263 | 1949 | 734 | 466 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 323.6K IT + 52.0K OT = 375.6K TT | Cost: 0.091$ + 0.057$ = 0.148$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | huggingface |
| Model | thinkingmachines/Inkling-Small |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 62.31 % |
| Test time | 372.18 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{
"type": {
"type": "Dissertation or thesis"
},
"author": {
"last_name": "Wolfsohn",
"first_name": "Wilhelm"
},
"publication": {
"title": "Ein Fall von Melaena neonatorum",
"year": 1900,
"place": "Leipzig",
"pages": "31",
"publisher": "B. Georgi",
"format": "8°"
},
"library_reference": {
"shelfmark": "Med. Diss. v. (20. Jan.) 1900",
"subjects": ""
}
}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.72 | 0.62 | 0.81 | 0.65 | 263 | 1567 | 378 | 848 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 508.9K IT + 410.5K OT = 919.4K TT | Cost: 0.229$ + 0.493$ = 0.722$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | huggingface |
| Model | thinkingmachines/Inkling |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 70.66 % |
| Test time | 33.34 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
The user wants me to extract bibliographic information from the image and return it as JSON matching the provided schema.
Looking at the image:
- It's a card about "Oberbeck, Erich: Die Stichomythie im deutschen Drama. (Renaissance und Klassizismus.) Eine Untersuchung über ihr Wesen und ihre Technik. Göttingen 1919: Dieterich. 79 S. 8°"
- Then: "Göttingen, Phil. Diss. v. 18. Juni 1918 [1920], Ref. Weißenfels"
- Then biographical info: "[Geb. 30. Mai 91 Dützfleth; Wohnort: Kuxhaven; Staatsangeh.: Hamburg; Vorbildung: G. Altona Reife 10; Studium: Berlin 7, Göttingen 2 S.; Rig. 25. Juli 17.] [U 20. 3570"
Now applying rules:
1. Card Type: Does it contain "s." on a separate line? Yes, "79 S. 8°" - wait, "s." is present but not on a separate line? Actually "79 S. 8°" is on the same line. But wait, the rule says: "If a card contains the note 's.' on a separate line, it is a 'Reference'." Here "79 S." is part of the publication line. But also there's "Göttingen 2 S." in the bio. However, the card is clearly a dissertation card ("Phil. Diss."). The rule might be interpreted differently, but looking at the card, it's a dissertation record. Actually wait, the rule says if "s." is on a separate line. Here "79 S. 8°" is on one line. But there's also "2 S." in the bio line. However, the card is about a dissertation. Actually, I think the card type should be "Dissertation or thesis" because it's a "Phil. Diss." record. The "s." rule might apply to reference cards that just cite something. But let me check: the card contains "79 S." which includes "S." but not on a separate line by itself. Actually "79 S." is together. So it's not "s." on a separate line. Thus it's "Dissertation or thesis".
Wait, but the user said: "If a card contains the note 's.' on a separate line, it is a 'Reference'." In this card, is there "s." alone? No. So type is "Dissertation or thesis".
2. Author: Oberbeck, Erich. So last_name: "Oberbeck", first_name: "Erich".
3. Publication:
- title: "Die Stichomythie im deutschen Drama. (Renaissance und Klassizismus.) Eine Untersuchung über ihr Wesen und ihre Technik."
- year: 1919 (from "Göttingen 1919")
- place: "Göttingen"
- pages: "79" (remove " S." suffix -> "79")
- publisher: "Dieterich"
- format: "8°"
Wait, the schema requires year as string? Actually in the schema provided at the end, year is "string". But in the initial instructions, it said "year: Publication year as integer". However, the final schema says "year": {"title": "Year", "type": "string"}. I need to match the exact schema provided at the end. The user said "Return ONLY the JSON object" and "You MUST respond with valid JSON matching this exact schema". The schema shows year as string. But the initial extraction rules said integer. There's a conflict. Let me check the schema carefully.
In the schema:
```
"publication": {
...
"year": {"title": "Year", "type": "string"},
...
}
```
So year should be a string according to the schema. But the initial rules said "year: Publication year as integer". However, the user said "You MUST respond with valid JSON matching this exact schema". So I should follow the schema: year as string. Actually wait, in the initial format example:
```json
"year": integer,
```
But then the schema at the end says string. The user explicitly said "You MUST respond with valid JSON matching this exact schema". So I will use string for year? Or should I use integer? The schema is the authoritative format for the response. But the initial instructions said integer. Let me look at the schema again: it says `"year": {"title": "Year", "type": "string"}`. So I must output a string. Actually, wait, in JSON, if I put 1919 without quotes, it's an integer, which wouldn't match the schema type "string". So I should put "1919" as a string.
But wait, the initial extraction rules said: "year: Publication year as integer" and "omit year fields entirely if not present". But the schema requires year (it's in required: ["title", "year"]). So I must include it. Given the conflict, I should prioritize the schema which requires it as string. Actually, maybe the schema is just a generic JSON schema and the user wants the initial format? The user said: "return it as a structured JSON object with the following exact format" and then showed a format with integer. Then at the end said "You MUST respond with valid JSON matching this exact schema" with the schema showing string. This is ambiguous. However, usually in these prompts, the schema at the end is the validation schema. But the initial format is what they want. Actually, the initial format is:
```json
{
...
"publication": {
...
"year": integer,
...
},
...
}
```
And the schema at the end is a JSON Schema that describes the object. In that schema, year is string. But maybe the user made a mistake in the schema. However, they said "matching this exact schema". So I should adhere to the schema. But if I put an integer, it violates the schema. If I put a string, it violates the initial example. Which one is more likely to be checked? Usually these prompts have an automatic validator using the schema. So I should match the schema: year as string.
Wait, actually looking closely at the schema:
```
"Publication": {
...
"properties": {
"title": {"title": "Title", "type": "string"},
"year": {"title": "Year", "type": "string"},
...
},
"required": ["title", "year"],
...
}
```
Yes, it's string. So I'll output "1919".
But wait, there's also "editor" in the schema that wasn't in the initial format. The initial format didn't have editor. The schema includes it but it's not required. I should omit it or set to null? The schema allows null. But the initial format doesn't include it. Since the user said "with the following exact format" initially, but then "matching this exact schema" at the end, I should probably include only the fields in the initial format, but arranged according to the schema? Actually the schema defines the object structure. The initial format shows:
```json
{
"type": { ... },
"author": { ... },
"publication": { ... },
"library_reference": { ... }
}
```
The schema also has these exact top-level fields. The publication in the schema has extra "editor" field, but since it's not required, I can omit it. Actually in JSON Schema, if a property is not in "required", it can be omitted. So I will omit "editor".
Now for library_reference:
- shelfmark: "Diss." or "AT"? The card says "Göttingen, Phil. Diss. v. 18. Juni 1918 [1920], Ref. Weißenfels" and at bottom right "[U 20. 3570]". The shelfmark might be "U 20. 3570" or "Diss. v. 18. Juni 1918". The rule says: "Often begins with 'Diss.' or 'AT', may be marked with 'Standort:'". Here we have "Phil. Diss." mentioned. But also there's "[U 20. 3570]" which looks like a shelfmark. Actually, in library cards, the shelfmark is often in brackets at the bottom right. So shelfmark: "U 20. 3570". But also there's "Göttingen, Phil. Diss. v. 18. Juni 1918 [1920]" which describes the dissertation. The rule says shelfmark often begins with "Diss." or "AT". Here "Phil. Diss." is part of the reference. But I think the shelfmark is "U 20. 3570". However, the card also mentions "Ref. Weißenfels" which might be part of subjects? Actually "Ref. Weißenfels" is likely the referee (Gutachter). But the rule says subjects are "Subject classifications or keywords". "Ref. Weißenfels" is not a subject. The biographical details are to be ignored.
Wait, let's re-read the card carefully:
"Oberbeck, Erich: Die Stichomythie im deutschen Drama. (Renaissance und Klassizismus.) Eine Untersuchung über ihr Wesen und ihre Technik. Göttingen 1919: Dieterich. 79 S. 8°
Göttingen, Phil. Diss. v. 18. Juni 1918 [1920], Ref. Weißenfels
[Geb. 30. Mai 91 Dützfleth; Wohnort: Kuxhaven; Staatsangeh.: Hamburg; Vorbildung: G. Altona Reife 10
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.77 | 0.71 | 0.77 | 0.76 | 263 | 1837 | 540 | 578 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 539.5K IT + 292.0K OT = 831.6K TT | Cost: 0.540$ + 1.183$ = 1.722$ |
index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic
| Provider | huggingface |
| Model | meta-models/Muse-Glimmer-30B |
| Temperature | 0.0 |
| Dataclass | Document |
| Normalized Score | 88.94 % |
| Test time | 388.46 seconds |
Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:
```json
{
"type": {
"type": "Dissertation or thesis" OR "Reference"
},
"author": {
"last_name": "string",
"first_name": "string"
},
"publication": {
"title": "string",
"year": integer,
"place": "string or empty string",
"pages": "string or empty string",
"publisher": "string or empty string",
"format": "string or empty string"
},
"library_reference": {
"shelfmark": "string or empty string",
"subjects": "string or empty string"
}
}
```
EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".
2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.
3. **Publication**:
- title: The main title of the work
- year: Publication year as integer
- place: Publication place
- pages: Page count (remove " S." suffix if present)
- publisher: Publishing house/institution
- format: Usually "8°", "8'", or "4°" - single value only
4. **Library Reference**:
- shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
- subjects: Subject classifications or keywords
5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.
6. **Ignore**: Disregard any information that doesn't fit into these categories.
Return ONLY the JSON object, no additional text or explanation.
{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Teufel","first_name":"Georg"},"publication":{"title":"Ueber einen Fall von multiplen Missbildungen mit besonderer Berücksichtigung der congenitalen Hüftluxation und deren operativer Behandlung","year":"1888","place":"Strassburg","pages":"56","publisher":"Univ.-Buchdr. v. J. H. Ed. Heitz (Heitz & Mündel)","format":"8°","editor":null},"library_reference":{"shelfmark":"Strassburg, Med. Fak., Inaug.-Diss. v. 1888","subjects":""}}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.90 | 0.89 | 0.89 | 0.91 | 263 | 2188 | 282 | 227 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 503.1K IT + 664.1K OT = 1.2M TT | Cost: 0.176$ + 0.996$ = 1.172$ |