RISE Humanities Data Benchmark, 0.5.5-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results
Show compact results Refine Search New Search
Download JSON Download CSV

Your search for Benchmark 'library_cards__true' with Search Hidden 'False' returned 119 results, showing page 2 of 12.
Result 11 of 119

Test T1485 at 2026-08-14

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Providerhuggingface
Modelswiss-ai/apertus-v1.5-70b
  
Temperature0.0
DataclassDocument
  
Normalized Score44.05 %
Test time186.68 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{
  "type": {
    "type": "Dissertation or thesis"
  },
  "author": {
    "last_name": "Müller",
    "first_name": "Johannes"
  },
  "publication": {
    "title": "Kapitalaanschub in ein Handelsgewerbe",
    "year": 1921,
    "place": "",
    "pages": "45 S.",
    "publisher": "",
    "format": "8°"
  },
  "library_reference": {
    "shelfmark": "Diss. v. 28. Nov. 1921 [1422]",
    "subjects": ""
  }
}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.45 0.44 0.45 0.44 263 1060 1288 1355
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 811.0K IT + 46.0K OT = 856.9K TTCost: 0.665$ + 0.134$ = 0.799$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 12 of 119

Test T1500 at 2026-08-14

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Providerhuggingface
Modelswiss-ai/apertus-v1.5-8b
  
Temperature0.0
DataclassDocument
  
Normalized Score48.67 %
Test time14.53 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Wolters","first_name":"Maria"},"publication":{"title":"Ueber die 'Bedeutung der Verebung in der Pathogenese der Dystrophia adiposa-genitalis.","year":"1936","place":"","pages":"31","publisher":"","format":"8°","editor":"Boss."},"library_reference":{"shelfmark":"Diss. 1937, 8, 597","subjects":""}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.49 0.49 0.46 0.52 263 1260 1494 1155
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 794.8K IT + 40.1K OT = 834.9K TTCost: 0.079$ + 0.008$ = 0.088$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 13 of 119

Test T1455 at 2026-08-04

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Providerx-ai
Modelgrok-4.5
  
Temperature0.0
DataclassDocument
  
Normalized Score89.98 %
Test time16.39 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Van der Meulen","first_name":"Ageus Jacobus"},"publication":{"title":"Studies over het ministerie van Van de Spiegel.","year":"1905","place":"Leiden","pages":"XI,559","publisher":"Kooyker","format":"8°","editor":null},"library_reference":{"shelfmark":"AT 27","subjects":null}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.91 0.90 0.90 0.92 263 2219 248 196
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 523.1K IT + 43.6K OT = 566.7K TTCost: 1.046$ + 0.261$ = 1.308$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 14 of 119

Test T1470 at 2026-08-04

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Provideropenrouter
Modelqwen/qwen3.8-max
  
Temperature0.0
DataclassDocument
  
Normalized Score88.73 %
Test time157.63 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{
  "type": {
    "type": "Dissertation or thesis"
  },
  "author": {
    "last_name": "Schwieler",
    "first_name": "Gösta H"
  },
  "publication": {
    "title": "Respiratory regulation during postnatal development in cats and rabbits and some of its morphological substrate",
    "year": 1968,
    "place": "Stockholm",
    "pages": "123",
    "publisher": "Akad.avh.Stockholm",
    "format": "8'"
  },
  "library_reference": {
    "shelfmark": "Diss.1969,1129",
    "subjects": "2/Hauskatze.Atmungsorgane,Lunge/Kaninchen.Atmungsorgane,Lunge/ M NK GK Diss"
  }
}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.90 0.89 0.89 0.91 263 2192 269 223
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 476.9K IT + 839.5K OT = 1.3M TTCost: 0.532$ + 2.805$ = 5.942$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 15 of 119

Test T1410 at 2026-07-25

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Provideranthropic
Modelclaude-opus-5
  
Temperature0.0
DataclassDocument
  
Normalized Score75.10 %
Test time6.13 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"schema": {"type": {"type": "Dissertation or thesis"}, "author": {"last_name": "Weber", "first_name": "Charlotte"}, "publication": {"title": "Ursachen der Kinderversorgung. Eine Untersuchung auf Grund der Versorgungsf\u00e4lle der F\u00fcrsorgedirektion des Kantons Bern", "year": "1951", "place": "Bern", "pages": "150", "publisher": "Verbandsdr.", "format": "8'"}, "library_reference": {"shelfmark": "Diss. Bern 1951", "subjects": ""}}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.76 0.75 0.73 0.80 263 1926 718 489
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 769.4K IT + 74.2K OT = 843.6K TTCost: 3.847$ + 1.856$ = 5.703$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 16 of 119

Test T1440 at 2026-07-25

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Providergenai
Modelgemini-3.5-flash-lite
  
Temperature0.0
DataclassDocument
  
Normalized Score86.35 %
Test time1.48 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Teufel","first_name":"Georg"},"publication":{"title":"Ueber einen Fall von multiplen Missbildungen mit besonderer Berücksichtigung der congenitalen Hüftluxation und deren operativer Behandlung.","year":"1888","place":"Strassburg","pages":"56","publisher":"Univ.-Buchdr. v. J. H. Ed. Heitz (Heitz & Mündel)","format":"8°","editor":null},"library_reference":{"shelfmark":null,"subjects":"Strassburg, Med. Fak., Inaug.-Diss. v. 1888"}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.87 0.86 0.84 0.90 263 2182 419 233
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 406.2K IT + 49.5K OT = 455.7K TTCost: 0.122$ + 0.124$ = 0.246$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 17 of 119

Test T1380 at 2026-07-25

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Provideropenai
Modelgpt-5.6-luna
  
Temperature0.0
DataclassDocument
  
Normalized Score84.45 %
Test time6.05 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Trockel","first_name":"Walter"},"publication":{"title":"Ein Grenzwertsatz für den Kern","year":"1975","place":"Bonn","pages":"63","publisher":"","format":"8°","editor":null},"library_reference":{"shelfmark":"Diss.1976,170","subjects":"Tausch; Wirtschaftsmathematik"}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.85 0.84 0.83 0.87 263 2101 431 314
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 444.4K IT + 124.5K OT = 568.9K TTCost: 0.444$ + 0.747$ = 1.191$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 18 of 119

Test T1350 at 2026-07-25

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Provideropenai
Modelgpt-5.6-sol
  
Temperature0.0
DataclassDocument
  
Normalized Score86.76 %
Test time3.82 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Sünkel","first_name":"Werner"},"publication":{"title":"Eine Zweikanal-Rechnung für die elastische Streuung von He³ an He³","year":"1968","place":"Tübingen","pages":"56","publisher":"","format":"8'","editor":null},"library_reference":{"shelfmark":"Diss.1969,780","subjects":""}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.87 0.87 0.87 0.88 263 2127 330 288
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 444.4K IT + 90.5K OT = 534.9K TTCost: 2.222$ + 2.713$ = 4.936$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 19 of 119

Test T1365 at 2026-07-25

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Provideropenai
Modelgpt-5.6-terra
  
Temperature0.0
DataclassDocument
  
Normalized Score84.56 %
Test time7.36 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Ammann","first_name":"Isaac"},"publication":{"title":"Dissertatio inauguralis medica exhibens historiam succinctam de morbis virginum","year":"1770","place":"Argentorati","pages":"42","publisher":"","format":"4°","editor":null},"library_reference":{"shelfmark":"Diss. 107, No. 15","subjects":""}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.85 0.85 0.84 0.86 263 2078 387 337
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 444.4K IT + 75.0K OT = 519.4K TTCost: 1.111$ + 1.125$ = 2.236$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752
Result 20 of 119

Test T1395 at 2026-07-25

index-card information-extraction typed, printed, handwritten 20, 19 de, fr, en, la, el, fi, sv, pl index bibliographic

Configuration
Provideropenrouter
Modelmoonshotai/kimi-k3
  
Temperature0.0
DataclassDocument
  
Normalized Score88.48 %
Test time51.92 seconds
Prompt 

Extract the bibliographic information about a historical dissertation from this index card and return it as a structured JSON object with the following exact format:

```json
{
  "type": {
    "type": "Dissertation or thesis" OR "Reference"
  },
  "author": {
    "last_name": "string",
    "first_name": "string"
  },
  "publication": {
    "title": "string",
    "year": integer,
    "place": "string or empty string",
    "pages": "string or empty string",
    "publisher": "string or empty string",
    "format": "string or empty string"
  },
  "library_reference": {
    "shelfmark": "string or empty string",
    "subjects": "string or empty string"
  }
}
```

EXTRACTION RULES:
1. **Card Type**: If a card contains the note "s." on a separate line, it is a "Reference". Otherwise, it is a "Dissertation or thesis".

2. **Author**: Extract last_name and first_name. If only one name is given, put it in last_name and leave first_name empty.

3. **Publication**:
   - title: The main title of the work
   - year: Publication year as integer
   - place: Publication place
   - pages: Page count (remove " S." suffix if present)
   - publisher: Publishing house/institution
   - format: Usually "8°", "8'", or "4°" - single value only

4. **Library Reference**:
   - shelfmark: Often begins with "Diss." or "AT", may be marked with "Standort:"
   - subjects: Subject classifications or keywords

5. **Missing Information**: Use empty string "" for missing text fields, omit year fields entirely if not present.

6. **Ignore**: Disregard any information that doesn't fit into these categories.

Return ONLY the JSON object, no additional text or explanation.

Results 

{"type":{"type":"Dissertation or thesis"},"author":{"last_name":"Senarclens","first_name":"Victor de"},"publication":{"title":"Contribution à l'étude de l'épithéliome malin du Chorion et de ses rapports avec la môle hydatique","year":">>> 1902 <<<","place":"Lausanne","pages":"52","publisher":"Service annexe de gynécologie et Institut pathologique de Lausanne","format":"8°","editor":""},"library_reference":{"shelfmark":"Lausanne, Thèse méd. 1901/1902","subjects":""}}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.89 0.88 0.88 0.90 263 2168 286 247
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/a, n/a. Tokens: 516.1K IT + 850.7K OT = 1.4M TTCost: 1.536$ + 12.679$ = 14.302$
Cite: Hindermann, Marti, Alberto, et al., (2025). RISE-UNIBAS/humanities_data_benchmark, 10.5281/zenodo.16941752