RISE Humanities Data Benchmark, 0.5.2-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results

Your search for Benchmark 'fraktur_adverts__true' with Search Hidden 'False' returned 142 results, showing page 6 of 15.
Result 51 of 142

Test T0503 at 2026-01-25

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Providergenai
Modelgemini-3-flash-preview
  
Temperature0.0
DataclassDocument
  
Normalized Score95.90 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.97 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 8.1K IT + 10.3K OT = 18.3K TTCost: 0.004$0.031$0.035$
Result 52 of 142

Test T0547 at 2026-01-25

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Providermistral
Modelministral-14b-2512
  
Temperature0.0
DataclassDocument
  
Normalized Score0.00 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.00 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 3.9K IT + 55 OT = 4.0K TTCost: 0.001$0.000$0.001$
Result 53 of 142

Test T0121 at 2026-01-25

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Provideropenai
Modelgpt-5-mini-2025-08-07
  
Temperature0.0
DataclassDocument
  
Normalized Score79.40 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.84 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 10.9K IT + 24.2K OT = 35.1K TTCost: 0.003$0.048$0.051$
Result 54 of 142

Test T0137 at 2026-01-25

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Provideropenai
Modelo3-2025-04-16
  
Temperature0.0
DataclassDocument
  
Normalized Score16.50 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.17 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 7.4K IT + 12.7K OT = 20.2K TTCost: 0.015$0.102$0.117$
Result 55 of 142

Test T0569 at 2026-01-25

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Providermistral
Modelmagistral-small-2509
  
Temperature0.0
DataclassDocument
  
Normalized Score0.00 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.00 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 3.9K IT + 403 OT = 4.3K TTCost: 0.002$0.000$0.002$
Result 56 of 142

Test T0407 at 2026-01-25

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Provideropenai
Modelgpt-5.1-2025-11-13
  
Temperature0.0
DataclassDocument
  
Normalized Score45.60 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.58 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 7.2K IT + 7.9K OT = 15.1K TTCost: 0.009$0.079$0.088$
Result 57 of 142

Test T0514 at 2026-01-24

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Provideranthropic
Modelclaude-opus-4-5-20251101
  
Temperature0.0
DataclassDocument
  
Normalized Score38.00 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.38 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 6.0K IT + 5.8K OT = 11.9K TTCost: 0.030$0.146$0.176$
Result 58 of 142

Test T0084 at 2026-01-24

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Provideropenai
Modelgpt-4.1-mini-2025-04-14
  
Temperature0.0
DataclassDocument
  
Normalized Score95.70 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.97 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 13.5K IT + 6.4K OT = 19.8K TTCost: 0.005$0.010$0.016$
Result 59 of 142

Test T0178 at 2026-01-24

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Providermistral
Modelmistral-medium-2505
  
Temperature0.0
DataclassDocument
  
Normalized Score0.00 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.00 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 3.9K IT + 834 OT = 4.8K TTCost: 0.002$0.002$0.003$
Result 60 of 142

Test T0099 at 2026-01-24

{'document-type': ['book-page'], 'writing': ['printed'], 'century': [18, 19], 'language': ['de'], 'layout': ['prose'], 'script-style': ['fraktur'], 'task': ['transcription']}

Configuration
Provideranthropic
Modelclaude-sonnet-4-20250514
  
Temperature0.0
DataclassDocument
  
Normalized Score13.10 %
Test timeunknown seconds
Prompt

## IDENTITY AND PURPOSE

You are an OCR and information extraction system trained to process historical newspaper pages printed in 18th-century German using Fraktur type. The pages contain mostly classified advertisements. Your task is to identify and extract each advertisement *exactly as printed*, including historical spellings, typographic errors, punctuation, and formatting.


## INSTRUCTIONS

- Extract **all advertisements** from the input image, one after the other, following the sequence on the page.
- Maintain the **original spelling**, capitalization, and any **typos or non-standard forms**.
- Follow these transcription rules: 
  - the long s (ſ) is transcribed as "s"
  - "/" is transcribed as ","
- Use the masthead of the newspaper only to extract the date, ignore other content.
- The layout is typically **two-column**; extract ads from both columns, including the ad number.
- Return the result as a **JSON object** in the specified format and **nothing else** (no explanations, summaries, or additional text).
- For each advertisement, include:
  - `"date"`: the publication date of the page in ISO 8061 format (YYYY-MM-DD)
  - `"tags_section"`: the heading under which the advertisement appears
  - `"text"`: the full advertisement text

## EXAMPLE OUTPUT

{
  "advertisements": [
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "5. Ein kleines, jedoch listiges Lehrbuch der Zauberkunst, lange im Gebrauche des jungen Bartolomeus Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zum Verkauff offeriert",
      "text": "6. Ein rarer, mit Edelsteinen besetzter Saxophon-Kasten, aus dem Besitze der Jungfer Lisa Simpson."
    },
    {
      "date": "1731-01-02",
      "tags_section": "Es werden zu Entleihen begehrt",
      "text": "7. Ein gar prachtvoller, jedoch etwas zerlesener Band mit Rezepten von Margaretha Simpsonin."
    }
  ]
}

Results

no valid result

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
0.14 n/a n/a n/a n/a n/a n/a n/a n/a
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 5.5K IT + 11.0K OT = 16.5K TTCost: 0.016$0.165$0.181$