RISE Humanities Data Benchmark, 0.5.5-pre1

Search Test Runs

 

A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.

A test run includes:

  • Prompt and role definition – what the model was asked to do and from what perspective (e.g. “as a historian”).
  • Model configuration – provider, model version, temperature, and other generation parameters.
  • Results – the model’s actual response and its evaluation (scores such as F1 or accuracy).
  • Usage and cost data – token counts and calculated API costs.
  • Metadata – information like the test date, benchmark name, and person who executed it.

Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.

Search Results

Your search for Benchmark 'company_lists__true' with Search Hidden 'False' returned 212 results, showing page 1 of 22.
Result 1 of 212

Test T1638 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Provideropenrouter
Modelz-ai/glm-5v-turbo
  
Temperature0.5
DataclassListPage
  
Normalized Score55.00 %
Test time78.63 seconds
Prompt

The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.

About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.

About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".

About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.

{
  "entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
  "company_name": "The name of the company or person",
  "location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
  ]
}

Results

{"page_id":"IX","entries":[{"entry_id":"IX-1","company_name":"Abegg & Co. Ltd.","location":"Zurich"},{"entry_id":"IX-2","company_name":"Acontra A.G.","location":"Zurich"},{"entry_id":"IX-3","company_name":"Adam & Co. Ltd.","location":"Port Louis, Mauritius"},{"entry_id":"IX-4","company_name":"Adroka Ltd.","location":"Basle"},{"entry_id":"IX-5","company_name":"Aebi (Robert) Ltd.","location":"Zurich"},{"entry_id":"IX-6","company_name":"Aksa Ltd.","location":"Würenlos, Aargau"},{"entry_id":"IX-7","company_name":"Albeko Shoe Machinery Ltd.","location":"Basle"},{"entry_id":"IX-8","company_name":"Albion House Ltd.","location":"Lausanne"},{"entry_id":"IX-9","company_name":"Albiswerk Zurich A.G.","location":"Zurich"},{"entry_id":"IX-10","company_name":"Aluminium Commercial S.A.","location":"Zurich"},{"entry_id":"IX-11","company_name":"Aluminium Limited (Canada) S.A.","location":"Geneva"},{"entry_id":"IX-12","company_name":"Aluminiumwerke A.G.","location":"Rorschach"},{"entry_id":"IX-13","company_name":"Aluminium Industrie A.G.","location":"Zurich"},{"entry_id":"IX-14","company_name":"Amalgamations Ltd.","location":"Madras, India"},{"entry_id":"IX-15","company_name":"Amag Automobil- und Motoren A.G.","location":"Zurich"},{"entry_id":"IX-16","company_name":"Amida Ltd.","location":"Grenchen"},{"entry_id":"IX-17","company_name":"Amidro (Swiss Druggists' & Chem. Purchasing Assoc.)","location":"Bienne"},{"entry_id":"IX-18","company_name":"Ampack, Hungerbühler & Lemm K.G.","location":"Rorschach"},{"entry_id":"IX-19","company_name":"Amsler (Alfred J.) & Co.","location":"Schaffhausen"},{"entry_id":"IX-20","company_name":"Andres (Rudolf) A.G.","location":"St. Gallen"},{"entry_id":"IX-21","company_name":"André & Co. Ltd.","location":"Lausanne"},{"entry_id":"IX-22","company_name":"Angst & Pfister Ltd.","location":"Zurich"},{"entry_id":"IX-23","company_name":"Annabelle (François Bloch)","location":"Lausanne"},{"entry_id":"IX-24","company_name":"Anra S. à r.l.","location":"Territet-Montreux"},{"entry_id":"IX-25","company_name":"Anselmi jr. (Carl)","location":"Solothurn"},{"entry_id":"IX-26","company_name":"Aplin & Barrett Ltd.","location":"Yeovil, Somerset"},{"entry_id":"IX-27","company_name":"Appleyard (John W.) & Sons Ltd.","location":"Morley, Yorks"},{"entry_id":"IX-28","company_name":"Arts Ménagers S.A.","location":"Geneva"},{"entry_id":"IX-29","company_name":"Association of Swiss Machine Tool and Tool Dealers","location":"Zurich"},{"entry_id":"IX-30","company_name":"«Astra» Fat & Oil Works Ltd.","location":"Steffisburg"},{"entry_id":"IX-31","company_name":"Atlantis Verlag A.G.","location":"Zurich"},{"entry_id":"IX-32","company_name":"Attenhofer (A.)","location":"Zurich"},{"entry_id":"IX-33","company_name":"Aubry Brothers, Ciny Watch Ltd.","location":"Le Noirmont"},{"entry_id":"IX-34","company_name":"Baader (Gustav)","location":"Basle"},{"entry_id":"IX-35","company_name":"Bachmann (Henri)","location":"Bienne"},{"entry_id":"IX-36","company_name":"Bachofen (Willy A.)","location":"Basle"},{"entry_id":"IX-37","company_name":"Baezner & Co","location":"Geneva"},{"entry_id":"IX-38","company_name":"Baker (John) & Bessemer Ltd.","location":"Rotherham"},{"entry_id":"IX-39","company_name":"Bally (C. F.) Ltd.","location":"Zurich"},{"entry_id":"IX-40","company_name":"Bally's Shoe Factories Ltd.","location":"Schönenwerd"},{"entry_id":"IX-41","company_name":"Bâloise Marine Insurance Co. Ltd.","location":"Basle"},{"entry_id":"IX-42","company_name":"Bangarter (A.) & Co. Ltd.","location":"Lyss"},{"entry_id":"IX-43","company_name":"Banque Galland & Co. S.A.","location":"Lausanne"},{"entry_id":"IX-44","company_name":"Banque de Paris et des Pays Bas S.A.","location":"Geneva"},{"entry_id":"IX-45","company_name":"Barclays Bank Ltd.","location":"London"},{"entry_id":"IX-46","company_name":"Barco Trading Co. Ltd.","location":"Geneva"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.55 0.55 0.56 0.55 14 509 408 410
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 62.0K IT + 41.8K OT = 103.8K TTCost: 0.074$0.167$0.242$
Result 2 of 212

Test T1624 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Provideropenrouter
Modelmeta/muse-spark-1.2
  
Temperature0.5
DataclassListPage
  
Normalized Score26.53 %
Test time11.84 seconds
Prompt

- Answer in valid JSON.
- The page ID is given as {page_id}.

Results

{"page_id":"{page_id}.","entries":[{"entry_id":".","company_name":"Swiss Bank Corporation","location":"Lausanne"},{"entry_id":".","company_name":"Swiss Bank Corporation","location":"Neuch\u0000e2tel"},{"entry_id":".","company_name":"Union Bank of Switzerland","location":"Z\u0000fcrich"},{"entry_id":".","company_name":"Barcham Green & Sons","location":"Hayle Mill Maidstone, Kent"},{"entry_id":".","company_name":"Herren & Guerchet","location":"Rue Jean-Petitot 10 Gen\u0000e8ve"},{"entry_id":".","company_name":"Louis Kohler","location":"11, St. Pierre, Lausanne"},{"entry_id":".","company_name":"Giuliniwerke A. G.","location":"Basel"},{"entry_id":".","company_name":"Georges Michoud All kinds","location":"Lausanne"},{"entry_id":".","company_name":"August Fr\u0000f6hlich Leather","location":"Bahnhofstrasse 110 Z\u0000fcrich"},{"entry_id":".","company_name":"Altherr & Guex","location":"Flawil"},{"entry_id":".","company_name":"Arbon Embroidery Works Ltd.","location":"Arbon"},{"entry_id":".","company_name":"Grieder & Co.","location":"Paradeplatz Z\u0000fcrich"},{"entry_id":".","company_name":"A. C. Pearse Ltd.","location":"14, Pilcher Gate Nottingham"},{"entry_id":".","company_name":"Zuberb\u0000fchler & Co., A. G.","location":"Zurzach"},{"entry_id":".","company_name":"Sulzer Fr\u0000e8res S. A.","location":"Winterthur"},{"entry_id":".","company_name":"Schw. Seidengazefabrik A. G.","location":"Thal, bei Rheineck"},{"entry_id":".","company_name":"Schw. Seidengazefabrik A. G.","location":"Z\u0000fcrich"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.28 0.27 0.29 0.27 15 263 634 729
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 55.8K IT + 30.5K OT = 86.3K TTCost: 0.070$0.130$0.199$
Result 3 of 212

Test T1639 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Provideropenrouter
Modelz-ai/glm-5v-turbo
  
Temperature0.5
DataclassListPage
  
Normalized Score45.55 %
Test time41.66 seconds
Prompt

- Answer in valid JSON.
- The page ID is given as {page_id}.

Results

{
  "page_id": "61",
  "entries": [
    {
      "entry_id": "1",
      "company_name": "Nardin, Ulysse",
      "location": ""
    },
    {
      "entry_id": "2",
      "company_name": "Natural, Le Coultre & Cie., S. A.",
      "location": ""
    },
    {
      "entry_id": "3",
      "company_name": "Nestlé & Anglo-Swiss Condensed Milk Co., Ltd.",
      "location": ""
    },
    {
      "entry_id": "4",
      "company_name": "Neuburger & Co.",
      "location": ""
    },
    {
      "entry_id": "5",
      "company_name": "North British Rubber Co., Ltd., The",
      "location": ""
    },
    {
      "entry_id": "6",
      "company_name": "Obrist & Co.",
      "location": ""
    },
    {
      "entry_id": "7",
      "company_name": "Paillard, E. & Cie., S. A.",
      "location": ""
    },
    {
      "entry_id": "8",
      "company_name": "Paillard & Benoit",
      "location": ""
    },
    {
      "entry_id": "9",
      "company_name": "Palace Hotel",
      "location": "Montana"
    },
    {
      "entry_id": "10",
      "company_name": "Pearse, A. C. & Co., Ltd.",
      "location": ""
    },
    {
      "entry_id": "11",
      "company_name": "Perrin & Cie.",
      "location": ""
    },
    {
      "entry_id": "12",
      "company_name": "Peter, Cailler, Kohler",
      "location": ""
    },
    {
      "entry_id": "13",
      "company_name": "Philippi & Hermann",
      "location": ""
    },
    {
      "entry_id": "14",
      "company_name": "Planta, J. & Co.",
      "location": ""
    },
    {
      "entry_id": "15",
      "company_name": "Porteous, James & Co., Ltd.",
      "location": ""
    },
    {
      "entry_id": "16",
      "company_name": "Preisig, Walter & Cie",
      "location": ""
    },
    {
      "entry_id": "17",
      "company_name": "Quasi-Arc Co., Ltd., The",
      "location": ""
    },
    {
      "entry_id": "18",
      "company_name": "Record Dreadnought Watch Co., S. A.",
      "location": ""
    },
    {
      "entry_id": "19",
      "company_name": "Reichenbach & Cie.",
      "location": ""
    },
    {
      "entry_id": "20",
      "company_name": "Reinhart & Co.",
      "location": ""
    },
    {
      "entry_id": "21",
      "company_name": "Rinderknecht, J.",
      "location": ""
    },
    {
      "entry_id": "22",
      "company_name": "Rolling & Drawing Mills Ltd.",
      "location": ""
    },
    {
      "entry_id": "23",
      "company_name": "Rolls, Frank V.",
      "location": ""
    },
    {
      "entry_id": "24",
      "company_name": "Rolo & Co.",
      "location": ""
    },
    {
      "entry_id": "25",
      "company_name": ""Roneo" A. G.",
      "location": ""
    },
    {
      "entry_id": "26",
      "company_name": "Roussy, Théodore",
      "location": ""
    },
    {
      "entry_id": "27",
      "company_name": "Rüegger & Co.",
      "location": ""
    },
    {
      "entry_id": "28",
      "company_name": "Ryff & Co., Ltd.",
      "location": ""
    },
    {
      "entry_id": "29",
      "company_name": "Sandoz Chemical Co., Ltd., The",
      "location": ""
    },
    {
      "entry_id": "30",
      "company_name": "Sandreuter & Co.",
      "location": ""
    },
    {
      "entry_id": "31",
      "company_name": "Sarasin, Rodolphe & Co., Ltd.",
      "location": ""
    },
    {
      "entry_id": "32",
      "company_name": "Sarasin Sons Ltd.",
      "location": ""
    },
    {
      "entry_id": "33",
      "company_name": "Sarasin, W. & Co.",
      "location": ""
    }
  ]
}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.44 0.46 0.44 0.44 11 285 358 357
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 56.9K IT + 24.3K OT = 81.2K TTCost: 0.068$0.097$0.165$
Result 4 of 212

Test T1593 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Providergenai
Modelgemini-3.7-flash
  
Temperature0.5
DataclassListPage
  
Normalized Score49.13 %
Test time6.18 seconds
Prompt

The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.

About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.

About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".

About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.

{
  "entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
  "company_name": "The name of the company or person",
  "location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
  ]
}

Results

{"page_id":"page_89","entries":[{"entry_id":"page_89-1","company_name":"C. Troeber-Ruesch","location":"Küsnacht (Zurich)"},{"entry_id":"page_89-2","company_name":"Unitraco Ltd.","location":"Zurich"},{"entry_id":"page_89-3","company_name":"H. Ursprung","location":"Zurich"},{"entry_id":"page_89-4","company_name":"Wettler & Frey","location":"Zurich 6"},{"entry_id":"page_89-5","company_name":"Paul Wüthrich","location":"Lausanne"},{"entry_id":"page_89-6","company_name":"Hultons Press Ltd.","location":"London E. C. 4"},{"entry_id":"page_89-7","company_name":"George Fischer Ltd.","location":"Schaffhausen"},{"entry_id":"page_89-8","company_name":"Louis Martin","location":"Lausanne"},{"entry_id":"page_89-9","company_name":"Techag Ltd.","location":"Zurich"},{"entry_id":"page_89-10","company_name":"Usine Fega","location":"Zurich"},{"entry_id":"page_89-11","company_name":"R. Amiguet-Peter","location":"Lausanne"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.50 0.49 0.50 0.49 15 490 495 502
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 22.3K IT + 17.0K OT = 39.3K TTCost: 0.017$0.064$0.080$
Result 5 of 212

Test T1608 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Providerx-ai
Modelgrok-4.6
  
Temperature0.5
DataclassListPage
  
Normalized Score61.20 %
Test time28.06 seconds
Prompt

The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.

About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.

About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".

About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.

{
  "entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
  "company_name": "The name of the company or person",
  "location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
  ]
}

Results

{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"C. Troeber-Ruesch","location":"Küsnacht (Zurich)"},{"entry_id":"{page_id}-2","company_name":"Unitraco Ltd.","location":"Zurich"},{"entry_id":"{page_id}-3","company_name":"H. Ursprung","location":"Zurich"},{"entry_id":"{page_id}-4","company_name":"Wettler & Frey","location":"Zurich 6"},{"entry_id":"{page_id}-5","company_name":"Paul Wüthrich","location":"Lausanne"},{"entry_id":"{page_id}-6","company_name":"Hultons Press Ltd.","location":"London E. C. 4"},{"entry_id":"{page_id}-7","company_name":"George Fischer Ltd.","location":"Schaffhausen"},{"entry_id":"{page_id}-8","company_name":"Louis Martin","location":"Lausanne"},{"entry_id":"{page_id}-9","company_name":"Techag Ltd.","location":"Zurich"},{"entry_id":"{page_id}-10","company_name":"Usine Fega","location":"Zurich"},{"entry_id":"{page_id}-11","company_name":"R. Amiguet-Peter","location":"Lausanne"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.61 0.61 0.62 0.61 15 608 379 384
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 47.4K IT + 11.4K OT = 58.8K TTCost: 0.095$0.068$0.163$
Result 6 of 212

Test T1623 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Provideropenrouter
Modelmeta/muse-spark-1.2
  
Temperature0.5
DataclassListPage
  
Normalized Score56.67 %
Test time33.62 seconds
Prompt

The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.

About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.

About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".

About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.

{
  "entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
  "company_name": "The name of the company or person",
  "location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
  ]
}

Results

{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"General Superintendence Co. Ltd.","location":"Geneva"},{"entry_id":"{page_id}-2","company_name":"Gondrand Brothers Ltd.","location":"Basle"},{"entry_id":"{page_id}-3","company_name":"Goth & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-4","company_name":"Charles Im Obersteg & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-5","company_name":"John Im Obersteg & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-6","company_name":"Jacky, Maeder & Co.","location":"Zurich"},{"entry_id":"{page_id}-7","company_name":"Jacky, Maeder & Co.","location":"Basle"},{"entry_id":"{page_id}-8","company_name":"Lavanchy & Co. Ltd.","location":"Lausanne"},{"entry_id":"{page_id}-9","company_name":"Marunion S.A.","location":"Zurich"},{"entry_id":"{page_id}-10","company_name":"MAT Transport Ltd.","location":"Basle"},{"entry_id":"{page_id}-11","company_name":"Müller-Gysin Ltd.","location":"Basle"},{"entry_id":"{page_id}-12","company_name":"Natural Ltd.","location":"Basle"},{"entry_id":"{page_id}-13","company_name":"A. Natural, Le Coultre & Co. Ltd.","location":"Geneva"},{"entry_id":"{page_id}-14","company_name":"Roba Shipping Agency & Warehouse Ltd.","location":"Basle"},{"entry_id":"{page_id}-15","company_name":"J. Véron, Grauer & Co. Ltd.","location":"Geneva"},{"entry_id":"{page_id}-16","company_name":"Weltifurrer International Transport Co. Ltd.","location":"Zurich"},{"entry_id":"{page_id}-17","company_name":"A. Welti-Furrer Ltd.","location":"Zurich"},{"entry_id":"{page_id}-18","company_name":"World Transport Ltd.","location":"Basle"},{"entry_id":"{page_id}-19","company_name":"George Fischer Ltd.","location":"Schaffhausen"},{"entry_id":"{page_id}-20","company_name":"Morganite Exports Ltd.","location":"London"},{"entry_id":"{page_id}-21","company_name":"Louis de Roll Iron Works Ltd.","location":"Gerlafingen"},{"entry_id":"{page_id}-22","company_name":"Sulzer Brothers Ltd.","location":"Winterthur"},{"entry_id":"{page_id}-23","company_name":"Schubarth & Co.","location":"Basle"},{"entry_id":"{page_id}-24","company_name":"Marunion S.A.","location":"Zurich"},{"entry_id":"{page_id}-25","company_name":"W. Simon","location":"Zurich"},{"entry_id":"{page_id}-26","company_name":"Emil Flachsmann","location":"Zurich"},{"entry_id":"{page_id}-27","company_name":"H. Gallmann & Co.","location":"Winterthur"},{"entry_id":"{page_id}-28","company_name":"L. E. Michel S.A.","location":"Malley-Lausanne"},{"entry_id":"{page_id}-29","company_name":"P. H. Stehelin & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-30","company_name":"Gustav Baader","location":"Basle"},{"entry_id":"{page_id}-31","company_name":"Aug. Birchmeier's Söhne","location":"Murgenthal"},{"entry_id":"{page_id}-32","company_name":"J. Coray","location":"Zurich"},{"entry_id":"{page_id}-33","company_name":"Guhl & Scheibler Ltd.","location":"Basle"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.56 0.57 0.56 0.57 15 566 446 426
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 60.2K IT + 43.8K OT = 104.0K TTCost: 0.075$0.186$0.261$
Result 7 of 212

Test T1609 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Providerx-ai
Modelgrok-4.6
  
Temperature0.5
DataclassListPage
  
Normalized Score43.07 %
Test time51.19 seconds
Prompt

- Answer in valid JSON.
- The page ID is given as {page_id}.

Results

{"page_id":"28","entries":[{"entry_id":"1","company_name":"Hermann Thorens","location":"Ste. Croix."},{"entry_id":"2","company_name":"\"Simex\" Soc. Coop.","location":"3, Place St. François, Lausanne."},{"entry_id":"3","company_name":"Fabrique d’Automobiles “Berna” S. A.","location":"Olten."},{"entry_id":"4","company_name":"Emil Birkhäuser & Co.","location":"11, Elisabethenstrasse, Basle."},{"entry_id":"5","company_name":"Imprimerie de l’Ere Nouvelle","location":"12, Rue de la Louve, Lausanne."},{"entry_id":"6","company_name":"Kreis & Co.","location":"21, Petersgraben, Basle."},{"entry_id":"7","company_name":"Säuberlin & Pfeiffer S. A.","location":"Vevey."},{"entry_id":"8","company_name":"Fr. Schenkel-Wyss","location":"60, Huttenstrasse, Zurich."},{"entry_id":"9","company_name":"Lovell & Christmas, Ltd.","location":"West Smithfield, London, E. C. 1."},{"entry_id":"10","company_name":"E. Osswald (Wholesale)","location":"16, Kreuzplatz, Zurich."},{"entry_id":"11","company_name":"Winandy & Co.","location":"16, Rue de Bourg, Lausanne."},{"entry_id":"12","company_name":"Pro Lemano","location":"Grand Chêne, Lausanne."},{"entry_id":"13","company_name":"Th. Roussy, “Librairie Anglaise”","location":"3, Rue de Bourg, Lausanne."},{"entry_id":"14","company_name":"Schindler & Co.","location":"Lucerne."},{"entry_id":"15","company_name":"Sulzer Frères S. A.","location":"Winterthur."},{"entry_id":"16","company_name":"Sulzer Frères S. A.","location":"Winterthur."},{"entry_id":"17","company_name":"Basle Warehouse Co., Ltd.","location":"1, Erlenstrasse, Basle."},{"entry_id":"18","company_name":"Canadian Pacific Railway Co.","location":"62-65, Charing Cross, London S. W. 1."},{"entry_id":"19","company_name":"Crowe & Co., Ltd.","location":"67, Nauenstrasse, Basle."},{"entry_id":"20","company_name":"Danzas & Co., A. G.","location":"Basle."},{"entry_id":"21","company_name":"Dominion Express Company of Canada","location":"62—65, Charing Cross, London S. W. 1."},{"entry_id":"22","company_name":"Galland & Co.","location":"3, Place St. François, Lausanne."},{"entry_id":"23","company_name":"Goth & Co. (General Agents of the Canadian Pacific Railway Co. and of the Dominion Express Co. of Canada)","location":"45, Elisabethenstrasse, Basle."},{"entry_id":"24","company_name":"Charles ImObersteg & Co.","location":"30, Aeschengraben, Basle."}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.43 0.43 0.43 0.43 15 427 559 565
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 42.3K IT + 10.9K OT = 53.2K TTCost: 0.085$0.065$0.150$
Result 8 of 212

Test T1594 at 2026-08-18

book-page information-extraction printed 20 en, de list company

Configuration
Providergenai
Modelgemini-3.7-flash
  
Temperature0.5
DataclassListPage
  
Normalized Score39.33 %
Test time6.68 seconds
Prompt

- Answer in valid JSON.
- The page ID is given as {page_id}.

Results

{"page_id":"IX","entries":[{"entry_id":"IX-1","company_name":"Abegg & Co. Ltd.","location":"Zurich"},{"entry_id":"IX-2","company_name":"Acontra A.G.","location":"Zurich"},{"entry_id":"IX-3","company_name":"Adam & Co. Ltd.","location":"Port Louis, Mauritius"},{"entry_id":"IX-4","company_name":"Adroka Ltd.","location":"Basle"},{"entry_id":"IX-5","company_name":"Aebi (Robert) Ltd.","location":"Zurich"},{"entry_id":"IX-6","company_name":"Aksa Ltd.","location":"Würenlos, Aargau"},{"entry_id":"IX-7","company_name":"Albeko Shoe Machinery Ltd.","location":"Basle"},{"entry_id":"IX-8","company_name":"Albion House Ltd.","location":"Lausanne"},{"entry_id":"IX-9","company_name":"Albiswerk Zurich A.G.","location":"Zurich"},{"entry_id":"IX-10","company_name":"Aluminium Commercial S.A.","location":"Zurich"},{"entry_id":"IX-11","company_name":"Aluminium Limited (Canada) S.A.","location":"Geneva"},{"entry_id":"IX-12","company_name":"Aluminiumwerke A.G.","location":"Rorschach"},{"entry_id":"IX-13","company_name":"Aluminium Industrie A.G.","location":"Zurich"},{"entry_id":"IX-14","company_name":"Amalgamations Ltd.","location":"Madras, India"},{"entry_id":"IX-15","company_name":"Amag Automobil- und Motoren A.G.","location":"Zurich"},{"entry_id":"IX-16","company_name":"Amida Ltd.","location":"Grenchen"},{"entry_id":"IX-17","company_name":"Amidro (Swiss Druggists' & Chem. Purchasing Assoc.)","location":"Bienne"},{"entry_id":"IX-18","company_name":"Ampack, Hungerbühler & Lemm K.G.","location":"Rorschach"},{"entry_id":"IX-19","company_name":"Amsler (Alfred J.) & Co.","location":"Schaffhausen"},{"entry_id":"IX-20","company_name":"Anderes (Rudolf) A.G.","location":"St. Gallen"},{"entry_id":"IX-21","company_name":"André & Co. Ltd.","location":"Lausanne"},{"entry_id":"IX-22","company_name":"Angst & Pfister Ltd.","location":"Zurich"},{"entry_id":"IX-23","company_name":"Annabelle (François Bloch)","location":"Lausanne"},{"entry_id":"IX-24","company_name":"Anra S. à r. l.","location":"Territet-Montreux"},{"entry_id":"IX-25","company_name":"Anselmi jr. (Carl)","location":"Solothurn"},{"entry_id":"IX-26","company_name":"Aplin & Barrett Ltd.","location":"Yeovil, Somerset"},{"entry_id":"IX-27","company_name":"Appleyard (John W.) & Sons Ltd.","location":"Morley, Yorks"},{"entry_id":"IX-28","company_name":"Arts Ménagers S.A.","location":"Geneva"},{"entry_id":"IX-29","company_name":"Association of Swiss Machine Tool and Tool Dealers","location":"Zurich"},{"entry_id":"IX-30","company_name":"«Astra» Fat & Oil Works Ltd.","location":"Steffisburg"},{"entry_id":"IX-31","company_name":"Atlantis Verlag A.G.","location":"Zurich"},{"entry_id":"IX-32","company_name":"Attenhofer (A.)","location":"Zurich"},{"entry_id":"IX-33","company_name":"Aubry Brothers, Ciny Watch Ltd.","location":"Le Noirmont"},{"entry_id":"IX-34","company_name":"Baader (Gustav)","location":"Basle"},{"entry_id":"IX-35","company_name":"Bachmann (Henri)","location":"Bienne"},{"entry_id":"IX-36","company_name":"Bachofen (Willy A.)","location":"Basle"},{"entry_id":"IX-37","company_name":"Baezner & Co.","location":"Geneva"},{"entry_id":"IX-38","company_name":"Baker (John) & Bessemer Ltd.","location":"Rotherham"},{"entry_id":"IX-39","company_name":"Bally (C. F.) Ltd.","location":"Zurich"},{"entry_id":"IX-40","company_name":"Bally's Shoe Factories Ltd.","location":"Schönenwerd"},{"entry_id":"IX-41","company_name":"Bâloise Marine Insurance Co. Ltd.","location":"Basle"},{"entry_id":"IX-42","company_name":"Bangerter (A.) & Co. Ltd.","location":"Lyss"},{"entry_id":"IX-43","company_name":"Banque Galland & Co. S.A.","location":"Lausanne"},{"entry_id":"IX-44","company_name":"Banque de Paris et des Pays Bas S.A.","location":"Geneva"},{"entry_id":"IX-45","company_name":"Barclays Bank Ltd.","location":"London"},{"entry_id":"IX-46","company_name":"Barco Trading Co. Ltd.","location":"Geneva"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.41 0.39 0.41 0.41 15 403 586 589
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 16.7K IT + 16.9K OT = 33.6K TTCost: 0.013$0.063$0.076$
Result 9 of 212

Test T1578 at 2026-08-16

book-page information-extraction printed 20 en, de list company

Configuration
Providerhuggingface
Modelmeta-models/Muse-Glimmer-30B
  
Temperature0.5
DataclassListPage
  
Normalized Score57.60 %
Test time70.02 seconds
Prompt

The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.

About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.

About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".

About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.

{
  "entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
  "company_name": "The name of the company or person",
  "location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
  ]
}

Results

{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":"null"},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":"null"},{"entry_id":"24-3","company_name":"Singer & Buchenhorner A.-G.","location":"null"},{"entry_id":"24-4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"null"},{"entry_id":"24-5","company_name":"Ausstattungsmagazine Ed. Senft","location":"null"},{"entry_id":"24-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"null"},{"entry_id":"24-7","company_name":"Adolf Bürgin","location":"null"},{"entry_id":"24-8","company_name":"Bussinger & Cie.","location":"null"},{"entry_id":"24-9","company_name":"Fränkel & Voellmy","location":"null"},{"entry_id":"24-10","company_name":"H. Hartmann","location":"null"},{"entry_id":"24-11","company_name":"Kunsthandlung Rath","location":"null"},{"entry_id":"24-12","company_name":"M. Russ","location":"null"},{"entry_id":"24-13","company_name":"Alf. Schray","location":"null"},{"entry_id":"24-14","company_name":"Wolf & Glatz","location":"null"},{"entry_id":"24-15","company_name":"Gebr. Antonietti","location":"null"},{"entry_id":"24-16","company_name":"A.-G. Bächtoid & Cie.","location":"null"},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"null"},{"entry_id":"24-18","company_name":"G. Meidinger & Cie.","location":"null"},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller","location":"null"},{"entry_id":"24-20","company_name":"Hug & Co.","location":"null"},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"null"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"null"},{"entry_id":"24-23","company_name":"Henri Grunauer","location":"null"},{"entry_id":"24-24","company_name":"Hans Ritter-Soder","location":"null"},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder","location":"null"},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":"null"},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":"null"},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner","location":"Niederndorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"null"},{"entry_id":"24-30","company_name":"G. P. Richter","location":"null"},{"entry_id":"24-31","company_name":"E. H. Schacke","location":"null"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.58 0.58 0.58 0.57 15 569 417 423
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 60.3K IT + 54.1K OT = 114.4K TTCost: 0.021$0.081$0.102$
Result 10 of 212

Test T1548 at 2026-08-16

book-page information-extraction printed 20 en, de list company

Configuration
Providerhuggingface
Modelthinkingmachines/Inkling-Small
  
Temperature0.5
DataclassListPage
  
Normalized Score53.00 %
Test time63.97 seconds
Prompt

The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.

About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.

About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".

About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.

{
  "entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
  "company_name": "The name of the company or person",
  "location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
  ]
}

Results

{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"Kunsthandlung Rath.","location":"null"},{"entry_id":"{page_id}-2","company_name":"Würsdorfer & Cie.","location":"null"},{"entry_id":"{page_id}-3","company_name":"Hotz, Volk & Cie.","location":"Neuewelt"},{"entry_id":"{page_id}-4","company_name":"Emil Mertz.","location":"null"},{"entry_id":"{page_id}-5","company_name":"Rudolf Gysin & Co.","location":"null"},{"entry_id":"{page_id}-6","company_name":"Merz & Co.","location":"null"},{"entry_id":"{page_id}-7","company_name":"Eisenmöbelfabrik Pratteln L. & C.","location":"null"},{"entry_id":"{page_id}-8","company_name":"Arnold.","location":"null"},{"entry_id":"{page_id}-9","company_name":"Emil Haefely & Cie. A.-G.","location":"null"},{"entry_id":"{page_id}-10","company_name":"R. Gschwind & Cie.","location":"Oberwil"},{"entry_id":"{page_id}-11","company_name":"Droeser & Sohn.","location":"null"},{"entry_id":"{page_id}-12","company_name":"Affolter, Christen & Co. A.-G.","location":"null"},{"entry_id":"{page_id}-13","company_name":"Frobenius A.-G.","location":"null"},{"entry_id":"{page_id}-14","company_name":"Franz Wittmer.","location":"null"},{"entry_id":"{page_id}-15","company_name":"Affolter, Christen & Co. A.-G.","location":"null"},{"entry_id":"{page_id}-16","company_name":"Preiswerk, Esser & Cie.","location":"null"},{"entry_id":"{page_id}-17","company_name":"R. Gschwind & Cie.","location":"Oberwil"},{"entry_id":"{page_id}-18","company_name":"Walz & Cie.","location":"null"},{"entry_id":"{page_id}-19","company_name":"Ch. H. Pfister & Co.","location":"null"},{"entry_id":"{page_id}-20","company_name":"Ch. H. Pfister & Co.","location":"null"},{"entry_id":"{page_id}-21","company_name":"Gas- und Wasserwerk Basel.","location":"Basel"},{"entry_id":"{page_id}-22","company_name":"Perrin & Küng.","location":"Maisprach"},{"entry_id":"{page_id}-23","company_name":"A.-G. Verblendstein-Fabrik Lausen","location":"Lausen"},{"entry_id":"{page_id}-24","company_name":"Basler Stempelfabrik Fr. Keller.","location":"null"},{"entry_id":"{page_id}-25","company_name":"Memmel & Cie.","location":"null"},{"entry_id":"{page_id}-26","company_name":"Aktiengesellschaft für Textilindustrie (Taschentücher, Monogramme, Wäschezchen, Konfektionsetiquetten).","location":"null"},{"entry_id":"{page_id}-27","company_name":"N. Schterenbuch.","location":"null"},{"entry_id":"{page_id}-28","company_name":"F. Schaer-Ganz.","location":"null"},{"entry_id":"{page_id}-29","company_name":"P. Steiger.","location":"null"},{"entry_id":"{page_id}-30","company_name":"A.-G. James Jaquet.","location":"null"}]}

Scoring
Fuzzy Score F1 micro / macro Micro precision/recall Tue/False Positives
n/a 0.53 0.53 0.53 0.53 15 529 462 463
      Micro Precision Micro Recall Instances TP FP FN
Costs / Pricing
Pricing Date: n/an/aTokens: 32.9K IT + 61.6K OT = 94.4K TTCost: 0.015$0.074$0.089$