A test run is a single execution of a benchmark test using a defined model configuration.
Each run represents how a particular large language model (LLM) — such as GPT-4, Claude-3, or Gemini — performed on a given task at a specific time, with specific settings.
A test run includes:
Together, test runs make it possible to compare models, providers, and configurations across benchmarks in a transparent and reproducible way.
book-page information-extraction printed 20 en, de list company
| Provider | openrouter |
| Model | z-ai/glm-5v-turbo |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 55.00 % |
| Test time | 78.63 seconds |
The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.
About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.
About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".
About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.
{
"entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
"company_name": "The name of the company or person",
"location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
]
}
{"page_id":"IX","entries":[{"entry_id":"IX-1","company_name":"Abegg & Co. Ltd.","location":"Zurich"},{"entry_id":"IX-2","company_name":"Acontra A.G.","location":"Zurich"},{"entry_id":"IX-3","company_name":"Adam & Co. Ltd.","location":"Port Louis, Mauritius"},{"entry_id":"IX-4","company_name":"Adroka Ltd.","location":"Basle"},{"entry_id":"IX-5","company_name":"Aebi (Robert) Ltd.","location":"Zurich"},{"entry_id":"IX-6","company_name":"Aksa Ltd.","location":"Würenlos, Aargau"},{"entry_id":"IX-7","company_name":"Albeko Shoe Machinery Ltd.","location":"Basle"},{"entry_id":"IX-8","company_name":"Albion House Ltd.","location":"Lausanne"},{"entry_id":"IX-9","company_name":"Albiswerk Zurich A.G.","location":"Zurich"},{"entry_id":"IX-10","company_name":"Aluminium Commercial S.A.","location":"Zurich"},{"entry_id":"IX-11","company_name":"Aluminium Limited (Canada) S.A.","location":"Geneva"},{"entry_id":"IX-12","company_name":"Aluminiumwerke A.G.","location":"Rorschach"},{"entry_id":"IX-13","company_name":"Aluminium Industrie A.G.","location":"Zurich"},{"entry_id":"IX-14","company_name":"Amalgamations Ltd.","location":"Madras, India"},{"entry_id":"IX-15","company_name":"Amag Automobil- und Motoren A.G.","location":"Zurich"},{"entry_id":"IX-16","company_name":"Amida Ltd.","location":"Grenchen"},{"entry_id":"IX-17","company_name":"Amidro (Swiss Druggists' & Chem. Purchasing Assoc.)","location":"Bienne"},{"entry_id":"IX-18","company_name":"Ampack, Hungerbühler & Lemm K.G.","location":"Rorschach"},{"entry_id":"IX-19","company_name":"Amsler (Alfred J.) & Co.","location":"Schaffhausen"},{"entry_id":"IX-20","company_name":"Andres (Rudolf) A.G.","location":"St. Gallen"},{"entry_id":"IX-21","company_name":"André & Co. Ltd.","location":"Lausanne"},{"entry_id":"IX-22","company_name":"Angst & Pfister Ltd.","location":"Zurich"},{"entry_id":"IX-23","company_name":"Annabelle (François Bloch)","location":"Lausanne"},{"entry_id":"IX-24","company_name":"Anra S. à r.l.","location":"Territet-Montreux"},{"entry_id":"IX-25","company_name":"Anselmi jr. (Carl)","location":"Solothurn"},{"entry_id":"IX-26","company_name":"Aplin & Barrett Ltd.","location":"Yeovil, Somerset"},{"entry_id":"IX-27","company_name":"Appleyard (John W.) & Sons Ltd.","location":"Morley, Yorks"},{"entry_id":"IX-28","company_name":"Arts Ménagers S.A.","location":"Geneva"},{"entry_id":"IX-29","company_name":"Association of Swiss Machine Tool and Tool Dealers","location":"Zurich"},{"entry_id":"IX-30","company_name":"«Astra» Fat & Oil Works Ltd.","location":"Steffisburg"},{"entry_id":"IX-31","company_name":"Atlantis Verlag A.G.","location":"Zurich"},{"entry_id":"IX-32","company_name":"Attenhofer (A.)","location":"Zurich"},{"entry_id":"IX-33","company_name":"Aubry Brothers, Ciny Watch Ltd.","location":"Le Noirmont"},{"entry_id":"IX-34","company_name":"Baader (Gustav)","location":"Basle"},{"entry_id":"IX-35","company_name":"Bachmann (Henri)","location":"Bienne"},{"entry_id":"IX-36","company_name":"Bachofen (Willy A.)","location":"Basle"},{"entry_id":"IX-37","company_name":"Baezner & Co","location":"Geneva"},{"entry_id":"IX-38","company_name":"Baker (John) & Bessemer Ltd.","location":"Rotherham"},{"entry_id":"IX-39","company_name":"Bally (C. F.) Ltd.","location":"Zurich"},{"entry_id":"IX-40","company_name":"Bally's Shoe Factories Ltd.","location":"Schönenwerd"},{"entry_id":"IX-41","company_name":"Bâloise Marine Insurance Co. Ltd.","location":"Basle"},{"entry_id":"IX-42","company_name":"Bangarter (A.) & Co. Ltd.","location":"Lyss"},{"entry_id":"IX-43","company_name":"Banque Galland & Co. S.A.","location":"Lausanne"},{"entry_id":"IX-44","company_name":"Banque de Paris et des Pays Bas S.A.","location":"Geneva"},{"entry_id":"IX-45","company_name":"Barclays Bank Ltd.","location":"London"},{"entry_id":"IX-46","company_name":"Barco Trading Co. Ltd.","location":"Geneva"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.55 | 0.55 | 0.56 | 0.55 | 14 | 509 | 408 | 410 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 62.0K IT + 41.8K OT = 103.8K TT | Cost: 0.074$ + 0.167$ = 0.242$ |
book-page information-extraction printed 20 en, de list company
| Provider | openrouter |
| Model | meta/muse-spark-1.2 |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 26.53 % |
| Test time | 11.84 seconds |
- Answer in valid JSON.
- The page ID is given as {page_id}.
{"page_id":"{page_id}.","entries":[{"entry_id":".","company_name":"Swiss Bank Corporation","location":"Lausanne"},{"entry_id":".","company_name":"Swiss Bank Corporation","location":"Neuch\u0000e2tel"},{"entry_id":".","company_name":"Union Bank of Switzerland","location":"Z\u0000fcrich"},{"entry_id":".","company_name":"Barcham Green & Sons","location":"Hayle Mill Maidstone, Kent"},{"entry_id":".","company_name":"Herren & Guerchet","location":"Rue Jean-Petitot 10 Gen\u0000e8ve"},{"entry_id":".","company_name":"Louis Kohler","location":"11, St. Pierre, Lausanne"},{"entry_id":".","company_name":"Giuliniwerke A. G.","location":"Basel"},{"entry_id":".","company_name":"Georges Michoud All kinds","location":"Lausanne"},{"entry_id":".","company_name":"August Fr\u0000f6hlich Leather","location":"Bahnhofstrasse 110 Z\u0000fcrich"},{"entry_id":".","company_name":"Altherr & Guex","location":"Flawil"},{"entry_id":".","company_name":"Arbon Embroidery Works Ltd.","location":"Arbon"},{"entry_id":".","company_name":"Grieder & Co.","location":"Paradeplatz Z\u0000fcrich"},{"entry_id":".","company_name":"A. C. Pearse Ltd.","location":"14, Pilcher Gate Nottingham"},{"entry_id":".","company_name":"Zuberb\u0000fchler & Co., A. G.","location":"Zurzach"},{"entry_id":".","company_name":"Sulzer Fr\u0000e8res S. A.","location":"Winterthur"},{"entry_id":".","company_name":"Schw. Seidengazefabrik A. G.","location":"Thal, bei Rheineck"},{"entry_id":".","company_name":"Schw. Seidengazefabrik A. G.","location":"Z\u0000fcrich"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.28 | 0.27 | 0.29 | 0.27 | 15 | 263 | 634 | 729 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 55.8K IT + 30.5K OT = 86.3K TT | Cost: 0.070$ + 0.130$ = 0.199$ |
book-page information-extraction printed 20 en, de list company
| Provider | openrouter |
| Model | z-ai/glm-5v-turbo |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 45.55 % |
| Test time | 41.66 seconds |
- Answer in valid JSON.
- The page ID is given as {page_id}.
{
"page_id": "61",
"entries": [
{
"entry_id": "1",
"company_name": "Nardin, Ulysse",
"location": ""
},
{
"entry_id": "2",
"company_name": "Natural, Le Coultre & Cie., S. A.",
"location": ""
},
{
"entry_id": "3",
"company_name": "Nestlé & Anglo-Swiss Condensed Milk Co., Ltd.",
"location": ""
},
{
"entry_id": "4",
"company_name": "Neuburger & Co.",
"location": ""
},
{
"entry_id": "5",
"company_name": "North British Rubber Co., Ltd., The",
"location": ""
},
{
"entry_id": "6",
"company_name": "Obrist & Co.",
"location": ""
},
{
"entry_id": "7",
"company_name": "Paillard, E. & Cie., S. A.",
"location": ""
},
{
"entry_id": "8",
"company_name": "Paillard & Benoit",
"location": ""
},
{
"entry_id": "9",
"company_name": "Palace Hotel",
"location": "Montana"
},
{
"entry_id": "10",
"company_name": "Pearse, A. C. & Co., Ltd.",
"location": ""
},
{
"entry_id": "11",
"company_name": "Perrin & Cie.",
"location": ""
},
{
"entry_id": "12",
"company_name": "Peter, Cailler, Kohler",
"location": ""
},
{
"entry_id": "13",
"company_name": "Philippi & Hermann",
"location": ""
},
{
"entry_id": "14",
"company_name": "Planta, J. & Co.",
"location": ""
},
{
"entry_id": "15",
"company_name": "Porteous, James & Co., Ltd.",
"location": ""
},
{
"entry_id": "16",
"company_name": "Preisig, Walter & Cie",
"location": ""
},
{
"entry_id": "17",
"company_name": "Quasi-Arc Co., Ltd., The",
"location": ""
},
{
"entry_id": "18",
"company_name": "Record Dreadnought Watch Co., S. A.",
"location": ""
},
{
"entry_id": "19",
"company_name": "Reichenbach & Cie.",
"location": ""
},
{
"entry_id": "20",
"company_name": "Reinhart & Co.",
"location": ""
},
{
"entry_id": "21",
"company_name": "Rinderknecht, J.",
"location": ""
},
{
"entry_id": "22",
"company_name": "Rolling & Drawing Mills Ltd.",
"location": ""
},
{
"entry_id": "23",
"company_name": "Rolls, Frank V.",
"location": ""
},
{
"entry_id": "24",
"company_name": "Rolo & Co.",
"location": ""
},
{
"entry_id": "25",
"company_name": ""Roneo" A. G.",
"location": ""
},
{
"entry_id": "26",
"company_name": "Roussy, Théodore",
"location": ""
},
{
"entry_id": "27",
"company_name": "Rüegger & Co.",
"location": ""
},
{
"entry_id": "28",
"company_name": "Ryff & Co., Ltd.",
"location": ""
},
{
"entry_id": "29",
"company_name": "Sandoz Chemical Co., Ltd., The",
"location": ""
},
{
"entry_id": "30",
"company_name": "Sandreuter & Co.",
"location": ""
},
{
"entry_id": "31",
"company_name": "Sarasin, Rodolphe & Co., Ltd.",
"location": ""
},
{
"entry_id": "32",
"company_name": "Sarasin Sons Ltd.",
"location": ""
},
{
"entry_id": "33",
"company_name": "Sarasin, W. & Co.",
"location": ""
}
]
}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.44 | 0.46 | 0.44 | 0.44 | 11 | 285 | 358 | 357 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 56.9K IT + 24.3K OT = 81.2K TT | Cost: 0.068$ + 0.097$ = 0.165$ |
book-page information-extraction printed 20 en, de list company
| Provider | genai |
| Model | gemini-3.7-flash |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 49.13 % |
| Test time | 6.18 seconds |
The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.
About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.
About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".
About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.
{
"entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
"company_name": "The name of the company or person",
"location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
]
}
{"page_id":"page_89","entries":[{"entry_id":"page_89-1","company_name":"C. Troeber-Ruesch","location":"Küsnacht (Zurich)"},{"entry_id":"page_89-2","company_name":"Unitraco Ltd.","location":"Zurich"},{"entry_id":"page_89-3","company_name":"H. Ursprung","location":"Zurich"},{"entry_id":"page_89-4","company_name":"Wettler & Frey","location":"Zurich 6"},{"entry_id":"page_89-5","company_name":"Paul Wüthrich","location":"Lausanne"},{"entry_id":"page_89-6","company_name":"Hultons Press Ltd.","location":"London E. C. 4"},{"entry_id":"page_89-7","company_name":"George Fischer Ltd.","location":"Schaffhausen"},{"entry_id":"page_89-8","company_name":"Louis Martin","location":"Lausanne"},{"entry_id":"page_89-9","company_name":"Techag Ltd.","location":"Zurich"},{"entry_id":"page_89-10","company_name":"Usine Fega","location":"Zurich"},{"entry_id":"page_89-11","company_name":"R. Amiguet-Peter","location":"Lausanne"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.50 | 0.49 | 0.50 | 0.49 | 15 | 490 | 495 | 502 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 22.3K IT + 17.0K OT = 39.3K TT | Cost: 0.017$ + 0.064$ = 0.080$ |
book-page information-extraction printed 20 en, de list company
| Provider | x-ai |
| Model | grok-4.6 |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 61.20 % |
| Test time | 28.06 seconds |
The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.
About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.
About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".
About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.
{
"entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
"company_name": "The name of the company or person",
"location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
]
}
{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"C. Troeber-Ruesch","location":"Küsnacht (Zurich)"},{"entry_id":"{page_id}-2","company_name":"Unitraco Ltd.","location":"Zurich"},{"entry_id":"{page_id}-3","company_name":"H. Ursprung","location":"Zurich"},{"entry_id":"{page_id}-4","company_name":"Wettler & Frey","location":"Zurich 6"},{"entry_id":"{page_id}-5","company_name":"Paul Wüthrich","location":"Lausanne"},{"entry_id":"{page_id}-6","company_name":"Hultons Press Ltd.","location":"London E. C. 4"},{"entry_id":"{page_id}-7","company_name":"George Fischer Ltd.","location":"Schaffhausen"},{"entry_id":"{page_id}-8","company_name":"Louis Martin","location":"Lausanne"},{"entry_id":"{page_id}-9","company_name":"Techag Ltd.","location":"Zurich"},{"entry_id":"{page_id}-10","company_name":"Usine Fega","location":"Zurich"},{"entry_id":"{page_id}-11","company_name":"R. Amiguet-Peter","location":"Lausanne"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.61 | 0.61 | 0.62 | 0.61 | 15 | 608 | 379 | 384 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 47.4K IT + 11.4K OT = 58.8K TT | Cost: 0.095$ + 0.068$ = 0.163$ |
book-page information-extraction printed 20 en, de list company
| Provider | openrouter |
| Model | meta/muse-spark-1.2 |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 56.67 % |
| Test time | 33.62 seconds |
The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.
About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.
About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".
About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.
{
"entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
"company_name": "The name of the company or person",
"location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
]
}
{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"General Superintendence Co. Ltd.","location":"Geneva"},{"entry_id":"{page_id}-2","company_name":"Gondrand Brothers Ltd.","location":"Basle"},{"entry_id":"{page_id}-3","company_name":"Goth & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-4","company_name":"Charles Im Obersteg & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-5","company_name":"John Im Obersteg & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-6","company_name":"Jacky, Maeder & Co.","location":"Zurich"},{"entry_id":"{page_id}-7","company_name":"Jacky, Maeder & Co.","location":"Basle"},{"entry_id":"{page_id}-8","company_name":"Lavanchy & Co. Ltd.","location":"Lausanne"},{"entry_id":"{page_id}-9","company_name":"Marunion S.A.","location":"Zurich"},{"entry_id":"{page_id}-10","company_name":"MAT Transport Ltd.","location":"Basle"},{"entry_id":"{page_id}-11","company_name":"Müller-Gysin Ltd.","location":"Basle"},{"entry_id":"{page_id}-12","company_name":"Natural Ltd.","location":"Basle"},{"entry_id":"{page_id}-13","company_name":"A. Natural, Le Coultre & Co. Ltd.","location":"Geneva"},{"entry_id":"{page_id}-14","company_name":"Roba Shipping Agency & Warehouse Ltd.","location":"Basle"},{"entry_id":"{page_id}-15","company_name":"J. Véron, Grauer & Co. Ltd.","location":"Geneva"},{"entry_id":"{page_id}-16","company_name":"Weltifurrer International Transport Co. Ltd.","location":"Zurich"},{"entry_id":"{page_id}-17","company_name":"A. Welti-Furrer Ltd.","location":"Zurich"},{"entry_id":"{page_id}-18","company_name":"World Transport Ltd.","location":"Basle"},{"entry_id":"{page_id}-19","company_name":"George Fischer Ltd.","location":"Schaffhausen"},{"entry_id":"{page_id}-20","company_name":"Morganite Exports Ltd.","location":"London"},{"entry_id":"{page_id}-21","company_name":"Louis de Roll Iron Works Ltd.","location":"Gerlafingen"},{"entry_id":"{page_id}-22","company_name":"Sulzer Brothers Ltd.","location":"Winterthur"},{"entry_id":"{page_id}-23","company_name":"Schubarth & Co.","location":"Basle"},{"entry_id":"{page_id}-24","company_name":"Marunion S.A.","location":"Zurich"},{"entry_id":"{page_id}-25","company_name":"W. Simon","location":"Zurich"},{"entry_id":"{page_id}-26","company_name":"Emil Flachsmann","location":"Zurich"},{"entry_id":"{page_id}-27","company_name":"H. Gallmann & Co.","location":"Winterthur"},{"entry_id":"{page_id}-28","company_name":"L. E. Michel S.A.","location":"Malley-Lausanne"},{"entry_id":"{page_id}-29","company_name":"P. H. Stehelin & Co. Ltd.","location":"Basle"},{"entry_id":"{page_id}-30","company_name":"Gustav Baader","location":"Basle"},{"entry_id":"{page_id}-31","company_name":"Aug. Birchmeier's Söhne","location":"Murgenthal"},{"entry_id":"{page_id}-32","company_name":"J. Coray","location":"Zurich"},{"entry_id":"{page_id}-33","company_name":"Guhl & Scheibler Ltd.","location":"Basle"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.56 | 0.57 | 0.56 | 0.57 | 15 | 566 | 446 | 426 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 60.2K IT + 43.8K OT = 104.0K TT | Cost: 0.075$ + 0.186$ = 0.261$ |
book-page information-extraction printed 20 en, de list company
| Provider | x-ai |
| Model | grok-4.6 |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 43.07 % |
| Test time | 51.19 seconds |
- Answer in valid JSON.
- The page ID is given as {page_id}.
{"page_id":"28","entries":[{"entry_id":"1","company_name":"Hermann Thorens","location":"Ste. Croix."},{"entry_id":"2","company_name":"\"Simex\" Soc. Coop.","location":"3, Place St. François, Lausanne."},{"entry_id":"3","company_name":"Fabrique d’Automobiles “Berna” S. A.","location":"Olten."},{"entry_id":"4","company_name":"Emil Birkhäuser & Co.","location":"11, Elisabethenstrasse, Basle."},{"entry_id":"5","company_name":"Imprimerie de l’Ere Nouvelle","location":"12, Rue de la Louve, Lausanne."},{"entry_id":"6","company_name":"Kreis & Co.","location":"21, Petersgraben, Basle."},{"entry_id":"7","company_name":"Säuberlin & Pfeiffer S. A.","location":"Vevey."},{"entry_id":"8","company_name":"Fr. Schenkel-Wyss","location":"60, Huttenstrasse, Zurich."},{"entry_id":"9","company_name":"Lovell & Christmas, Ltd.","location":"West Smithfield, London, E. C. 1."},{"entry_id":"10","company_name":"E. Osswald (Wholesale)","location":"16, Kreuzplatz, Zurich."},{"entry_id":"11","company_name":"Winandy & Co.","location":"16, Rue de Bourg, Lausanne."},{"entry_id":"12","company_name":"Pro Lemano","location":"Grand Chêne, Lausanne."},{"entry_id":"13","company_name":"Th. Roussy, “Librairie Anglaise”","location":"3, Rue de Bourg, Lausanne."},{"entry_id":"14","company_name":"Schindler & Co.","location":"Lucerne."},{"entry_id":"15","company_name":"Sulzer Frères S. A.","location":"Winterthur."},{"entry_id":"16","company_name":"Sulzer Frères S. A.","location":"Winterthur."},{"entry_id":"17","company_name":"Basle Warehouse Co., Ltd.","location":"1, Erlenstrasse, Basle."},{"entry_id":"18","company_name":"Canadian Pacific Railway Co.","location":"62-65, Charing Cross, London S. W. 1."},{"entry_id":"19","company_name":"Crowe & Co., Ltd.","location":"67, Nauenstrasse, Basle."},{"entry_id":"20","company_name":"Danzas & Co., A. G.","location":"Basle."},{"entry_id":"21","company_name":"Dominion Express Company of Canada","location":"62—65, Charing Cross, London S. W. 1."},{"entry_id":"22","company_name":"Galland & Co.","location":"3, Place St. François, Lausanne."},{"entry_id":"23","company_name":"Goth & Co. (General Agents of the Canadian Pacific Railway Co. and of the Dominion Express Co. of Canada)","location":"45, Elisabethenstrasse, Basle."},{"entry_id":"24","company_name":"Charles ImObersteg & Co.","location":"30, Aeschengraben, Basle."}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.43 | 0.43 | 0.43 | 0.43 | 15 | 427 | 559 | 565 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 42.3K IT + 10.9K OT = 53.2K TT | Cost: 0.085$ + 0.065$ = 0.150$ |
book-page information-extraction printed 20 en, de list company
| Provider | genai |
| Model | gemini-3.7-flash |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 39.33 % |
| Test time | 6.68 seconds |
- Answer in valid JSON.
- The page ID is given as {page_id}.
{"page_id":"IX","entries":[{"entry_id":"IX-1","company_name":"Abegg & Co. Ltd.","location":"Zurich"},{"entry_id":"IX-2","company_name":"Acontra A.G.","location":"Zurich"},{"entry_id":"IX-3","company_name":"Adam & Co. Ltd.","location":"Port Louis, Mauritius"},{"entry_id":"IX-4","company_name":"Adroka Ltd.","location":"Basle"},{"entry_id":"IX-5","company_name":"Aebi (Robert) Ltd.","location":"Zurich"},{"entry_id":"IX-6","company_name":"Aksa Ltd.","location":"Würenlos, Aargau"},{"entry_id":"IX-7","company_name":"Albeko Shoe Machinery Ltd.","location":"Basle"},{"entry_id":"IX-8","company_name":"Albion House Ltd.","location":"Lausanne"},{"entry_id":"IX-9","company_name":"Albiswerk Zurich A.G.","location":"Zurich"},{"entry_id":"IX-10","company_name":"Aluminium Commercial S.A.","location":"Zurich"},{"entry_id":"IX-11","company_name":"Aluminium Limited (Canada) S.A.","location":"Geneva"},{"entry_id":"IX-12","company_name":"Aluminiumwerke A.G.","location":"Rorschach"},{"entry_id":"IX-13","company_name":"Aluminium Industrie A.G.","location":"Zurich"},{"entry_id":"IX-14","company_name":"Amalgamations Ltd.","location":"Madras, India"},{"entry_id":"IX-15","company_name":"Amag Automobil- und Motoren A.G.","location":"Zurich"},{"entry_id":"IX-16","company_name":"Amida Ltd.","location":"Grenchen"},{"entry_id":"IX-17","company_name":"Amidro (Swiss Druggists' & Chem. Purchasing Assoc.)","location":"Bienne"},{"entry_id":"IX-18","company_name":"Ampack, Hungerbühler & Lemm K.G.","location":"Rorschach"},{"entry_id":"IX-19","company_name":"Amsler (Alfred J.) & Co.","location":"Schaffhausen"},{"entry_id":"IX-20","company_name":"Anderes (Rudolf) A.G.","location":"St. Gallen"},{"entry_id":"IX-21","company_name":"André & Co. Ltd.","location":"Lausanne"},{"entry_id":"IX-22","company_name":"Angst & Pfister Ltd.","location":"Zurich"},{"entry_id":"IX-23","company_name":"Annabelle (François Bloch)","location":"Lausanne"},{"entry_id":"IX-24","company_name":"Anra S. à r. l.","location":"Territet-Montreux"},{"entry_id":"IX-25","company_name":"Anselmi jr. (Carl)","location":"Solothurn"},{"entry_id":"IX-26","company_name":"Aplin & Barrett Ltd.","location":"Yeovil, Somerset"},{"entry_id":"IX-27","company_name":"Appleyard (John W.) & Sons Ltd.","location":"Morley, Yorks"},{"entry_id":"IX-28","company_name":"Arts Ménagers S.A.","location":"Geneva"},{"entry_id":"IX-29","company_name":"Association of Swiss Machine Tool and Tool Dealers","location":"Zurich"},{"entry_id":"IX-30","company_name":"«Astra» Fat & Oil Works Ltd.","location":"Steffisburg"},{"entry_id":"IX-31","company_name":"Atlantis Verlag A.G.","location":"Zurich"},{"entry_id":"IX-32","company_name":"Attenhofer (A.)","location":"Zurich"},{"entry_id":"IX-33","company_name":"Aubry Brothers, Ciny Watch Ltd.","location":"Le Noirmont"},{"entry_id":"IX-34","company_name":"Baader (Gustav)","location":"Basle"},{"entry_id":"IX-35","company_name":"Bachmann (Henri)","location":"Bienne"},{"entry_id":"IX-36","company_name":"Bachofen (Willy A.)","location":"Basle"},{"entry_id":"IX-37","company_name":"Baezner & Co.","location":"Geneva"},{"entry_id":"IX-38","company_name":"Baker (John) & Bessemer Ltd.","location":"Rotherham"},{"entry_id":"IX-39","company_name":"Bally (C. F.) Ltd.","location":"Zurich"},{"entry_id":"IX-40","company_name":"Bally's Shoe Factories Ltd.","location":"Schönenwerd"},{"entry_id":"IX-41","company_name":"Bâloise Marine Insurance Co. Ltd.","location":"Basle"},{"entry_id":"IX-42","company_name":"Bangerter (A.) & Co. Ltd.","location":"Lyss"},{"entry_id":"IX-43","company_name":"Banque Galland & Co. S.A.","location":"Lausanne"},{"entry_id":"IX-44","company_name":"Banque de Paris et des Pays Bas S.A.","location":"Geneva"},{"entry_id":"IX-45","company_name":"Barclays Bank Ltd.","location":"London"},{"entry_id":"IX-46","company_name":"Barco Trading Co. Ltd.","location":"Geneva"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.41 | 0.39 | 0.41 | 0.41 | 15 | 403 | 586 | 589 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 16.7K IT + 16.9K OT = 33.6K TT | Cost: 0.013$ + 0.063$ = 0.076$ |
book-page information-extraction printed 20 en, de list company
| Provider | huggingface |
| Model | meta-models/Muse-Glimmer-30B |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 57.60 % |
| Test time | 70.02 seconds |
The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.
About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.
About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".
About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.
{
"entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
"company_name": "The name of the company or person",
"location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
]
}
{"page_id":"24","entries":[{"entry_id":"24-1","company_name":"A. Boreux","location":"null"},{"entry_id":"24-2","company_name":"Brunnenverwaltung Eptingen","location":"null"},{"entry_id":"24-3","company_name":"Singer & Buchenhorner A.-G.","location":"null"},{"entry_id":"24-4","company_name":"Alf. Dürr, J. H. Neukomm's Nachf.","location":"null"},{"entry_id":"24-5","company_name":"Ausstattungsmagazine Ed. Senft","location":"null"},{"entry_id":"24-6","company_name":"Basler Möbelfabrik A.-G. vorm. Hermann Wagner & Cie.","location":"null"},{"entry_id":"24-7","company_name":"Adolf Bürgin","location":"null"},{"entry_id":"24-8","company_name":"Bussinger & Cie.","location":"null"},{"entry_id":"24-9","company_name":"Fränkel & Voellmy","location":"null"},{"entry_id":"24-10","company_name":"H. Hartmann","location":"null"},{"entry_id":"24-11","company_name":"Kunsthandlung Rath","location":"null"},{"entry_id":"24-12","company_name":"M. Russ","location":"null"},{"entry_id":"24-13","company_name":"Alf. Schray","location":"null"},{"entry_id":"24-14","company_name":"Wolf & Glatz","location":"null"},{"entry_id":"24-15","company_name":"Gebr. Antonietti","location":"null"},{"entry_id":"24-16","company_name":"A.-G. Bächtoid & Cie.","location":"null"},{"entry_id":"24-17","company_name":"Maschinenfabrik Burckhardt A.-G.","location":"null"},{"entry_id":"24-18","company_name":"G. Meidinger & Cie.","location":"null"},{"entry_id":"24-19","company_name":"A.-G. für Motorlastwagen Soller","location":"null"},{"entry_id":"24-20","company_name":"Hug & Co.","location":"null"},{"entry_id":"24-21","company_name":"Schweiz. Geigenbaugesellschaft Liestal","location":"null"},{"entry_id":"24-22","company_name":"Schweizerisches Musikhaus Bannwart & Cie.","location":"null"},{"entry_id":"24-23","company_name":"Henri Grunauer","location":"null"},{"entry_id":"24-24","company_name":"Hans Ritter-Soder","location":"null"},{"entry_id":"24-25","company_name":"Alb. Singeisen-Halder","location":"null"},{"entry_id":"24-26","company_name":"Ch. H. Pfister & Co.","location":"null"},{"entry_id":"24-27","company_name":"E. Müry & Cie.","location":"null"},{"entry_id":"24-28","company_name":"Affentranger, Haas & Plattner","location":"Niederndorf"},{"entry_id":"24-29","company_name":"Schweizerische Nähmaschinenfabrik A.-G.","location":"null"},{"entry_id":"24-30","company_name":"G. P. Richter","location":"null"},{"entry_id":"24-31","company_name":"E. H. Schacke","location":"null"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.58 | 0.58 | 0.58 | 0.57 | 15 | 569 | 417 | 423 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 60.3K IT + 54.1K OT = 114.4K TT | Cost: 0.021$ + 0.081$ = 0.102$ |
book-page information-extraction printed 20 en, de list company
| Provider | huggingface |
| Model | thinkingmachines/Inkling-Small |
| Temperature | 0.5 |
| Dataclass | ListPage |
| Normalized Score | 53.00 % |
| Test time | 63.97 seconds |
The image you are presented with stems from a digitized book containing lists of companies.
Your task is to extract structured information about each company listed on the page.
About the source:
- The image stems from a trade index of the British Swiss Chamber of Commerce.
- The image can show an alphabetical or a thematic list of companies.
- The companies are mostly located in Switzerland and the UK.
- The image stems from a trade index between 1925 and 1958.
- Most pages have one column but some years have two columns.
- The source itself is in English and German but the company names can be in English, German, French or Italian.
About the entries:
- Each entry describes a single company or person.
- Alphabetical entries have filling dots between the company name and the page number. Dots and page numbers are not part of the data and should be ignored.
- Alphabetical entries seldom to never have locations.
- Thematic entries often have locations.
- Thematic entries are listed under headings that describe the type of business.
- Some thematic headings are only references to other headings, e.g. "X, s. Y".
About the output:
- Answer in valid JSON. The JSON should be an array of objects with the following fields:
- The page ID is given as {page_id}.
- Do not add country information, if it is not directly written with the location.
{
"entry_id": "A unique identifier for the entry, e.g. '{page_id}-1'",
"company_name": "The name of the company or person",
"location": "The location of the company, e.g. 'Zurich' or 'London, UK'. If no location is given, set to null."
]
}
{"page_id":"{page_id}","entries":[{"entry_id":"{page_id}-1","company_name":"Kunsthandlung Rath.","location":"null"},{"entry_id":"{page_id}-2","company_name":"Würsdorfer & Cie.","location":"null"},{"entry_id":"{page_id}-3","company_name":"Hotz, Volk & Cie.","location":"Neuewelt"},{"entry_id":"{page_id}-4","company_name":"Emil Mertz.","location":"null"},{"entry_id":"{page_id}-5","company_name":"Rudolf Gysin & Co.","location":"null"},{"entry_id":"{page_id}-6","company_name":"Merz & Co.","location":"null"},{"entry_id":"{page_id}-7","company_name":"Eisenmöbelfabrik Pratteln L. & C.","location":"null"},{"entry_id":"{page_id}-8","company_name":"Arnold.","location":"null"},{"entry_id":"{page_id}-9","company_name":"Emil Haefely & Cie. A.-G.","location":"null"},{"entry_id":"{page_id}-10","company_name":"R. Gschwind & Cie.","location":"Oberwil"},{"entry_id":"{page_id}-11","company_name":"Droeser & Sohn.","location":"null"},{"entry_id":"{page_id}-12","company_name":"Affolter, Christen & Co. A.-G.","location":"null"},{"entry_id":"{page_id}-13","company_name":"Frobenius A.-G.","location":"null"},{"entry_id":"{page_id}-14","company_name":"Franz Wittmer.","location":"null"},{"entry_id":"{page_id}-15","company_name":"Affolter, Christen & Co. A.-G.","location":"null"},{"entry_id":"{page_id}-16","company_name":"Preiswerk, Esser & Cie.","location":"null"},{"entry_id":"{page_id}-17","company_name":"R. Gschwind & Cie.","location":"Oberwil"},{"entry_id":"{page_id}-18","company_name":"Walz & Cie.","location":"null"},{"entry_id":"{page_id}-19","company_name":"Ch. H. Pfister & Co.","location":"null"},{"entry_id":"{page_id}-20","company_name":"Ch. H. Pfister & Co.","location":"null"},{"entry_id":"{page_id}-21","company_name":"Gas- und Wasserwerk Basel.","location":"Basel"},{"entry_id":"{page_id}-22","company_name":"Perrin & Küng.","location":"Maisprach"},{"entry_id":"{page_id}-23","company_name":"A.-G. Verblendstein-Fabrik Lausen","location":"Lausen"},{"entry_id":"{page_id}-24","company_name":"Basler Stempelfabrik Fr. Keller.","location":"null"},{"entry_id":"{page_id}-25","company_name":"Memmel & Cie.","location":"null"},{"entry_id":"{page_id}-26","company_name":"Aktiengesellschaft für Textilindustrie (Taschentücher, Monogramme, Wäschezchen, Konfektionsetiquetten).","location":"null"},{"entry_id":"{page_id}-27","company_name":"N. Schterenbuch.","location":"null"},{"entry_id":"{page_id}-28","company_name":"F. Schaer-Ganz.","location":"null"},{"entry_id":"{page_id}-29","company_name":"P. Steiger.","location":"null"},{"entry_id":"{page_id}-30","company_name":"A.-G. James Jaquet.","location":"null"}]}
| Fuzzy Score | F1 micro / macro | Micro precision/recall | Tue/False Positives | |||||
| n/a | 0.53 | 0.53 | 0.53 | 0.53 | 15 | 529 | 462 | 463 |
| Micro Precision | Micro Recall | Instances | TP | FP | FN | |||
| Pricing Date: n/a, n/a. | Tokens: 32.9K IT + 61.6K OT = 94.4K TT | Cost: 0.015$ + 0.074$ = 0.089$ |