Company Lists

This benchmark evaluates the performance of large language models on extracting structured company information (company name and location) from historical trade indexes. The benchmark consists of 16 pages from trade indexes, issued between 1921 and 1958 by the British Chamber of Commerce for Switzerland, preserved at the Swiss Economic Archives in Basel. The complete record comprises 14 single trade indexes. These sources consist of lists of chamber members, accompanied by additional details such as addresses, locations, and-in later years- telephone numbers and telegraphic codes. Each volume contains two distinct types of lists: first, an alphabetical directory indicating the company name and the page reference within the index, and second, the actual trade index, which provides full entries containing the relevant commercial information. The layout of these indexes varies considerably over time, ranging from multi-column tables to more linear list formats. 

The objective of this benchmark was to extract those data fields that occur consistently across all entries, namely the company name and the location. The task required the model to identify these elements across differing typographic layouts and information densities, and to classify them into a structured format suitable for further processing.

{
   "page_id": "156089_1321092_18",
   "entries": [
       {
           "entry_id": "156089_1321092_18-1",
           "company_name": "Färbereien Schetty, A.-G.",
           "location": "Basle"
       },
       {
           "entry_id": "156089_1321092_18-2",
           "company_name": "G. Kappeler A.-G.",
           "location": "Zofingen"
       },
       {
           "entry_id": "156089_1321092_18-3",
           "company_name": "J. A. Crabtree & Co., Ltd.",
           "location": "Walsall, England"
       }, ...

class TypeValuePair(BaseModel):
   type: str
   value: str
 
class Entry(BaseModel):
   entry_id: str
   company_name: str
   location: str
   additional_information: Optional[List[TypeValuePair]] = None
 
class ListPage(BaseModel):
   page_id: str
   entries: List[Entry]