Reconciling hundreds of partner restaurants across four sources

Twistag built an AI data pipeline for a leading global beverage company that reconciled hundreds of European partner restaurants across four conflicting data sources and more than ten languages into a single directory, fully automated and repeatable within three months. Records on TripAdvisor, TheFork, Google Reviews and venue websites disagreed on names, addresses, opening hours and cuisine categories, and deciding whether two listings describe the same physical restaurant is a judgment call rather than a field comparison. Twistag used conventional ETL for normalisation and deduplication, and reserved OpenAI calls for the judgment work: entity matching, field-by-field conflict resolution, and description synthesis in the brand's voice.

A global beverage company's on-premise presence is its restaurant network — the venues where consumers encounter the product in context. The company wanted that network navigable: a directory where someone could find partner restaurants by city, cuisine or dining style. The data made that harder than it sounds. Information for hundreds of venues sat scattered across TripAdvisor, TheFork, Google Reviews and individual restaurant websites, and no two sources agreed on anything — names, opening hours, cuisine categories and addresses all conflicted, often across more than ten European languages. "Ristorante Da Mario" on TripAdvisor, "Da Mario Restaurant" on Google, "da mario" on the restaurant's own site: sometimes the same place, sometimes not. Manually verifying each restaurant would have required a team of researchers working for months on the first market alone. The client is not named publicly.

Twistag's most important architectural decision was where to use AI and where not to. The approach was surgical: conventional ETL for normalisation and straightforward deduplication, AI only for cases genuinely requiring contextual judgment. That keeps AI cost proportional to complexity rather than to volume, and keeps system behaviour predictable.

The pipeline runs on Google Cloud Platform with Cloud Run for serverless scraping workers and Pub/Sub managing the event queue between stages. Puppeteer handles JavaScript-rendered platforms such as TripAdvisor and TheFork; lighter HTTP scrapers handle static restaurant websites; the Google Search API serves as a discovery layer. Adding a regional source means deploying a new scraper container without touching the rest of the pipeline. Raw scraped data then passes through a deterministic ETL layer — address standardisation, cuisine taxonomy mapping, language detection, obvious duplicate removal — and each field receives a confidence score based on the source's historical accuracy for that field type, the recency of the data and consistency across sources.

The reconciliation engine runs on OpenAI for three tasks: entity matching, deciding whether listings from different sources refer to the same physical restaurant using address proximity, cuisine alignment and review content patterns as well as name similarity; conflict resolution field by field rather than source by source, because Google tends to have fresher hours, TripAdvisor more granular cuisine classification, and the restaurant's own site the most accurate name; and content generation, synthesising a coherent profile in the brand's voice. Every reconciliation task is structured as explicit input, context and expected output, which makes decisions auditable.

Stack: OpenAI, Google Cloud Platform, Cloud Run, Pub/Sub, Puppeteer, Google Search API. Services: AI-ready data platforms, cloud modernisation, applied AI and reconciliation pipelines.

The first phase covered hundreds of partner restaurants across multiple European markets in three months, indexed across more than ten languages and reconciled across four major data sources. Each profile is the output of AI-driven reconciliation across at least two independent sources, with confidence scores on every field showing the team where manual review is worth spending. The more durable result is the architecture: adding a new European market means deploying scrapers for any region-specific sources and running the pipeline unchanged. Twistag has not published a figure for directory usage, partner engagement or verified coverage accuracy.

Phase two and beyond are operational expansions, not engineering projects.