Data foundations agents can use

The real bottleneck

When an AI project slips, the cause is rarely the model. It is usually the data. The agent cannot reach the documents it needs, the same customer appears under three IDs, or a field everyone relies on turns out to be filled in by hand and often wrong.

Gartner’s research puts numbers on this. In a 2024 survey of data management leaders, 63% of organisations said they did not have, or were not sure they had, the right data management practices for AI. Gartner predicts that through 2026, organisations will abandon 60% of AI projects that are not supported by AI-ready data.

That sounds like an argument for a large data programme before any AI work. It is the opposite. The companies that move fastest fix the data a specific use case needs, as part of building it, and reuse that work for the next one.

What “AI-ready” means for an agent

Data is ready for an agent when four things are true for the use case in question:

1. Reachable. The agent can read the sources it needs, through an API, a database connection or a document store, with permissions that a security team has approved.
2. Joined. The same customer, supplier, product or site can be recognised across sources, even when each system records it differently.
3. Trusted. The business knows which sources and fields are reliable, and the agent can tell when it is working with something uncertain.
4. Current. The data is fresh enough for the decision. A scheduling agent needs today’s bookings. A contract agent can work from last month’s archive.

Unstructured data counts. Contracts, invoices, emails, policies, call notes and PDFs hold much of the knowledge agents need, and language models can read them. What they need is a store the agent can search, with access controls that match who is allowed to see what.

Access first

In Twistag’s experience, the longest delays in the first weeks come from access, well before data quality becomes the issue. A request for read access to an ERP, a document archive or a CRM can take weeks when nobody owns the decision.

Three steps shorten this:

• List every source in week one. For the first use case, write down each system and document store the agent must read, and the few it must write to.
• Name an owner for each. The person who can approve access, and the person in IT who can set it up.
• Start with read-only. Read access is easier to approve and enough for most proofs. Write access comes later, often through an approval step.

The agent readiness check below turns this into twelve questions with a red, amber or green answer.

Joining the records

Most mid-market and enterprise companies hold the same objects in several systems. A guest exists in the booking engine, the property system and the loyalty app. A supplier exists in the ERP, the procurement tool and an email thread. An agent needs to know these are the same thing.

Two Twistag engagements show how this is done in practice.

A global beverage company wanted a directory of its partner restaurants across Europe, searchable by city, cuisine and dining style. The information on hundreds of venues came from four sources: TripAdvisor, TheFork, Google Reviews and the restaurants’ own websites, in more than ten languages. Names, addresses, opening hours and cuisines often disagreed. Twistag split the work in two. Deterministic code standardised addresses, mapped cuisines to one list, detected languages and removed obvious duplicates. A language model handled the judgement calls: deciding whether two listings were the same restaurant, and resolving conflicts field by field. Each field got a confidence score based on the source’s accuracy for that kind of field, how recent it was and whether sources agreed. The result was a fully automated, repeatable pipeline in three months.

Datatalks, a customer data platform Twistag has engineered for six years, faced the same problem at scale. Fan data for more than 50 sports organisations sat across ticketing, e-commerce, CRM, point-of-sale and social systems, so one fan appeared as several unconnected records. An identity-resolution engine now matches people across sources that record them by email, phone or social handle, and the platform holds more than 6 million unified fan profiles.

The pattern in both cases: rules handle what rules can, a model handles judgement, and every merged record carries a confidence score so people know what to check.

Model the business first

The fastest way to make data reusable is to agree a simple model of the business objects the agents will work with: customers, suppliers, products, orders, contracts, sites. Each object gets one definition, one ID and a known source for each field.

A European RegTech start-up serving the cosmetics industry shows why this pays. Twistag designed the data model first, as a single source of truth for raw materials, suppliers, formulations, regulatory documents and customer requirements. Four agents then ran on that model: regulatory inquiries, supplier diligence, formulation and procurement. Each new agent reused the same foundation. Regulatory inquiries that took three to four hours now take under an hour, a 75% saving.

The hotel group in Chapter 1 followed the same path. Before the optimiser could suggest a single rotation, a data layer had to bring bookings, occupancy and staffing from every property, on different property systems, into one view.

A data model does not need a long programme. It can start with the three or four objects the first agent needs, and grow with each use case.

Trust and freshness

Agents make mistakes when they treat uncertain data as certain. Three practices reduce this:

• Mark the trusted sources. For each field, decide which system wins when sources disagree.
• Score confidence. When records are merged or extracted from documents, keep a confidence score and send low-confidence items to a person.
• Set refresh rates by decision. Some agents need live data, others a daily or weekly refresh. Over-engineering freshness adds cost without value.

Source.app, a marketing attribution platform Twistag built, pulls data from six marketing tools into one normalised model and refreshes it every four hours. That rhythm matches the decisions campaign managers make each day.

Security and permissions

An agent should see what the person it works for is allowed to see, and nothing more. That means:

• respecting the same permissions as the source systems, especially for documents;
• keeping personal data out of prompts and logs unless the use case needs it;
• recording which sources each answer came from, so it can be checked later.

Chapter 8 covers these controls in the context of governance and regulation.

Common mistakes

• A data programme before any use case. Months of cleaning data nobody has asked for, with no agent at the end.
• Cleaned samples for the proof. The agent works on the sample and fails on the real archive.
• Treating documents as out of scope. Much of the knowledge an agent needs lives in PDFs, emails and contracts.
• No owner for the data model. Every team adds its own version of “customer”, and the next agent starts again.
• Ignoring permissions. An agent that shows a junior employee a board paper ends the programme faster than any technical failure.

What to do this month

1. For the first use case, list every source the agent must read and write, with an owner for each.
2. Request read access in week one, and track how long it takes.
3. Agree the three or four business objects the first agent needs, with one ID and one trusted source per field.
4. Score each source red, amber or green with the agent readiness check, and fix the reds first.

Questions for the leadership team

1. For the first use case, which documents, records and systems does the agent need to read, and which does it need to write to?
2. Who owns each of those sources, and how long does it take today to get read access?
3. Where does the same customer, supplier or product appear under different names or IDs in different systems?
4. Which fields does the business trust, and which does everyone check by hand?
5. Who will own the shared data model once the first agent is live?