Conversational commerce architecture: what replaces e-commerce search

Last updated: July 2026

Classical e-commerce search is a keyword box that returns a ranked list. It has been that way for twenty years and it is losing the shopper who types "shoes I can run in that are good for flat feet under 150 euros" and gets nothing useful. Conversational commerce is what replaces that shopper's experience: a system that understands the description, plans the searches that answer it, reasons over the results, and returns a recommendation with the shopper's constraints explained. This post is the architecture behind that experience — the three layers that ship, the retrieval patterns underneath, the cost pattern that keeps it affordable, and the boundary between conversational commerce and a chatbot bolted onto a product catalogue. It's the pillar for our cluster on AI-native product engineering.

Key takeaways

• Conversational commerce is not a chatbot on a catalogue. A chatbot fragments the shopping journey (browse → open chat → browse again). Conversational commerce integrates natural-language input into the core discovery surface. The shopper's description IS the query.
• Three layers ship: intent understanding, retrieval, response generation. Intent understanding classifies and normalises the query. Retrieval combines vector search, keyword, and structured filters. Response generation produces a curated recommendation grounded in the catalogue, not a ranked list.
• Classify before you invoke the LLM. Simple queries ("Nike Pegasus 41") route to keyword search — no LLM needed. Complex intent queries ("shoes for flat feet, wide toe box, under €150") route to the reasoning layer. This alone cuts LLM cost by 60-80% (Alhena AI, 2026).
• Vector search is necessary but not sufficient. Vector search solved relevance. Generative discovery solves decisions. The LLM reasons over the retrieved candidates and explains the recommendation to the shopper. Both layers ship together.
• Grounding in the catalogue is non-negotiable. Every recommendation resolves to a specific SKU, a specific stock level, and a specific price. Hallucinated products are a category-killing failure mode. Structured output enforced against the catalogue schema is the guard.

Why keyword search is losing

The classical e-commerce search pipeline is a tokeniser, an inverted index, a ranking function, and a facets sidebar. It works when the shopper knows the product's proper name. It fails when the shopper knows what they want the product to do.

The gap is widening in 2026 because shoppers have been trained by LLM chat interfaces to describe rather than name. A shopper who spent thirty minutes chatting with an AI assistant last Tuesday is not going to shrink their vocabulary when they arrive on a retailer's product page. They will type in the same descriptive shape. If the retailer's search returns "no results" or an obviously irrelevant list, the shopper leaves.

The industry response has been three-fold. Semantic search, which uses vector embeddings to match on meaning rather than tokens. Chatbots, which sit alongside the catalogue as a separate surface. And a growing set of retailers building conversational commerce as the primary product discovery experience — the shopper's description IS the query, and the response is a curated recommendation grounded in the retailer's catalogue.

The first two are half-measures. Semantic search still returns a ranked list; the reasoning does not happen. Chatbots fragment the journey — the shopper opens a chat to describe the problem, then has to close it to browse the recommended products. Conversational commerce is the integration.

What conversational commerce actually is

Conversational commerce is a product discovery surface where the shopper describes what they need in natural language, and the system returns a curated recommendation — one to five products, each grounded in the catalogue with stock, price, and reasoning about why it fits. The interaction can be single-turn ("I need a rain jacket for city cycling") or multi-turn ("also under €200"), and the recommendations refine as the conversation progresses.

Three properties distinguish it from a chatbot bolted onto a catalogue:

The input surface is primary, not secondary. The shopper does not have to open a chat window. The search box accepts descriptive queries and routes them through the same understanding layer. Existing keyword queries still work — the shopper does not have to change behaviour.

The output is a recommendation, not a list. Classical search returns twenty ranked results and leaves the ranking rationale opaque. Conversational commerce returns one to five products with a reason attached to each: "this pair has a wide toe box, is under €150, and reviewers with flat feet rated it highly." The reasoning is what turns discovery into a decision.

The catalogue grounding is enforced. Every recommendation resolves to a real SKU. The LLM does not invent products, does not misstate stock, does not misquote price. Structured output against the catalogue schema is the guardrail. Hallucinated products in a retail context are worse than no result — they destroy trust in the recommendation surface.

The three-layer architecture

Every production conversational commerce system we have looked at converges on the same three-layer shape.

Layer 1: Intent understanding

The intent layer classifies and normalises the query. Its job is to decide what the query needs before invoking any expensive downstream layer.

Classification. Is this a keyword query ("Nike Pegasus 41")? A descriptive query ("running shoes for flat feet")? A multi-turn continuation ("also in a smaller size")? A comparison request ("this one vs the other one")? A support question ("when will this ship")? Each class routes to a different pipeline.

Normalisation. For descriptive queries that will go to the reasoning layer, the intent layer extracts structured constraints — price range, size, colour, category, brand — and converts them into filters. The remaining descriptive content stays as free text for the retrieval and reasoning layers to work on.

The cost pattern lives here. Classification is a cheap model call (small open-weight model or a rules-based classifier). Only complex intent queries route to the expensive reasoning layer. Keyword queries skip the LLM entirely. This is the token economics discipline from our cost post applied to conversational commerce.

Layer 2: Retrieval

The retrieval layer's job is to produce a set of candidate products the reasoning layer can decide over. Three retrieval mechanisms combine, always in parallel:

Vector search for semantic relevance. Product descriptions, review summaries, and structured attributes get embedded into a vector store (Pinecone, Weaviate, Qdrant, pgvector, or the retailer's existing choice). The query embedding retrieves the nearest neighbours in embedding space. This layer handles the "wide toe box" part of "wide toe box running shoes" — the store has products described in terms of "roomy forefoot" or "generous toe area," and vector search matches on meaning.

Keyword search for exact-name and identifier matching. Product names, SKUs, brand names — a shopper who typed "Nike Pegasus 41" should get the Pegasus 41, not something semantically similar. Elasticsearch, OpenSearch, or the vector store's hybrid mode. This layer is fast and cheap and handles a large share of real traffic.

Structured filters for the constraints the intent layer extracted. Price under €150, size 42, in stock. Filters apply as hard cuts on top of the vector and keyword results. This is where the catalogue schema pays back — every product has known-good structured fields, and filtering is deterministic.

The combined result is a candidate set of 20-100 products, ranked by a hybrid score that combines vector similarity, keyword relevance, and structured-filter compliance. This candidate set is what the reasoning layer decides over.

Layer 3: Reasoning and response generation

The reasoning layer takes the candidate set and produces the recommendation. This is where the LLM lives, and where the decision replaces the ranked list.

The reasoning step. The LLM is prompted with the shopper's original query, the extracted constraints, the top candidate products (with their structured attributes and short descriptions), and any conversation history. It selects one to five products from the candidate set and explains the selection.

The grounding constraint. The LLM's response fits a structured output schema. Each recommended product references a specific SKU from the candidate set — the LLM cannot invent products, cannot misstate attributes. Post-generation validation checks that every referenced SKU exists in the catalogue and that stock and price match the current state.

The response shape. One to five products with a per-product reason ("wide toe box, under €150, high rating from flat-foot reviewers") plus a short overall summary of the shopper's constraint set. The output can render as a curated grid, a comparison table, or a chat-like sequence, depending on where the surface lives in the retailer's UI.

The cost pattern that makes it viable

Conversational commerce at retail scale sees millions of queries per day. Running every query through the reasoning layer would cost more than the incremental revenue justifies. The industry has converged on the classify-first pattern to solve this:

• Keyword queries ("Nike Pegasus 41") route to keyword search only. No LLM call. Sub-cent cost per query.
• Structured queries ("running shoes size 42 under €150") route to keyword + filter. Still no LLM. Cheap.
• Simple descriptive queries ("cheap running shoes") route to vector search only, with a small classifier LLM re-ranking. Cents per query.
• Complex descriptive queries ("running shoes for flat feet with wide toe box under €150 that reviewers rate for long distances") route to the full three-layer pipeline. This is where the reasoning LLM runs. Highest cost — but the query volume is a small percentage of total traffic.

The classification step itself is a cheap small-model call — sub-cent per query. The result: LLM cost per query lands in single-digit cents on average across all traffic, even though the complex-query subset costs closer to ten cents each. This is the pattern that makes conversational commerce a positive-margin feature rather than a cost sinkhole.

Where conversational commerce stops (and chatbots begin)

There is a legitimate boundary between conversational commerce and post-purchase support. Conversational commerce is a discovery surface — its job is to help the shopper choose. Post-purchase questions (shipping status, returns, warranty) belong in a support chat, not in the discovery surface. Conflating the two produces a discovery experience that also does support badly and a support experience that also does discovery badly.

The pattern that works is a shared identity layer with two surfaces. The shopper's session, their catalogue history, and their preferences persist across both. When they land in the discovery surface, the recommendations know their history. When they land in support, the support agent knows what they bought. But the surfaces themselves are optimised for different jobs — recommendation and reasoning versus troubleshooting and resolution.

The implementation pattern

Shipping a conversational commerce system into production tends to follow the same sequence:

Phase 1 — Catalogue readiness audit. Do the products have structured attributes? Are the descriptions rich enough for vector embedding? Are stock and price data reliable? A catalogue that is not agent-ready cannot support conversational discovery, regardless of the front-end architecture. This is the agent-readable data governance work extended to retail.

Phase 2 — Retrieval layer first. Ship vector search, keyword search, and structured filtering as the underlying retrieval before touching the reasoning layer. The retrieval layer alone is already better than the classical search for a large share of queries.

Phase 3 — Intent classification and cost gating. Ship the classification layer that routes queries to the right pipeline. This is what makes the reasoning layer economically viable.

Phase 4 — Reasoning layer and grounded output. Add the LLM reasoning step with structured output enforcement. Start with a narrow query class (descriptive complex queries) and expand once the eval pipeline is stable.

Phase 5 — Continuous evaluation. Golden datasets in CI catch regressions. LLM-as-a-judge evaluators score live traffic. Human review of edge cases feeds the next iteration. The observability and evaluation pattern from our OTel post applies unchanged — a conversational commerce system is an agent, and it needs the same instrumentation.

What this shifts for the retailer

Conversational commerce is not a bolted-on feature. It is a change to what the search surface does. The retailer that ships it well will see three things move:

Discovery conversion goes up on descriptive queries — the shoppers who used to get "no results" and leave now get a curated recommendation and buy. This is the primary upside.

Support tickets on "how do I find X" go down — the discovery surface answers the question the shopper would have asked support. This is a downstream saving.

The catalogue investment pays back differently. Rich product descriptions, structured attributes, and quality review summaries all matter more than they did before. The retailer that has been investing in catalogue quality gets a compounding return. The retailer that has been treating the catalogue as a data dump discovers that conversational discovery does not work on a dump.

Every retailer running an e-commerce store in 2026 is on the wrong side of at least one of these shifts. Naming which one you are on wrong side of is the first Phase 1 conversation.