AI agent observability: the OpenTelemetry + Langfuse pattern

Last updated: July 2026

Agent observability used to be a vendor debate. In 2026 it is a specification: the OpenTelemetry GenAI Semantic Conventions, maintained by a Special Interest Group under the CNCF, now defines what a trace event captures across six layers — LLM client calls, agent orchestration, MCP tool calls, workflow composition, content capture, and quality evaluation. Langfuse, Arize, AWS Bedrock AgentCore, and Datadog LLM observability all now consume traces on the standard OTLP endpoint. The vendor question has flipped from "which format" to "which backend fits your other tooling." This post is the engineer's tour of what a compliant agent trace actually contains, the pattern for shipping observability from day one, and where observability stops — because eval is the layer above it. It's a cluster child of our pillar on agentic AI workflow services and it operationalises the observability surface from our governance reference.

Key takeaways

• OpenTelemetry GenAI is the 2026 standard. The GenAI SIG formed in April 2024 and the spec is now at v1.41. Every serious agent framework instruments against it; every serious observability platform consumes it. If your stack does not emit OTel-compliant traces, you are on the wrong side of the standard.
• The span tree is invoke_agent at the top, chat and execute_tool as children. The nesting mirrors the agent's control flow. An operator reading the trace sees planning, tool calls, retrievals, and outputs as a single navigable timeline.
• Six standard attributes carry the load. gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.response.finish_reasons, and the corresponding cost and latency fields. Everything downstream — cost dashboards, quality alerts, compliance audits — reads from these.
• Content capture is off by default. Prompts, tool arguments, and outputs contain sensitive data. The spec keeps them out of traces unless you opt in per environment. Opting in is a security decision, not a default.
• Langfuse is now an OTel backend, not an alternative to OTel. Its /api/public/otel endpoint consumes OTLP traces from any instrumented agent. Same for Arize, Datadog, and the AWS-managed observability service. The lock-in that used to exist at instrumentation time is gone.

What a compliant agent trace looks like

Every action an agent takes emits a span. Spans nest to form a tree. Reading the tree tells you what the agent did, in what order, with what result, at what cost.

The top-level span is invoke_agent. It represents the whole task — a user request, an incoming event, a scheduled trigger. Its attributes carry the agent identity, the user or upstream agent on whose behalf the action ran, the total token count, the total cost, the total latency, and the final outcome.

Inside invoke_agent, the agent's planning and execution steps emit child spans. chat spans wrap LLM calls — one per call, with the model version, the input and output token counts, the finish reason, and (if content capture is enabled) the prompt and response. execute_tool spans wrap tool invocations — the tool name, the arguments, the return value, and the latency. retrieval spans (for RAG steps) carry the query, the retrieved chunks, and (where the platform supports it) the relevance score.

Multiagent architectures nest the pattern. When agent A calls agent B, agent B's invoke_agent span is a child of one of agent A's spans. The context of who called whom propagates through the trace, so an operator debugging a failure can follow the call chain from the top-level task down to the specific tool call that returned unexpected data.

The six layers OTel GenAI now covers

The original spec covered LLM client calls only. Since 2024 it has expanded to cover the full agent surface:

Layer 1 — LLM client call. The chat span. Model, tokens, finish reason, latency, cost. This is where the spec started and where every implementation still begins.

Layer 2 — Agent orchestration. The invoke_agent span and its children. Planning steps, tool selection, control flow. This is where the multi-step agent's behaviour becomes legible.

Layer 3 — MCP tool calling. Model Context Protocol tool invocations get their own execute_tool spans with MCP-specific attributes. As MCP has become the standard for exposing enterprise data to agents in 2026, this layer is where enterprise-context traces show up.

Layer 4 — Workflow composition. For multi-step or multi-agent workflows, the spec now captures workflow-level events — approval waits, human-review handoffs, batch job completions.

Layer 5 — Content capture. Opt-in. When enabled, spans include the full prompt, the full response, the tool schemas, the tool arguments, the tool results. This is what makes traces useful for debugging quality issues. It is also what makes them a regulated data source, so the opt-in default is deliberate.

Layer 6 — Quality evaluation. Newly added in v1.41. Spans can attach evaluation scores — the LLM-as-a-judge score, the golden-dataset match, the human-review verdict. Observability and eval used to be separate systems. In the current spec they are the same event graph.

The attributes every trace carries

The spec defines a small set of standard attributes that every instrumented agent emits. The list is short by design:

• gen_ai.request.model — the model called, e.g. claude-sonnet-4-5 or gpt-5-mini
• gen_ai.request.max_tokens, temperature, top_p — the request parameters
• gen_ai.response.finish_reasons — why the model stopped: stop, tool_calls, length, content_filter
• gen_ai.usage.input_tokens and gen_ai.usage.output_tokens — the token counts
• gen_ai.usage.cost — the cost in dollars (added by the instrumentation layer, not the model)
• gen_ai.agent.name and gen_ai.agent.id — the agent identity
• gen_ai.tool.name, gen_ai.tool.arguments, gen_ai.tool.result — tool call details (arguments and result gated by content capture)

Everything downstream — the token economics dashboards from our cost post, the human-review queue from our operation center post, the audit trail from our governance reference — reads from these attributes. Standardising them at the trace layer is what makes those downstream surfaces portable across observability backends.

The privacy default: content capture is off

The spec keeps prompts, tool arguments, and outputs out of traces by default. Turning content capture on is a per-environment decision, made against the specific regulatory frame the workload runs in.

Two patterns work in production. Environment-scoped capture: content capture is on in development and staging, off in production. Engineers debug against staging traces with full content; the production trace store never sees sensitive input. This works for high-throughput consumer workloads where the value of debugging any individual production event is low.

Redaction-in-flight: content capture is on in production, but a redaction step in the trace pipeline replaces PII, credentials, and regulated field values with hashes or category markers before the trace lands in the store. This works for enterprise workloads where debugging real production events matters more than absolute content minimisation. The redaction pipeline itself becomes an audit surface.

Which pattern fits is a Phase 1 conversation between engineering, security, and the compliance officer. The right answer varies by industry, by jurisdiction, and by the specific data classes the agent handles.

Langfuse, Arize, Bedrock AgentCore, Datadog — the backend layer

The observability backend is where traces live, dashboards render, and alerts fire. Four options are common in 2026:

• Langfuse. Open-source, OTel-compliant, strong on eval integration and prompt versioning. Fits teams that want visibility into prompt-level changes as first-class events.
• Arize AI. Enterprise-focused, strong on drift detection and continuous evaluation. Fits teams with ML operations maturity already in place.
• AWS Bedrock AgentCore. Managed service, tightly integrated with the Bedrock agent stack. Fits enterprises whose agents run on Bedrock and whose observability tooling is otherwise AWS-native.
• Datadog LLM observability. Native to the Datadog stack. Fits enterprises whose infrastructure observability is already Datadog and want the LLM layer to unify with everything else.

All four consume OTel-compliant traces on the OTLP endpoint. Switching between them costs an environment variable, not a re-instrumentation. That is the practical benefit of the standard: the backend choice is deferrable, and reversible when your enterprise's tooling shifts.

The pattern for shipping observability from day one

Observability is a launch requirement, not a launch follow-up. The pattern we use in delivery:

Instrument at the framework layer, not the application. The agent framework (LangChain, CrewAI, Google ADK, the Microsoft Agent Framework) is the natural instrumentation point. Native OTel support in the framework means the application code stays clean and the traces stay consistent across every agent in the fleet.

Emit to OTLP from Phase 2. The OTLP collector runs in the same infrastructure as the agent. Traces stream to it, then out to whichever backend the enterprise picked. If the backend choice is not finalised in Phase 2, the collector buffers to disk while the decision lands — the collection layer is not blocked on the decision.

Enable content capture in staging only. Production stays privacy-conscious by default. Content capture in production is a Phase 3 decision, and it requires the redaction pipeline to be shipped first.

Ship the trace-to-alert path early. A latency spike, a cost regression, a finish-reason distribution shift — the alerting rules for these get defined in Phase 2 so they fire from the first day of production traffic. Fixing the alerts later means missing the first two weeks of production behaviour, which is when the most surprises hit.

Where observability stops (and evaluation begins)

Observability tells you what the agent did. Evaluation tells you whether what it did was right. The two systems overlap — v1.41 of the OTel spec explicitly includes evaluation scores as span attributes — but they are not the same layer.

Observability's job is to record. Every span captures what happened. The observer decides what to look at.

Evaluation's job is to judge. Golden datasets in CI catch regressions before deploy. Online LLM-as-a-judge evaluators score live traffic continuously. Human review of edge cases feeds the next golden update. The evaluation results attach back to the trace as attributes, so the two systems form one queryable graph.

The teams that ship agents into production successfully treat observability as necessary and evaluation as sufficient. The trace shows you the agent's decision. The eval score tells you whether the decision was correct. Both belong in the same event store; both belong in the same operator's workflow. That is what OTel GenAI v1.41 makes possible, and what the pattern above lands.