An Agentic AI system in production without proper observability is going to fail in ways your team cannot diagnose. Not "occasionally" — routinely, and unpredictably. Regular APM tells you the p99 latency was 3.2 seconds. It does not tell you why the agent decided to call the refund tool with a negative amount.
What traditional observability doesn't cover
CloudWatch, Datadog, New Relic — all excellent for what they were designed for. What they don't cover:
- The model's reasoning path. Which tools did it consider before choosing one? What was the reasoning between turns?
- The tool-invocation payload. What arguments did the model pass, and why?
- The retrieved context. Which chunks came back from the knowledge base? Were they actually relevant to the query?
- The rejection reasons. When a guardrail fires, what content triggered it?
- The multi-turn conversation state. What did the agent "remember" from earlier turns and how did that shape this turn?
None of that shows up in an APM dashboard by default. You have to instrument it.
The layers of an Agentic AI trace
For every user-facing turn, you want to record:
- The user input. Verbatim, but with PII redacted.
- The prompt sent to the model. Full context including retrieved KB chunks and prior turn summaries.
- The model's raw output. Before any post-processing.
- The tool invocations. Each call, its arguments, and its return value.
- The guardrail outcomes. Which policies evaluated, which fired, at what severity.
- The final response. After post-processing, before it goes to the user.
- Timing. Per-step latency, not just total.
That's roughly 7 log entries per turn. For a busy production system, that's a lot of log volume. Plan for it.
Where to store this
Two options for enterprise deployments:
- CloudWatch Logs with structured JSON. Simplest, integrates with CloudTrail, cheapest to start. Log Insights queries work but get slow at scale.
- OpenSearch or a dedicated observability backend. Structured, queryable, expensive. Worth it when you're routinely debugging conversations.
For most POC-to-production paths, start with CloudWatch, migrate to OpenSearch when you hit the query-speed wall. Both work.
The critical field: correlation ID
Every turn gets a UUID. Every log entry from that turn — the input, the prompt, the tool calls, the guardrail evaluations, the response — carries the same UUID. This is the single most important design decision.
Without it, you cannot reconstruct a conversation. With it, "show me everything that happened for turn X" is a one-line query.
At the API-gateway level, generate the correlation ID. Pass it through every downstream component as a header, then embed it into every log entry.
What to redact
PII you must redact:
- Email addresses (unless the interaction was via email).
- Phone numbers.
- Names in some regulatory contexts (India DPDP, GDPR).
- Any customer identifier the model was given as context.
PII you may not need to redact:
- Session IDs (they're your own identifiers).
- Prompt IDs.
- Model version.
The redaction happens in the logging layer, not in the model output. Redact before you log, not after.
The debugging pattern that works
A user reports: "the agent gave me the wrong refund amount." You go to the logs.
- Query by user session ID → get all correlation IDs from that session.
- Query by correlation ID → get the full turn trace.
- Look at retrieved KB chunks → is the reference data even in the knowledge base correctly?
- Look at tool invocations → did the model call the right tool with the right args?
- Look at model output → is the reasoning path visible or opaque?
Most Agentic AI bugs I've debugged in production fell into three categories:
- Bad retrieval. The knowledge base returned the wrong chunk. Fixed by improving chunking or reranking.
- Mis-parameterized tool call. The model passed reasonable-looking but wrong arguments. Fixed by tightening the tool schema or adding validation.
- Prompt drift. The system prompt was subtly changed and behaviour shifted. Fixed by treating prompts as versioned artifacts, not free text.
Without observability, none of these are diagnosable. You just see "the agent is wrong sometimes."
Alerts that matter
- Guardrail fire rate spikes. A sudden increase means someone is trying to jailbreak, or a prompt change introduced a new safety issue.
- Tool-invocation error rate. The model calling tools that fail is a diagnostic signal about model behaviour changes.
- p95 turn latency drift. A model upgrade you didn't initiate can shift latency by hundreds of ms.
- KB retrieval empty-result rate. If retrievals are returning nothing, either the index is broken or user queries have drifted from your knowledge base's scope.
Alert on rates and trends, not absolutes. A single tool failure is normal. A sudden 10x increase is a signal.
What I under-invested in early
Trace visualisation. For the first two production rollouts, we were querying raw logs to reconstruct conversations. It worked but was painfully slow. What eventually made a difference was a small internal tool — one screen, correlation ID as input, timeline view of the turn as output. Nothing fancy. Just faster than log-splunking.
If you're going to be debugging Agentic AI in production, budget a week to build the equivalent. It pays back within the first incident.
Bottom line
Agentic AI observability is not standard APM. It is a discipline you build alongside the system. Every hour you spend on observability before production launch is worth ten hours during the first three months of production. Every team I've seen skip this in the name of "shipping faster" has come back to build it later, more painfully.