Back to Blog

AWS Bedrock + AgentCore: An Agentic AI Architecture That Ships

July 20, 2026

Most Agentic AI demos on AWS collapse in production for the same three reasons: unclear session state, no guardrail on tool invocation, and observability that stops at the LLM call. Bedrock + AgentCore fixes the first two directly if you use them idiomatically. The third is on you.

Here's the architecture I ship, and where each component earns its place.

The layered model

At a high level, an Agentic AI system on AWS looks like this:

  1. Frontend — a lightweight web/chat UI, an API endpoint, or a Slack/Teams surface. Nothing model-specific here.
  2. Session & orchestration — AgentCore handles multi-step agent loops, tool routing, and state.
  3. Foundation model — Bedrock invokes Claude, Nova, Titan, or a Mistral variant. Model choice is per-agent, not per-application.
  4. Tools / actions — Lambda functions, API Gateway, or direct AWS SDK calls, exposed as OpenAPI schemas to the agent.
  5. Knowledge base — Bedrock Knowledge Bases, backed by OpenSearch Serverless or Aurora Postgres with pgvector.
  6. Guardrails — Bedrock Guardrails for content filtering, PII redaction, denied-topic enforcement.
  7. Observability — CloudWatch Logs + CloudTrail + a custom trace store (DynamoDB or a proper trace backend).

Where AgentCore earns its keep

AgentCore is the piece that stops teams from rebuilding Router + Memory + Tool-schema-validation for the fifth time.

  • Session memory with per-user isolation, TTL, and encryption baked in.
  • Tool invocation via typed OpenAPI schemas — the model sees exactly what functions it can call and what inputs are valid.
  • Multi-agent coordination — a supervisor pattern where a router agent delegates to specialist agents (DevOps, FinOps, Support) without you writing the state machine.

You can do all of this with LangGraph or CrewAI on top of Bedrock InvokeModel directly. You'll ship 2x slower, and the audit story is harder.

Where Bedrock Knowledge Bases earn theirs

Ingesting docs into a vector store yourself works for a POC. In production the operational surface is what kills you: incremental ingestion, source-URL tracking for citations, syncing when the source S3 bucket changes, chunk-size tuning per collection.

Bedrock Knowledge Bases pins all of this to managed infrastructure. The trade-off is less control over chunking strategy — for niche formats (PDFs with tables, code repos), a custom pipeline is still better.

Guardrails: the piece nobody demos, everybody needs

Every enterprise Agentic AI POC hits the same review gate before production: what stops it from leaking PII, generating a legal opinion, or executing a destructive action?

Bedrock Guardrails answers three of those:

  • Content filters — configurable severity thresholds for hate, insults, sexual, violence, misconduct.
  • Denied topics — natural-language definitions (e.g., "medical advice") the model refuses to engage with.
  • PII redaction — inbound and outbound scrubbing for a fixed set of PII types.

What Guardrails does not do: prevent the tool from performing the destructive action. That's your responsibility — narrow the IAM role, add per-tool budget limits, and require human-in-the-loop for anything irreversible.

The reference architecture

User → API Gateway → Lambda (dispatcher)
                      ↓
                   Bedrock AgentCore (Session, Router)
                      ↓
                   [Tool Lambdas]  ←→  [Bedrock KB]
                      ↓
                   Bedrock InvokeModel (Claude / Nova)
                      ↓
                   Bedrock Guardrails (in-line)
                      ↓
                   Response streams back

Every arrow is CloudTrail-logged. AgentCore emits step-level traces. Guardrails emits its own audit stream. That's your observability floor — you'll want to add per-request trace IDs so a single conversation can be reconstructed end-to-end.

What breaks in production, and how

1. Cost blowouts from context bloat. Each turn the agent may re-inject the full tool schema + prior turns + KB context. A conversational agent averaging 5 turns can burn 40k input tokens before you notice. Mitigate with per-agent token budgets and summarisation prompts.

2. Tool-invocation hallucinations. Even with strict schemas, models occasionally invent tool names or pass typed-string values where enums are expected. AgentCore's schema validator catches most of this — but log the raw model output before validation so you can trace failure modes.

3. Multi-agent deadlocks. Supervisor-of-specialists patterns can loop when two specialists disagree. Impose a max-turn ceiling and log every hand-off.

4. Latency drift. A model change (Claude 3 → 3.5 → 3.7) can shift per-turn latency by 300–800ms with no code change. Alert on p95 turn latency, not just error rate.

When not to use AgentCore

  • Single-turn Q&A over docs — Bedrock Knowledge Bases + a plain InvokeModel is simpler.
  • Highly deterministic workflows — Step Functions with LLM calls at nodes is easier to reason about than an agentic loop.
  • Batch inference — Bedrock Batch API, not AgentCore.

Agentic patterns are worth it when the sequence of tool calls isn't knowable in advance, and when the value of the answer is high enough to tolerate the latency of a multi-step loop.

Bottom line

If you're standing up an Agentic AI capability inside an existing AWS estate, Bedrock + AgentCore + Guardrails is the shortest path from POC to a production posture that survives an audit. You'll still write the observability layer and the tool-level authorization boundaries — but you won't be reinventing the agent runtime.

LinkedIn