Retrieval-augmented generation on AWS has two paths. Bedrock's managed Knowledge Bases feature, or a custom pipeline you build on OpenSearch / Aurora with pgvector / Pinecone / anything else. Both work. They're not interchangeable.
What Bedrock Knowledge Bases handles for you
- Ingestion from S3 (documents, PDFs, text, HTML, CSVs).
- Chunking (with a small number of configurable options).
- Embedding generation via Bedrock embedding models.
- Vector store provisioning (backed by OpenSearch Serverless, Aurora PostgreSQL with pgvector, or Pinecone).
- Metadata filtering.
- Reranking (with the newer models).
- Session-tied conversational retrieval when connected via AgentCore.
The value proposition is "you drop a bucket of PDFs and get retrieval." That's largely accurate.
Where Knowledge Bases hits its limits
Chunking strategy. You get a handful of options — fixed size, hierarchical, semantic. What you don't get is arbitrary control over how a document is decomposed. If your source documents have unusual structure (tables in PDFs, code blocks in markdown, legal contract clauses), the default chunkers will underperform.
Embedding model choice. You use Bedrock's embedding models. Fine if your domain is well-covered by them. Constrained if you need a domain-specific embedding (medical, legal, code).
Retrieval logic. You get similarity search plus optional metadata filters. If you need hybrid keyword + vector, or graph-augmented retrieval, or query rewriting — you're layering your own logic on top and losing some of the managed simplicity.
Reindexing cost. When your source S3 bucket changes, reindexing is triggered. This is fine at small scale. At larger scale (10s of GB, changing daily), it becomes an operational cost line.
What a custom RAG pipeline gives you
- Full control over chunking. Semantic parsers, custom heuristics per document type, overlap tuning.
- Choice of embedding model.
- Hybrid retrieval (BM25 plus vector plus reranking).
- Query rewriting and query decomposition.
- Selective reindexing at document level rather than bucket level.
- Ability to add graph or knowledge-graph augmentation.
What it costs you:
- 2–4 weeks of engineering to get to feature parity with Knowledge Bases.
- Ongoing ops burden — you're now running an ingestion pipeline, a vector store, and a retrieval service.
- The overhead of keeping up with model releases yourself.
The decision heuristic
Use Knowledge Bases when:
- Your source documents are well-behaved (Word docs, PDFs with normal structure, HTML pages).
- Your domain is general enterprise content (policies, product manuals, customer support articles).
- The team building the RAG doesn't have deep ML experience.
- Time to first working system matters.
- You're operating on AWS and don't need portability.
Roll your own when:
- Your documents have unusual structure (heavy tables, code, legal clauses).
- You need domain-specific embeddings.
- You need retrieval logic beyond similarity + metadata filter.
- Your operating environment includes non-AWS components.
- You have ML engineers on the team who will run it.
The under-appreciated middle path
You can use Knowledge Bases for the 80% of documents that fit its patterns and a custom pipeline for the 20% that don't. This is what I've done twice. It looks like two retrievers behind a single router — the agent decides which to query based on the question. It's not elegant. It works.
What actually determines retrieval quality
Not the vector store. Not the embedding model. Not the chunker.
- Source document quality. Garbage in, garbage out. If your PDFs are scanned images with no OCR, no retrieval strategy will save you.
- Chunk semantic completeness. Each chunk should be understandable on its own. If a chunk depends on the previous paragraph for context, retrieval will fail on it.
- Metadata. Chunks tagged with document type, section, and date allow the retriever to filter. Metadata is almost always underused.
- Reranking. A rerank pass over top-K candidates is disproportionately more impactful than a better initial retrieval.
Get these right and either implementation performs. Get them wrong and neither will.
Bottom line
If someone asks "should we use Bedrock Knowledge Bases?" the honest answer is "depends on your documents." I have shipped both. The Knowledge Bases wins were on generic corporate content. The custom RAG wins were on specialized document types where the domain expertise was in the chunking, not the retrieval.
The good news is you can start with Knowledge Bases, see where it falls short for your content, and move to custom for those specific cases without throwing the initial investment away. The bad news is teams that start with custom RAG rarely simplify back to Knowledge Bases even when they could have.