S3 Vectors vs OpenSearch Serverless for RAG on AWS

This is a choice you only face on the customer-managed path — if you have not ruled out the fully managed option yet, building RAG on your S3 documents with Bedrock is the prior question, because a Managed Knowledge Base picks the store for you.

If you are building RAG on AWS and need somewhere to put your vectors yourself, S3 Vectors is the right default for most workloads, and Amazon OpenSearch Serverless is the exception you move to for a specific reason — sustained query volume, sub-10 millisecond latency, or search capabilities beyond similarity. Both plug into Amazon Bedrock Knowledge Bases, so the decision is about cost and latency behaviour rather than about which retrieval framework you are committing to.

The Short Answer

Your situation	Store
RAG queried intermittently — internal tools, support bots, document search	S3 Vectors
Large or fast-growing corpus where storage economics dominate	S3 Vectors
Vectors that must stay queryable but are rarely queried	S3 Vectors
Sustained query volume through the working day	OpenSearch Serverless
Single-digit millisecond retrieval inside a request path	OpenSearch Serverless
You need hybrid search, aggregations, faceting, advanced filtering	OpenSearch Serverless
Full corpus is cold, a hot subset is queried constantly	Both, tiered
A few hundred documents total	Probably neither — see the last section

The rest of this post is why, and the two mechanics that decide most real cases: how S3 Vectors bills queries, and what “scale to zero” actually costs you.

What S3 Vectors Is

S3 Vectors adds a new bucket type — a vector bucket — with dedicated APIs for storing and querying vectors, and no infrastructure to provision. Inside a vector bucket you create vector indexes.

• Up to 2 billion vectors per index, and up to 10,000 indexes per vector bucket
• Distance metric is Cosine or Euclidean, chosen at index creation
• Writes are strongly consistent — a query immediately after a write sees that write
• Standard S3 durability of 99.999999999% (11 nines)
• Metadata up to 40 KB per vector, of which 2 KB can be filterable

One decision to get right the first time: index name, dimension count, distance metric, and non-filterable metadata keys are immutable after creation. Changing any of them means creating a new index and re-uploading everything. Be deliberate about dimensions before loading a few hundred million vectors — your embedding model choice is effectively locked in alongside them.

What OpenSearch Serverless Is

A managed search and vector engine. Capacity is measured in OpenSearch Compute Units (OCUs), each 6 GiB of memory plus corresponding vCPU, and it gives you a full search engine rather than similarity lookup alone — hybrid search, aggregations, faceted search, advanced filtering.

It comes in two generations, and the difference matters enough that most guidance written before mid-2026 is now misleading.

Classic collections require a minimum of 1 OCU for indexing and 1 OCU for search. A vector search collection cannot pool OCUs with other collection types, so that minimum is dedicated and charged whether the collection serves one query or none — roughly $350 a month at idle, which for a long time was the single fact that decided this comparison.

NextGen collections, generally available since May 2026, remove that floor. Minimum OCU defaults to zero for both indexing and search, and after 10 minutes with no incoming requests the collection scales to zero. Indexing and search scale down and wake independently. AWS reports it auto-scales 20 times faster than the previous generation and saves up to 60% against provisioning for peak.

If you have read that OpenSearch Serverless carries a hard several-hundred-dollar monthly floor, that was true and is now specific to Classic collections.

Why S3 Vectors Is the Default for RAG

Two reasons, and the first is about how queries are billed.

The cost model has no idle charge at all

S3 Vectors prices on three dimensions:

PUT cost is billed on the logical GB uploaded, where each vector’s size is its vector data plus metadata plus key. Multiple vectors fit in a single PUT, so batching directly reduces upload cost.

Storage cost is logical storage summed across indexes, and the arithmetic is simple enough to do on paper:

• Vector data: 4 bytes per dimension — a 1024-dimension vector is 4 KB
• Metadata: filterable and non-filterable, both counted
• Key: 1 byte per character

Query cost is a per-API charge plus a $/TB data-processing charge based on average vector size multiplied by the number of vectors in the index you are querying.

That last one is the mechanic most write-ups skip, and it is worth being precise about: query cost scales with the size of the index, not with the number of results you asked for. You are billed for searching the corpus, not for retrieving the answer. The same query against a 500-million-vector index costs meaningfully more than against a 100,000-vector one.

This is an object-storage cost model applied to vectors, and it is why there is no idle charge — nothing is provisioned and nothing is held hot, so you pay for stored bytes and queries accrue only when you actually search. The economics also improve with scale rather than degrading: $/TB pricing improves above 100,000 vectors, and AWS has since cut query charges by up to 80% on indexes above 10 million vectors.

The latency it gives up does not matter to RAG

Nothing is kept warm, and that is the trade. S3 Vectors delivers sub-second latency for cold queries and under 100 ms warm, holding at billion-vector scale. AWS’s guidance is to choose it where roughly 100 ms or more is acceptable, and explicitly not for sub-10 ms requirements.

For RAG that is close to free. Retrieval sits in front of a model that will spend several seconds generating a response, so 100 ms of retrieval disappears into the noise. The requirement only bites when retrieval sits in a user-facing request path with a tight budget — type-ahead, an in-page recommendation call, a synchronous API with a 50 ms SLA.

The Cold-Start Trap

Once the Classic price floor is off the table, the obvious move looks like NextGen with a zero minimum: a full search engine, no idle cost. There is a catch, and it is documented plainly:

Expect 10–30 seconds of latency on the first request to each component while capacity is restored.

The 10-minute idle window is not configurable.

Put that next to the alternative:

	Idle cost	First request after idle	Warm latency
S3 Vectors	Storage only, no compute	Sub-second	Under 100 ms
OpenSearch Serverless NextGen (min 0 OCU)	Storage only, no compute	10–30 seconds	Single-digit ms
OpenSearch Serverless Classic	~$350/month	Always warm	Single-digit ms

For a workload queried a few times a day — which describes most internal RAG — that penalty lands on a real user nearly every time they use it. A support assistant that takes 20 seconds to answer the first question of the morning reads as broken, however fast the second one is.

So scale-to-zero did not make OpenSearch Serverless the natural home for intermittent RAG. It removed the idle cost and charged for it on the cold path. On the path intermittent workloads actually travel, S3 Vectors is roughly an order of magnitude faster.

Setting a minimum OCU above zero removes the cold start, but restores an idle bill — back to a version of the Classic trade with more knobs.

When to Run Both

AWS positions these as tiers rather than competitors, and the integrations follow. S3 Vectors works with managed Amazon OpenSearch Service so you can keep the storage economics while still using OpenSearch APIs, and vectors can be exported from S3 Vectors to OpenSearch Serverless for workloads that need real-time performance.

The pattern: S3 Vectors as the durable cold tier holding the full corpus, OpenSearch serving the hot subset that is genuinely queried at latency. BMW Group runs a variant of this — 20 petabytes queried in natural language, S3 Vectors for semantic similarity alongside Amazon Athena for SQL.

Worth reaching for when the corpus is large but access is heavily skewed: most of it archival, a small slice hot. If access is uniformly infrequent, one tier is enough, and it is S3 Vectors.

When You Need Neither

A corpus of a few hundred documents — a company blog, internal runbooks, product documentation — often fits inside a single model context window.

At that scale retrieval actively degrades answer quality. Chunking splits arguments into fragments and hands back the top few, where reading the whole document preserves the reasoning. Exact search across a few dozen files is instant, lossless and free. Semantic search is what you reach for when exact search stops scaling. If you are still deciding whether vector retrieval is the right shape for your data at all, knowledge graphs, RAG and GraphRAG compared is the prior question.

Building the pipeline before that point adds embedding jobs, sync logic and a retrieval layer in exchange for worse answers. Measure the corpus before choosing a store for it.

Summary

• Default to S3 Vectors for RAG. Sub-second cold, under 100 ms warm, no idle charge — and RAG’s latency budget is set by the model, not by retrieval.
• Move to OpenSearch Serverless for sustained query volume, sub-10 ms retrieval in a request path, or search features beyond similarity.
• S3 Vectors bills queries against index size, not result count. An object-storage model, which is why idle costs nothing.
• The ~$350/month floor is Classic-only. NextGen defaults to zero minimum OCU and scales to zero after 10 minutes.
• Scale-to-zero costs 10–30 seconds on the first request. For intermittent RAG that lands on a real user, and it is the strongest single argument for S3 Vectors.
• Tier them when the corpus is large and access is skewed. Use neither below a few hundred documents.
• Index configuration is immutable — dimensions, distance metric and non-filterable metadata keys are fixed at creation.

If you are costing out a full Bedrock deployment, the vector store is one line among several — the Bedrock cost structure breakdown covers the rest.

Pricing figures and service limits move. Verify current numbers on the S3 pricing page and the OpenSearch Service pricing page before committing to an architecture.