Which Tier Does Your Vector Workload Live On?

TL;DR

• Since Bedrock Knowledge Bases launched in late 2023, the default vector store was OpenSearch Serverless with a ~$700/month floor you paid whether you queried it or not. Amazon S3 Vectors added a true pay-per-use tier, and the question changed to which tier of the cost/latency curve your workload belongs on.
• The tiers sort cleanly: hot (OpenSearch, sub-50ms, always-on), warm (Aurora + pgvector, beside relational data), cold (S3 Vectors, ~100ms, pay-per-use). Picking a store is picking a tier, not picking a winner.
• The cold tier is a category the market already validated. Turbopuffer built a ~$100M business storing vectors on object storage for Cursor, Notion, and Anthropic; Cursor cut its vector-database cost 95% moving to it. S3 Vectors is AWS making that same architecture a managed primitive.
• Agent workloads belong on the cold tier, and the reason is structural: an agent’s retrieval is one step inside a multi-second reasoning loop, so 100ms versus 10ms is noise. The latency that disqualifies a customer-facing search box is invisible to an agent.
• S3 Vectors is not a database replacement. It is semantic-only, has no hybrid search, and trades single-digit-ms latency for cost. Know the three limits before you build on it.

When Amazon Bedrock Knowledge Bases launched at the end of 2023, the default vector store was OpenSearch Serverless, and it came with a bill before you stored anything. The floor was roughly $700 a month — four OpenSearch Compute Units (two for indexing, two for search with standby) at about $0.24 per unit-hour, running whether you queried once a day or a thousand times a second. For a production search system, that floor is rounding error. For a personal knowledge base or an internal document search, it is the reason the project never ships.

Amazon S3 Vectors, generally available since December 2025, prices the way the rest of S3 does: no minimum charge, pay for what you store and what you query, nothing while idle. One 30-project analysis puts the entry point at roughly $0.60 for a first month against OpenSearch Serverless’s hundreds, and finds S3 Vectors 15 to 66 times cheaper across small and medium workloads. OpenSearch has since narrowed its own gap — fractional compute units and a scale-to-zero tier cut the idle cost — but the arrival of a true pay-per-use tier is what reset the default. The question stopped being “which vector database wins” and became “which tier of the cost and latency curve does this workload belong on.”

The Tiers

Every AWS-native vector option now sorts onto one curve that trades latency for cost. Picking a store is picking a point on that curve.

Amazon Bedrock Knowledge Bases sits across the whole curve as the managed RAG layer, and it supports every store as a backend — OpenSearch Serverless and Managed Cluster, Aurora, Neptune Analytics, Pinecone, MongoDB Atlas, Redis, and S3 Vectors. You choose the tier; Bedrock runs the ingestion, chunking, embedding, and retrieval on top of it. The store is a decision about cost and latency, not a decision about capability.

The shift since S3 Vectors arrived is that the top and bottom of the curve now combine instead of compete. You keep hot vectors in OpenSearch for the queries that need speed and spill the long tail into S3 Vectors for cost, and AWS supports importing directly from one to the other. “Pick one database” was the wrong frame; “which tier for which slice of the data” is the right one.

The Cold Tier Is Already a Business

S3 Vectors can read like an AWS experiment if you have not been watching the vector-database market. It is AWS’s managed entry into a pattern that a startup has already turned into a company.

The storage math behind the cold tier is not subtle. When you embed text, the vectors are much larger than the source — Turbopuffer measures 1KB of text expanding to roughly 16KB of vector data after chunking and embedding. Keeping all of that in memory or on SSD is where the bill comes from. Object storage runs about $0.02 per GB against roughly 16 times that for SSD, and most vectors in a real corpus are read rarely. Storing the cold majority on object storage and caching only the hot slice is the whole idea.

Turbopuffer built its company on exactly that idea. It stores vectors on S3, GCS, or Azure Blob and caches hot data on SSD and RAM by access pattern. Sacra estimates it reached about $100M in annualized revenue by March 2026, up 2,400% year over year, serving Cursor, Notion, Anthropic, Linear, and Superhuman. When Cursor moved its codebase index to this architecture, it reported cutting vector-database cost by 95% — because most codebases are queried infrequently, and paying in-memory rates to keep them all live was the expensive mistake.

Turbopuffer is one vendor of several. LanceDB is an open-source, file-based vector store that runs directly on any S3-compatible backend, and AWS itself published an architecture for 1B+ vectors on LanceDB, S3, and Lambda where the only fixed cost is the S3 storage footprint and each query costs fractions of a cent. Spice.ai integrated S3 Vectors into its open-source engine. The industry even converged on shared vocabulary for it — hot, warm, and cold vector tiers — because enough teams are building this way that the pattern needed names.

S3 Vectors is AWS making that architecture a managed primitive: the cold tier without running the caching layer yourself. The reason to trust the tier is that a $100M business, a marquee set of AI customers, and AWS’s own reference architecture all point at the same economics.

Why Agents Belong on the Cold Tier

Here is the insight that makes S3 Vectors the default for agent work, and it is easy to miss if you only look at the latency number.

A customer-facing search box lives or dies on latency. A user typed, a box must fill, and 100 milliseconds versus 10 is the difference between snappy and sluggish. That workload belongs on the hot tier, and the OpenSearch bill is the cost of the experience.

An agent is a different shape of workload. When an agent retrieves, the retrieval is one step inside a reasoning loop that already takes seconds — the model is thinking, calling tools, reading results, thinking again. As one analysis puts it, adding a vector lookup that takes 100ms instead of 10ms is rarely the bottleneck when the model-side latency of a single tool call is already 500ms to several seconds. The 90 milliseconds you save on the hot tier vanish inside a loop that was never going to be fast. You paid the always-on floor to optimize a step that is not on the critical path.

This is why AWS positions S3 Vectors as purpose-built for agent memory: agent memory grows continuously, is queried in bursts, and tolerates hundreds of milliseconds because it lives inside a slow loop. The cold tier fits the shape of the work. The hot tier would be paying a premium for speed the agent cannot use.

The access pattern is the same one that made the cold tier a business. Cursor’s codebases sit dormant most of the time and get queried in bursts; agent memory has the identical profile — mostly idle, occasionally read, never on a human’s critical path. The workload that justified an object-storage-native database for code search is the workload most agents already have.

Where S3 Vectors Fits

The official use cases share one profile: large or growing corpora, moderate or bursty query rates, latency budgets measured in hundreds of milliseconds, and a preference for zero infrastructure. That profile shows up in five recurring patterns:

• Semantic search over large collections — documents, media, medical images, video archives where cost per vector dominates.
• RAG long tail — the bulk of a knowledge base that is queried occasionally, with a hot tier in front only if some subset gets constant traffic.
• Agent memory — persistent, growing, bursty, latency-tolerant. The canonical fit.
• Batch evaluation corpora — embeddings you query in batches, not in real time.
• Cold tier of a tiered design — the durable, cheap floor under an OpenSearch hot tier.

My own case is the RAG long tail. I put my blog — 70-plus posts — into a Bedrock Knowledge Base on S3 Vectors. Small corpus, queried when I am writing and want to know what I already said, entirely latency-tolerant. A retrieval returns the relevant posts with strong semantic scores:

query: "do agent memories fade while skills persist"
  0.825  memories-fade-skills-persist.md
  0.753  skills-are-git-native-distribution.md
  0.725  agent-sprawl-is-a-skills-problem.md
  0.714  what-is-a-skill.md

I reach that knowledge base from my agents through one tool on a gateway — a pattern worth its own post, How AgentCore Gateway Turns Any API Into an Agent Tool. The point here is the storage decision underneath it: this workload is cold-tier, and the cold tier no longer costs $700 to enter.

The Three Limits

S3 Vectors buys its cost advantage with tradeoffs, and a design that ignores them will disappoint.

Semantic-only, no hybrid search. AWS documents that S3 Vectors does not support hybrid search — the combination of vector similarity with keyword matching. For content full of exact tokens like part numbers or version strings, pure semantic recall scores poorly, and OpenSearch with lexical scoring remains the right tier. My prose has no such tokens, so the limit does not bite, but a corpus of technical specs would feel it on the first query.

Latency is a floor, not a target. Warm queries land around 100 milliseconds, cold ones sub-second — fine inside an agent loop, disqualifying for an interactive search box. When part of the workload needs speed, the answer is not to abandon S3 Vectors but to tier: keep the bulk cold, promote the hot subset to OpenSearch, which AWS supports importing into directly.

Metadata is capped. S3 Vectors limits custom metadata per vector, which constrains how much filtering you can push into the store. For rich per-document filtering at scale, that cap is a real design input, not a footnote. I take apart exactly what that cap does to ingestion and filtering in What Actually Gets Stored When You Put a Vector in S3 Vectors.

So What

Stop asking which vector database is best. Start asking which tier your workload sits on, because the answer is usually obvious once you look at the shape of the queries. Customer-facing and latency-critical goes hot. Sitting next to relational data goes warm. Large, growing, bursty, and patient — which describes most internal RAG and nearly all agent memory — goes cold.

The reason this matters now is that the cold tier stopped being expensive. The $700 floor was quietly deciding architectures — pushing people toward one always-on database because standing up anything felt like the same fixed cost. Remove the floor and the honest answer for most agent and internal workloads is the cheap tier, running at pennies, tolerating a latency the agent never notices. The premium tier is still there when you need speed. Most of the time, you do not.