Vector database

A vector database stores data as embeddings, lists of numbers that capture the meaning of a piece of text, an image or another item, and finds the stored items closest in meaning to a query. It’s what lets an AI system find the paragraph about “terminating the contract early” when you searched for “how do we get out of this deal.”

Why it matters

Vector databases are the usual storage layer behind retrieval-augmented generation (RAG). Documents are split into chunks, each chunk is turned into an embedding, and the embeddings go into the database. When someone asks a question, the question is embedded too, the database returns the nearest chunks, and those chunks are handed to the model as context. That’s how a model answers questions about documents it was never trained on, without loading all of them into its context window.

How it works, briefly

Two pieces of text with similar meaning end up with embeddings that sit close together in a space with hundreds or thousands of dimensions. The database’s job is to find the nearest neighbors to a query quickly, even across millions of items. Most use approximate search, which trades a little accuracy for a lot of speed.

Dedicated vector databases exist, and many general databases, including Postgres with an extension, now store and search vectors too. For a lot of teams, the database they already run is enough.

Where it falls short

Vector search finds text that sounds similar, which is not always text that answers the question. It struggles with exact matches like product codes, names and numbers, so most production systems combine it with keyword search. It also loses structure: it can find a paragraph about a customer, but it can’t tell you how that customer relates to an open deal and a support ticket. Chunking choices and permissions matter as much as the database itself. A search index that ignores who is allowed to see what will happily retrieve documents a user shouldn’t read.

Bring your agent. Bring your team.