AI 101 — What is a vector database?

A vector database is a database purpose-built to store and search "embeddings" — long lists of numbers that capture the meaning of a piece of text, an image, or audio — so it can find things that are semantically similar rather than things that merely match your keywords.

If that sounds abstract, consider where you have already met one. Every "chat with your documents" feature works because a system converts your question into numbers, compares those numbers against millions of stored ones, and pulls back the closest matches as context for the answer. The same trick powers recommendation feeds, duplicate detection, image search, and the long-term memory that lets an agent recall what it did last week. Our explainer on What is RAG? describes how retrieval grounds a model's answers; the vector database is the machinery doing the finding.

Why it matters right now

Retrieval has quietly become load-bearing infrastructure for AI. When a company points a chatbot at its internal documentation, the quality of the whole system rests on whether relevant paragraphs actually come back from the search. Get that wrong and even a frontier model will confidently answer from the wrong context — or hallucinate around the gap.

That is why nearly every serious RAG pipeline runs on a vector database, and why the category exploded: dedicated products like Pinecone, Weaviate, Qdrant, Milvus, and Chroma compete for the workload, while cloud giants bolted vector search onto existing offerings such as Elasticsearch and Google Cloud's Vertex AI services. Even PostgreSQL, the world's favorite general-purpose relational database, gained a popular open-source extension called pgvector that stores and searches embeddings directly. The pattern repeats across the industry: if software holds knowledge, someone is adding vector search to it.

The mental model

An embedding places a piece of content at coordinates in "meaning space." A sentence about laptops sits near other laptop talk, regardless of exact wording; a photo of a retriever lands near photos of other dogs. Similar meaning means nearby points — that single property is what makes semantic search possible.

A vector database does two jobs with those coordinates. First it indexes millions or billions of them so that "which stored vectors sit closest to this query?" returns in milliseconds instead of hours. Brute-force comparison dies at scale, so these databases use approximate-nearest-neighbor indexes such as HNSW — a multi-layer graph, published by researchers Yury Malkov and Dmitry Yashunin, that hops across the space like a courier using highways then side streets. Second, it behaves like a real database: rows can be inserted, updated, deleted, and filtered by ordinary metadata — "search only contracts from this year" — alongside the similarity math. Many also blend vector results with classic keyword matching, a combination called hybrid search, because exact terms still matter for names, part numbers, and error codes.

The library analogy

Keyword search is a card catalog: find the drawer labeled exactly "VOLTAIRE", miss everything filed under the author's pen name. A vector database is a librarian who has read every book and understands your intent — ask for "books about questioning authority through wit" and she hands you the right shelf even though no title contains those words.

The approximate part is honest too: our librarian occasionally skips a book she should have handed over. Well-tuned systems get the important ones right far more often than they miss, which is the trade that makes searching billions of items instant.

Common misconceptions

"It's just a place to store embeddings." Storage is the easy half. The value lies in the ANN index structures, metadata filtering, and hybrid ranking — the engineering that turns similarity math into sub-second search at scale.

"I need a new database to use vectors." Not always. pgvector inside PostgreSQL handles many workloads comfortably; dedicated vector databases earn their keep at large scale or under demanding latency requirements. Match the tool to the job.

"Approximate means unreliable." The approximation trades a sliver of recall for orders-of-magnitude speed, and quality is tunable. It is the same bargain web search made decades ago.

"Adding one fixes hallucinations." Retrieval only helps when good documents exist and the search returns them; poor data in, poor answers out. It narrows the problem, it does not eliminate it.

Where to learn more

Start with Pinecone's and Elastic's introductory guides, then read the original HNSW paper if the indexing ideas hook you. To place this concept in the bigger picture, pair it with our explainers on What is RAG? and What is a context window? — retrieval decides what fits into that window, and both determine how much a model really knows when it answers.

Related reading: What is RAG? shows the pipeline this database powers, and What is a context window? explains where retrieved documents end up once found.

If your team added vector search this year, did it actually improve answers — or just add infrastructure? Tell us in the comments.

Sources: Pinecone — What is a Vector Database? · Elastic — What is vector search? · Malkov & Yashunin — HNSW paper (arXiv) · pgvector (GitHub)