What Is a Vector Database? How Vector Databases Work, Benefits, Use Cases, and Examples

A vector database stores data as embeddings — lists of numbers that capture meaning — and retrieves records by similarity rather than exact match. Given a query vector, it returns the nearest vectors in the index. That is how a search for "cancel my subscription" finds a document titled "ending your plan".

Key takeaways

• Vector databases answer "what is closest?", not "what is equal?".
• They rely on approximate nearest neighbour (ANN) indexes — HNSW and IVF are the two you will meet — trading a little recall for a large speed gain.
• They are the retrieval half of retrieval-augmented generation.
• Memory, not CPU, is the usual cost driver. Quantisation is the first lever to pull.
• Hybrid search — keyword plus vector — beats pure vector search on identifiers, codes and rare proper nouns.

What is an embedding, and why does it matter?

An embedding is a fixed-length list of numbers produced by a model that represents the meaning of a piece of text, an image, or audio. Text with similar meaning produces vectors that sit close together in that space, even when the two pieces of text share no words.

Dimensions vary by model: OpenAI text-embedding-3-small outputs 1536 numbers per input and text-embedding-3-large outputs 3072, while the open-source all-MiniLM-L6-v2 outputs 384. Higher dimensions usually mean better nuance and more memory. Two vectors from different models are never comparable, so changing the embedding model means re-indexing everything.

How does a vector database work?

1. Chunk the source data. Long documents are split into passages, because one embedding for a 40-page PDF represents nothing usefully.
2. Embed each chunk. An embedding model converts every chunk into a vector, stored alongside its text and metadata — source, author, date, tenant, permissions.
3. Build the index. The database organises vectors into an ANN structure so queries do not compare against every record.
4. Embed the query. At search time the user's question goes through the same model, producing a vector in the same space.
5. Search and filter. The engine walks the index for the nearest k vectors, applying metadata filters so results respect tenancy and permissions.
6. Return payloads. You get back the original text and metadata plus a similarity score — ready to feed a model or render as results.

Where filtering happens matters. Pre-filtering narrows candidates before the ANN walk and guarantees correctness but can be slow; post-filtering searches first and discards afterwards, which is fast but can return fewer than k results. Mature engines implement filtered search inside the graph traversal to get both.

Indexing algorithms: HNSW, IVF and quantisation

• Flat (brute force). Compare the query with every vector. Exact, trivially correct, and fine up to roughly 100,000 vectors. Use it as your accuracy baseline.
• HNSW. A multi-layer proximity graph. Search starts on a sparse top layer and descends, hopping toward closer neighbours. Excellent recall and latency; the memory cost is the graph itself. Tune M and efConstruction at build time, efSearch at query time.
• IVF. Cluster vectors, then search only the nearest few clusters. Lower memory than HNSW, with recall governed by how many clusters you probe.
• Product quantisation (PQ) and IVF-PQ. Compress vectors into short codes. Dramatic memory savings, some accuracy loss — the standard approach at billion scale.
• Scalar and binary quantisation. Store each dimension as an int8 or a single bit. Binary quantisation can cut memory around 32x and is often paired with a rescoring pass over full-precision vectors.
• DiskANN. Keeps most of the index on SSD rather than RAM, trading some latency for far cheaper capacity.

Every ANN index is a recall-versus-latency dial. Measure recall@k against a flat index on a sample before you tune anything — a system that is fast and wrong is worse than the keyword search it replaced.

Distance metrics: cosine, dot product and Euclidean

• Cosine similarity. Measures angle, ignoring magnitude. The default for text embeddings.
• Dot product. Accounts for magnitude as well as direction. Identical to cosine when vectors are normalised, and cheaper to compute.
• Euclidean (L2) distance. Straight-line distance. Common for image and audio embeddings.

Use whichever metric your embedding model was trained with. Mismatching it is a silent accuracy bug — nothing errors, the results are just quietly worse.

Vector database vs relational database vs keyword search

These are complements, not rivals. Production retrieval usually runs BM25 and vector search together and fuses the two ranked lists with reciprocal rank fusion.

What are the benefits of a vector database?

• Semantic recall. Finds the right passage even when the user's wording shares nothing with the document.
• Sub-100ms search at scale. ANN indexes keep latency roughly logarithmic as the collection grows.
• Any modality. Text, images, audio and code all embed into the same kind of structure, enabling search-by-example.
• Metadata filtering built in. Tenancy, permissions, recency and language constraints apply inside the search rather than after it.
• Live updates. Add or delete a document and it is searchable immediately — no retraining, unlike fine-tuning.
• Grounding for models. It gives a language model verifiable source passages instead of relying on memorised parameters.

Use cases and real-world examples

• RAG chatbots over internal knowledge. Policies, runbooks and manuals retrieved per question — the most common deployment by far.
• Product and content recommendation. "More like this" computed from item embeddings rather than hand-built rules.
• Image and video search. Retrieve by visual similarity or by a text description of the picture.
• Duplicate and near-duplicate detection. Support tickets, CRM records, job listings and fraud signals that are worded differently but mean the same thing.
• Code search. Find the function that does a thing, without knowing what it was named.
• Agent long-term memory. Store past interactions as vectors and retrieve the relevant ones, typically exposed to the agent through an MCP server.
• Anomaly detection. A vector far from every cluster is a candidate outlier.

Popular vector databases compared

• pgvector. A PostgreSQL extension. Vectors sit beside your relational data with real transactions and joins. The correct default if you already run Postgres.
• Qdrant. Open source, written in Rust, strong filtered search and quantisation support. Good self-hosted choice.
• Milvus. Open source and built for very large, distributed collections with multiple index types.
• Weaviate. Open source with built-in vectorisation modules and hybrid search out of the box.
• Pinecone. Fully managed and serverless. Least operational work, at the price of a vendor dependency.
• Chroma. Lightweight and developer-friendly, ideal for prototypes and local development.
• Elasticsearch / OpenSearch. Mature keyword search with kNN added — attractive when you already operate a cluster and want hybrid search in one place.
• FAISS. Not a database but the reference similarity search library from Meta AI. Embed it when you want the index without the server.

Limitations and challenges

• Memory cost. One million 1536-dimension float32 vectors is roughly 6 GB before index overhead. Budget for RAM, then quantise.
• Re-embedding on model change. Switching embedding models invalidates the entire index. Plan for a dual-write migration.
• Weak on exact identifiers. SKUs, error codes and version numbers are keyword problems; use hybrid search.
• Chunking decides quality. Bad chunk boundaries cap retrieval accuracy no matter how good the index is.
• Approximate means approximate. ANN can miss a relevant result. Measure recall@k rather than assuming it.
• Security is on you. Embeddings of sensitive text are sensitive. Enforce per-tenant filters at query time, not in application code afterwards.

How to choose and get started

1. Count your vectors honestly. Under a few million and already on Postgres? Use pgvector and stop shopping.
2. Write the filter requirements down first. Tenancy, permissions and recency filters separate the engines far more than raw speed does.
3. Pick an embedding model and freeze it for the evaluation, so you are comparing databases and not models.
4. Build a flat-index baseline on a sample and record recall@10. That is your ground truth.
5. Tune HNSW against that baseline until recall is acceptable, then measure p95 latency at realistic concurrency.
6. Add hybrid search before you add complexity. BM25 plus vectors with rank fusion fixes more failures than any index tuning.
7. Then quantise and re-measure. Binary or scalar quantisation with rescoring usually keeps recall within a point or two at a fraction of the memory.

Frequently Asked Questions (FAQs)

What is a vector database?

A vector database stores data as embeddings - lists of numbers that capture meaning - and retrieves records by similarity rather than exact match. Given a query vector it returns the nearest vectors in the index, which is how systems find content that means the same thing in different words.

How is a vector database different from a relational database?

A relational database answers questions about exact values: rows where status equals shipped. A vector database answers questions about closeness: the twenty documents most similar in meaning to this question. One uses B-tree indexes on scalar columns, the other uses approximate nearest neighbour indexes over high-dimensional vectors.

What is an embedding?

An embedding is a fixed-length list of numbers produced by a model that represents the meaning of text, an image or audio. Similar content produces vectors that sit close together. OpenAI text-embedding-3-small outputs 1536 numbers per input; all-MiniLM-L6-v2 outputs 384.

What is HNSW and why do vector databases use it?

HNSW, Hierarchical Navigable Small World, is a graph index that connects each vector to its neighbours across several layers. A search hops greedily from a sparse top layer down to a dense bottom one, reaching the nearest neighbours in logarithmic time instead of comparing against every vector.

Do I need a dedicated vector database or is pgvector enough?

If you already run PostgreSQL and hold under a few million vectors, pgvector is usually enough and keeps everything in one system with real transactions. Dedicated engines such as Qdrant, Milvus or Pinecone earn their place at tens of millions of vectors, heavy filtering, or strict latency targets.

What is hybrid search?

Hybrid search runs a keyword search such as BM25 and a vector search together, then merges the two ranked lists, commonly with reciprocal rank fusion. It fixes the main weakness of pure vector search, which is matching exact identifiers, product codes and rare proper nouns.

How much does a vector database cost to run?

Cost is driven by memory, because most indexes live in RAM. One million 1536-dimension float32 vectors is roughly 6 GB before index overhead. Scalar or binary quantisation cuts that by four to thirty-two times with a modest recall loss, which is usually the first optimisation worth making.

Conclusion

A vector database is the storage layer that makes meaning searchable. It does not replace your relational database or your keyword index — it answers a different question, and the best systems run all three together. The engineering decisions that actually determine quality are unglamorous: how you chunk the source material, which embedding model you commit to, whether filters run inside the search, and whether you ever measured recall against an exact baseline. Get those right and almost any modern engine will serve you well; get them wrong and no amount of index tuning will save the results.

Building semantic search or RAG on your own data?

DI Solutions designs the chunking, embedding and retrieval pipeline, benchmarks recall properly, and ships it with tenancy and permissions enforced at query time — hire our AI engineering team.

Reference links

• HNSW paper (Malkov & Yashunin)
• pgvector
• FAISS by Meta AI
• Qdrant documentation