Other index types
HNSW and IVF cover most needs, but you'll meet other index types in documentation and benchmarks. Knowing what each is for helps you read a database's feature list with confidence.
Flat (brute force)
No index structure at all: compare against every vector. Exact, simple, and fast enough for small collections — or for a small subset left after a strict filter. Also the ground truth for measuring other indexes.
DiskANN (Vamana graphs)
A graph index designed to live mostly on SSD instead of RAM. It keeps a compressed version of each vector in memory to guide the search, and fetches full vectors from disk only for the final candidates. The graph is built so that searches need few disk reads.
When it shines: very large collections where keeping everything in RAM would be too expensive, and fast SSDs are available.
Locality-sensitive hashing (LSH)
Uses special hash functions designed so that similar vectors tend to land in the same bucket. Search hashes the query and looks only in matching buckets.
When it shines: some streaming and very high-throughput settings, and as a conceptual building block. For typical embedding search, graph and IVF indexes usually give better recall for the same cost, so LSH is less common in modern vector databases.
Tree-based indexes
Split the space recursively into regions, like a decision tree. Classic KD-trees work well in low dimensions but degrade badly as dimensions grow. Annoy, a library created at Spotify, uses forests of random projection trees and memory-mapped files — handy for read-heavy, rarely updated datasets.
GPU indexes
GPUs can run brute-force and IVF-style searches massively in parallel. Libraries such as FAISS offer GPU implementations, and some databases use GPUs to accelerate index building or high-throughput batch queries.
When it shines: very high query volumes, large batch jobs (for example, deduplicating a whole dataset), and fast index builds.
Choosing an index
| Situation | Reasonable choice |
|---|---|
| Fewer than ~100k vectors | Flat |
| General-purpose, fits in RAM | HNSW |
| HNSW but memory is tight | HNSW + scalar quantization |
| Hundreds of millions to billions, RAM-constrained | IVF-PQ, or DiskANN on SSD |
| Read-mostly, rarely updated, simple deployment | Annoy-style memory-mapped trees, or HNSW |
| Massive batch similarity jobs | GPU flat or IVF |
| Strict filters leave a small subset | Flat search over the filtered subset |
Most of the time you won't choose an index algorithm directly — you'll choose a database, and it will offer one or two well-tuned options. Knowing the families lets you understand the trade-offs it has made for you.
Benchmarks such as ANN-Benchmarks compare many algorithms on public datasets as speed-versus-recall curves. They're useful for intuition, but always re-test on your own data and filters.
Senior Full-Stack Engineer · AI & AWS. I build production search, RAG and AI systems on AWS and Postgres.