vector database

At a billion vectors the index choice is a memory budget problem before it's a recall problem. Flat search is out, HNSW may not fit, and PQ trades recall for RAM. Here is the decision a staff engineer makes on the whiteboard.

When you shard a vector index or route requests across model replicas, naive 'hash mod N' reshuffles everything when N changes. Consistent hashing moves only a small fraction. Here is the implementation.

The service behind every RAG stack and semantic search box. The signal is your ANN index choice and how you handle the three things that quietly break it: metadata filtering, live updates, and scale.

Offering RAG as a product means handling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. Learn the ingestion and query planes, how to keep tenants isolated, and how to give customers eval and observability.