Vector Quantization

Vector quantization maps a numerical vector to a representative vector from a finite collection called a codebook. Instead of storing every original coordinate, a system stores a compact code identifying the representative. This is lossy compression: the reconstructed vector generally differs from the original. For embedding search, the practical objective is to reduce the cost of storing and comparing vectors while preserving enough information to retrieve relevant neighbors.

A single codebook covering an entire high-dimensional vector can be impractical at useful accuracy levels. Product quantization addresses this by dividing vectors into subvectors and encoding each subvector with its own codebook. For example, a 768-dimensional embedding stored as 32-bit floating-point numbers requires 3,072 bytes of raw vector data. Dividing it into 96 subvectors and using 256 representatives per subvector allows each representative identifier to fit in one byte, producing a 96-byte code. That is a theoretical 32-fold reduction in the vector payload, not a guarantee of a 32-fold reduction in database size or cloud charges.

Real capacity planning must include codebooks, document metadata, identifiers, graph links or other index structures, replicas, and any retained original vectors. Ten million vectors in the example require approximately 30.72 GB of raw float32 data versus 0.96 GB of product-quantized codes, using decimal units. However, an implementation that retains full-precision vectors on disk for rescoring still pays for that storage. Its main benefit might be a smaller in-memory search index rather than equivalent savings across every storage tier.

In a cloud deployment, engineers typically generate embeddings, select a supported index and quantization method, train codebooks where required, and build the compressed index. Training data should represent the production embedding distribution. Changing the embedding model or substantially changing the document population can require retraining codebooks and rebuilding the index. Quantization does not inherently reduce the number of embedding-model calls or the tokens processed to create embeddings.

The central trade-off is retrieval quality against resource consumption. Quantization error can change distance estimates and reorder neighbors. A common mitigation retrieves a larger candidate set using compressed vectors, then rescores those candidates using original vectors. This can recover accuracy, but increases computation, storage reads, and sometimes tail latency. Rescoring cannot recover a relevant document that never entered the candidate set.

Benchmark a representative query set against an uncompressed baseline. Track recall at k, application-level relevance, p95 and p99 latency, throughput, index size, resident memory, index-build duration, and cost per query. Compression can improve throughput when memory bandwidth or capacity is the bottleneck, but encoding, distance estimation, and rescoring introduce work, so lower latency is not automatic. Confirm that reduced resource consumption permits smaller instances, fewer nodes, or lower metered usage before counting it as a billing reduction.

A common misconception is that all embedding compression methods are identical. Scalar quantization reduces precision coordinate by coordinate, while classical vector quantization encodes groups of coordinates jointly; product quantization is a structured form of vector quantization. Dimensionality reduction instead changes the number of coordinates. These techniques can sometimes be combined, but their accuracy effects must be evaluated together. For AI cost questions, distinguish vector-index optimization from prompt-token optimization and embedding-generation optimization: compressing stored embeddings does not directly shorten an LLM prompt or eliminate repeated embedding computation.