TurboQuant brings ultra-low-bit compression to vector search

Vector databases face a growing memory problem: storing embeddings as float32 becomes expensive as collections expand. A new Hugging Face Blog community article examines TurboQuant, a compression method designed to reduce that cost without sacrificing as much nearest-neighbor accuracy.

For a collection of one million 1,024-dimensional vectors, float32 storage requires about 4.096 GB before accounting for metadata or the HNSW index. TurboQuant reduces the vector codes to roughly 0.512 GB at four bits per coordinate—an 8× reduction. Its available formats include TQ4, TQ2, TQ1.5 and TQ1; a 1,024-dimensional vector requires 512 bytes in TQ4, 256 bytes in TQ2 and 128 bytes in TQ1, compared with 4,096 bytes in float32.

Why the approach matters

Rather than quantizing each embedding in its original coordinate system, TurboQuant first applies a random orthogonal rotation. That preserves distances and dot products while spreading values more evenly across coordinates. Values are then mapped to a fixed Lloyd-Max codebook, avoiding per-dataset codebook training.

The method is now available as an optional quantization layer in Qdrant 1.18. Qdrant can search compressed vectors, rescore candidates with original vectors, retain vector norms and use integer-based SIMD scoring. Those additions aim to improve distance estimates while keeping comparisons fast.

The article positions TQ4 as an option when scalar quantization’s 4× savings are insufficient, and TQ1 or TQ2 for tighter memory budgets where binary quantization’s potential 32× reduction may hurt recall. Developers should still benchmark their own embeddings, queries, HNSW configuration and rescoring strategy before choosing a format.

Source: Hugging Face Blog