Storage Amplification in Vector Search
The Problem: The "Storage Amplification" Tax of Vector Search
When you build applications powered by LLMs - such as codebase search in Cursor or workspace Q&A in Notion, you must convert unstructured text, code, or images into high-dimensional vector embeddings. This introduces a massive data scaling bottleneck known as storage space amplification.
-
The Size Explosion: A paragraph of text might only take about 1 kilobyte of space. However, once processed by an embedding model, it is converted into a high-dimensional vector (such as a 1536-dimensional array of floating-point numbers). This resulting vector occupies around 6 kilobytes of space.
-
Chunking Amplification: To get accurate search results, you cannot embed a whole document at once; you have to chop it up into smaller chunks (like paragraphs or sentences). A simple 1 kilobyte article split into four chunks suddenly requires 24 kilobytes of vector data - a 24x space amplification.
-
The Economic Problem: High-dimensional vectors are highly resistant to standard compression techniques because almost every floating-point number carries unique semantic information. Traditionally, databases were forced to store these vectors entirely in DRAM (memory) to search them quickly. DRAM costs roughly $2 per gigabyte, compared to cloud object storage (S3) which costs 2 cents per gigabyte.
Storing and searching billions of vectors in memory quickly becomes economically impossible for most companies. Thus, finding a way to compress vectors so they can be stored cheaply and searched efficiently is critical.
Quantization Techniques
To solve the problem of compressing vectors, various techniques were invented. Some of the common techniques are:
Related reads
- RaBitQ — unbiased error bounds with ultra-compact bit strings
- Product Quantization (PQ) — compression via subspace clustering
- DRAM — why the most expensive tier forces the compression