Vector Databases at Billion Scale

4.1 The workload

This chapter is different from the previous three. Those built concepts; this one builds a system. Everything here is anchored to one concrete workload, and every number is derived rather than asserted.

Why this scale changes the engineering

A million vectors is a laptop exercise. Any library works, defaults are fine, and mistakes are cheap because rebuilding takes minutes. At 1.3 billion, four things become true at once:

• Memory is the binding constraint, not CPU. The decisive question is how many bytes each vector occupies, because that sets your machine count and therefore most of your bill.
• Full rebuilds stop being routine. An operation taking 40 seconds per million takes about 14 hours at this scale. "Just re-index" ceases to be a casual suggestion and becomes a planned migration.
• Defaults are actively wrong. Qdrant's out-of-the-box settings target datasets thousands of times smaller. Shipping them here produces an expensive, slow cluster.
• Architecture decisions become expensive to reverse. The collection layout in section 4.10 is the clearest example: choosing wrong means re-ingesting everything later.

4.2 Why not a normal database

A reasonable first question: you already run Postgres. Why add another system?

Because relational indexes answer a fundamentally different question. A B-tree index makes equality and range queries fast: find the row where id = 5, or price between 20 and 40. It works by keeping values in sorted order, which requires that "sorted order" exists.

What brute force actually costs here

Worth computing, because it establishes why approximation is mandatory rather than merely convenient.

comparisons        800,000,000
multiply-adds      800,000,000 x 768  =  614 billion operations
memory read        800,000,000 x 3 KB =  2.46 TB streamed per query

Even at a very optimistic 50 GB/s of memory bandwidth, reading the data
alone takes ~49 seconds. Per query. For one user.

That is the entire justification for this chapter. Exact search does not scale, so we trade a small amount of accuracy for several orders of magnitude of speed.

4.3 Exact versus approximate search

4.4 Index families, and why Qdrant uses HNSW

Every ANN index makes the same bargain: examine a small fraction of the vectors, and accept that you will occasionally miss a true neighbour. What separates the families is how they decide which fraction to examine — and each strategy fails in its own characteristic way. Knowing all of them matters, because the reason HNSW wins for this workload is also the reason it is expensive.

Partition indexes: IVF

The oldest and most intuitive approach. Run k-means over a sample of the data to find cluster centroids, assign each vector to its nearest one, and store an inverted list per cluster. At query time, compare against the centroids — a few thousand comparisons — then scan only the vectors inside the closest nprobe clusters.

Hashing: LSH

LSH takes a different angle: rather than partitioning the space by proximity to learned centroids, it uses random projections. Generate a set of random hyperplanes through the origin, and record which side of each plane a vector falls on. Vectors pointing in similar directions almost always agree on most of those bits, so a vector's bit string works as a bucket key.

Graph indexes: HNSW

Graph indexes discard partitioning entirely. Instead, every vector stores links to some of its nearest neighbours, and search becomes navigation: start somewhere, repeatedly step to whichever neighbour is closer to the query, and stop when no neighbour improves. HNSW adds a hierarchy on top of that idea.

How the graph is searched

The idea borrows from how you would navigate a country without a map: take long-haul flights to get near the right region, then progressively shorter hops.

Compression: product quantization

PQ is not an index. It is a way of making vectors smaller so that whichever index you chose can hold more of them, and it appears here because "which index family" and "how are the vectors stored" are decisions people routinely conflate in interviews.

Breaking the memory ceiling: DiskANN

Every option so far assumes the index lives in RAM. At 1.3 billion vectors that assumption is what drives the entire capacity plan in 4.9 — and DiskANN is the family that rejects it.

Choosing between them

4.5 The HNSW parameters that matter

Qdrant exposes a handful of numbers that govern the recall, speed and memory of your index. Getting these right is most of the tuning work, and the defaults are wrong at our scale. Before the table of what to set, it is worth understanding what the graph builder actually does with each one — because every parameter here is a direct lever on a specific step of the construction or search algorithm, and knowing which step is what turns tuning from guesswork into reasoning.

What Qdrant does when you insert a vector

Qdrant builds the graph incrementally. Every point that arrives goes through the same five steps, and m and ef_construct are the two dials on that procedure.

The neighbour selection heuristic

Step 4 is where graph quality is won or lost, and it is the part most descriptions of HNSW skip. The obvious implementation — keep the m closest candidates — produces a measurably worse graph than the one Qdrant builds.

What Qdrant does when you search

Search reuses the same machinery in reverse, and ef is its only dial. The graph is fixed by this point; nothing you pass at query time changes it.

full_scan_threshold: the parameter whose units are a trap

This one is different in kind from the other three. It does not tune the graph at all — it decides whether the graph is used in the first place. Below the threshold, Qdrant abandons HNSW and brute-forces the candidate set, which is both faster and exact for small enough sets.

The parameters, side by side

Recommended values for this workload

4.6 Sizing from first principles

This is the section to internalise. Everything about cost, machine count and feasibility follows from one formula, and it is simple enough to do on a whiteboard in an interview.

bytes_per_vector  =  (dimensions x bytes_per_dimension)   # the vector
                  +  (2 x m x 4 x 1.08)                   # the HNSW graph

total_RAM         =  bytes_per_vector x vector_count

The second term deserves explanation because it is the part people forget. Layer 0 of the graph stores up to 2 x m neighbour identifiers per vector — twice m, because the base layer is built with doubled connectivity. Each identifier is a 4-byte integer. The upper layers add roughly 8% on top. With m=16 that is 138 bytes per vector, and critically this cost does not shrink when you quantize. The graph is made of pointers, not of vector data.

The unquantized baseline

Start with float32, which is what you get if you change nothing.

TITLES   768 dims x 4 bytes  = 3,072 B  + 138 B graph = 3,210 B/vector
         x 800,000,000                               = 2,568 GB

IMAGES  1024 dims x 4 bytes  = 4,096 B  + 138 B graph = 4,234 B/vector
         x 500,000,000                               = 2,117 GB

TOTAL                                                 = 4,685 GB  (~4.7 TB)

Nearly five terabytes of RAM. On 256 GB nodes that is 19 machines holding one copy of the data, before replication, before payload storage, before any headroom. Add a single replica for availability and you are at 38.

4.7 Quantization

A float32 number uses 4 bytes to store roughly 7 decimal digits of precision. Embedding values are almost all small numbers between about -1 and 1, and nearest-neighbour ranking only needs to know which vectors are closer, not their exact distances. That precision is largely wasted.

Rescoring: why binary is not as lossy as it looks

Binary quantization keeps one bit per dimension — effectively just the sign. That sounds catastrophic, and on its own it is mediocre. The trick is a two-stage search.

1. Search the binary index, but ask for more results than you need Request the top 100 when you want 10. This is oversampling. Binary distance is computed with bitwise XOR and a popcount instruction, which is extraordinarily fast — often 10 to 40 times faster than float comparison.
2. Fetch the full-precision vectors for those 100 candidates Read them from disk. This is 100 random reads, which on NVMe is a few milliseconds and is the reason this technique needs fast storage.
3. Re-rank the 100 by exact distance and return the top 10 The expensive precise comparison now runs against 100 vectors rather than 800 million.

TurboQuant and the turbo4 datatype

The three modes above were the whole picture until recently. Qdrant has since added TurboQuant, and it changes the default advice, so it is worth understanding what it does differently rather than treating it as a fourth row in the table.

What you actually have to deploy

A reasonable worry at this point, especially for anyone who has operated product quantization: if TurboQuant is a Google method with a rotation and a codebook, where does that codebook come from? Is there a model to install next to Qdrant, or a training job to run over the catalogue before anything can be written? Neither. There is nothing to install and nothing to fit.

This is the substantive operational difference between TurboQuant and product quantization, and it is easy to miss because both are described as “codebook” methods. PQ's codebook is learned from your vectors: at the 48-slice configuration from 4.4, it is roughly 200,000 floats derived by k-means on a sample of your catalogue. TurboQuant's is sixteen numbers that are the same for every collection in the world.

4.8 Matryoshka truncation

Quantization reduces the bits per dimension. Matryoshka representation learning reduces the number of dimensions, and it applies to your text embeddings specifically.

4.9 The full capacity plan

Combining sections 4.6 to 4.8, here is the recommended configuration and the reasoning behind each choice.

TITLES  256 dims / 8 = 32 B binary  + 138 B graph = 170 B/vector
        x 800,000,000                             = 136 GB

IMAGES  1024 dims / 8 = 128 B binary + 138 B graph = 266 B/vector
        x 500,000,000                              = 133 GB

TOTAL RAM                                          = 269 GB
NVMe (float32 originals for rescoring)             = 4.5 TB

index + graph               269 GB
payload (~200 B/vector)     260 GB   (disk-backed, partially cached)
                          -------
working set                 529 GB
x2 replication            1,058 GB
/0.70 headroom            1,512 GB

=> 6 nodes at 256 GB RAM, each with ~2 TB NVMe

4.10 Four markets: the core architecture decision

You have Canada, the United States, Mexico and Chile. Do you build four collections, or one collection with a market field in the payload? This is the decision that is most expensive to reverse, so it deserves the most careful thought.

There is a third option most teams do not know about, and it is usually the right one. Take the two obvious options first.

The problem with Option B that is not obvious

Filtering in a vector database is not like filtering in SQL. In Postgres, a WHERE market = 'MX' narrows the search and makes it faster. In an HNSW index, filtering can make it dramatically slower, and the reason is worth understanding properly.

How to choose, concretely

Creating the collection

The configuration that implements everything decided so far: two collections, market as a tenant key, binary quantization with originals on disk for rescoring.

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://qdrant:6333", timeout=120)

# --- Titles: 800M vectors, MRL-truncated 768 -> 256 -------------------
client.create_collection(
    collection_name="product_titles",
    vectors_config=models.VectorParams(
        size=256,
        distance=models.Distance.COSINE,
        # Keep float32 originals on disk. RAM holds only the binary
        # codes; these are read back for rescoring the candidates.
        on_disk=True,
    ),
    quantization_config=models.BinaryQuantization(
        binary=models.BinaryQuantizationConfig(always_ram=True),
    ),
    hnsw_config=models.HnswConfigDiff(
        m=16,
        ef_construct=200,
        # Also build per-tenant graph links, so a market-filtered
        # search traverses mostly-relevant neighbours. The global
        # graph stays (m=16): with only four markets, unscoped
        # queries still need a path. See 4.19 for when to drop it.
        payload_m=16,
    ),
    optimizers_config=models.OptimizersConfigDiff(
        # 1-2 GB segments. Too small = many files; too large = slow merges.
        max_segment_size=2_000_000,
        default_segment_number=0,
        memmap_threshold=100_000,
    ),
    shard_number=12,
    replication_factor=2,
    on_disk_payload=True,
)

# --- The tenant index. This is the important line. -------------------
#
# is_tenant=True tells Qdrant to physically co-locate points that
# share a market value, so a filtered search walks a graph that is
# mostly relevant instead of discarding 90% of what it visits.
# Without this, single-collection multitenancy performs badly.
client.create_payload_index(
    collection_name="product_titles",
    field_name="market",
    field_schema=models.KeywordIndexParams(
        type="keyword",
        is_tenant=True,
    ),
)

# Ordinary filter indexes for the other fields we query on.
for field, schema in [
    ("category_id", "keyword"),
    ("in_stock", "bool"),
    ("price_cents", "integer"),
]:
    client.create_payload_index(
        collection_name="product_titles",
        field_name=field,
        field_schema=schema,
    )

# --- Images: identical shape, but 1024 dims and no truncation --------
client.create_collection(
    collection_name="product_images",
    vectors_config=models.VectorParams(
        size=1024,          # DINOv3 has no MRL: cannot truncate
        distance=models.Distance.COSINE,
        on_disk=True,
    ),
    quantization_config=models.BinaryQuantization(
        binary=models.BinaryQuantizationConfig(always_ram=True),
    ),
    hnsw_config=models.HnswConfigDiff(
        m=16, ef_construct=256, payload_m=16,
    ),
    shard_number=8,
    replication_factor=2,
    on_disk_payload=True,
)

So is the index global, or per market?

A question worth stopping on, because the answer decides whether cross-market search still works. If the tenant key makes Qdrant build a separate graph for each market, then a query that spans markets has four graphs to deal with and no obvious way to combine them. If instead the index stays global, it is not clear what the tenant key bought you. Neither answer is quite right.

That single design answers both questions at once. A market-filtered search follows the same-market edges and rarely wastes a hop, because at every node there is a neighbour that will survive the filter. An unfiltered, cross-market search ignores those extra edges entirely and walks the global ones, behaving exactly as it would in a collection with no tenant key at all.

4.11 Sharding and replication

A shard is a horizontal slice of a collection that lives on one node and holds its own independent HNSW graph. Sharding is how you spread 800 million vectors across machines.

Choosing a shard count

Shard keys, and the trap in using market as one

Qdrant lets you route points to specific shards by a payload value, called a shard key. It is tempting to shard by market. Resist it.

shard_us    480M vectors   <- overloaded, slow, the bottleneck
shard_mx    160M vectors
shard_ca    120M vectors
shard_cl     40M vectors   <- nearly idle, wasting a node

4.12 Metadata filtering

Section 4.10 introduced why filtering hurts graph search. This section covers what to do about it, because in a real storefront almost every query is filtered: in stock, ships to this region, within this category, above this rating.

The three filtering strategies

4.13 Two modalities, two collections

You have text vectors from EmbeddingGemma and image vectors from DINOv3. A recurring question is whether these can share a collection. They cannot, and the reason is worth stating precisely.

4.14 Bootstrapping 1.3 billion vectors

Initial load is the operation most likely to go badly, because the naive approach is both slow and produces a worse index than necessary.

The correct sequence

1. Create the collection with indexing disabled Set indexing_threshold=0. Qdrant now accepts points as plain storage without touching the graph. Ingest runs at full speed and stays at full speed.
2. Upload in parallel, in batches Many workers, each sending batches of a few hundred points. Details in section 4.15.
3. Verify the count before indexing Check that the point count matches your source. Discovering a gap after a 14-hour index build is a bad day.
4. Enable indexing and let it build once Set indexing_threshold back to 20000. Qdrant builds each segment's graph once, in parallel across segments and shards. One pass, correct result.
5. Wait for green, then measure recall Poll until collection status is green. Then run your recall benchmark before sending production traffic.

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://qdrant:6333", prefer_grpc=True)
COLLECTION = "product_titles"


def set_indexing(enabled: bool) -> None:
    """Toggle HNSW construction.

    threshold=0 means "never index", which is what makes bulk load fast.
    Restoring it to 20000 triggers a single clean build per segment.
    """
    client.update_collection(
        collection_name=COLLECTION,
        optimizer_config=models.OptimizersConfigDiff(
            indexing_threshold=20_000 if enabled else 0,
        ),
    )


# 1. Turn indexing off before the first point arrives.
set_indexing(False)

# 2. ... run the parallel loaders from section 4.15 here ...

# 3. Verify before paying for the build.
expected = 800_000_000
actual = client.count(COLLECTION, exact=True).count
if actual != expected:
    raise SystemExit(f"aborting: {actual:,} loaded, expected {expected:,}")

# 4. One build pass.
set_indexing(True)

import time

def wait_until_green(poll_seconds: int = 60) -> None:
    """Block until Qdrant reports the collection fully indexed.

    Expect hours at this scale. Log progress rather than waiting
    silently -- an operator watching a blank terminal for 14 hours
    will eventually restart something they should not.
    """
    while True:
        info = client.get_collection(COLLECTION)
        if info.status == models.CollectionStatus.GREEN:
            print("indexing complete")
            return
        print(
            f"status={info.status} "
            f"indexed={info.indexed_vectors_count:,} "
            f"of {info.points_count:,}"
        )
        time.sleep(poll_seconds)

wait_until_green()

4.15 Batch upserts and incremental updates

After bootstrap, the system lives in a steady state of change: new products arrive, titles get corrected, items go out of stock, whole categories get re-merchandised. How you batch those writes determines both throughput and whether search stays available while they land.

Choosing a batch size

The most common tuning question, and the answer is less obvious than "bigger is better".

from concurrent.futures import ThreadPoolExecutor, as_completed
from itertools import islice
import time
import uuid

from qdrant_client import QdrantClient, models

BATCH_SIZE = 256   # ~1 MB per request at 256 dims plus payload
WORKERS = 16       # roughly 2 per client CPU core
MAX_RETRIES = 5

client = QdrantClient(
    url="http://qdrant:6333",
    prefer_grpc=True,   # notably faster than HTTP for bulk work
    timeout=120,
)


def chunked(iterable, size):
    """Yield lists of `size` items without materialising the source."""
    it = iter(iterable)
    while batch := list(islice(it, size)):
        yield batch


def stable_id(product_id: str, market: str) -> str:
    """Derive a deterministic point ID from the business keys.

    This is what makes the pipeline idempotent: a replayed batch
    overwrites the same points instead of duplicating them. Never use
    uuid4() here -- a retry after a timeout would double-insert.
    """
    return str(uuid.uuid5(uuid.NAMESPACE_URL, f"{market}/{product_id}"))

def upsert_batch(rows: list[dict]) -> int:
    """Upsert one batch, retrying with exponential backoff.

    Raises only after MAX_RETRIES, so a transient node restart or a
    brief GC pause does not kill a multi-hour job.
    """
    points = [
        models.PointStruct(
            id=stable_id(row["product_id"], row["market"]),
            vector=row["vector"],
            payload={
                "product_id": row["product_id"],
                "market": row["market"],        # the tenant key
                "category_id": row["category_id"],
                "in_stock": row["in_stock"],
            },
        )
        for row in rows
    ]

    for attempt in range(MAX_RETRIES):
        try:
            client.upsert(
                collection_name="product_titles",
                points=points,
                wait=False,   # never block in a bulk pipeline
            )
            return len(points)
        except Exception:
            if attempt == MAX_RETRIES - 1:
                raise
            # Back off so a struggling cluster gets room to recover
            # instead of being hammered by every worker at once.
            time.sleep(2 ** attempt)
    return 0


def load_all(rows) -> int:
    """Drive batches across a bounded thread pool.

    Parallelism matters more than batch size: a single writer cannot
    saturate a sharded cluster no matter how large its batches are.
    """
    written = 0
    with ThreadPoolExecutor(max_workers=WORKERS) as pool:
        futures = [
            pool.submit(upsert_batch, batch)
            for batch in chunked(rows, BATCH_SIZE)
        ]
        for future in as_completed(futures):
            written += future.result()
            if written % 1_000_000 < BATCH_SIZE:
                print(f"{written:,} points written")
    return written

Update patterns: pick the cheapest one that works

Not every change requires rewriting a vector. Recognising which kind of update you have saves an enormous amount of work.

02:00  pull the day's catalogue delta          ~4M rows changed
02:20  compare title hashes                    ~180K genuinely new text
02:30  embed those 180K on GPU                 ~12 min
02:45  upsert vectors (16 workers, batch 256)  ~3 min
02:50  set_payload for the other 3.8M rows     ~8 min, no re-embedding
03:00  verify counts, sample recall
03:15  done

4.16 Re-indexing without downtime

Sooner or later you will need to rebuild everything: a better embedding model ships, you decide to change dimensions, or you need different HNSW parameters. At 1.3 billion vectors this is a migration, and it must happen while the current system keeps serving.

The alias swap

The standard technique, and the reason to use aliases from day one rather than pointing your application at collection names directly.

1. Your application never names a collection directly It queries the alias titles_live, which currently points at titles_v1. If you did not do this at the start, do it now — creating an alias for an existing collection is instant.
2. Build the replacement alongside Create titles_v2 with the new configuration and bootstrap it using the procedure from 4.14. The live collection continues serving throughout. You need capacity for both simultaneously, which is the main cost of this approach.
3. Catch up on the delta The build took hours, and the catalogue changed during it. Replay every update since the build started. This is why your pipeline should be idempotent and why points need an updated_at field.
4. Validate before switching Run your recall benchmark against v2. Compare results for real production queries side by side. A model upgrade that improves benchmark scores can still rank your catalogue worse.
5. Swap the alias atomically One API call repoints titles_live to titles_v2. No downtime, no deployment.
6. Keep v1 for a while Storage is cheap relative to an incident. Retain it for a week so rollback is another single alias call, then delete.

4.17 Monitoring and failure modes

Vector databases fail in ways that do not raise errors. A degraded index returns results happily; they are simply worse. That makes monitoring unusually important, because your users will notice before your dashboards do unless you watch the right things.

4.18 The optimisation runbook, in order

Everything up to here explained mechanisms one at a time. This section is the sequence: what to do first, what to measure, and how to stop — ending at the smallest, cheapest configuration that still meets a recall target you set deliberately rather than discovered by accident.

Step 0: build the benchmark before touching anything

You cannot optimise against a number you don't have. This step is unglamorous, takes about a day, and is the reason the rest of the process works at all.

The order to try things in

Each step below is cheaper or safer than the one after it. Run the benchmark after every step, and stop as soon as you are under your RAM budget with recall still above target — there is no prize for using every technique in the chapter.

Worked example: the MRL truncation study

Section 4.8 gave the memory arithmetic for truncating EmbeddingGemma's 768 dimensions. This is how to actually run the study rather than trusting the table — and the shape of the answer, which is the part that generalises.

# Sweep MRL truncation dimensions against fixed ground truth.
# One variable changes: the dimension count. Everything else is held.
import numpy as np

def truncate(vectors, dim):
    # MRL packs importance into the leading dimensions, so a prefix
    # is a valid embedding. Renormalise: cosine assumes unit length.
    cut = vectors[:, :dim]
    return cut / np.linalg.norm(cut, axis=1, keepdims=True)

def recall_at_k(got, truth, k=10):
    return np.mean([
        len(set(g[:k]) & set(t[:k])) / k
        for g, t in zip(got, truth)
    ])

for dim in [768, 512, 256, 128, 64]:
    # Rebuild the index at this dimension, query, compare to the
    # SAME float32 full-dimension ground truth every time.
    got = search_all(truncate(corpus, dim), truncate(queries, dim))
    print(dim, round(recall_at_k(got, ground_truth), 4))

Two-stage memory tiering: the last big RAM saving

By this point the vectors are as small as they are going to get. The remaining question is not how to compress further but which copy lives in RAM — and Qdrant lets you answer that separately for the compact representation and the originals.

# Originals cold and stored as 4-bit; 1-bit index pinned in RAM.
# Stage 1 searches the pinned 1-bit index -- no disk, no float math.
# Stage 2 rescores candidates against the 4-bit originals from disk.
client.create_collection(
    collection_name="product_titles",
    vectors_config=models.VectorParams(
        size=256,
        distance=models.Distance.COSINE,
        memory=models.Memory.COLD,
        datatype=models.Datatype.TURBO4,
    ),
    quantization_config=models.TurboQuantization(
        turbo=models.TurboQuantQuantizationConfig(
            bits=models.TurboQuantBitSize.BITS1,
            memory=models.Memory.PINNED,
        ),
    ),
)

# Oversampling and rescoring are query-time knobs: tune them against
# the benchmark without rebuilding the index.
client.query_points(
    collection_name="product_titles",
    query=vector,
    limit=10,
    search_params=models.SearchParams(
        quantization=models.QuantizationSearchParams(
            rescore=True,
            oversampling=3.0,   # fetch 30 candidates, return 10
        ),
    ),
)

4.19 Bootstrapping order and per-tenant indexing

Section 4.14 covered the load sequence and 4.10 covered tenant keys. This section is the part that only becomes visible once both are true at once: the interaction between them, and the switch that turns global indexing off entirely.

Why indexing order dominates the load

Turning off global indexing

The tenant key in 4.10 makes filtered search cheaper by adding same-tenant links on top of the global graph. There is a stronger version of the same idea: stop building the global graph at all, and keep only the per-tenant links.

# Per-tenant graphs, no global index.
# Only correct if EVERY query filters on the tenant field.
client.create_collection(
    collection_name="product_titles",
    vectors_config=models.VectorParams(
        size=256, distance=models.Distance.COSINE,
    ),
    hnsw_config=models.HnswConfigDiff(
        m=0,           # no global graph
        payload_m=16,  # one graph per tenant instead
    ),
)

# Declare the tenant field BEFORE loading any points, so Qdrant
# co-locates each tenant's data on disk as it is written.
client.create_payload_index(
    collection_name="product_titles",
    field_name="market",
    field_schema=models.KeywordIndexParams(
        type=models.KeywordIndexType.KEYWORD,
        is_tenant=True,
    ),
)

The full sequence, start to finish

1. Create the collection with vector indexing disabled indexing_threshold=0, with the quantization and memory tiers from 4.18 already configured. These are structural — changing them later means rebuilding.
2. Create payload indexes, tenant field first Before any points are written, so co-location happens on the way in rather than during a later rewrite.
3. Load in parallel batches Many workers, a few hundred points each (4.15). Ingest stays at full speed because nothing is building a graph yet.
4. Verify counts against the source Before indexing, not after. Discovering a gap once the build has run costs the whole build.
5. Restore indexing_threshold and let it build once Per segment, per shard, in parallel — and per tenant if m=0 is set.
6. Wait for green, then run the 4.18 benchmark Against the real recall target, with real filters, before production traffic arrives. This is the last point at which a bad compression decision is cheap to undo.

4.20 Key takeaways

4.21 Interview drills

These are the questions asked of people who claim billion-scale vector experience. The distinguishing feature of a strong answer here is arithmetic — being able to size something on a whiteboard rather than gesturing at concepts.

Expand all

Where this leaves you

You can now size a billion-scale vector cluster from first principles, justify every parameter you set, and answer the four-market question with reasoning rather than instinct. That is the infrastructure layer of a semantic search system.

What it does not give you is quality. A perfectly sized index running a mediocre retrieval strategy still returns mediocre results, and no amount of tuning m and ef will fix a bad ranking approach. Chapter 5 covers what sits on top: hybrid search combining vectors with keyword matching, cross-encoder reranking, and how to evaluate whether any of it is genuinely working.