Sizing Lakebase Search — Lessons from a Real App and 4 Million Passages

This post organizes the Databricks blog Sizing Lakebase Search: lessons from a real app and 4 million passages (Philipp Tiefenbacher, October 2026) into a reference you can scan when sizing a deployment. All numbers come from the original post.

The benchmark has two subjects:

• Slide Hub — an internal Databricks app that field engineers use to find and reuse slides from about 700 internal presentations. It has 23k slide rows, real users and real traffic, and an MCP server gives AI agents the same search.
• MIRACL English — a public passage corpus built from Wikipedia, used in 1M, 2M and 4M subsets to see how behavior changes with scale.

Summary

• Lakebase Search runs vector and BM25 keyword search as Postgres extensions inside the application database, so there is no separate search system and no second transaction path.
• Sizing the Local File Cache to roughly match the vector index keeps 95–98% of pages in memory and drops warm vector latency to single-digit milliseconds.
• Reducing the candidate pool from 1,000 to 200 preserves 90.5% of the top-10 results at about one-third of the page reads.
• Compute is billed only while it runs. A 14-minute load test serving 109k requests used $0.02 of Lakebase Postgres compute. An always-on 0.5 CU floor ($31–40/month) is the recommended production baseline to avoid cold starts.

1. One database for the app and its search

Lakebase Search became generally available in September 2026. It is a set of Postgres extensions that run next to the application tables.

CREATE EXTENSION lakebase_vector;   -- embeddings + lakebase_ann index (ANN search)
CREATE EXTENSION lakebase_text;     -- lakebase_bm25 index (BM25 keyword search)
-- lakebase_tokenizer is optional

Slide Hub's slide rows sit in the same serverless Lakebase Postgres instance as votes, telemetry, change history, previews and both indexes. The app reads data and indexes through one connection and one permission model.

2. The search query

Slide Hub runs hybrid search: vector search for meaning plus BM25 for query words. One SQL statement carries the whole ranking policy — it ranks each candidate list, fuses the rankings, applies app filters and joins the votes, sharing one connection, one set of grants and one commit with the app logic.

• Candidates: 200 from each method.
• Fusion: Reciprocal Rank Fusion with k=60 — a slide scores 1/(60 + rank) in each list, and the scores add.
• Similarity gate: 0.70 rejects vector matches too far from the query for this embedding model.
• Weights: title tiers add weight when the query matches a title; notes get weight 0.5.
• Scope filters: restrict results to assets the user may see.

3. What limits the app under load

A search request uses two computes, and either can be the limit.

• Lakebase Postgres compute, sized in Compute Units. One CU is about 2 GB of RAM with its share of CPU and local SSD. Autoscaling ranges from 0.5 to 64 CU (max at most 16 CU above min); fixed-size computes go up to 112 CU; computes up to 32 CU can scale to zero. Recently read 8 KB pages stay in the Local File Cache (LFC), which grows with CU size and can use up to 75% of memory.
• Databricks Apps compute, fixed per app. MEDIUM: up to 2 vCPU / 6 GB. LARGE: up to 4 vCPU / 12 GB. Each Python worker holds a pool of 8 database connections.

Throughput and median latency at 500 users:

• Up to 4 CU, Lakebase is the limit. At 1 CU the search statement averaged 146 ms with a 1.5 GB cache; at 8 CU it averaged 34 ms with a 12 GB cache.
• From 8 CU, the app is the limit. 16 CU with the same MEDIUM app adds only 4%; a LARGE app at 16 CU reaches at least 662 req/s, where the load generator became the limit.
• The connection pool matters. At 50 users: 45 req/s and 2.4 s p95 without the pool, 85 req/s and 0.7 s p95 with it. The embedding cache made no measurable difference.

Takeaway: find the limit before you scale. Below 8 CU, add Lakebase CU. From 8 CU, give the app more workers.

4. Candidate pools and cold starts

The candidate limit sets how many pages a query touches, and on a cold cache that sets latency. Twenty public queries were measured in three states — cold (first query after scale-to-zero), first-touch (first run on an awake compute) and repeat.

Slide Hub moved from 1,000 to 200 candidates. End to end from the same region, a cold search at 200 candidates took 2.9 s and a repeat 40 ms. The cold time is about 1 s waiting for the endpoint to wake, then page reads from storage until the cache warms; the first query on a new connection also pays about 75 ms of planning.

Takeaway: keep the candidate pool small. 200 candidates keep 90.5% of the top 10, read about a third of the pages, and halve cold latency.

5. Scale: 1M to 4M passages of MIRACL English

Data and index

• The full corpus has 32,893,221 passages, embedded with gte-large-en-v1.5 (1024 dimensions) on one A10G GPU at 379 passages/s — 2.95 GPU-hours versus $48.87 on a model-serving endpoint, with matching vectors (mean cosine 1.0000).
• lakebase_ann index size: 5.8 GB at 1M, 11.5 GB at 2M, 23.0 GB at 4M. A binary COPY loaded 4M rows in 19 minutes at 8 CU.

The cache ratio

Lakebase Postgres holds about 1.6 GB of file cache per CU.

• Cache ratio = cache size ÷ vector index size (1.0 means the index just fits).
• Hit rate = share of the pages a query reads that do not come from storage.

Query mixes: uniform (never repeats — worst case), skewed (Zipf over 20,000 titles — typical search traffic) and cold (right after wake-up). "Warm" results follow a warm-up until the hit rate stops changing.

Findings

Takeaway: size the cache from the vector index, not the row count. A cache about as large as the index serves 95–98% of pages without storage.

Recall against exact search

lakebase_ann is approximate; lakebase_ann.probes sets how many IVF partitions a query reads. At default probes, recall@10 was 0.74 / 0.75 / 0.75 at 1M / 2M / 4M.

The quality build took 52 minutes at 16 CU. At 500 probes it reached 0.91 in 13 ms, which takes 1,000 probes and 20 ms on the default build. Raising lakebase_ann.epsilon to 4 added less than 0.3 points. Lakebase Search supports a single index of over 1 billion vectors. Recall depends on data and model, so measure on your own data.

6. Cost

• Lakebase was about 1% of the Slide Hub 30-day bill. A suspended compute incurs no compute cost; storage is billed separately. Slide Hub's production database scales to zero because a slower first search after 5 idle minutes is acceptable internally.
• The 14-minute in-region load ramp cost $0.13 at list price ($0.11 app, $0.02 Lakebase, autoscaling 0.5–1 CU, 0.21 CU-hours) and served 109,602 requests without errors up to 1,000 users.
• One CU-hour is 0.213 DBU — about $0.085 (Premium) and $0.11 (Enterprise) list price in AWS US East.

Takeaway: for production, set the minimum to 0.5 CU instead of scaling to zero. It costs about $31–40 a month and removes the ~1 s wake-up and empty-cache start. Let dev and test branches scale to zero.

7. Choosing among Databricks searches

Sizing checklist

1. Measure the vector index size; pick CU for the cache ratio you want (about 1.6 GB of cache per CU).
2. Keep the candidate pool small (200 is a good default).
3. Under load, scale Lakebase CU until the app becomes the limit, then add app workers.
4. Use a connection pool.
5. Build the vector index with build_mode = 'quality'; raise probes only if you need more recall.
6. In production, keep a 0.5 CU minimum; let dev/test scale to zero.

Reference

• Databricks Blog: Sizing Lakebase Search: lessons from a real app and 4 million passages