Notion engineers Preeti Gondi, Mickey Liu, Nathan Louie, Calder Lund and Jacob Sager published a detailed retrospective on how Notion's vector search infrastructure evolved since Notion AI Q&A launched in November 2023. The headline claim: 10x scale growth alongside a 90% cost reduction over two years.

At launch, ingestion ran on two paths: an offline Apache Spark batch pipeline for chunking and embedding existing documents, and an online Kafka-based path for near real-time page edits. The vector database used dedicated 'pod' clusters, sharded by workspace ID similar to Notion's Postgres setup. Within a month, demand from a multi-million-workspace waitlist pushed indexes near capacity. Instead of Postgres-style incremental re-sharding, Notion introduced 'generation' IDs — when a shard set neared capacity, a new set was provisioned and new workspaces routed there, avoiding re-shard downtime. Onboarding throughput improved via Airflow scheduling and Spark tuning: daily onboarding capacity rose 600x, active workspaces grew 15x, and vector DB capacity expanded 8x, clearing the waitlist by April 2024.

In May 2024, Notion migrated from the pod-based architecture to a serverless one decoupling storage and compute, cutting costs 50% from peak and removing capacity-provisioning bottlenecks. Still spending millions annually, the team evaluated alternative vector databases and, between May 2024 and January 2025, migrated its multi-billion-object workload to turbopuffer, an object-storage-based engine supporting both managed and bring-your-own-cloud deployments. The migration included a full re-index, an upgraded embeddings model, removal of sharding/generation logic (turbopuffer treats each namespace as an independent index), and a gradual generation-by-generation cutover. Results: 60% lower search-engine spend, 35% lower AWS EMR compute costs, and p50 query latency improved from 70-100ms to 50-70ms.

In July 2025, the 'Page State Project' tackled redundant re-embedding: previously any edit to a page — even metadata like permissions — triggered full re-chunking and re-embedding. The new system stores two 64-bit xxHash values per span (text and metadata) in DynamoDB, comparing them on each edit. Pure text changes only re-embed the changed spans; pure metadata changes (e.g., permission updates) skip embedding entirely and issue a cheaper PATCH to the vector DB. This cut data volume by 70%, lowering both embeddings API and vector DB write costs.

Since July 2025, Notion has been migrating its near-real-time embeddings pipeline from Spark/EMR to Ray running on Anyscale, addressing a 'double compute' problem (paying for both Spark preprocessing and per-token embedding API fees), unreliable third-party embedding APIs, and clunky custom pipelining built to avoid rate limits. Ray lets Notion self-host open-source embedding models, unifies preprocessing and inference on one compute layer with GPU/CPU pipelining, and Anyscale's managed workspaces let engineers use tools like Cursor and VS Code without provisioning infrastructure. Notion expects a 90%+ reduction in embeddings infrastructure costs from this migration, still in progress. For query-time embedding, Ray Serve hosts open-source embedding models in persistent GPU deployments with configurable batching and autoscaling defined in YAML.

Looking ahead, Notion plans to connect more third-party data sources, keep evaluating new embedding models (enabled by Ray's flexibility), continue pipeline optimization, and use vector search to power upcoming 'Custom Agents' that draw context from a user's workspace and connected apps.