Skip to content
Atlas by Comet Lab
Infrastructure layer

Vector databases, compared

A vector database stores embeddings and finds the closest matches to a query: the memory behind retrieval. Most projects need less than they expect, so the first question is whether a database you already run can do the job before you take on a new one.

Learn the idea How AI uses tools and memory on Horizon

Open this layer in the explorer

At a glance

Every tool in this layer, with where it fits and the facts that usually decide it.

Tool Where it fits Licence Self-hostable Hosted API Maturity
pgvector Embedded vector search The right call when your data already lives in Postgres and the vector workload is moderate: retrieval without a second database. Open source Yes No Established
Chroma Embedded vector database The easiest on-ramp for a RAG prototype, and a fine production store for small to medium workloads. Open source Yes Yes Growing
Elasticsearch Search engine A fit when you already run Elasticsearch and want hybrid keyword-plus-vector search in the system you operate. Source available Yes Yes Mature
LanceDB Embedded vector database A fit when you want fast, embedded vector search with no server, especially over data on object storage. Open source Yes Yes Growing
Milvus Dedicated vector database A fit when vector search must operate at very large scale, into the billions of vectors. Open source Yes Yes Established
Pinecone Managed vector database A fit when a team wants vector search with zero operational burden and a hosted service is acceptable. Proprietary No Yes Established
Qdrant Dedicated vector database A strong pick when retrieval needs to combine similarity with metadata filtering, or has outgrown an in-database extension. Open source Yes Yes Established
Redis In-memory vector search A fit when you already run Redis and want low-latency vector search without operating a separate database. Source available Yes Yes Mature
turbopuffer Serverless vector search A fit for large search workloads where cost matters and not all of the data is hot at once. Proprietary No Yes Growing
Vespa Search and serving engine A fit for large-scale search and recommendation where vectors, keywords, and learned ranking must all work together. Open source Yes Yes Mature
Weaviate Dedicated vector database A fit when you want hybrid search and the option to let the database handle vectorisation for you. Open source Yes Yes Established

How to choose

Start from your situation. Where a line says “our”, it comes from Comet Lab’s own builds.

  1. Your data already lives in Postgres

    Retrieval without a second database. Our default vector store: most projects never outgrow it.

  2. Retrieval needs metadata filtering, or has outgrown pgvector

    Qdrant is our pick when a project genuinely outgrows pgvector; Weaviate adds hybrid search and can handle vectorisation for you.

  3. No operations at all, and a hosted service is allowed

    Pinecone removes the operational work; turbopuffer keeps costs low for large workloads where not all the data is hot.

  4. Billions of vectors, or search with learned ranking

    Built for very large scale, and overkill for the small and medium workloads most RAG projects have.

  5. You already run Elasticsearch or Redis

    Add vector search to the system you operate: hybrid keyword and vector search in Elasticsearch, low-latency search in Redis.

  6. Embedded, with no server

    Chroma is the easiest start for a RAG prototype; LanceDB is fast embedded search over data on object storage.