Awesome Vector Search Engine
A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search
Standalone Service
- Apache Cassandra 5.0 – Vector search (cep-30), Strict Serialisable ACID (cep-15), horizontally scaling database
- Qdrant - Vector Similarity Search Engine with extended filtering support
- Vald - A Highly Scalable Distributed Vector Search Engine
- Milvus - A cloud-native vector database with high-performance and high scalability.
- Weaviate - A cloud-native, real-time vector search engine
- OpenDistro Elasticsearch KNN - A machine learning plugin which supports an approximate k-NN search algorithm for Open Distro for Elasticsearch
- Elastiknn - Elasticsearch plugin for nearest neighbor search
- Epsilla - A High Performance Vector Database Management System, Hippocampus For AI
- Vearch - A scalable distributed system for efficient similarity search of deep learning vectors
- pgANN - Fast Approximate Nearest Neighbor (ANN) searches with a PostgreSQL database
- Jina - Jina allows you to build deep learning-powered search-as-a-service.
- Infinity - The AI-native database built for LLM applications, providing incredibly fast vector and full-text search
- Aquila DB - Distribution focused k-NN search algorithm
- Redis HNSW - A redis module for similarity search based on HNSW
- Solr - Apache Solr - has a Dense Vector Search feature as of Solr 9.0
- Marqo - A semantic search engine which supports tensor search (sequence of vectors)
- txtai - Build semantic search applications and workflows
- Semantra - A multipurpose tool for semantically searching documents.
- SuperDuperDB - Bring AI to your favorite database
- TensorDB - High Performance Vector Database Supporting Heterogeneous Computing
- JVector - a pure Java, zero dependency, embedded vector search engine, used by DataStax Astra DB and Apache Cassandra.
- VQLite - Simple and Lightweight Vector Search Engine
- Vexvault - 100% browser based, open source, scalable, simple, zero-cost vector search
- Vespa.ai - Text search engine and ... fast approximate vector search (ANN)
- Vespa's large-scale ANN search using HNSW-IF indexes is described here
Library
- LangStream - LangStream is an open-source project that combines the best of event-based architectures with the latest Gen AI technologies.
- CassIO - CassIO is the ultimate solution for seamlessly integrating Apache Cassandra® with generative artificial intelligence and other machine learning workloads
- JVector - A pure Java, zero dependency, embedded vector search engine used by some of the advanced distributed databases such as DataStax Astra DB & Apache Cassandra™
- Faiss - A library for efficient similarity search and clustering of dense vectors
- Distributed Faiss - Work with FAISS indexes which don't fit into a single server memory
- Autofaiss - Automatically create Faiss knn indices
- ScaNN - A library efficient vector similarity search at scale.
- NMSLIB - Non-Metric Space Library, an efficient similarity search library for generic non-metric spaces
- Annoy - C++ library with Python bindings to search for points
- FLANN - Library written in C++ and contains bindings for the following languages: C, MATLAB, Python, and Ruby
- LLM App - Open-source Python library for a real-time data KNN (K-Nearest Neighbors) indexing
- MRPT - Fast nearest neighbor search with random projection
- RPForest - Python library for approximate nearest neighbours search
- pgvector - Open-source vector similarity search extension for Postgres
- PASE - Ultra-High-Dimensional approximate nearest neighbor search extension for Postgres
- Pyserini - Toolkit for reproducible information retrieval research with sparse and dense representations
- NGT - Provides commands and a library for performing high-speed approximate nearest neighbor
- NearPy - Approximate search using different locality-sensitive hashing methods
- TOROS N2 - lightweight approximate Nearest Neighbor library
- PUFFINN - Parameterless and Universal Fast FInding of Nearest Neighbors
- SPTAG - A distributed approximate nearest neighborhood search (ANN) library
- PyNNDescent - A python nearest neighbor descent for approximate k nearest neighbors
- TarsosLSH - A Java library implementing practical nearest neighbour search algorithm for multidimensional vectors
- TorchPQ - Efficient implementations of Product Quantization and its variants using Pytorch and CUDA
- Granne - Graph-based retrieval of approximate nearest neighbors witten in rust
- Embeddinghub - A database built for machine learning embeddings
- Hora - Efficient approximate nearest neighbor search algorithm collections library written in Rust
- Voy - A WASM vector similarity search engine written in Rust
- Chroma - The open-source embedding database for building LLM apps in Python or JavaScript with memory
- USearch - Smaller & Faster Vector Search Engine for C++, Python, JavaScript, Rust, Java, GoLang, Wolfram
- Golang vector stores collection - Chroma, PGVector interfaces
- Scalable Vector Search (SVS) - A performance library for vector similarity search
Cloud Service
- Epsilla Cloud - The fully managed serverless vector database with 10X faster, cheaper and better.
- DataStax Astra Vector - Multi-cloud, serverless vector DBaaS
- Relevance AI - Vector Platform From Experimentation To Deployment
- Pinecone - Managed vector search with filtering, live index updates, horizontal scaling, and a lot more
- MyScale - A managed vector database based on ClickHouse
- Redis Cloud - Managed vector database in Redis
- Zilliz Cloud - Cloud-native service for Milvus
Research Papers
List of methods on how approximate vector search algorithm can be implemented more effciently.
- SPANN: Highly-efficient Billion-scale Approximate Nearest Neighborhood Search - NEURIPS 2021
- Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighbors - ECCV 2018
- Accelerating Large-Scale Inference with Anisotropic Vector Quantization
- Billion-scale similarity search with GPUs
- Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs
- Optimization of Indexing Based on k-Nearest Neighbor Graph for Proximity Search in High-dimensional Data
- On Approximately Searching for Similar Word Embeddings - ACL 2016
