Vector Stores

A vector store holds documents alongside their embeddings and lets you retrieve the ones most similar to a query. In Rig they are the storage layer behind RAG: you embed your data once, insert it into a store, and then query the store for the closest matches at prompt time.

Every store integration implements the same small set of traits, so the code you write to insert and query documents is almost identical whether you use the built-in in-memory store or a database like MongoDB, LanceDB, or Qdrant. This page covers those shared concepts; the per-store pages cover setup and backend-specific options.

Minimal example

The in-memory store ships in rig, so it needs no extra crates or running database. This is the smallest end-to-end flow: embed documents, index them, and search.

use rig::client::{ProviderClient, EmbeddingsClient};
use rig::providers::openai;
use rig::embeddings::EmbeddingsBuilder;
use rig::vector_store::in_memory_store::InMemoryVectorStore;
use rig::vector_store::{VectorStoreIndex, VectorSearchRequest};
use rig::Embed;
use serde::{Deserialize, Serialize};

#[derive(Embed, Clone, Serialize, Deserialize, Debug, PartialEq, Eq)]
struct WordDefinition {
    id: String,
    #[embed]
    definition: String,
}

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let openai = openai::Client::from_env()?;
    let model = openai.embedding_model(openai::TEXT_EMBEDDING_3_SMALL);

    let documents = vec![
        WordDefinition { id: "doc0".into(), definition: "A flurbo is a green alien.".into() },
        WordDefinition { id: "doc1".into(), definition: "A glarb-glarb is an ancient farming tool.".into() },
    ];

    // Generate an embedding for each document's `#[embed]` field.
    let embeddings = EmbeddingsBuilder::new(model.clone())
        .documents(documents)?
        .build()
        .await?;

    // Store the documents and build an index over them.
    let index = InMemoryVectorStore::from_documents(embeddings).index(model);

    // Query for the closest matches.
    let req = VectorSearchRequest::builder()
        .query("What is a flurbo?")
        .samples(1)
        .build();

    let results = index.top_n::<WordDefinition>(req).await?;
    for (score, id, doc) in results {
        println!("{score:.3} {id}: {}", doc.definition);
    }

    Ok(())
}

Swapping to a persistent database only changes how you construct the store and index — the EmbeddingsBuilder and VectorSearchRequest steps stay the same.

Embedding your documents

Before anything can go into a store, each document needs one or more embeddings. Derive Embed on your type and mark the field(s) to embed with #[embed], then let EmbeddingsBuilder batch the embedding calls.

use rig::embeddings::EmbeddingsBuilder;
use rig::Embed;
use serde::{Deserialize, Serialize};

#[derive(Embed, Clone, Serialize, Deserialize)]
struct Document {
    id: String,
    #[embed]
    content: String,
}

let embeddings = EmbeddingsBuilder::new(model.clone())
    .documents(docs)?   // a Vec<Document>
    .build()
    .await?;

build().await? returns a Vec<(Document, OneOrMany<Embedding>)> — each document paired with its embedding(s). OneOrMany lets a single document carry multiple embeddings (for example one per chunk), and stores rank a document by its best-matching embedding. To add a single document at a time, use .document(doc)? instead of .documents(...)?.

Core traits

Rig’s vector-store abstractions live in rig::vector_store. Two traits do the work.

VectorStoreIndex

VectorStoreIndex is the read side: query a store by similarity. Types that implement it also automatically implement Rig’s Tool trait, so any index can be handed to an agent as a searchable knowledge base.

pub trait VectorStoreIndex: Send + Sync {
    /// Return the top-`n` documents, deserialized into `T`, with similarity scores and ids.
    async fn top_n<T: for<'a> Deserialize<'a> + Send>(
        &self,
        req: VectorSearchRequest,
    ) -> Result<Vec<(f64, String, T)>, VectorStoreError>;

    /// Same ranking, but return only scores and ids.
    async fn top_n_ids(
        &self,
        req: VectorSearchRequest,
    ) -> Result<Vec<(f64, String)>, VectorStoreError>;
}

VectorStoreIndexDyn is a type-erased version of the same trait for dynamic-dispatch scenarios (for example a boxed dyn VectorStoreIndexDyn).

InsertDocuments

InsertDocuments is the write side: add documents and their embeddings to a store. It replaces the older VectorStore trait.

pub trait InsertDocuments: Send + Sync {
    async fn add_documents(
        &mut self,
        documents: Vec<(String, OneOrMany<Embedding>)>,
    ) -> Result<(), VectorStoreError>;
}

You typically pass the output of EmbeddingsBuilder::build() straight into add_documents. Some stores also expose convenience constructors (for example the in-memory store’s from_documents) that insert on construction.

Querying with VectorSearchRequest

All queries go through VectorSearchRequest, built with a fluent builder. At minimum you provide the query text and the number of results (samples).

use rig::vector_store::VectorSearchRequest;

let req = VectorSearchRequest::builder()
    .query("search query")
    .samples(5)
    .build();

let results = index.top_n::<Document>(req).await?;

Add a threshold to drop weak matches:

let req: VectorSearchRequest = VectorSearchRequest::builder()
    .query("search query")
    .samples(5)
    .threshold(0.7)
    .build();

Filters

For backends that support metadata filtering, attach a Filter. The Filter type gives a backend-agnostic way to express conditions; each store translates it into its native query language.

use rig::vector_store::request::{Filter, SearchFilter};

let req = VectorSearchRequest::builder()
    .query("search query")
    .samples(5)
    .filter(Filter::eq("category", serde_json::Value::from("science")))
    .build();

Filter supports equality, comparison, and boolean-combinator variants (Eq, Ne, Gt, Lt, And, Or, and others). Not every backend implements every variant — the in-memory store offers basic filtering, while dedicated databases (MongoDB, Qdrant, LanceDB) support richer conditions.

Vector stores as agent tools

Because every VectorStoreIndex also implements Tool, you can attach an index directly to an agent. The agent decides when to search, issues a query, and receives the matching documents back.

let agent = openai.agent("gpt-5.5")
    .preamble("You can search a knowledge base to answer questions.")
    .tool(index)
    .build();

For attaching context automatically on every prompt instead (classic RAG), see dynamic_context.

Error handling

Store operations return VectorStoreError. Common variants include EmbeddingError (embedding generation failed), JsonError (document (de)serialization), DatastoreError (backend storage failure), MissingIdError (unknown document id), and FilterError (building or converting a filter).

Comparing the stores

All stores share the traits above; they differ in persistence, scaling, and filtering power.

Feature	In-Memory	MongoDB	Qdrant	LanceDB	SQLite
Persistence	No	Yes	Yes	Yes	Yes
Horizontal scaling	No	Yes	Yes	No	No
Setup complexity	Low	Medium	Medium	Low	Low
Memory usage	High	Low	Medium	Low	Low
Query speed	Fast	Medium	Fast	Fast	Medium
Filtering	Basic	Rich	Rich	Rich	SQL

A rough guide:

In-memory — development, tests, and small datasets. No setup, no persistence.
LanceDB — embedded, file-based store with columnar storage; good local-to-cloud path.
SQLite — single-file, embedded store with SQL filtering; persistent without running a separate service.
MongoDB / Qdrant — production workloads needing persistence, rich filtering, and horizontal scaling.
Neo4j — when your data is already a graph and you want vector search alongside graph queries.
SurrealDB — a single database for documents, graph, and vector search.