Dynamic model creation

Sometimes you need to pick a model provider at runtime — for example a service that lets each user choose their own provider. Rust’s static type system makes this harder than in a dynamic language like Python, because every value needs a concrete type, but it is entirely doable with a couple of small tradeoffs.

For background on how Rig models providers and clients, see Providers & Clients.

Enum dispatch

The simplest approach is a single enum whose variants wrap an Agent for each provider, with a prompt method that dispatches to the inner agent:

use rig::agent::Agent;
use rig::completion::{Prompt, PromptError};
use rig::providers::{anthropic, openai};

enum Agents {
    Anthropic(Agent<anthropic::completion::CompletionModel>),
    OpenAI(Agent<openai::completion::CompletionModel>),
}

impl Agents {
    async fn prompt(&self, prompt: &str) -> Result<String, PromptError> {
        match self {
            Self::Anthropic(agent) => agent.prompt(prompt).await,
            Self::OpenAI(agent) => agent.prompt(prompt).await,
        }
    }
}

This is the most convenient option, but matching on every variant each time you touch a client gets tedious — especially if you want to support every provider Rig does.

A provider registry

To make the enum easier to use, build a registry that maps provider names to function pointers. Each function constructs an Agents value from a shared config, so you can create agents dynamically from a string key:

use std::collections::HashMap;
use rig::client::{CompletionClient, ProviderClient};
use rig::providers::{
    anthropic::{self, completion::CLAUDE_SONNET_4_6},
    openai::{self, GPT_5_5},
};

struct AgentConfig<'a> {
    name: &'a str,
    preamble: &'a str,
}

// In production you might use a dedicated `RegistryKey` type instead of
// arbitrary strings for improved type safety.
struct ProviderRegistry(HashMap<&'static str, fn(&AgentConfig) -> Agents>);

/// Build an `Agents::Anthropic` variant.
fn anthropic_agent(AgentConfig { name, preamble }: &AgentConfig) -> Agents {
    let agent = anthropic::Client::from_env()
        .expect("ANTHROPIC_API_KEY must be set")
        .agent(CLAUDE_SONNET_4_6)
        .name(name)
        .preamble(preamble)
        .build();

    Agents::Anthropic(agent)
}

/// Build an `Agents::OpenAI` variant.
fn openai_agent(AgentConfig { name, preamble }: &AgentConfig) -> Agents {
    let agent = openai::Client::from_env()
        .expect("OPENAI_API_KEY must be set")
        .completions_api()
        .agent(GPT_5_5)
        .name(name)
        .preamble(preamble)
        .build();

    Agents::OpenAI(agent)
}

impl ProviderRegistry {
    /// Register the Anthropic and OpenAI constructors.
    pub fn new() -> Self {
        Self(HashMap::from_iter([
            ("anthropic", anthropic_agent as fn(&AgentConfig) -> Agents),
            ("openai", openai_agent as fn(&AgentConfig) -> Agents),
        ]))
    }

    /// Construct an agent for `provider`, or `None` if it isn't registered.
    pub fn agent(&self, provider: &str, agent_config: &AgentConfig) -> Option<Agents> {
        self.0.get(provider).map(|p| p(agent_config))
    }
}

With the registry in place, choosing a provider at runtime is a string lookup:

#[tokio::main]
async fn main() {
    let registry = ProviderRegistry::new();
    let prompt = "How much does 4oz of parmesan cheese weigh?";

    let helpful_cfg = AgentConfig {
        name: "Assistant",
        preamble: "You are a helpful assistant",
    };

    let openai_agent = registry.agent("openai", &helpful_cfg).unwrap();
    let oai_response = openai_agent.prompt(prompt).await.unwrap();
    println!("Helpful response (OpenAI): {oai_response}");

    let unhelpful_cfg = AgentConfig {
        name: "Assistant",
        preamble: "You are an unhelpful assistant",
    };

    let anthropic_agent = registry.agent("anthropic", &unhelpful_cfg).unwrap();
    let anthropic_response = anthropic_agent.prompt(prompt).await.unwrap();
    println!("Unhelpful response (Anthropic): {anthropic_response}");
}

Tradeoffs

A dynamic factory like this is convenient, but it comes with real costs:

Erasing the concrete model type creates a pocket of type unsafety that is harder to maintain and extend.
You lose access to any model-specific methods that aren’t part of the shared enum interface.
You have to update the enum and registry whenever you want to support a new provider.
There’s a small runtime cost to the indirection (usually negligible here).

If you’re running a service that offers users a choice of provider, these tradeoffs are often worth it. If you only ever use one provider, prefer a concrete Agent<M> type — see Model routing for a typed router built on that.

Dynamic model creation

Enum dispatch

A provider registry

Tradeoffs

See also