unified-llm

A unified Rust client library for multiple LLM providers (OpenAI, Anthropic, Google Gemini). Write provider-agnostic code and switch models by changing a single string identifier.

Architecture

The library is organized into four layers:

Layer 4: High-Level API         generate(), stream(), generate_object()
Layer 3: Core Client            Client, provider routing, middleware hooks
Layer 2: Provider Utilities     Shared helpers (SSE parsing, retry, etc.)
Layer 1: Provider Specification ProviderAdapter trait, shared types

Installation

Add to your Cargo.toml:

[dependencies]
unified-llm = { path = "crates/unified-llm" }
tokio = { version = "1", features = ["full"] }

Usage Examples

Simple Generation

use unified_llm::generate::{generate, GenerateParams};

#[tokio::main]
async fn main() {
    let result = generate(
        GenerateParams::new("claude-opus-4-6")
            .prompt("Explain quantum computing in one paragraph")
    ).await.unwrap();

    println!("{}", result.text);
    println!("Tokens used: {}", result.usage.total_tokens);
}

Generation with System Message

use unified_llm::generate::{generate, GenerateParams};

let result = generate(
    GenerateParams::new("claude-opus-4-6")
        .system("You are a helpful coding assistant.")
        .prompt("Write a Rust function to check if a number is prime")
).await.unwrap();

Generation with Tools

use unified_llm::generate::{generate, GenerateParams};
use unified_llm::tools::Tool;

let weather_tool = Tool::active(
    "get_weather",
    "Get the current weather for a location",
    serde_json::json!({
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City name, e.g. 'San Francisco, CA'"
            }
        },
        "required": ["location"]
    }),
    |args| async move {
        let location = args["location"].as_str().unwrap_or("unknown");
        Ok(serde_json::json!(format!("72F and sunny in {}", location)))
    },
);

let result = generate(
    GenerateParams::new("claude-opus-4-6")
        .system("You are a helpful assistant with access to weather data.")
        .prompt("What is the weather in San Francisco?")
        .tools(vec![weather_tool])
        .max_tool_rounds(5)
).await.unwrap();

println!("{}", result.text);
println!("Steps taken: {}", result.steps.len());
println!("Total tokens: {}", result.total_usage.total_tokens);

Streaming

use unified_llm::generate::{stream_generate, GenerateParams};
use unified_llm::types::StreamEventType;
use futures::StreamExt;

let mut stream = stream_generate(
    GenerateParams::new("claude-opus-4-6")
        .prompt("Write a haiku about coding")
).await.unwrap();

while let Some(event) = stream.next().await {
    let event = event.unwrap();
    if event.r#type == StreamEventType::TextDelta {
        print!("{}", event.delta.unwrap_or_default());
    }
}

Structured Output

use unified_llm::generate::{generate_object, GenerateParams};

let schema = serde_json::json!({
    "type": "object",
    "properties": {
        "name": { "type": "string" },
        "age": { "type": "integer" }
    },
    "required": ["name", "age"]
});

let result = generate_object(
    GenerateParams::new("gpt-5.2")
        .prompt("Extract: 'Alice is 30 years old'"),
    schema,
).await.unwrap();

let output = result.output.unwrap();
assert_eq!(output["name"], "Alice");
assert_eq!(output["age"], 30);

Client Configuration

use unified_llm::client::Client;
use std::sync::Arc;

// From environment variables (reads OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)
let client = Client::from_env();

// Or configure explicitly
let mut client = Client::new(
    std::collections::HashMap::new(),
    None,
    vec![],
);
// Register adapters...

// Use with generate
let result = generate(
    GenerateParams::new("claude-opus-4-6")
        .prompt("Hello")
        .client(Arc::new(client))
).await.unwrap();

Model Catalog

use unified_llm::catalog::{get_model_info, list_models, get_latest_model};

// Look up a model
let info = get_model_info("claude-opus-4-6").unwrap();
println!("{} ({})", info.display_name, info.provider);
println!("Context window: {} tokens", info.context_window);

// Look up by alias
let info = get_model_info("opus").unwrap();
assert_eq!(info.id, "claude-opus-4-6");

// List all models for a provider
let anthropic_models = list_models(Some("anthropic"));
for model in &anthropic_models {
    println!("  {} - {}", model.id, model.display_name);
}

// Get the latest model for a provider
let best = get_latest_model("openai", Some("reasoning")).unwrap();
println!("Best OpenAI reasoning model: {}", best.id);

Retry Logic

use unified_llm::retry::retry;
use unified_llm::types::RetryPolicy;

let policy = RetryPolicy {
    max_retries: 3,
    base_delay: 1.0,
    max_delay: 60.0,
    backoff_multiplier: 2.0,
    jitter: true,
};

let response = retry(&policy, || {
    let c = client.clone();
    let r = request.clone();
    async move { c.complete(&r).await }
}).await.unwrap();

Error Handling

use unified_llm::error::SdkError;

match result {
    Ok(response) => println!("{}", response.text()),
    Err(SdkError::RateLimit { retry_after, .. }) => {
        println!("Rate limited. Retry after {:?}s", retry_after);
    }
    Err(SdkError::Authentication { message, .. }) => {
        println!("Auth error: {}", message);
    }
    Err(e) if e.retryable() => {
        println!("Transient error, can retry: {}", e);
    }
    Err(e) => {
        println!("Fatal error: {}", e);
    }
}

Modules

Module	Description
types	Core data types: Message, Request, Response, Usage, StreamEvent, etc.
error	Error hierarchy with retryability classification
client	Client with provider routing and middleware
provider	ProviderAdapter trait
middleware	Middleware trait for cross-cutting concerns
tools	Tool definitions and parallel execution
retry	Retry with exponential backoff and jitter
generate	High-level API: generate(), stream(), generate_object()
catalog	Model catalog with lookup functions

Supported Providers

Provider	API	Environment Variable
OpenAI	Responses API (/v1/responses)	OPENAI_API_KEY
Anthropic	Messages API (/v1/messages)	ANTHROPIC_API_KEY
Gemini	Gemini API (/v1beta/...)	GEMINI_API_KEY

License

MIT