# CLAUDE.md

This file defines the rules for Rust work in LiteLLM.

## Crates (exactly three — see AGENTS.md)

`litellm-core` describes work; `litellm-ai-gateway` executes it; `litellm-python-bridge`
exposes it to the Python SDK. A crate is a **layer**, not a route — add modules, not crates.

## Core Boundary

`litellm-core` is the pure translation layer; the `litellm-ai-gateway` host executes work.

Route-level Rust structure mirrors LiteLLM's Python responsibilities:
- `core/src/<route>/` owns the route contract, shared types, and provider
  template traits. For OCR, this means `core/src/ocr`.
- `core/src/providers/<provider>/<route>/transformation.rs` owns the
  provider-specific transform. For Mistral OCR, this means
  `core/src/providers/mistral/ocr/transformation.rs`.
- Network execution lives in the host crate `ai-gateway` (`ai-gateway/src/io/`),
  never inside `core`.

Allowed in `core`:
- Pure request transforms
- Pure response transforms
- Pure stream chunk normalization
- Shared data types and validation errors
- Deterministic token/cost helper logic

Not allowed in `core`:
- Network calls
- Environment variable or secret reads
- Filesystem access
- Database or cache access
- Provider SDK signing or auth flows
- Logging callbacks, spend writes, or custom callbacks
- Global mutable runtime state

Python owns rollout state and fallback while Rust is being introduced. Rust
paths must be off by default until parity tests prove equivalence with Python.

## Production Bar

Rust code in this workspace is held to a strict parity and robustness bar from
the first PR:

- Correctness parity is proven with tests. Do not rely on README claims or
  manual inspection for a port that mirrors Python behavior.
- Every provider transform must have unit tests for supported-parameter
  filtering, request body shape, response normalization, missing/null fields,
  and bad-input errors.
- When Rust is exposed through Python, add Python tests that prove disabled,
  enabled, and unavailable-bridge fallback behavior.
- Avoid panics on user/provider input. Return typed errors and let the host map
  them to Python exceptions or HTTP responses.
- OCR handles documents that often contain personal data. Do not log document
  contents, base64 payloads, provider response bodies, or secrets.
- Error messages must be useful but data-minimized. Truncate or sanitize any
  upstream body before it crosses a host boundary.
- Treat empty or whitespace-only credentials, URLs, and config values as absent
  at the host/config resolution layer.
- Preserve Python output shape intentionally. If a field is always serialized as
  `null` for Python parity, leave a short comment explaining that parity choice.

## Host I/O Rules

These rules apply when adding future crates or modules that execute network I/O,
such as `ai-gateway`, router hosts, or standalone servers:

- Set connect and full-request timeouts. No unbounded waits.
- Reuse HTTP clients; do not construct clients per request.
- Prefer rustls TLS for portable Python wheels and Linux images unless there is
  a documented reason not to.
- Add request IDs and structured tracing at the host layer, without logging OCR
  document contents or secrets.
- Do not echo raw upstream response bodies to callers. Sanitize and bound them.
- Avoid `expect`/`unwrap` in server startup and request paths unless the panic is
  impossible by construction and documented.

## Constants

Magic numbers and fixed strings go in a crate-level `constants.rs`, never
hardcoded inline — the Rust mirror of Python's `litellm/constants.py`.

- Each crate that needs them has `src/constants.rs` (declared `mod constants;`);
  import from it (`use crate::constants::...`). Don't scatter `const` values at
  the top of feature modules.
- An env-overridable tunable still lives in `constants.rs` as its `DEFAULT_*`
  value; the env read (with fallback to that default) happens at the host/config
  resolution layer, not in `core`/`providers`.
- Exception: a value that is purely local to one function and has no meaning
  elsewhere may stay inline, but prefer `constants.rs` when in doubt.

## Checks

Run these before pushing Rust changes. The same checks run in GitHub Actions
for changes under `litellm-rust/`.

```bash
cd litellm-rust
cargo fmt --check
# the ai-gateway binary + server code is behind the `server` feature
cargo clippy -p litellm-ai-gateway --all-targets --features server -- -D warnings
cargo clippy -p litellm-core -p litellm-python-bridge --all-targets -- -D warnings
cargo test --workspace
```

When a Rust path is exposed through Python, add Python parity tests that compare
the existing Python output with the Rust-backed output.