Merge remote-tracking branch 'origin/main' into litellm_least_busy_burst_reservation

# Conflicts:
#	tests/unit/router_strategy/test_router_routing_groups.py

.agents/skills/database-access/SKILL.md

@@ -1,54 +1,73 @@
 ---
 name: database-access
-description: Add database queries, extract existing Prisma calls into repositories, or explicitly port queries to Rust and SQLx without expanding migration scope
+description: Organize database repositories and proxy dependency wiring, manage connection ownership, or port Prisma operations to Rust and SQLx without expanding migration scope
 metadata:
   internal: true
 ---
 
 # Database access
 
+## Situations
+
+- Adding a query or an operation to a repository, read [adding a query](references/python-repository.md#adding-a-query)
+- Moving existing Prisma calls behind a repository, read [moving existing access](references/python-repository.md#moving-existing-access-into-a-repository)
+- Porting a repository from Prisma to Rust and SQLx, read [porting to SQLx](references/python-repository.md#porting-a-repository-to-sqlx)
+- Injecting a repository into a route, read [Python and FastAPI](references/python-fastapi.md) or [Rust and Axum](references/rust-axum.md)
+
 ## Adding new database access
 
 - Put Python database access behind an operation in `litellm/repositories`
-- Give callers typed inputs and outputs, not Prisma clients, models, or query dictionaries
-- Keep query execution and backend selection inside the repository
+- Give each repository its own FastAPI provider, so a route builds only the repositories it uses
+- A provider borrows existing handles and performs no database I/O during construction
+- Define the operations a route calls as a Protocol in `litellm/models/<feature>.py`, next to its domain types
+  - The Protocol takes and returns domain types, never Prisma models, raw rows or serialized payloads
+  - Parsing and serialization of stored values live in the repository
+  - Add a Protocol when a repository gets a provider, not up front for every repository
+- Keep query execution inside the repository, with one implementation per repository and no backend switch or fallback
+- A repository is either all Prisma or all native, never a mix of both
 - Prisma is fine for the implementation, adding a query does not require a Rust port
 - Native Rust callers use `litellm-rust/crates/db` directly
 
-## Asked to move existing access behind a repository
+## Framework wiring
 
-- Identify the calls being moved and their transaction boundary
-- Move the existing Prisma implementation into the relevant repository
-- Replace those direct database calls with repository operations
-- Preserve return values, errors, and transaction behavior, with focused regression tests
-- Keep this an extraction, not a backend migration or unrelated cleanup
+- Python routes receive a per-repository dependency such as `UserBannerRepositoryDep` and call its operations, such as `banners.get_banner()`
+- `DatabaseDep` lives in `litellm/proxy/deps/database.py`, repository providers in `litellm/proxy/deps/repositories.py`
+  - A provider returns the feature's Protocol, so routes never see which backend implements it
+  - Dependencies used by one feature only, like the banner audit binding, stay next to that feature's routes
+- Repositories never import FastAPI, so jobs and helpers can use them without a request
+- Use named FastAPI providers with `Annotated[..., Depends(...)]`, ordinary helpers and jobs receive explicit arguments
+- Preserve each route's no-database behavior, authorization precedence and background-task scheduling
+- Read [Python and FastAPI](references/python-fastapi.md) for provider examples, lifespan state and route verification
+- Read [Rust and Axum](references/rust-axum.md) when adding database access to Rust handlers
 
-## An equivalent Rust or SQLx query already exists
+## Connection ownership
 
-- Leave it and the current backend selection unchanged unless integration is explicitly requested
-- Its existence alone does not justify switching a Prisma caller to Rust
-- Repository extraction can keep using Prisma even when the Rust query is available
+- Keep connection setup and shutdown separate from repositories, which own no pools
+- Reuse existing handles, never connect per request or disconnect from a repository operation
+- Give each pool one owner for setup and shutdown, borrowed handles never close it
+- Finish requests, jobs and queued database work before closing connections
+- Clean up resources opened before a startup failure, without a second disconnect fallback
+- Add a backend to the existing lifecycle owner and pass its handle only to repositories that need it
+- Preserve cancellation, timeout, error and transaction semantics across the native bridge
 
-## Asked to port a Prisma query to Rust or SQLx
+## Changing ownership or reloading resources
 
-- Identify the existing inputs, outputs, errors, and transaction boundary
-- Check `litellm-rust/crates/db` for an equivalent query and reuse it if its behavior matches
-- Otherwise follow `litellm-rust/crates/db/AGENTS.md`
-  - Put static SQL in `queries/` and compile-check it with SQLx
-  - Keep typed inputs and reusable row projections in `src/`
-  - Refresh SQLx query metadata after SQL changes
-- Keep Python conversion in the Python bridge, not in `litellm-rust/crates/db`
-- When caller integration is requested, expose the Rust operation through the bridge and call it from the repository
-  - A query-only port can land without switching callers
-- Verify behavior against real PostgreSQL before switching callers
-  - Check relevant null handling, conflicts, ordering, errors, and transaction semantics
-- Remove the old implementation only when no callers need it
+- Inventory initialization, reload, tests, background jobs and shutdown together
+- Move every writer when changing authoritative storage, copying a replaceable global into state is insufficient
+- A frozen repository does not make its clients immutable
+- Decide whether each request captures one resource generation or reads it again during processing
+- Build and validate replacements before publishing them, preserving the old generation on failure
+- Serialize reloads and coordinate publication with shutdown
+- Drain streams and background users before retiring old resources, retaining every generation grows resource use
+- Define multi-worker coordination separately from an in-process swap
+- Choose mechanisms such as `ArcSwap` only after defining publication, capture and retirement
 
 ## Migration context and boundaries
 
-- Long-term direction is to centralize Python database access in repositories, then replace their Prisma implementations incrementally with Rust APIs
-- Mixed Prisma and Rust implementations are expected during the transition, unrelated existing calls can stay as they are
-- Both backends may have separate pools through PgBouncer, size them together and verify pooling-mode compatibility
+- Long-term direction is to centralize Python database access in repositories, then replace their Prisma implementations with Rust APIs one repository at a time
+- Prisma and native repositories coexist during the transition, unrelated existing calls can stay as they are
+- Prisma and SQLx keep separate client pools to PgBouncer, which pools connections to PostgreSQL
+  - Size both layers together and preserve reader/writer routing and pooling-mode compatibility
 - PgBouncer does not share transactions between Prisma and SQLx, keep each atomic unit of work on one backend and transaction
 - Never retry a failed Rust write through Prisma when it may already have committed
 - Prisma owns schema migrations today, eventual SQLx migration ownership and Prisma removal are separate planned steps, not part of a query port

.agents/skills/database-access/references/python-fastapi.md

@@ -0,0 +1,101 @@
+# Python and FastAPI
+
+## Existing pattern
+
+- `UserBannerStore` in `litellm/models/user_banner.py` is the Protocol the banner routes call
+- `get_database` in `litellm/proxy/deps/database.py` reads the lifespan-owned handle from request state
+- `get_user_banner_repository` in `litellm/proxy/deps/repositories.py` builds the banner repository from that handle
+- Choose each repository's replica routing once in its provider, never at call sites
+- A handler receives `UserBannerStore | None`, never a Prisma client or a concrete repository
+- `UserBannerRepository` implements `UserBannerStore` structurally, keeping the existing Prisma table, serialization and configuration-sync behavior
+- The banner audit provider binds the legacy audit callable at request time, without performing audit I/O
+- A native repository can later implement the same Protocol without changing handler signatures, see [porting a repository to SQLx](python-repository.md#porting-a-repository-to-sqlx)
+
+## Dependency injection
+
+- Add one provider and one `Annotated` alias per repository in `litellm/proxy/deps/repositories.py`
+- Annotate the provider and alias with the feature's Protocol, not the concrete repository
+- A route depends only on the repositories it calls, so a request never builds repositories it does not use
+- Do not build repositories once at lifespan, a reconnect can replace the Prisma client they wrap
+
+```python
+class UserBannerStore(Protocol):
+    async def get_banner(self) -> UserBanner: ...
+
+    async def save_banner(self, banner: UserBanner) -> None: ...
+```
+
+```python
+async def get_user_banner_repository(database: DatabaseDep) -> UserBannerStore | None:
+    return None if database is None else UserBannerRepository(database.prisma)
+
+
+UserBannerRepositoryDep: TypeAlias = Annotated[UserBannerStore | None, Depends(get_user_banner_repository)]
+```
+
+- FastAPI caches each provider within a request, so routes and sub-dependencies share one instance
+- Ordinary helpers and jobs receive explicit arguments, never unresolved `Depends` defaults
+- A route still reading the `prisma_client` global or building a repository inline switches to the repository's `...RepositoryDep`, keeping its no-database behavior and error precedence
+
+```python
+async def get_user_banner(banners: UserBannerRepositoryDep) -> UserBanner:
+    if banners is None:
+        return UserBanner()
+    return await banners.get_banner()
+```
+
+## Lifespan state
+
+- Prisma startup owns connection setup, authentication refresh, health checks and migrations
+- `Database` holds the initialized Prisma client and the `NativeDatabase` handle, without holding fields per feature
+  - The native handle opens no pool yet, its `connect` and `close` are noops
+- The proxy lifespan builds the handle with `Database.open`, registers `Database.aclose` on its exit stack and publishes it in request state
+  - Callers only open and close `Database`, which closes Prisma and the native handle together, and `open` closes Prisma itself when the native connect fails
+- Starlette shallow-copies lifespan state into request state, which is distinct from `app.state`
+- The database module never imports a repository, legacy globals borrow its handle without owning shutdown
+- Drain requests, jobs, logging, spend and audit work before database teardown
+- The exit stack closes the handle on exit, including startup failures after the client is created
+- When adding native resources, initialize them in the serving process, respecting runtime fork restrictions
+- Await native shutdown instead of depending on Python garbage collection
+- Apply the [connection ownership rules](../SKILL.md#connection-ownership) when changing the owner
+
+## Timing and errors
+
+- FastAPI resolves dependencies before entering the handler
+- Handler exception handling cannot catch failures raised during dependency resolution
+- A universal required-database dependency can override authorization and validation behavior
+- Banner GET without a database returns the default disabled banner
+- Banner PATCH checks the admin role before returning the existing missing-database error
+- Invalid stored banner data falls back to the default, but storage failures still propagate
+- Banner writes finish before the success audit is queued as a FastAPI background task
+- Queue post-response work with `BackgroundTasks`, not bare `asyncio.create_task`, so the in-flight drain waits for it before the database closes
+- Do not move background work into an awaited provider merely to remove an import
+
+## Verification
+
+- Exercise real FastAPI handlers using `dependency_overrides` for external boundaries
+- Keep production parsing and persistence code real
+- Restore overrides after each test
+- Verify complete response bodies, status codes and OpenAPI against the base
+- Include combined failure conditions to pin error precedence
+- Check saved values, fresh revisions, audit contents and failed-write behavior
+- Check lifespan state with a running lifespan, not only a TestClient that skips startup
+- Verify providers do not connect, query, disconnect or audit eagerly
+- Use real PostgreSQL and live proxy requests when changing persistence behavior
+
+## Scope
+
+- Preserve direct Python call sites or migrate them explicitly
+- Inventory imports, state writers, side effects and error ordering before migrating a feature
+- Preserve configuration sync and serialization, and state which legacy reads remain
+- Do not migrate every route in a large module just to establish the pattern
+- Do not invent providers for resources without consumers
+
+## Framework references
+
+- [Lifespan events](https://fastapi.tiangolo.com/advanced/events/) and [Starlette lifespan state](https://starlette.dev/lifespan/)
+- [`SessionDep`-style `Annotated` aliases](https://fastapi.tiangolo.com/tutorial/sql-databases/#create-a-session-dependency), which `DatabaseDep` and `UserBannerRepositoryDep` follow
+- [Dependency composition and caching](https://fastapi.tiangolo.com/tutorial/dependencies/sub-dependencies/)
+- [Background tasks and dependencies with `yield`](https://fastapi.tiangolo.com/advanced/advanced-dependencies/#background-tasks-and-dependencies-with-yield-technical-details)
+- [Dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
+- [Testing lifespan](https://fastapi.tiangolo.com/advanced/testing-events/)

.agents/skills/database-access/references/python-repository.md

@@ -0,0 +1,119 @@
+# Python repositories
+
+## Adding a query
+
+Use this when a feature needs a read or write that no repository offers yet, including one more operation on an existing repository
+
+### Before starting
+
+- Search `litellm/repositories` for a repository that already owns the table, and extend it instead of adding a second one
+- Check `litellm/models/<feature>.py` for an existing Protocol and domain types
+- Decide the route's behavior when no database is configured, and where that check sits relative to authorization and validation
+
+### Steps
+
+1. Domain types and Protocol in `litellm/models/<feature>.py`
+   - The Protocol takes and returns domain types, never Prisma models, raw rows or serialized payloads
+   - Add the Protocol together with the provider, not for repositories nothing injects
+2. Repository in `litellm/repositories/<feature>_repository.py`
+   - Implement the operation with Prisma, a Rust port is not required
+   - Compose the Prisma client and type the tables it touches with `TableActions` from `prisma_protocols.py`, never inherit `PrismaTableRepository`
+   - Parse and serialize stored values here, so callers only see domain types
+   - Never import FastAPI, connect, disconnect or choose replicas per call
+3. Provider and `Annotated` alias in `litellm/proxy/deps/repositories.py`, see [dependency injection](python-fastapi.md#dependency-injection)
+4. The route takes `<Feature>RepositoryDep`, handles `None` with the behavior decided above, and calls the operation
+5. Jobs and helpers that need the operation receive the repository as an explicit argument, built by their caller from the handle it already holds
+
+### Done when
+
+- Handler tests drive the real route with `dependency_overrides` and the real repository code, see [verification](python-fastapi.md#verification)
+- Tests cover the no-database path, error precedence, and what was actually stored
+- Persistence changes are checked against real PostgreSQL through a live proxy
+
+### Out of scope
+
+- Porting the query to Rust, see [porting a repository to SQLx](#porting-a-repository-to-sqlx)
+- Moving unrelated direct Prisma calls in the same module
+
+## Moving existing access into a repository
+
+Use this when code calls Prisma directly, through `prisma_client.db.<table>` or a helper, and the calls should live behind a repository. The backend stays Prisma
+
+### Before starting
+
+- List every call being moved, its callers, and the transaction or batch it runs in
+- List the module's imports, global state writers, side effects such as audit and config sync, and error ordering
+- Record current return values, raised errors and response bodies, these are what the change must preserve
+
+### Steps
+
+1. Move the existing Prisma code into the relevant repository in `litellm/repositories` without rewriting the queries
+2. Add domain types and a Protocol in `litellm/models/<feature>.py` when a route will inject the repository
+3. Replace each direct call with the repository operation, through a provider for routes and explicit arguments for jobs and helpers, see [Python and FastAPI](python-fastapi.md)
+4. Calls sharing a transaction move into the same repository or unit of work in one change
+5. State which legacy reads remain and why, when the change does not move all of them
+
+### Done when
+
+- Focused regression tests pin the preserved return values, errors and transaction behavior
+- Response bodies, status codes and OpenAPI match the base
+- Configuration sync and serialization behave the same, including invalid stored data
+
+### Out of scope
+
+- Switching the backend, even when an equivalent Rust query exists
+- Converting other routes in the module just to establish the pattern
+- Cleanups unrelated to the moved calls
+
+## Porting a repository to SQLx
+
+Status: TBD. No repository has been ported yet. `database.native` is a `NativeDatabase` from `crates/python-bridge/src/db.rs` whose `connect` and `close` are noops that open no pool, and the PyO3 repository surface does not exist. A real pool takes typed writer/reader settings and a token provider, never the Prisma `DATABASE_URL`, which carries Prisma-only params and rotating IAM tokens. The constraints below are settled, the steps are a sketch to revise after the first port lands
+
+Use this only when a port is explicitly requested. Repository extraction does not require one
+
+### When not to port
+
+- An equivalent Rust or SQLx query existing is not a reason to switch a Prisma caller
+- Leave existing backend selection unchanged unless integration is explicitly requested
+- Never port part of a repository, it is either all Prisma or all native
+
+### Before starting
+
+- Identify the repository's inputs, outputs, errors and transaction boundary
+- List repositories that share a transaction with it, they move together
+- Check `litellm-rust/crates/db` for an equivalent query and reuse it when its behavior matches
+
+### Steps
+
+1. Rust queries, following `litellm-rust/crates/db/AGENTS.md`
+   - Static SQL in `queries/`, compile-checked with SQLx, typed inputs and row projections in `src/`
+   - Dynamic filters and sort keys become closed Rust enums that select static SQL, every value is bound
+   - Refresh SQLx query metadata after SQL changes
+2. PyO3 surface exposing typed operations, never SQL text, table names or column names from Python. TBD: where it lives and how the handle reaches it
+3. Native repository implementing the same Protocol in `litellm/repositories`
+   - Converts typed PyO3 results to domain types, preserving defaults, invalid-data handling and storage errors
+   - Python conversion stays in the bridge, not in `crates/db`
+4. Switch the provider in `litellm/proxy/deps/repositories.py`, the Protocol and routes stay unchanged
+
+```diff
+ async def get_user_banner_repository(database: DatabaseDep) -> UserBannerStore | None:
+-    return None if database is None else UserBannerRepository(database.prisma)
++    return None if database is None else NativeUserBannerRepository(database.native)
+```
+
+5. Delete the Prisma repository in the same change and switch every caller, including jobs and helpers
+
+### Done when
+
+- Behavior is verified against real PostgreSQL, covering null handling, conflicts, ordering, errors and transaction semantics
+- `DATABASE_URL=... cargo test -p litellm-db --locked -- --include-ignored` passes against a Prisma-migrated database
+- Live proxy requests show unchanged responses. TBD: a Python-side real-PostgreSQL test harness
+
+### Rollback
+
+- Roll back the release, never retry an uncertain native write through Prisma
+
+### Out of scope
+
+- SQLx owning schema migrations, and removing Prisma
+- Converting unrelated routes to dependency injection

.agents/skills/database-access/references/rust-axum.md

@@ -0,0 +1,4 @@
+# Rust and Axum
+
+- This should use same `crates/db` what is exposed to Python world.
+- Details are TBD.

.github/workflows/test-rust-db.yml

@@ -42,6 +42,11 @@ concurrency:
 
 env:
   CARGO_TERM_COLOR: always
+  RUSTC_WRAPPER: sccache
+  SCCACHE_GHA_ENABLED: "true"
+  SCCACHE_GHA_VERSION: litellm-ci-v1
+  SCCACHE_GHA_RW_MODE: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
+  CARGO_INCREMENTAL: "0"
 
 jobs:
   db-queries:
@@ -87,6 +92,11 @@ jobs:
           workspaces: litellm-rust
           cache-on-failure: true
 
+      - name: Set up sccache
+        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
+        with:
+          version: "v0.18.0"
+
       - name: Install Python dependencies
         working-directory: .
         run: uv sync --frozen --extra extra-proxy

.github/workflows/test-rust.yml

@@ -78,6 +78,11 @@ concurrency:
 
 env:
   CARGO_TERM_COLOR: always
+  RUSTC_WRAPPER: sccache
+  SCCACHE_GHA_ENABLED: "true"
+  SCCACHE_GHA_VERSION: litellm-ci-v1
+  SCCACHE_GHA_RW_MODE: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
+  CARGO_INCREMENTAL: "0"
 
 jobs:
   rust-lint:
@@ -101,6 +106,11 @@ jobs:
           cache-on-failure: true
           save-if: ${{ github.ref == 'refs/heads/main' }}
 
+      - name: Set up sccache
+        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
+        with:
+          version: "v0.18.0"
+
       - run: cargo clippy --workspace --all-targets --locked -- -D warnings
 
   rust-test:
@@ -143,6 +153,11 @@ jobs:
           cache-on-failure: true
           save-if: ${{ github.ref == 'refs/heads/main' }}
 
+      - name: Set up sccache
+        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
+        with:
+          version: "v0.18.0"
+
       - name: Check generated trace contracts
         working-directory: .
         run: uv run scripts/generate_trace_types.py --check
@@ -189,6 +204,11 @@ jobs:
           cache-on-failure: true
           save-if: ${{ github.ref == 'refs/heads/main' }}
 
+      - name: Set up sccache
+        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
+        with:
+          version: "v0.18.0"
+
       - run: uv build --wheel --out-dir dist
 
       - run: python .github/scripts/verify_linux_native_wheel.py dist/*.whl

.github/workflows/test-unit.yml

@@ -73,9 +73,21 @@ jobs:
         timeout-minutes: 5
         uses: ./.github/actions/cache-cargo-build
 
+      - name: Set up sccache
+        if: steps.changes.outputs.decision != 'skip' && steps.rust-bridge-cache.outputs.cache-hit != 'true'
+        uses: mozilla-actions/sccache-action@d651010b8da762cde178750d8eda7b5febfe147a
+        with:
+          version: "v0.18.0"
+
       - name: Build editable Rust bridge
         if: steps.changes.outputs.decision != 'skip' && steps.rust-bridge-cache.outputs.cache-hit != 'true'
         timeout-minutes: 15
+        env:
+          RUSTC_WRAPPER: sccache
+          SCCACHE_GHA_ENABLED: "true"
+          SCCACHE_GHA_VERSION: litellm-ci-v1
+          SCCACHE_GHA_RW_MODE: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'READ_WRITE' || 'READ_ONLY' }}
+          CARGO_INCREMENTAL: "0"
         run: |
           mkdir -p rust-bridge-dist
           uv run --no-project --with maturin==1.15.0 python -c 'import maturin; maturin.build_editable("rust-bridge-dist")'
@@ -565,6 +577,7 @@ jobs:
               tests/unit/proxy/rerank_endpoints
               tests/unit/proxy/realtime_endpoints
               tests/unit/proxy/ui_crud_endpoints
+              tests/unit/proxy/deps
               tests/unit/proxy/config_resolvers
               tests/unit/proxy/utils
             python-version: "3.12"

AGENTS.md

@@ -90,6 +90,8 @@ Do not put names of customers or customer company names in code, PR descriptions
 
 CI supply-chain safety: Never pipe a remote script into a shell (`curl ... | bash`, `wget ... | sh`). Download the artifact to a file, verify its SHA-256 checksum, then install. Pin every external tool to a specific version with a full URL (not `latest` or `stable`). Verify checksums for all downloaded binaries, using the provider's official `.sha256` / `.sha256sum` sidecar when available. These rules apply to every download in CI
 
+For database operations, connection ownership and proxy dependency wiring, follow [database-access](.agents/skills/database-access/SKILL.md)
+
 Prisma migrations apply synchronously at proxy boot, before it serves traffic, so a migration must only change schema, never rewrite rows. No `UPDATE`, `DELETE` or `MERGE`, and no `INSERT ... SELECT`: on a spend-log-sized table any of those is minutes of downtime plus a doubled heap that plain autovacuum won't give back. `tests/code_coverage_tests/check_migrations_no_data_rewrites.py` enforces this. When a rewrite is genuinely bounded and has to ship inside the migration, mark the statement `-- data-migration-ok: <what bounds it>`
 
 Follow these coding conventions for new/updated code (a three-line fix in a legacy file shouldn't trigger huge drive-by refactors):

CONTRIBUTING.md

@@ -276,6 +276,8 @@ When the provider reports its own usage or spend numbers, compare the proxy's nu
 
 Provider work (a new provider, a new parameter, a new usage or cost field) is the most common kind of contribution, and it is where PRs get reworked most often. Follow these rules
 
+**Preserve zero prices.** In [`model_prices_and_context_window.json`](model_prices_and_context_window.json), an explicit `0.0` means that the priced operation is free. A missing field or `null` means no rate is specified, so the relevant calculator may use a fallback. For example, when no batch input rate is configured, batch cache writes use `cache_creation_input_token_cost` if set, otherwise the regular input rate, then apply the batch discount. A zero ordinary input rate does not make separately priced cache reads or writes free. Test `rate is None` before substituting a fallback, since `row.get("cache_creation_input_token_cost") or fallback` would discard a valid zero. Speech keeps character pricing ahead of token pricing even when the character rate is zero. If neither rate is configured, speech cost calculation raises an error. Its model lookup preserves unset rates instead of treating compatibility defaults as explicit zero prices. Perplexity keeps a configured legacy search rate ahead of the current search rate, including zero. When registering custom prices, `None` leaves an existing rate unchanged and `0.0` overrides it
+
 **Find the existing machinery before you add plumbing.** LiteLLM already has generic paths that move data between the endpoints, the provider transformations, and cost tracking. A second, hand-rolled path for the same data is a common reason a provider PR gets rewritten before merge. [`ARCHITECTURE.md`](ARCHITECTURE.md) shows where the layers live, and the decision records in [`docs/adrs/`](docs/adrs/AGENTS.md) explain why they are shaped that way. For example, [the ADR on provider usage fields](docs/adrs/adr_001_provider_usage_fields_ride_as_usage_extras.md) covers how provider-specific usage fields reach cost tracking. If you cannot find a mechanism for what you need, say so in the PR or ask in [#pr-review on Slack](https://join.slack.com/t/litellmossslack/shared_invite/zt-3o7nkuyfr-p_kbNJj8taRfXGgQI1~YyA) before you build one
 
 **Bill a new provider usage field from the usage object.** Leave the field on the usage object as an extra and read it in that provider's `cost_calculator.py`. A provider without one gets a new `cost_calculator.py` and an arm for it in `cost_per_token` (`litellm/cost_calculator.py`), which picks the calculator by provider. If the provider's transform builds `Usage` from named fields, as the Anthropic, Gemini, and Bedrock Converse ones do, pass the new field there too. Streamed requests rebuild usage in `ChunkProcessor.calculate_usage` (`litellm/litellm_core_utils/streaming_chunk_builder_utils.py`), which does not copy extras yet the way the Responses-to-chat bridge does (minus a short exclusion list), so until it does, bill the field on streams by naming it there, as Anthropic's `inference_geo` is. A provider that reports its own charge for the request, as OpenRouter does in `usage.cost`, is billed at that charge from `_hidden_params` before any calculator runs, unless the deployment sets its own prices. When the field counts a built-in tool such as web search, add the provider to `get_cost_for_web_search_request` in `litellm/llms/__init__.py`, and add a gate to `litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py` only when no existing gate fires. Then show the field's cost in your proof on every endpoint it reaches, streaming included

enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py

@@ -10,6 +10,7 @@ from dataclasses import replace as dataclasses_replace
 from datetime import datetime, timedelta
 from types import MappingProxyType
 from typing import TYPE_CHECKING, Final, List, Literal, Optional, Protocol, Tuple, cast
+from pydantic import TypeAdapter, ValidationError
 
 from litellm_enterprise.proxy.common_utils.managed_object_attribution import (
     ManagedObjectCreatorAttribution,
@@ -50,6 +51,7 @@ def _error_names_resource(error: Exception, resource_id: str) -> bool:
 CHECK_BATCH_COST_USER_AGENT = "LiteLLM Proxy/CheckBatchCost"
 RECENT_BATCH_POLL_WINDOW: Final = timedelta(hours=48)
 GIVE_UP_POLLING_AFTER: Final = timedelta(days=2 * MANAGED_OBJECT_STALENESS_CUTOFF_DAYS)
+_FILE_OBJECT_ADAPTER: Final[TypeAdapter[dict[str, object]]] = TypeAdapter(dict[str, object])
 
 
 PROVIDER_TERMINAL_BATCH_STATUSES: Final[Tuple[str, ...]] = (
@@ -550,6 +552,10 @@ class CheckBatchCost:
         llm_provider: str,
         bare_model_name: str,
     ) -> Optional[Tuple[str, str]]:
+        stored_model_id: Final = self._get_stored_model_id(job)
+        if stored_model_id is not None and self._batch_deployment_exists(stored_model_id):
+            return stored_model_id, job.unified_object_id
+
         deployment_id = self._get_deployment_id_for_bare_model(bare_model_name, llm_provider)
         if deployment_id is None:
             verbose_proxy_logger.info(
@@ -662,23 +668,36 @@ class CheckBatchCost:
         )
 
     @staticmethod
-    def _get_input_file_id(job: "ManagedObjectRow") -> Optional[str]:
-        from litellm.types.utils import LiteLLMBatch
-
+    def _parse_file_object(job: "ManagedObjectRow") -> Optional[dict[str, object]]:
         file_object: Final = job.file_object
         try:
-            parsed_file_object: Final[object] = (
+            return _FILE_OBJECT_ADAPTER.validate_python(
                 json.loads(file_object) if isinstance(file_object, str) else file_object
             )
-        except (json.JSONDecodeError, ValueError):
+        except (json.JSONDecodeError, ValueError, ValidationError):
             return None
-        if not isinstance(parsed_file_object, dict):
+
+    @classmethod
+    def _get_input_file_id(cls, job: "ManagedObjectRow") -> Optional[str]:
+        from litellm.types.utils import LiteLLMBatch
+
+        parsed_file_object: Final = cls._parse_file_object(job)
+        if parsed_file_object is None:
             return None
         try:
             return LiteLLMBatch.model_validate(parsed_file_object).input_file_id
         except Exception:
             return None
 
+    @classmethod
+    def _get_stored_model_id(cls, job: "ManagedObjectRow") -> Optional[str]:
+        """The deployment that created the batch, written into file_object by the managed
+        files hook at create time. Older rows have no such key and fall back to matching
+        the bare model name against the router."""
+        parsed_file_object: Final = cls._parse_file_object(job)
+        model_id: Final = parsed_file_object.get("model_id") if parsed_file_object else None
+        return model_id if isinstance(model_id, str) and model_id else None
+
     async def _track_completed_batch_cost(
         self,
         job: "ManagedObjectRow",

enterprise/litellm_enterprise/proxy/hooks/managed_files.py

@@ -21,6 +21,7 @@ from typing import (
     TypedDict,
     Union,
     cast,
+    runtime_checkable,
 )
 from uuid import NAMESPACE_URL, uuid5
 
@@ -374,6 +375,12 @@ class _ManagedResourcePrismaClient(Protocol):
     def db(self) -> _ManagedResourceDatabase: ...
 
 
+@runtime_checkable
+class _WriterRoutedDatabase(Protocol):
+    @property
+    def writer(self) -> _ManagedResourceDatabase: ...
+
+
 class _SchedulerWithJobLookup(Protocol):
     def get_job(self, job_id: str) -> object: ...
 
@@ -479,6 +486,11 @@ def _managed_object_table(prisma_client: _ManagedResourcePrismaClient) -> _Manag
     return prisma_client.db.litellm_managedobjecttable
 
 
+def _writer_database(db: _ManagedResourceDatabase) -> _ManagedResourceDatabase:
+    """Reads that feed a write must see the create that just happened; a lagging read replica may not"""
+    return db.writer if isinstance(db, _WriterRoutedDatabase) else db
+
+
 _JSON_VALUE_ADAPTER: Final[TypeAdapter[JsonValue]] = TypeAdapter(JsonValue)
 _POSTGRES_UNSTORABLE_CHARACTER: Final = "\x00"
 
@@ -512,6 +524,25 @@ def _without_model_output(
     )
 
 
+def _with_model_id(
+    file_object: LiteLLMBatch | LiteLLMFineTuningJob | ResponsesAPIResponse,
+    model_id: str | None,
+) -> LiteLLMBatch | LiteLLMFineTuningJob | ResponsesAPIResponse:
+    return file_object if model_id is None else file_object.model_copy(update={"model_id": model_id})
+
+
+async def _stored_model_id(managed_object_table: _ManagedObjectTableActions, unified_object_id: str) -> str | None:
+    row: Final = await managed_object_table.find_first(where={"unified_object_id": unified_object_id})
+    if row is None:
+        return None
+    try:
+        parsed: Final = _JSON_VALUE_ADAPTER.validate_python(_decode_json_blob(row.file_object))
+    except (json.JSONDecodeError, ValidationError):
+        return None
+    model_id: Final = parsed.get("model_id") if isinstance(parsed, dict) else None
+    return model_id if isinstance(model_id, str) and model_id else None
+
+
 def _storage_metadata_of(file_object: OpenAIFileObject | None) -> Mapping[str, str]:
     hidden_params: Final = cast(  # cast-ok: _hidden_params is an untyped attribute the upload path sets
         "Mapping[str, object]", getattr(file_object, "_hidden_params", None) or {}
@@ -697,9 +728,17 @@ class _PROXY_LiteLLMManagedFiles(CustomLogger, BaseFileEndpoints):
         persist_attribution: bool = False,
         create_if_missing: bool = True,
         batch_processed: bool = False,
+        creating_model_id: str | None = None,
     ) -> None:
         """Persist a managed object row, caching it and upserting it in the DB.
 
+        creating_model_id is the router deployment that created the object. It is kept
+        inside the stored file_object JSON as `model_id`, because the raw provider id an
+        unmanaged batch stores as unified_object_id says nothing about which deployment
+        made it, and the cost poller needs that deployment to read the result back. A
+        later observe of the row (create_if_missing=False) keeps the stored value rather
+        than overwriting it with whichever deployment served the retrieve.
+
         persist_attribution is set only by the batch and background response creates,
         the callers that can speak for the creator; it gates the api_key, request_tags, and,
         for a response, end_user_id columns that the cost poll jobs bill against, so a later poll or
@@ -720,7 +759,14 @@ class _PROXY_LiteLLMManagedFiles(CustomLogger, BaseFileEndpoints):
         It is written only in the upsert create branch.
         """
         verbose_logger.info(f"Storing LiteLLM Managed {file_purpose} object with id={unified_object_id} in cache")
-        stored_object: Final = _without_model_output(file_object)
+        managed_database: Final = _managed_database(self.prisma_client)
+        managed_object_table: Final = managed_database.litellm_managedobjecttable
+        preserved_model_id: Final = (
+            creating_model_id
+            if creating_model_id is not None or create_if_missing
+            else await _stored_model_id(_writer_database(managed_database).litellm_managedobjecttable, unified_object_id)
+        )
+        stored_object: Final = _with_model_id(_without_model_output(file_object), preserved_model_id)
         litellm_managed_object = LiteLLM_ManagedObjectTable(
             unified_object_id=unified_object_id,
             model_object_id=model_object_id,
@@ -752,7 +798,6 @@ class _PROXY_LiteLLMManagedFiles(CustomLogger, BaseFileEndpoints):
             "status": file_object.status,
             "updated_by": user_api_key_dict.user_id,
         }
-        managed_object_table: Final = _managed_object_table(self.prisma_client)
         if not create_if_missing:
             await managed_object_table.update_many(
                 where={"unified_object_id": unified_object_id},
@@ -2170,6 +2215,7 @@ class _PROXY_LiteLLMManagedFiles(CustomLogger, BaseFileEndpoints):
                 request_tags=request_tags_from_metadata(request_metadata if isinstance(request_metadata, dict) else {}),
                 persist_attribution=is_batch_create,
                 create_if_missing=is_batch_create,
+                creating_model_id=model_id if is_batch_create else None,
             )
             if not is_batch_create:
                 await self.store_batch_output_file_ownership(

litellm-rust/.agents/skills/rust-redaction/SKILL.md

@@ -0,0 +1,43 @@
+---
+name: rust-redaction
+description: Keep secrets out of Debug output in litellm-rust with SecretValue and veil. Use when a struct or enum holds a password, token, key, credential URL or credential provider, when writing or reviewing a Debug impl, or when a type with secrets gets logged or traced
+metadata:
+  internal: true
+---
+
+# Rust redaction
+
+Every secret in a Debug output is hidden the same way, so a reviewer can tell at a glance that a field is covered and a new field cannot slip through a hand-written impl
+
+## Where SecretValue lives
+
+`SecretValue` lives in `crates/secret-value` and is imported as `litellm_secret_value::SecretValue` everywhere. No crate re-exports it, so there is one import path. Depend on `litellm-secret-value` directly. Do not pull in `litellm-auth-types` or `litellm-auth` just to get it
+
+`crates/secret-value` holds only the type. Credential acquisition (`TokenProvider`, `CredentialPlan`, provider param tables) stays in `litellm-auth-types`, and generic clients such as `litellm-redis-client` must not depend on it
+
+## Rules
+
+- Never hand-write `impl Debug` to hide a value, and never invent markers like `"<redacted>"`, `"<configured>"` or `.map(|_| "...")`. A hand-written impl silently prints any field added later
+- A secret that crosses a crate boundary or lives in config is typed as `SecretValue`. It redacts itself, compares in constant time and only gives up the plaintext through `expose()`. A type whose only secrets are `SecretValue` fields derives plain `Debug`
+  ```rust
+  #[derive(Clone, Debug, PartialEq)]
+  pub struct HostEndpoint {
+      pub host: String,
+      pub port: u16,
+      pub username: Option<SecretValue>,
+      pub password: Option<SecretValue>,
+      pub tls: bool,
+      pub db: i64,
+  }
+  ```
+- Keep a secret as `SecretValue` until the last conversion into the third-party type (redis-rs, reqwest, an SDK) and call `expose()` there. Calling `expose()` early spreads plaintext `String`s through our own types
+- A secret that has to stay a raw type, because a serde wire shape or a third-party struct owns the field, is covered by deriving `veil::Redact` on the holder and marking the field `#[redact(with = "[REDACTED]")]`. `Redact` replaces `Debug`, so do not derive both
+- A URL that can carry userinfo is a secret as a whole. Do not parse it to show the host. Keep it as a `SecretValue`
+- A bag of unparsed settings that may hold credentials (`additional_fields`, raw kwargs) is redacted as a whole with `#[redact(with = "[REDACTED]")]`
+- An opaque handle over `Arc<dyn Trait>` whose trait has no `Debug` gets a newtype whose `Debug` prints only the type name with `finish_non_exhaustive()`. That impl shows nothing, so it is not a redaction impl and the first rule does not apply. A type holding such a handle can derive `Debug`
+- Never put a secret in a tracing field or an error message. Log the type with `?value`, or log only non-secret fields. See the rust-tracing and rust-error skills
+
+## Tests
+
+- One test per type that holds a secret, built the way production builds it, asserting the plaintext is absent from `format!("{value:?}")` and `[REDACTED]` is present
+- Use a distinctive plaintext (`"credential-value"`) so the absence check cannot pass by accident

litellm-rust/AGENTS.md

@@ -1,5 +1,7 @@
 # Rust workspace rules
 
+For keeping secrets out of Debug output, follow [.agents/skills/rust-redaction/SKILL.md](.agents/skills/rust-redaction/SKILL.md)
+
 For diagnostic tracing changes, follow [.agents/skills/rust-tracing/SKILL.md](.agents/skills/rust-tracing/SKILL.md)
 
 For string-valued enums and their Serde conversions, follow [.agents/skills/rust-string-enums/SKILL.md](.agents/skills/rust-string-enums/SKILL.md)

litellm-rust/Cargo.lock

@@ -3380,6 +3380,7 @@ dependencies = [
  "litellm-auth-azure",
  "litellm-auth-gcp",
  "litellm-auth-types",
+ "litellm-secret-value",
 ]
 
 [[package]]
@@ -3416,6 +3417,7 @@ dependencies = [
  "azure_identity",
  "litellm-auth-types",
  "litellm-error",
+ "litellm-secret-value",
  "moka",
  "rstest",
  "serde_json",
@@ -3435,6 +3437,7 @@ dependencies = [
  "http 1.4.2",
  "litellm-auth-types",
  "litellm-error",
+ "litellm-secret-value",
  "moka",
  "rstest",
  "serde",
@@ -3450,10 +3453,10 @@ name = "litellm-auth-types"
 version = "0.1.0"
 dependencies = [
  "litellm-error",
+ "litellm-secret-value",
  "rstest",
  "serde",
  "serde_json",
- "subtle",
  "thiserror 2.0.19",
  "tokio",
  "veil",
@@ -3523,6 +3526,7 @@ dependencies = [
  "litellm-cache-testing",
  "litellm-error",
  "litellm-http",
+ "litellm-secret-value",
  "percent-encoding",
  "reqwest",
  "rstest",
@@ -3668,10 +3672,10 @@ dependencies = [
 name = "litellm-callbacks-legacy-python"
 version = "0.1.0"
 dependencies = [
- "litellm-auth",
  "litellm-error",
  "litellm-host",
  "litellm-host-python",
+ "litellm-secret-value",
  "proptest",
  "pyo3",
  "rstest",
@@ -3692,9 +3696,9 @@ dependencies = [
 name = "litellm-config"
 version = "0.1.0"
 dependencies = [
- "litellm-auth-types",
  "litellm-error",
  "litellm-router-types",
+ "litellm-secret-value",
  "rstest",
  "serde",
  "serde_json",
@@ -3804,7 +3808,6 @@ dependencies = [
  "envy",
  "futures-util",
  "http-body-util",
- "litellm-auth-types",
  "litellm-circuit-breaker",
  "litellm-config",
  "litellm-error",
@@ -3817,6 +3820,7 @@ dependencies = [
  "litellm-llms",
  "litellm-redis-client",
  "litellm-redis-testing",
+ "litellm-secret-value",
  "litellm-secrets",
  "litellm-tracing",
  "rstest",
@@ -3839,9 +3843,9 @@ dependencies = [
  "axum",
  "axum-login",
  "futures-util",
- "litellm-auth-types",
  "litellm-config",
  "litellm-error",
+ "litellm-secret-value",
  "litellm-secrets",
  "rstest",
  "serde",
@@ -3879,6 +3883,7 @@ dependencies = [
  "litellm-llms",
  "litellm-llms-types",
  "litellm-router",
+ "litellm-secret-value",
  "litellm-secrets",
  "rstest",
  "serde",
@@ -3895,9 +3900,9 @@ name = "litellm-gateway-management"
 version = "0.1.0"
 dependencies = [
  "base64 0.22.1",
- "litellm-auth-types",
  "litellm-error",
  "litellm-gateway-auth",
+ "litellm-secret-value",
  "rand 0.8.7",
  "rstest",
  "thiserror 2.0.19",
@@ -3912,9 +3917,9 @@ dependencies = [
  "base64 0.22.1",
  "futures-util",
  "http 1.4.2",
- "litellm-auth-types",
  "litellm-config",
  "litellm-error",
+ "litellm-secret-value",
  "litellm-secrets",
  "moka",
  "rmcp",
@@ -3942,9 +3947,9 @@ dependencies = [
  "futures-util",
  "governor",
  "jsonwebtoken",
- "litellm-auth-types",
  "litellm-error",
  "litellm-gateway-auth",
+ "litellm-secret-value",
  "rand 0.8.7",
  "rstest",
  "serde",
@@ -3964,8 +3969,8 @@ name = "litellm-host"
 version = "0.1.0"
 dependencies = [
  "futures-util",
- "litellm-auth",
  "litellm-coroutine",
+ "litellm-secret-value",
  "rstest",
  "serde_json",
  "thiserror 2.0.19",
@@ -4094,6 +4099,7 @@ dependencies = [
  "litellm-inference-testing",
  "litellm-llms",
  "litellm-llms-types",
+ "litellm-secret-value",
  "litellm-secrets",
  "litellm-tracing",
  "rstest",
@@ -4124,6 +4130,7 @@ dependencies = [
  "litellm-llms",
  "litellm-llms-types",
  "litellm-router-types",
+ "litellm-secret-value",
  "litellm-secrets",
  "litellm-tracing",
  "reqwest",
@@ -4154,6 +4161,7 @@ dependencies = [
  "litellm-inference-testing",
  "litellm-llms",
  "litellm-llms-types",
+ "litellm-secret-value",
  "litellm-secrets",
  "litellm-tracing",
  "mime_guess",
@@ -4188,6 +4196,7 @@ dependencies = [
  "litellm-inference-testing",
  "litellm-llms",
  "litellm-llms-types",
+ "litellm-secret-value",
  "litellm-secrets",
  "litellm-tracing",
  "reqwest",
@@ -4207,6 +4216,7 @@ dependencies = [
  "litellm-auth",
  "litellm-http",
  "litellm-inference",
+ "litellm-secret-value",
  "litellm-secrets",
  "tokio",
 ]
@@ -4254,6 +4264,7 @@ dependencies = [
  "litellm-llms-types",
  "litellm-python-compat",
  "litellm-router-types",
+ "litellm-secret-value",
  "litellm-secrets",
  "reqwest",
  "rstest",
@@ -4347,6 +4358,7 @@ dependencies = [
  "litellm-llms-types",
  "litellm-model-catalog",
  "litellm-router-types",
+ "litellm-secret-value",
  "litellm-secrets",
  "litellm-secrets-aws",
  "litellm-secrets-types",
@@ -4396,6 +4408,7 @@ dependencies = [
  "futures-util",
  "litellm-circuit-breaker",
  "litellm-redis-testing",
+ "litellm-secret-value",
  "redis",
  "rstest",
  "thiserror 2.0.19",
@@ -4432,12 +4445,23 @@ name = "litellm-router-types"
 version = "0.1.0"
 dependencies = [
  "litellm-auth-types",
+ "litellm-secret-value",
  "rstest",
  "serde",
  "serde_json",
  "serde_with",
 ]
 
+[[package]]
+name = "litellm-secret-value"
+version = "0.1.0"
+dependencies = [
+ "rstest",
+ "serde",
+ "subtle",
+ "veil",
+]
+
 [[package]]
 name = "litellm-secrets"
 version = "0.1.0"
@@ -4451,6 +4475,7 @@ dependencies = [
  "litellm-error",
  "litellm-http",
  "litellm-python-compat",
+ "litellm-secret-value",
  "litellm-secrets-aws",
  "litellm-secrets-azure",
  "litellm-secrets-cyberark",
@@ -4479,6 +4504,7 @@ dependencies = [
  "base64 0.22.1",
  "litellm-auth-aws",
  "litellm-core-utils",
+ "litellm-secret-value",
  "litellm-secrets-types",
  "litellm-tracing",
  "rstest",
@@ -4498,6 +4524,7 @@ dependencies = [
  "litellm-auth-types",
  "litellm-core-utils",
  "litellm-http",
+ "litellm-secret-value",
  "litellm-secrets-types",
  "percent-encoding",
  "reqwest",
@@ -4519,6 +4546,7 @@ dependencies = [
  "base64 0.22.1",
  "litellm-core-utils",
  "litellm-http",
+ "litellm-secret-value",
  "litellm-secrets-types",
  "litellm-tracing",
  "moka",
@@ -4549,6 +4577,7 @@ dependencies = [
  "litellm-auth-types",
  "litellm-core-utils",
  "litellm-http",
+ "litellm-secret-value",
  "litellm-secrets-types",
  "moka",
  "percent-encoding",
@@ -4570,6 +4599,7 @@ dependencies = [
  "litellm-core-utils",
  "litellm-error",
  "litellm-http",
+ "litellm-secret-value",
  "litellm-secrets-types",
  "reqwest",
  "rstest",
@@ -4589,7 +4619,7 @@ dependencies = [
 name = "litellm-secrets-types"
 version = "0.1.0"
 dependencies = [
- "litellm-auth-types",
+ "litellm-secret-value",
  "moka",
  "rstest",
  "serde",

litellm-rust/Cargo.toml

@@ -37,6 +37,7 @@ litellm-callbacks-legacy-python = { path = "crates/callbacks-legacy-python" }
 litellm-framer = { path = "crates/framer" }
 litellm-auth = { path = "crates/auth" }
 litellm-auth-types = { path = "crates/auth-types" }
+litellm-secret-value = { path = "crates/secret-value" }
 litellm-auth-aws = { path = "crates/auth-aws" }
 litellm-auth-azure = { path = "crates/auth-azure" }
 litellm-auth-gcp = { path = "crates/auth-gcp" }

litellm-rust/crates/auth-azure/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-auth-types.workspace = true
 
 moka.workspace = true

litellm-rust/crates/auth-azure/src/native.rs

@@ -13,7 +13,8 @@ use azure_identity::{
 use sha2::{Digest, Sha256};
 
 use litellm_auth_types::{Error, ErrorDetail};
-use litellm_auth_types::{InputSource, ResolvedCredential, SecretValue, Sourced};
+use litellm_auth_types::{InputSource, ResolvedCredential, Sourced};
+use litellm_secret_value::SecretValue;
 
 use super::credential_provider_cache::{
     AzureCredentialProviderCache, AzureCredentialProviderCacheKey,
@@ -498,7 +499,8 @@ mod tests {
     use azure_core::{Bytes, Result};
 
     use super::{NativeAzureRequest, NativeAzureTokenAcquirer, ValidatedAzureRequest};
-    use litellm_auth_types::{InputSource, SecretValue, Sourced};
+    use litellm_auth_types::{InputSource, Sourced};
+    use litellm_secret_value::SecretValue;
 
     fn deployment<T>(value: T) -> Sourced<T> {
         Sourced::new(value, InputSource::Deployment)

litellm-rust/crates/auth-azure/src/resolve.rs

@@ -1,7 +1,9 @@
 use litellm_auth_types::{
-    CredentialFileRef, CredentialLookup, CredentialRef, InputSource, ResolvedCredential,
-    SecretValue, Sourced, TokenProviderHandle,
+    CredentialFileRef, CredentialLookup, CredentialRef, InputSource, ResolvedCredential, Sourced,
+    TokenProviderHandle,
 };
+use litellm_secret_value::SecretValue;
+
 use litellm_auth_types::{Error, ErrorDetail};
 use std::future::Future;
 use std::pin::Pin;
@@ -500,8 +502,9 @@ mod tests {
     use litellm_auth_types::ResolvedCredential;
     use litellm_auth_types::{
         CredentialFileRef, CredentialLookup, CredentialLookupFuture, CredentialRef,
-        CredentialResolver, CredentialResolverHandle, InputSource, SecretValue, Sourced,
+        CredentialResolver, CredentialResolverHandle, InputSource, Sourced,
     };
+    use litellm_secret_value::SecretValue;
 
     #[derive(Debug)]
     struct FileResolver;

litellm-rust/crates/auth-azure/src/types.rs

@@ -1,8 +1,10 @@
 use std::collections::BTreeMap;
 
 use litellm_auth_types::{
-    CredentialResolverHandle, Error, InputSource, SecretValue, Sourced, TokenProviderHandle,
+    CredentialResolverHandle, Error, InputSource, Sourced, TokenProviderHandle,
 };
+use litellm_secret_value::SecretValue;
+
 use serde_json::{Map, Value};
 use strum::EnumString;
 

litellm-rust/crates/auth-gcp/Cargo.toml

@@ -9,6 +9,7 @@ repository.workspace = true
 google-sdk = ["dep:google-cloud-auth", "dep:http"]
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-auth-types.workspace = true
 
 base64.workspace = true

litellm-rust/crates/auth-gcp/src/auth.rs

@@ -258,7 +258,7 @@ fn auth_acquisition_error(error: gcp_auth::Error) -> Error {
 
 #[cfg(test)]
 mod tests {
-    use litellm_auth_types::SecretValue;
+    use litellm_secret_value::SecretValue;
 
     use super::*;
 

litellm-rust/crates/auth-gcp/src/config.rs

@@ -4,7 +4,9 @@ use base64::{
     Engine, alphabet,
     engine::{DecodePaddingMode, GeneralPurpose, GeneralPurposeConfig},
 };
-use litellm_auth_types::{Error, ErrorDetail, InputSource, SecretValue, Sourced};
+use litellm_auth_types::{Error, ErrorDetail, InputSource, Sourced};
+use litellm_secret_value::SecretValue;
+
 use serde_json::{Map, Value};
 
 use litellm_auth_types::VertexParams;

litellm-rust/crates/auth-types/Cargo.toml

@@ -7,9 +7,9 @@ repository.workspace = true
 
 [dependencies]
 litellm-error.workspace = true
+litellm-secret-value.workspace = true
 serde.workspace = true
 serde_json.workspace = true
-subtle.workspace = true
 thiserror.workspace = true
 veil.workspace = true
 

litellm-rust/crates/auth-types/src/credential.rs

@@ -5,7 +5,8 @@ use std::sync::Arc;
 
 use veil::Redact;
 
-use crate::{Error, ResolvedCredential, SecretValue, TokenProviderHandle};
+use crate::{Error, ResolvedCredential, TokenProviderHandle};
+use litellm_secret_value::SecretValue;
 
 #[derive(Clone, Debug, PartialEq, Eq)]
 pub enum CredentialFileRef {
@@ -104,8 +105,8 @@ mod tests {
         CredentialLookup, CredentialLookupFuture, CredentialPlan, CredentialPlanResolution,
         CredentialRef, CredentialResolver, CredentialResolverHandle,
     };
-    use crate::SecretValue;
     use crate::{Error, ErrorDetail};
+    use litellm_secret_value::SecretValue;
 
     #[derive(Debug)]
     struct HostResolver;

litellm-rust/crates/auth-types/src/lib.rs

@@ -5,7 +5,6 @@ mod error;
 pub mod http;
 mod params;
 mod policy;
-mod secret;
 mod token;
 
 use serde::{Deserialize, Serialize};
@@ -55,5 +54,4 @@ pub use error::{Error, ErrorDetail};
 pub use http::CredentialPlacement;
 pub use params::{AwsParams, ParamKind, ParamSpec, VertexParams};
 pub use policy::{CredentialPlanKind, CredentialRule, ExistingHeaderBehavior, ProviderAuthPolicy};
-pub use secret::SecretValue;
 pub use token::{ResolvedCredential, TokenFuture, TokenProvider, TokenProviderHandle};

litellm-rust/crates/auth-types/src/policy.rs

@@ -71,7 +71,8 @@ impl ProviderAuthPolicy {
 #[cfg(test)]
 mod tests {
     use super::{CredentialPlanKind, CredentialRule, ExistingHeaderBehavior, ProviderAuthPolicy};
-    use crate::{CredentialPlacement, ResolvedCredential, SecretValue};
+    use crate::{CredentialPlacement, ResolvedCredential};
+    use litellm_secret_value::SecretValue;
 
     const RULES: &[CredentialRule] = &[CredentialRule {
         kind: CredentialPlanKind::Static,

litellm-rust/crates/auth-types/src/token.rs

@@ -5,7 +5,8 @@ use std::time::SystemTime;
 
 use veil::Redact;
 
-use crate::{Error, SecretValue};
+use crate::Error;
+use litellm_secret_value::SecretValue;
 
 #[derive(Clone, Debug, PartialEq, Eq)]
 pub enum ResolvedCredential {

litellm-rust/crates/auth-types/tests/token.rs

@@ -8,9 +8,9 @@ use std::{
     time::{Duration, SystemTime},
 };
 
-use litellm_auth_types::{
-    Error, ErrorDetail, ResolvedCredential, SecretValue, TokenProviderHandle,
-};
+use litellm_auth_types::{Error, ErrorDetail, ResolvedCredential, TokenProviderHandle};
+use litellm_secret_value::SecretValue;
+
 use rstest::rstest;
 
 fn credential(index: usize, access_token: bool) -> ResolvedCredential {

litellm-rust/crates/auth/Cargo.toml

@@ -16,3 +16,6 @@ litellm-auth-types.workspace = true
 litellm-auth-aws = { workspace = true, optional = true }
 litellm-auth-azure = { workspace = true, optional = true }
 litellm-auth-gcp = { workspace = true, optional = true }
+
+[dev-dependencies]
+litellm-secret-value.workspace = true

litellm-rust/crates/auth/tests/facade.rs

@@ -1,7 +1,8 @@
 use litellm_auth::{
     CredentialPlacement, CredentialPlanKind, CredentialRule, ExistingHeaderBehavior,
-    ProviderAuthPolicy, ResolvedCredential, SecretValue,
+    ProviderAuthPolicy, ResolvedCredential,
 };
+use litellm_secret_value::SecretValue;
 
 const RULES: &[CredentialRule] = &[CredentialRule {
     kind: CredentialPlanKind::Static,

litellm-rust/crates/cache-gcs/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-http.workspace = true
 futures-util.workspace = true
 litellm-auth-gcp.workspace = true

litellm-rust/crates/cache-gcs/src/token.rs

@@ -1,7 +1,9 @@
 use std::{future::Future, pin::Pin};
 
 use litellm_auth_gcp::{VertexAuth, VertexConfig};
-use litellm_auth_types::{InputSource, SecretValue, Sourced};
+use litellm_auth_types::{InputSource, Sourced};
+use litellm_secret_value::SecretValue;
+
 use litellm_cache::Error;
 
 pub trait TokenSource: Send + Sync + 'static {

litellm-rust/crates/callbacks-legacy-python/Cargo.toml

@@ -16,7 +16,7 @@ serde_json.workspace = true
 thiserror.workspace = true
 
 [dev-dependencies]
-litellm-auth.workspace = true
+litellm-secret-value.workspace = true
 proptest.workspace = true
 rstest.workspace = true
 serde_json.workspace = true

litellm-rust/crates/callbacks-legacy-python/src/adapter.rs

@@ -925,11 +925,11 @@ assert all(value is failure for name, value in logger.calls if name.endswith('_h
 mod payload_tests {
     use std::ffi::CStr;
 
-    use litellm_auth::SecretValue;
     use litellm_host::hooks::CallHooks;
     use litellm_host::interceptors::{RawResponse, RequestContext, WireRequest};
     use litellm_host::lifecycle::ExecutionEvent;
     use litellm_host_python::{HookStep, PythonCallEvent, PythonOwned, to_py};
+    use litellm_secret_value::SecretValue;
     use proptest::prelude::*;
     use pyo3::gc::{PyTraverseError, PyVisit};
     use pyo3::prelude::*;

litellm-rust/crates/config/Cargo.toml

@@ -6,7 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
-litellm-auth-types.workspace = true
+litellm-secret-value.workspace = true
 litellm-router-types.workspace = true
 serde.workspace = true
 serde_json.workspace = true

litellm-rust/crates/config/src/mcp.rs

@@ -1,6 +1,6 @@
 use std::{collections::BTreeMap, fmt};
 
-use litellm_auth_types::SecretValue;
+use litellm_secret_value::SecretValue;
 use serde::Deserialize;
 
 use crate::Object;

litellm-rust/crates/config/src/redis.rs

@@ -1,7 +1,7 @@
 use std::{collections::BTreeMap, fmt, time::Duration};
 
-use litellm_auth_types::SecretValue;
 use litellm_router_types::Spelled;
+use litellm_secret_value::SecretValue;
 use serde::{Deserialize, Deserializer, de::Error as _};
 
 use crate::{AdditionalFields, Error, Object, Value};

litellm-rust/crates/config/src/settings.rs

@@ -1,6 +1,6 @@
 use std::fmt;
 
-use litellm_auth_types::SecretValue;
+use litellm_secret_value::SecretValue;
 use serde::Deserialize;
 
 use crate::{

litellm-rust/crates/gateway-auth/Cargo.toml

@@ -6,9 +6,9 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 axum = { workspace = true, features = ["matched-path"] }
 axum-login.workspace = true
-litellm-auth-types.workspace = true
 litellm-config.workspace = true
 litellm-error.workspace = true
 litellm-secrets.workspace = true

litellm-rust/crates/gateway-auth/src/authentication.rs

@@ -1,8 +1,8 @@
 use std::{future::Future, pin::Pin, sync::Arc, time::SystemTime};
 
 use axum::http::{Request, request::Parts};
-use litellm_auth_types::SecretValue;
 use litellm_config::Config;
+use litellm_secret_value::SecretValue;
 use litellm_secrets::source::SecretSource;
 use sha2::{Digest, Sha256};
 use subtle::ConstantTimeEq;

litellm-rust/crates/gateway-auth/src/http.rs

@@ -4,7 +4,7 @@ use axum::{
     middleware::Next,
     response::Response,
 };
-use litellm_auth_types::SecretValue;
+use litellm_secret_value::SecretValue;
 
 use crate::{AccessRequest, Auth, AuthenticatedRequest, Credential, Error};
 

litellm-rust/crates/gateway-auth/src/keys.rs

@@ -1,6 +1,6 @@
 use std::{future::Future, pin::Pin, time::SystemTime};
 
-use litellm_auth_types::SecretValue;
+use litellm_secret_value::SecretValue;
 
 use crate::{Error, hash_token};
 

litellm-rust/crates/gateway-auth/src/ui.rs

@@ -3,7 +3,7 @@ use std::{convert::Infallible, sync::Arc};
 use axum::extract::FromRequestParts;
 use axum::http::{header::AUTHORIZATION, request::Parts};
 use axum_login::{AuthUser, AuthnBackend, UserId};
-use litellm_auth_types::SecretValue;
+use litellm_secret_value::SecretValue;
 use serde::Deserialize;
 use sha2::{Digest, Sha256};
 use subtle::ConstantTimeEq;

litellm-rust/crates/gateway-auth/tests/auth.rs

@@ -8,9 +8,9 @@ use axum::{
     routing::get,
 };
 use futures_util::future::BoxFuture;
-use litellm_auth_types::SecretValue;
 use litellm_config::Config;
 use litellm_gateway_auth::{Auth, RequireMasterKey, hash_token};
+use litellm_secret_value::SecretValue;
 use litellm_secrets::source::SecretSource;
 use rstest::{fixture, rstest};
 use tower::ServiceExt;

litellm-rust/crates/gateway-auth/tests/keys.rs

@@ -1,10 +1,10 @@
 use std::time::{Duration, SystemTime};
 
-use litellm_auth_types::SecretValue;
 use litellm_gateway_auth::{
     Error,
     keys::{KeyHash, KeyLookup, KeyStatus, LookupFuture, verify_key},
 };
+use litellm_secret_value::SecretValue;
 use rstest::{fixture, rstest};
 
 struct Lookup {

litellm-rust/crates/gateway-auth/tests/ui.rs

@@ -1,6 +1,6 @@
 use axum_login::{AuthUser, AuthnBackend};
-use litellm_auth_types::SecretValue;
 use litellm_gateway_auth::{UiBackend, UiCredentials};
+use litellm_secret_value::SecretValue;
 use rstest::{fixture, rstest};
 
 #[fixture]

litellm-rust/crates/gateway-inference/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-cache-response.workspace = true
 axum = { workspace = true, features = ["json", "multipart", "original-uri"] }
 base64.workspace = true

litellm-rust/crates/gateway-inference/src/ocr.rs

@@ -2,10 +2,10 @@ use litellm_gateway_auth::AuthenticatedRequest;
 use std::sync::Arc;
 
 use axum::{Json, extract::State, http::HeaderMap, response::IntoResponse};
-use litellm_auth::SecretValue;
 use litellm_inference_ocr::types::{LiteLLMOcrRequest, OcrConnectionInputs, OcrDocumentInput};
 use litellm_llms::base_llm::ocr::transformation::decode_request_value;
 use litellm_llms_types::formats::ocr::OcrDocument;
+use litellm_secret_value::SecretValue;
 use serde_json::Value;
 
 use crate::{

litellm-rust/crates/gateway-inference/tests/support/mod.rs

@@ -13,7 +13,9 @@ use futures_util::future::BoxFuture;
 use litellm_gateway_inference::{Deployment, Gateway, router};
 use litellm_http::{HttpClientPool, HttpSettings, Resolution, media::PublicDnsResolver};
 use litellm_inference::resources::CoreResources;
-use litellm_secrets::{SecretValue, source::SecretSource};
+use litellm_secret_value::SecretValue;
+use litellm_secrets::source::SecretSource;
+
 use serde_json::Value;
 use tokio::io::{AsyncReadExt, AsyncWriteExt};
 use tower::ServiceExt;

litellm-rust/crates/gateway-management/Cargo.toml

@@ -6,8 +6,8 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 base64.workspace = true
-litellm-auth-types.workspace = true
 litellm-error.workspace = true
 litellm-gateway-auth.workspace = true
 rand.workspace = true

litellm-rust/crates/gateway-management/src/keys.rs

@@ -1,8 +1,8 @@
 use std::{future::Future, pin::Pin, sync::Arc, time::SystemTime};
 
 use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
-use litellm_auth_types::SecretValue;
 use litellm_gateway_auth::keys::{KeyHash, KeyStatus};
+use litellm_secret_value::SecretValue;
 use rand::{RngCore, rngs::OsRng};
 
 use crate::Error;

litellm-rust/crates/gateway-management/tests/keys.rs

@@ -7,7 +7,6 @@ use std::{
 };
 
 use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
-use litellm_auth_types::SecretValue;
 use litellm_gateway_auth::{
     hash_token,
     keys::{KeyHash, KeyLookup, KeyStatus, verify_key},
@@ -16,6 +15,7 @@ use litellm_gateway_management::{
     Error,
     keys::{Keys, Revocation},
 };
+use litellm_secret_value::SecretValue;
 use rstest::{fixture, rstest};
 use support::{MemoryStore, Unavailable};
 

litellm-rust/crates/gateway-mcp/Cargo.toml

@@ -7,11 +7,11 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 axum = { workspace = true, features = ["json", "query"] }
 futures-util.workspace = true
 litellm-config.workspace = true
 litellm-secrets.workspace = true
-litellm-auth-types.workspace = true
 litellm-error.workspace = true
 base64.workspace = true
 http.workspace = true

litellm-rust/crates/gateway-mcp/src/configured.rs

@@ -8,8 +8,8 @@ use std::{
 use base64::{Engine, engine::general_purpose::STANDARD};
 use futures_util::future::try_join_all;
 use http::{HeaderName, HeaderValue};
-use litellm_auth_types::SecretValue;
 use litellm_config::{McpAuth, McpServer, McpTransport};
+use litellm_secret_value::SecretValue;
 use litellm_secrets::source::SecretSource;
 use rmcp::{
     RoleClient, ServiceExt,

litellm-rust/crates/gateway-ui/Cargo.toml

@@ -23,10 +23,10 @@ tower-http = { version = "0.6.11", features = ["fs", "set-header"] }
 tower-sessions.workspace = true
 
 [dev-dependencies]
+litellm-secret-value.workspace = true
 futures-util.workspace = true
 serde_json.workspace = true
 tower = { version = "0.5", features = ["util"] }
-litellm-auth-types.workspace = true
 rstest.workspace = true
 tempfile.workspace = true
 tokio.workspace = true

litellm-rust/crates/gateway-ui/tests/routes.rs

@@ -16,8 +16,8 @@ use tower_sessions::{
     session::{Id, Record},
 };
 
-use litellm_auth_types::SecretValue;
 use litellm_gateway_auth::UiBackend;
+use litellm_secret_value::SecretValue;
 
 struct App {
     router: Router,

litellm-rust/crates/gateway/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 axum.workspace = true
 envy = "0.4.2"
 http-body-util = "0.1"
@@ -17,7 +18,6 @@ thiserror.workspace = true
 futures-util.workspace = true
 litellm-gateway-auth.workspace = true
 litellm-gateway-ui.workspace = true
-litellm-auth-types.workspace = true
 litellm-config.workspace = true
 litellm-error.workspace = true
 litellm-circuit-breaker.workspace = true

litellm-rust/crates/gateway/src/redis.rs

@@ -152,12 +152,12 @@ async fn reachable(client: RedisClient) -> Option<RedisClient> {
 fn client(params: Params, breaker: Option<BreakerConfig>) -> Result<RedisClient, Error> {
     tracing::info!(endpoint = ?params.endpoint, namespace = ?params.namespace, "redis client configured");
     let endpoint = match params.endpoint {
-        RedisEndpoint::Url(url) => Endpoint::Url(url.expose().to_owned()),
+        RedisEndpoint::Url(url) => Endpoint::Url(url),
         RedisEndpoint::Host(host) => Endpoint::Host(HostEndpoint {
             host: host.host,
             port: host.port,
-            username: host.username.map(|value| value.expose().to_owned()),
-            password: host.password.map(|value| value.expose().to_owned()),
+            username: host.username,
+            password: host.password,
             tls: host.tls,
             db: host.db,
         }),
@@ -170,8 +170,8 @@ fn client(params: Params, breaker: Option<BreakerConfig>) -> Result<RedisClient,
                     port: node.port,
                 })
                 .collect(),
-            username: cluster.username.map(|value| value.expose().to_owned()),
-            password: cluster.password.map(|value| value.expose().to_owned()),
+            username: cluster.username,
+            password: cluster.password,
             tls: cluster.tls,
         }),
     };

litellm-rust/crates/gateway/src/secrets.rs

@@ -1,8 +1,8 @@
 use std::{collections::BTreeMap, sync::Arc};
 
 use futures_util::future::BoxFuture;
-use litellm_auth_types::SecretValue;
 use litellm_config::{Object, Value};
+use litellm_secret_value::SecretValue;
 use litellm_secrets::source::SecretSource;
 
 use crate::Error;

litellm-rust/crates/gateway/src/settings.rs

@@ -1,6 +1,6 @@
 use std::ffi::OsString;
 
-use litellm_auth_types::SecretValue;
+use litellm_secret_value::SecretValue;
 use litellm_tracing::Level;
 use serde::{Deserialize, de::DeserializeOwned};
 

litellm-rust/crates/gateway/tests/mcp.rs

@@ -424,7 +424,7 @@ impl litellm_secrets::source::SecretSource for NoSecrets {
         _: &'a str,
     ) -> futures_util::future::BoxFuture<
         'a,
-        Result<Option<litellm_auth_types::SecretValue>, litellm_secrets::Error>,
+        Result<Option<litellm_secret_value::SecretValue>, litellm_secrets::Error>,
     > {
         Box::pin(async { Ok(None) })
     }

litellm-rust/crates/gateway/tests/server.rs

@@ -159,7 +159,7 @@ async fn mounts_ui_without_exposing_credentials_or_authorizing_inference(inferen
     std::fs::write(assets.path().join("index.html"), "dashboard").unwrap();
     let backend = litellm_gateway_auth::UiBackend::new(
         "admin".into(),
-        litellm_auth_types::SecretValue::new("ui-password"),
+        litellm_secret_value::SecretValue::new("ui-password"),
     )
     .unwrap();
     let ui = litellm_gateway_ui::router(

litellm-rust/crates/host/Cargo.toml

@@ -6,8 +6,8 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 futures-util.workspace = true
-litellm-auth.workspace = true
 litellm-coroutine.workspace = true
 serde_json.workspace = true
 thiserror.workspace = true

litellm-rust/crates/host/src/interceptors.rs

@@ -22,7 +22,7 @@ pub struct RequestContext {
     /// Optional-param names that carry credentials and must be redacted when logged.
     pub secret_fields: Vec<String>,
     /// The credential the route resolved for the provider call.
-    pub api_key: Option<litellm_auth::SecretValue>,
+    pub api_key: Option<litellm_secret_value::SecretValue>,
 }
 
 #[derive(Clone, Debug, PartialEq, Eq)]

litellm-rust/crates/inference-chat/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-auth = { workspace = true, features = ["aws", "azure", "gcp"] }
 litellm-cache-response.workspace = true
 litellm-core-utils.workspace = true

litellm-rust/crates/inference-chat/src/handler.rs

@@ -58,7 +58,7 @@ pub(super) async fn execute(
         custom_llm_provider: provider_name.to_owned(),
         optional_params: Value::Object(call.optional_params),
         secret_fields: Vec::new(),
-        api_key: call.api_key.map(litellm_auth::SecretValue::new),
+        api_key: call.api_key.map(litellm_secret_value::SecretValue::new),
     };
     let authenticated = resolve_auth(auth, environment, &|key| secrets.get(key)).await?;
     let identity = litellm_host::interceptors::ProviderIdentity {

litellm-rust/crates/inference-chat/tests/caching.rs

@@ -140,7 +140,7 @@ impl litellm_secrets::source::SecretSource for ChangingSecrets {
         name: &'a str,
     ) -> futures_util::future::BoxFuture<
         'a,
-        Result<Option<litellm_secrets::SecretValue>, litellm_secrets::Error>,
+        Result<Option<litellm_secret_value::SecretValue>, litellm_secrets::Error>,
     > {
         Box::pin(async move {
             let revision = self.revision.load(Ordering::SeqCst);
@@ -154,7 +154,7 @@ impl litellm_secrets::source::SecretSource for ChangingSecrets {
             } else {
                 None
             };
-            Ok(value.map(litellm_secrets::SecretValue::new))
+            Ok(value.map(litellm_secret_value::SecretValue::new))
         })
     }
 }

litellm-rust/crates/inference-messages/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 bytes.workspace = true
 futures-util.workspace = true
 litellm-auth = { workspace = true, features = ["aws", "azure", "gcp"] }

litellm-rust/crates/inference-messages/src/handler.rs

@@ -2,7 +2,9 @@ use std::time::Duration;
 
 use bytes::Bytes;
 use futures_util::{FutureExt, StreamExt, TryStreamExt, stream::BoxStream};
-use litellm_auth::{AuthServices, SecretValue};
+use litellm_auth::AuthServices;
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::{
     dot_notation_indexing::delete_nested_value,
     get_provider_specific_headers::get_provider_specific_headers, settings::Lookup,

litellm-rust/crates/inference-messages/tests/caching.rs

@@ -107,7 +107,7 @@ impl litellm_secrets::source::SecretSource for ChangingSecrets {
         name: &'a str,
     ) -> futures_util::future::BoxFuture<
         'a,
-        Result<Option<litellm_secrets::SecretValue>, litellm_secrets::Error>,
+        Result<Option<litellm_secret_value::SecretValue>, litellm_secrets::Error>,
     > {
         Box::pin(async move {
             let revision = self.revision.load(Ordering::SeqCst);
@@ -121,7 +121,7 @@ impl litellm_secrets::source::SecretSource for ChangingSecrets {
             } else {
                 None
             };
-            Ok(value.map(litellm_secrets::SecretValue::new))
+            Ok(value.map(litellm_secret_value::SecretValue::new))
         })
     }
 }

litellm-rust/crates/inference-ocr/Cargo.toml

@@ -6,6 +6,7 @@ license.workspace = true
 repository.workspace = true
 
 [dependencies]
+litellm-secret-value.workspace = true
 base64.workspace = true
 bytes.workspace = true
 futures-util.workspace = true

litellm-rust/crates/inference-ocr/src/handler.rs

@@ -128,7 +128,7 @@ mod tests {
         decode_request(OcrWireRequest {
             model: model.into(),
             document,
-            api_key: Some(litellm_auth::SecretValue::new("test-key")),
+            api_key: Some(litellm_secret_value::SecretValue::new("test-key")),
             api_base: Some(base.into()),
             custom_llm_provider: None,
             extra_headers: None,

litellm-rust/crates/inference-ocr/src/provider_config.rs

@@ -1,4 +1,6 @@
-use litellm_auth::{InputSource, SecretValue, Sourced};
+use litellm_auth::{InputSource, Sourced};
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::get_llm_provider_logic::{
     CustomLlmProvider, LlmProviders, get_custom_llm_provider,
 };
@@ -407,7 +409,7 @@ mod tests {
     fn connection_resolution_preserves_dynamic_precedence_and_input_sources() {
         let connection = OcrConfigKind::Mistral.resolve_connection_params(OcrCredentialInputs {
             api_key: Some(Sourced::new(
-                litellm_auth::SecretValue::new("explicit-key"),
+                litellm_secret_value::SecretValue::new("explicit-key"),
                 InputSource::Deployment,
             )),
             api_base: Some(Sourced::new(
@@ -415,7 +417,7 @@ mod tests {
                 InputSource::Deployment,
             )),
             dynamic_api_key: Some(Sourced::new(
-                litellm_auth::SecretValue::new("dynamic-key"),
+                litellm_secret_value::SecretValue::new("dynamic-key"),
                 InputSource::Environment,
             )),
             dynamic_api_base: Some(Sourced::new(
@@ -455,7 +457,7 @@ mod tests {
     ) {
         let dynamic_key = dynamic_value.map(|value| {
             Sourced::new(
-                litellm_auth::SecretValue::new(value),
+                litellm_secret_value::SecretValue::new(value),
                 InputSource::Environment,
             )
         });
@@ -463,7 +465,7 @@ mod tests {
             dynamic_value.map(|value| Sourced::new(value.into(), InputSource::Environment));
         let connection = OcrConfigKind::Mistral.resolve_connection_params(OcrCredentialInputs {
             api_key: Some(Sourced::new(
-                litellm_auth::SecretValue::new("explicit-key"),
+                litellm_secret_value::SecretValue::new("explicit-key"),
                 InputSource::Deployment,
             )),
             api_base: Some(Sourced::new(
@@ -502,14 +504,14 @@ mod tests {
             OcrCredentialInputs {
                 api_key: explicit_key.map(|value| {
                     Sourced::new(
-                        litellm_auth::SecretValue::new(value),
+                        litellm_secret_value::SecretValue::new(value),
                         InputSource::Deployment,
                     )
                 }),
                 api_base: explicit_base
                     .map(|value| Sourced::new(value.into(), InputSource::Deployment)),
                 dynamic_api_key: Some(Sourced::new(
-                    litellm_auth::SecretValue::new("dynamic-key"),
+                    litellm_secret_value::SecretValue::new("dynamic-key"),
                     InputSource::Environment,
                 )),
                 dynamic_api_base: Some(Sourced::new(

litellm-rust/crates/inference-ocr/src/types.rs

@@ -1,7 +1,9 @@
 use std::{collections::BTreeMap, path::PathBuf, time::Duration};
 
 use bytes::Bytes;
-use litellm_auth::{InputSource, SecretValue, TokenProviderHandle};
+use litellm_auth::{InputSource, TokenProviderHandle};
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::call_arguments::CallArguments;
 use litellm_llms::base_llm::ocr::{
     error::Error,

litellm-rust/crates/inference-ocr/src/wire.rs

@@ -1,6 +1,8 @@
 use std::{collections::BTreeMap, time::Duration};
 
-use litellm_auth::{InputSource, SecretValue};
+use litellm_auth::InputSource;
+use litellm_secret_value::SecretValue;
+
 use litellm_llms::base_llm::ocr::{error::Error, transformation::decode_request_value};
 use litellm_llms_types::formats::ocr::OcrDocument;
 use serde::Deserialize;

litellm-rust/crates/inference-ocr/tests/ocr/azure_ai.rs

@@ -3,9 +3,9 @@ use std::sync::{
     atomic::{AtomicUsize, Ordering},
 };
 
-use litellm_auth::{
-    ResolvedCredential, SecretValue, TokenFuture, TokenProvider, TokenProviderHandle,
-};
+use litellm_auth::{ResolvedCredential, TokenFuture, TokenProvider, TokenProviderHandle};
+use litellm_secret_value::SecretValue;
+
 use rstest::rstest;
 
 use super::*;

litellm-rust/crates/inference-ocr/tests/ocr/lifecycle.rs

@@ -239,7 +239,7 @@ impl litellm_host_native::services::HostCallHandler<Ocr> for CallerTokenHost {
             OcrOp::AcquireAzureAdToken(reply) => {
                 self.trace.lock().unwrap().push("token".into());
                 reply.send(litellm_auth::ResolvedCredential::Static(
-                    litellm_auth::SecretValue::new("caller-token"),
+                    litellm_secret_value::SecretValue::new("caller-token"),
                 ));
                 Ok(())
             }

litellm-rust/crates/inference-ocr/tests/ocr/main.rs

@@ -80,7 +80,7 @@ fn wire(model: &str, base: &str, document: Value, options: Value) -> OcrWireRequ
     OcrWireRequest {
         model: model.into(),
         document,
-        api_key: Some(litellm_auth::SecretValue::new("test-key")),
+        api_key: Some(litellm_secret_value::SecretValue::new("test-key")),
         api_base: Some(base.into()),
         custom_llm_provider: None,
         extra_headers: None,

litellm-rust/crates/inference-responses/Cargo.toml

@@ -25,6 +25,7 @@ tokio-tungstenite.workspace = true
 tracing.workspace = true
 
 [dev-dependencies]
+litellm-secret-value.workspace = true
 litellm-http = { workspace = true, features = ["test-support"] }
 litellm-cache.workspace = true
 litellm-cache-memory.workspace = true

litellm-rust/crates/inference-responses/tests/caching.rs

@@ -98,7 +98,7 @@ impl litellm_secrets::source::SecretSource for ChangingSecrets {
         name: &'a str,
     ) -> futures_util::future::BoxFuture<
         'a,
-        Result<Option<litellm_secrets::SecretValue>, litellm_secrets::Error>,
+        Result<Option<litellm_secret_value::SecretValue>, litellm_secrets::Error>,
     > {
         Box::pin(async move {
             let revision = self.revision.load(Ordering::SeqCst);
@@ -112,7 +112,7 @@ impl litellm_secrets::source::SecretSource for ChangingSecrets {
             } else {
                 None
             };
-            Ok(value.map(litellm_secrets::SecretValue::new))
+            Ok(value.map(litellm_secret_value::SecretValue::new))
         })
     }
 }

litellm-rust/crates/inference-testing/Cargo.toml

@@ -7,6 +7,7 @@ repository.workspace = true
 publish = false
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-auth.workspace = true
 futures-util.workspace = true
 litellm-http.workspace = true

litellm-rust/crates/inference-testing/src/lib.rs

@@ -5,7 +5,8 @@ use litellm_http::{
     ClientVariant, HttpClientConfig, HttpClientPool, HttpSettings, Resolution,
     media::PublicDnsResolver,
 };
-use litellm_secrets::{SecretValue, source::SecretSource};
+use litellm_secret_value::SecretValue;
+use litellm_secrets::source::SecretSource;
 
 use litellm_inference::resources::CoreResources;
 

litellm-rust/crates/llms-types/src/providers/anthropic/beta.rs

@@ -199,7 +199,7 @@ impl AnthropicBeta {
                 &[BetaProvider::BedrockConverse, BetaProvider::Databricks]
             }
             Self::ContextManagement20250627 => &[BetaProvider::BedrockConverse],
-            Self::AdvisorTool20260301 | Self::Compact20260904 => &[
+            Self::AdvisorTool20260301 => &[
                 BetaProvider::AzureAi,
                 BetaProvider::BedrockConverse,
                 BetaProvider::Bedrock,
@@ -207,6 +207,12 @@ impl AnthropicBeta {
                 BetaProvider::VertexAi,
                 BetaProvider::Databricks,
             ],
+            Self::Compact20260904 => &[
+                BetaProvider::BedrockConverse,
+                BetaProvider::Bedrock,
+                BetaProvider::BedrockMantle,
+                BetaProvider::Databricks,
+            ],
             Self::Bash20241022
             | Self::Bash20250124
             | Self::McpServers20251204
@@ -256,14 +262,18 @@ impl AnthropicBeta {
             Self::InterleavedThinking20250514 | Self::WebSearch20250305 => {
                 &[BetaProvider::BedrockConverse, BetaProvider::Bedrock]
             }
-            Self::MidConversationOutputConfig20260701
-            | Self::MidConversationToolChanges20260701 => &[
+            Self::MidConversationOutputConfig20260701 => &[
                 BetaProvider::AzureAi,
                 BetaProvider::BedrockConverse,
                 BetaProvider::BedrockMantle,
                 BetaProvider::VertexAi,
                 BetaProvider::Databricks,
             ],
+            Self::MidConversationToolChanges20260701 => &[
+                BetaProvider::BedrockConverse,
+                BetaProvider::BedrockMantle,
+                BetaProvider::Databricks,
+            ],
             Self::Output128k20250219 | Self::TokenEfficientTools20250219 => &[
                 BetaProvider::AzureAi,
                 BetaProvider::BedrockConverse,
@@ -483,7 +493,7 @@ mod tests {
         let config = beta_headers_config();
         let config_columns: Vec<String> = config
             .keys()
-            .filter(|column| column.as_str() != "description")
+            .filter(|column| column.as_str() != "description" && !column.ends_with("_header_only"))
             .cloned()
             .collect();
         let beta_provider_columns: Vec<String> = BetaProvider::VARIANTS
@@ -502,12 +512,16 @@ mod tests {
         let config = beta_headers_config();
         for provider in BetaProvider::VARIANTS {
             for beta in &AnthropicBeta::KNOWN {
-                let expected = config
-                    .get(provider.as_ref())
-                    .and_then(serde_json::Value::as_object)
-                    .and_then(|column| column.get(beta.as_str()))
-                    .and_then(serde_json::Value::as_str)
-                    .map(str::to_string);
+                let cell = |column: &str| {
+                    config
+                        .get(column)
+                        .and_then(serde_json::Value::as_object)
+                        .and_then(|values| values.get(beta.as_str()))
+                        .and_then(serde_json::Value::as_str)
+                        .map(str::to_string)
+                };
+                let expected =
+                    cell(&format!("{provider}_header_only")).or_else(|| cell(provider.as_ref()));
                 let actual = beta.on(*provider).map(|name| name.to_string());
                 assert_eq!(actual, expected, "provider {provider}, beta {beta}");
                 if matches!(beta, AnthropicBeta::AdvancedToolUse20251120)

litellm-rust/crates/llms/Cargo.toml

@@ -9,6 +9,7 @@ repository.workspace = true
 test-support = ["litellm-http/test-support"]
 
 [dependencies]
+litellm-secret-value.workspace = true
 litellm-llms-types.workspace = true
 litellm-core-utils.workspace = true
 litellm-error.workspace = true

litellm-rust/crates/llms/src/anthropic/chat/transformation.rs

@@ -1,4 +1,3 @@
-use litellm_auth::SecretValue;
 use litellm_core_utils::decline::Decline;
 use litellm_core_utils::{
     core_helpers::{finish_reason_for, unix_now, usage_from_parts},
@@ -10,6 +9,7 @@ use litellm_llms_types::{
     },
     providers::anthropic::DEFAULT_HEADERS,
 };
+use litellm_secret_value::SecretValue;
 use serde::Deserialize;
 use serde_json::{Map, Value, json};
 

litellm-rust/crates/llms/src/anthropic/common_utils.rs

@@ -1,5 +1,7 @@
 use crate::base_llm::messages::context::MessagesModelCapabilities;
-use litellm_auth::{CredentialPlacement, SecretValue};
+use litellm_auth::CredentialPlacement;
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::settings::resolve_non_empty;
 use litellm_http::request::{
     has_header, header_value, header_values, with_header, without_headers,

litellm-rust/crates/llms/src/anthropic/messages/transformation.rs

@@ -1,7 +1,8 @@
 use litellm_auth::CredentialPlacement;
 use litellm_llms_types::{
     formats::messages::{
-        ContextEdit, ContextManagement, Message, MessagesOptionalParams, MessagesRequest, Speed,
+        ContentBlock, ContentBlockType, ContextEdit, ContextManagement, Message,
+        MessagesOptionalParams, MessagesRequest, Speed, ThinkingConfig, ThinkingDisplay,
     },
     providers::anthropic::{AnthropicBeta, BetaProvider, BetaSet, DEFAULT_HEADERS},
     recognized::Recognized,
@@ -203,6 +204,10 @@ fn feature_betas(request: &MessagesRequest) -> BetaSet {
             .then_some(AnthropicBeta::FastMode20260201),
         messages_carry_output_config(&request.messages)
             .then_some(AnthropicBeta::PerTurnControl20260701),
+        uses_thinking_display_updates(params)
+            .then_some(AnthropicBeta::ThinkingDisplayUpdates20260818),
+        messages_carry_tool_changes(&request.messages)
+            .then_some(AnthropicBeta::MidConversationToolChanges20260701),
         has_advisor_tool(tools).then_some(AnthropicBeta::AdvisorTool20260301),
         is_tool_search_used(tools).then_some(AnthropicBeta::AdvancedToolUse20251120),
     ]
@@ -253,6 +258,36 @@ fn messages_carry_output_config(messages: &[Message]) -> bool {
         .any(|message| message.extra.contains_key("output_config"))
 }
 
+fn uses_thinking_display_updates(params: &MessagesOptionalParams) -> bool {
+    let display = match params.thinking.as_ref().and_then(Recognized::known) {
+        Some(ThinkingConfig::Enabled(thinking)) => thinking.display.as_ref(),
+        Some(ThinkingConfig::Adaptive(thinking)) => thinking.display.as_ref(),
+        _ => None,
+    };
+    display.and_then(Recognized::known) == Some(&ThinkingDisplay::Updates)
+}
+
+fn is_tool_reference_change(block: &ContentBlock) -> bool {
+    matches!(
+        &block.block_type,
+        Some(ContentBlockType::Other(block_type))
+            if block_type == "tool_addition" || block_type == "tool_removal"
+    ) && block
+        .extra
+        .get("tool")
+        .and_then(|tool| tool.get("type"))
+        .and_then(Value::as_str)
+        == Some("tool_reference")
+}
+
+fn messages_carry_tool_changes(messages: &[Message]) -> bool {
+    messages
+        .iter()
+        .filter(|message| message.role == "system")
+        .flat_map(Message::blocks)
+        .any(is_tool_reference_change)
+}
+
 fn unsupported_param(model: &str, param: &str, value: &str, hint: &str) -> Error {
     Error::InvalidRequest(crate::ErrorDetail::UnsupportedParameter {
         model: model.into(),
@@ -1071,6 +1106,41 @@ mod tests {
         json!({"safeguards": [{"type": "dangerous_tool_use", "classifier_context": {"v": 1, "permission_mode": "auto"}}]}),
         &[]
     )]
+    #[case::adaptive_display_updates(
+        json!({"thinking": {"type": "adaptive", "display": "updates"}}),
+        &["thinking-display-updates-2026-08-18"]
+    )]
+    #[case::enabled_display_updates(
+        json!({"thinking": {"type": "enabled", "budget_tokens": 2048, "display": "updates"}}),
+        &["thinking-display-updates-2026-08-18"]
+    )]
+    #[case::summarized_display(json!({"thinking": {"type": "adaptive", "display": "summarized"}}), &[])]
+    #[case::disabled_thinking(json!({"thinking": {"type": "disabled"}}), &[])]
+    #[case::system_tool_addition(
+        json!({"messages": [
+            {"role": "user", "content": "Use the ping tool"},
+            {"role": "system", "content": [{"type": "tool_addition", "tool": {"type": "tool_reference", "name": "ping"}}]},
+        ]}),
+        &["mid-conversation-tool-changes-2026-07-01"]
+    )]
+    #[case::system_tool_removal(
+        json!({"messages": [
+            {"role": "system", "content": [{"type": "tool_removal", "tool": {"type": "tool_reference", "name": "ping"}}]},
+        ]}),
+        &["mid-conversation-tool-changes-2026-07-01"]
+    )]
+    #[case::user_tool_addition(
+        json!({"messages": [
+            {"role": "user", "content": [{"type": "tool_addition", "tool": {"type": "tool_reference", "name": "ping"}}]},
+        ]}),
+        &[]
+    )]
+    #[case::tool_addition_without_a_tool_reference(
+        json!({"messages": [
+            {"role": "system", "content": [{"type": "tool_addition", "tool": {"type": "tool_definition"}}]},
+        ]}),
+        &[]
+    )]
     fn feature_betas_follow_the_request(#[case] fields: Value, #[case] expected: &[&str]) {
         assert_eq!(feature_betas(&request(fields)), betas(expected));
     }
@@ -1093,6 +1163,33 @@ mod tests {
         );
     }
 
+    #[rstest]
+    #[case::vertex_ai(
+        BetaProvider::VertexAi,
+        &["mid-conversation-tool-changes-2026-07-01", "thinking-display-updates-2026-08-18"]
+    )]
+    #[case::azure_ai(
+        BetaProvider::AzureAi,
+        &["mid-conversation-tool-changes-2026-07-01", "thinking-display-updates-2026-08-18"]
+    )]
+    #[case::bedrock_converse(BetaProvider::BedrockConverse, &["thinking-display-updates-2026-08-18"])]
+    #[case::databricks(BetaProvider::Databricks, &["thinking-display-updates-2026-08-18"])]
+    fn tool_change_and_display_betas_reach_the_providers_that_accept_them(
+        #[case] provider: BetaProvider,
+        #[case] expected: &[&str],
+    ) {
+        let fields = json!({
+            "thinking": {"type": "adaptive", "display": "updates"},
+            "messages": [
+                {"role": "system", "content": [{"type": "tool_addition", "tool": {"type": "tool_reference", "name": "ping"}}]},
+            ],
+        });
+        assert_eq!(
+            provider_feature_betas(&request(fields), provider),
+            betas(expected)
+        );
+    }
+
     #[rstest]
     fn hosts_without_a_beta_map_column_get_safeguards_without_a_derived_beta() {
         let client = headers(&[("anthropic-beta", "claude-code-20250219")]);

litellm-rust/crates/llms/src/azure_ai/messages/transformation.rs

@@ -1,10 +1,10 @@
-use litellm_auth::SecretValue;
 use litellm_http::request::{has_bearer_auth, has_header};
 use litellm_llms_types::{
     formats::messages::{MessagesOptionalParams, MessagesRequest},
     providers::anthropic::{BetaProvider, DEFAULT_HEADERS, MESSAGES_PATH},
 };
 use litellm_router_types::LitellmParams;
+use litellm_secret_value::SecretValue;
 
 use crate::{
     Error,
@@ -545,6 +545,11 @@ mod tests {
         &[("anthropic-beta", "web-search-2025-03-05")],
         &[("x-api-key", "k"), ("anthropic-beta", "structured-outputs-2025-11-13,web-search-2025-03-05")]
     )]
+    #[case::compaction_needs_the_compact_2026_09_04_beta(
+        json!({"compaction": {"type": "summarize"}}),
+        &[],
+        &[("x-api-key", "k"), ("anthropic-beta", "compact-2026-09-04")]
+    )]
     #[case::no_feature_needs_a_beta(json!({}), &[], &[("x-api-key", "k")])]
     #[case::safeguards_need_the_dangerous_tool_use_beta(
         json!({"safeguards": [{"type": "dangerous_tool_use"}]}),

litellm-rust/crates/llms/src/azure_ai/ocr/document_intelligence/transformation.rs

@@ -1250,7 +1250,7 @@ mod tests {
     #[tokio::test]
     async fn request_endpoint_accepts_request_owned_key() {
         let connection = OcrConnection {
-            api_key: Some(litellm_auth::SecretValue::new("request-key")),
+            api_key: Some(litellm_secret_value::SecretValue::new("request-key")),
             api_key_source: InputSource::Request,
             api_base: Some("https://request.example".into()),
             api_base_source: InputSource::Request,

litellm-rust/crates/llms/src/azure_ai/ocr/transformation.rs

@@ -211,7 +211,7 @@ mod tests {
     #[fixture]
     fn connection() -> OcrConnection {
         OcrConnection {
-            api_key: Some(litellm_auth::SecretValue::new("request-key")),
+            api_key: Some(litellm_secret_value::SecretValue::new("request-key")),
             api_base: Some("https://example.com".into()),
             ..Default::default()
         }
@@ -312,7 +312,7 @@ mod tests {
     #[tokio::test]
     async fn request_endpoint_accepts_request_owned_key() {
         let connection = OcrConnection {
-            api_key: Some(litellm_auth::SecretValue::new("request-key")),
+            api_key: Some(litellm_secret_value::SecretValue::new("request-key")),
             api_key_source: InputSource::Request,
             api_base: Some("https://request.example".into()),
             api_base_source: InputSource::Request,

litellm-rust/crates/llms/src/base_llm/auth.rs

@@ -5,7 +5,9 @@
 //! that split: `validate_environment` shapes the forwarded headers and names the credential
 //! as an [`AuthScheme`], and [`resolve_auth`] turns the scheme into headers and a signer.
 
-use litellm_auth::{AuthServices, CredentialPlacement, SecretValue, TokenProviderHandle};
+use litellm_auth::{AuthServices, CredentialPlacement, TokenProviderHandle};
+use litellm_secret_value::SecretValue;
+
 use litellm_auth_aws::{AwsCredentialSource, SigV4Signer};
 use litellm_auth_gcp::VertexConfig;
 use litellm_http::request::with_header;

litellm-rust/crates/llms/src/base_llm/ocr/transformation.rs

@@ -5,7 +5,9 @@ use serde::{
 use serde_json::{Map, Value};
 use std::{collections::BTreeMap, future::Future, sync::Arc, time::Duration};
 
-use litellm_auth::{InputSource, SecretValue, Sourced, TokenProviderHandle};
+use litellm_auth::{InputSource, Sourced, TokenProviderHandle};
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::{call_arguments::CallArguments, settings::ProcessEnvironment};
 use litellm_http::outbound::RequestSigner;
 use litellm_llms_types::formats::ocr::{LiteLLMOcrResponse, OcrDocument, OcrResponseFormat};

litellm-rust/crates/llms/src/bedrock/chat/converse_transformation.rs

@@ -1,5 +1,7 @@
 use litellm_auth::AwsParams;
-use litellm_auth::{CredentialPlacement, SecretValue};
+use litellm_auth::CredentialPlacement;
+use litellm_secret_value::SecretValue;
+
 use litellm_auth_aws::{
     AwsCredentialSource, bedrock_model_id_and_region,
     constants::{AWS_BEARER_TOKEN_BEDROCK, BEDROCK_RUNTIME_ENDPOINT_TEMPLATE, BEDROCK_SERVICE},

litellm-rust/crates/llms/src/bedrock/claude_platform/messages/transformation.rs

@@ -1,6 +1,8 @@
 use std::sync::LazyLock;
 
-use litellm_auth::{AwsParams, CredentialPlacement, ParamKind, ParamSpec, SecretValue};
+use litellm_auth::{AwsParams, CredentialPlacement, ParamKind, ParamSpec};
+use litellm_secret_value::SecretValue;
+
 use litellm_auth_aws::{AwsCredentialSource, resolve_aws_region};
 use litellm_core_utils::settings::resolve_non_empty;
 use litellm_http::request::{header_value, with_header, without_headers};

litellm-rust/crates/llms/src/bedrock/messages/invoke_transformations/anthropic_claude_transformation.rs

@@ -7,7 +7,9 @@ use crate::{
 };
 use futures_util::StreamExt;
 use litellm_auth::AwsParams;
-use litellm_auth::{CredentialPlacement, SecretValue};
+use litellm_auth::CredentialPlacement;
+use litellm_secret_value::SecretValue;
+
 use litellm_auth_aws::{
     AwsCredentialSource, bedrock_model_id_and_region,
     constants::{

litellm-rust/crates/llms/src/bedrock_mantle/messages/transformation.rs

@@ -1,6 +1,8 @@
 use std::sync::LazyLock;
 
-use litellm_auth::{AwsParams, CredentialPlacement, SecretValue};
+use litellm_auth::{AwsParams, CredentialPlacement};
+use litellm_secret_value::SecretValue;
+
 use litellm_auth_aws::AwsCredentialSource;
 use litellm_auth_aws::bedrock_model_id_and_region;
 use litellm_core_utils::settings::resolve_non_empty;

litellm-rust/crates/llms/src/cohere/ocr/transformation.rs

@@ -735,7 +735,7 @@ mod tests {
         assert!(matches!(
             CohereParseConfig.resolve_headers(
                 &OcrConnection {
-                    api_key: Some(litellm_auth::SecretValue::new("  ")),
+                    api_key: Some(litellm_secret_value::SecretValue::new("  ")),
                     ..Default::default()
                 },
                 &|_| None,

litellm-rust/crates/llms/src/deepseek/messages/transformation.rs

@@ -1,4 +1,6 @@
-use litellm_auth::{CredentialPlacement, SecretValue};
+use litellm_auth::CredentialPlacement;
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::settings::resolve_non_empty;
 use litellm_llms_types::{
     formats::messages::{MessagesOptionalParams, MessagesRequest, MessagesTool},

litellm-rust/crates/llms/src/edenai/messages/transformation.rs

@@ -1,4 +1,6 @@
-use litellm_auth::{CredentialPlacement, SecretValue};
+use litellm_auth::CredentialPlacement;
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::settings::resolve_non_empty;
 use litellm_llms_types::formats::messages::{
     MessagesOptionalParams, MessagesRequest, MessagesResponse,

litellm-rust/crates/llms/src/minimax/messages/transformation.rs

@@ -1,4 +1,6 @@
-use litellm_auth::{CredentialPlacement, SecretValue};
+use litellm_auth::CredentialPlacement;
+use litellm_secret_value::SecretValue;
+
 use litellm_core_utils::settings::resolve_non_empty;
 use litellm_llms_types::{
     formats::messages::{MessagesOptionalParams, MessagesRequest},