* feat(cli): enable Claude Code gateway model discovery by default in lite claude
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cli): build agent env declaratively and document discovery key for lite up
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cli): keep build_agent_env within LIT002 type-discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Required lint failed ruff format on the ternary that unwraps a polars DataFrame or list before alias recovery
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Key Activity charts already fell back to user_email. The top-keys tables
and usage export still printed '-' or a truncated hash. They now use the
same alias-then-email label.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Recovery already resolved the owner email for double-hashed spend keys,
then Key Activity dropped it. The Usage payload now carries user_email
and the key label falls back to that email before key-hash-...
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
The Python 3.10 smoke check imports the proxy without prisma, so PrismaError
is loaded only inside the DB helper. Recovery now returns frozen mappings
and ReadOnly TypedDict fields so the type-discipline budget stays put.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
The Usage recovery path was adding four BLE001 hits and failing the
strict-rule budget. Soft-fail only on PrismaError so a down token table
still falls through to SpendLogs.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Lint CI failed because ruff format splits the Set alias import and collapses a couple of long lines.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Recovery now resolves the key owner's email from UserTable via the
recovered token user_id, and SpendLogsMetadata keeps user_api_key_user_email
so new batch/export consumers see email without a separate user join.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Extract the Usage reverse-hash / SpendLogs alias recovery into a shared
helper and apply it when CloudZero and Focus export DailyUserSpend rows,
so BI pulls get api_key_alias back for historical v1.99 double-hashed keys
instead of null.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Callback log replay also omitted user_api_key_hash, so it could double-hash
spend rows the same way batch costing did. On the read path, Usage key
metadata now reverse-hashes orphaned DailyUserSpend.api_key values against
VerificationToken and falls back to SpendLogs metadata so historical dirty
rows show their api_key_alias again instead of key-hash-...
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Batch cost attribution and the legacy queue endpoint already store the
VerificationToken hash in user_api_key, but omitted user_api_key_hash.
Since v1.99 the spend-log writer re-hashes any key without that provenance
flag, so DailyUserSpend.api_key no longer joins VerificationToken and Usage
shows key-hash-... rows with null api_key_alias / user_email.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
prisma's `delete` returns None when the `where` clause matched no row
instead of raising, and the handler never looked at the return value. It
went straight on to filter an in-memory list that never held the name and
answered 200 "Credential deleted successfully", so an operator scripting
credential cleanup could not tell a real deletion from a typo.
Look at what the repository returned and answer 404 with the name, the
same rejection PATCH /credentials/{credential_name} already gives. A
credential that only exists in the config yaml is never written to the
table, so it now answers 404 too, which is honest: reporting success for
it is the same lie, since it comes back on the next proxy boot.
Adds regression tests for the delete 404, the still-working delete, the
config-yaml-only credential, and for the raise-not-return fix on both
DELETE /credentials/{credential_name} and GET /credentials, which
serialized a rejection as the 200 response body.
The config stub shared one list object between save_config and get_config, so the
DB overlay handed the endpoint back the very list it had just appended to and both
tests passed with the product fix reverted. Store the settings as JSON the way the
litellm_config row does, and check the duplicate guard against a list that only
ever existed in the DB.
The monitor reads the queue on its first pass, before it ever waits on a
request, so dropping a request made while spend_log_flush_requested is still
None delays nothing. Reordering the loop to wait first would turn that drop
into a real delay for the Responses chaining flow, and now fails this test.
`POST /v1/agents/{id}/make_public` appended the agent id to
`litellm.public_agent_groups` and only then called `get_config()`, which
re-applies the DB's `litellm_settings` over the module globals and threw the
append away. The config it saved was therefore a no-op: the endpoint answered
200 with an empty `public_agent_groups`, the agent never reached
`GET /public/agent_hub`, and re-publishing never hit the "already public" 400.
Read the config first, derive the new list from the refreshed globals, save it,
then update the global
Also fixes the e2e model hub spec, which is flaky for a second reason: the
"Make Models Public" modal preselects the groups that are already public, so a
blind click on "Select All" cleared them and left "Next" disabled for the full
15s action timeout. Check the box instead of toggling it, and wait for "Next"
to be enabled before clicking
Covers the reintroduction of a second module-level cache for the guardrail
translation mappings: remapping the loader between two pre-call hooks must
change which handler runs, and the module must expose no assignable map of
its own.
Streaming /v1/messages against a model served through the chat-completions
bridge (every non-Anthropic provider other than OpenAI) minted its msg_ id
inside the stream wrapper, so the spend row landed under the provider's own
completion id and the caller could not find the call by the only id it saw.
The wrapper now mints the id once in its constructor and hands it to the
logging object, the same way the Responses-API bridge does.
`PrismaClient.spend_log_flush_requested` was an `asyncio.Event` built at
import time, so it bound to whichever event loop first awaited it and every
later loop got `RuntimeError: ... is bound to a different event loop` out of
`_wait_for_spend_log_flush_request`. The queue monitor's blanket `except
Exception` swallowed that into its error logger, so the flush silently never
happened and the row sat in the worker's queue until the next poll.
The monitor now creates its own Event inside the loop that awaits it and
hands it to the client, and `request_spend_log_flush` signals through the
client instead of the class. A request that arrives before the monitor is
running is dropped and loses nothing, because the monitor reads the queue on
its first pass before it ever waits.
In CI this showed up as the proxy-endpoints shard flaking on
test_monitor_spend_logs_queue_flushes_as_soon_as_one_is_requested whenever
--dist=loadscope put the health-endpoint tests, which boot a proxy TestClient
and start a monitor, on the same worker ahead of the spend-log tests.
TestStreamingScanDedup restored the reduced module-level translation
mapping on teardown via monkeypatch, so under --dist=loadscope the
worker that ran only that class carried the reduced mapping into the
streaming block test modules. Tag routing tests now assert the eligible
deployment set directly instead of sampling ten random picks. The
liveliness latency check measures steady-state polls after a warm-up
request rather than the first request through a fresh app.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The documented DISABLE_AIOHTTP_TRANSPORT env var already selects the httpx transport, so the extra module-global write was redundant. Types the monkeypatch fixture while here.
The unified guardrail cached the endpoint translation mappings in its own
module global on top of the loader's cache in litellm/llms. Tests wrote to
that second copy directly, so a teardown that restored a stale snapshot left
a test double installed for every later test on the same xdist worker, and
proxy-endpoints went red on whichever guardrail streaming test happened to
land after it.
Read through load_guardrail_translation_mappings() at each call site and give
the tests one seam to patch, so pytest owns every restore.
Failing closed at the sensitive-data masker's depth of 10 turned an ordinary
nested tool JSON schema into REDACTED leaves, because a list level costs two
depth. The walk now bounds on DEFAULT_MAX_RECURSE_DEPTH, which no real payload
reaches, and the masker's own limit is left alone.
A streaming /v1/messages call against a non-Anthropic model is served an SSE
message_start frame carrying a msg_ id the adapter mints locally, since the
Responses API upstream only issues a resp_ id. That value never left the
adapter, so the spend row was keyed on the bridged response id and
GET /spend/logs?request_id=msg_... came back empty.
The adapter now hands the id it minted to the logging object, and the
/v1/messages logging path keys the row on it.
The "Final returned optional params" line printed whatever the caller nested inside
extra_body, so a credential tucked in there reached stdout in plaintext one line after
the request line that already redacts it.
The call site now runs redact_credentials_in_payload behind a guard reading both of
print_verbose's consumers, litellm.set_verbose and the LiteLLM logger's DEBUG level, so
the line prints in exactly the cases it did before and the walk costs nothing when
nothing would read it.
The proxy's exception tails defaulted `type` and `param` to the four-character
string "None", which is neither a known OpenAI error type nor the JSON null the
nullable `param` field is typed as, so a client's error handler matched nothing
and fell into its generic branch.
Lifts the helpers PR #39521 added for the unified LLM endpoints into
litellm/proxy/common_utils/openai_error_payload.py and calls them from the file,
rerank, image, realtime, anthropic, and pass-through route families, plus the
shared handle_exception_on_proxy handler that the management, batches,
fine-tuning, credential, SCIM, guardrail, and customer routes funnel through.
The remaining families (proxy_server, auth, health, spend tracking, and
management endpoints) follow in separate PRs so each slice stays QA'able on a
live proxy.
Background streaming emits event types with no typed model, which arrive
as GenericEvent holding a plain dict. Only typed events had their nested
response id rewritten, so those frames advertised the raw internal id
while their siblings advertised the encrypted one. The raw shape skips
the ownership check, so any other key could retrieve or cancel that
response.
Rewrite the advertised id wherever an event carries one, whichever shape
it arrives in, so a future event type cannot reopen this.
The two tests that shipped with the fix both called ensure_crypto_provider
themselves, so deleting the call from connect_upstream left the whole suite
green, and swapping ring for aws-lc-rs did too.
Adds an integration test, which gets its own process, that dials wss:// at a
local plain-TCP listener through the public Responses WebSocket entrypoint and
asserts an Err plus an installed provider. Without the install in the dial it
panics with the original CryptoProvider message. A unit test now compares the
installed provider's cipher suites and key-exchange groups against ring's, so
the choice of backend is pinned rather than assumed.
Also names tls12 in the workspace rustls features: it already arrives through
reqwest and tokio-rustls, so the graph is unchanged, but a direct dependency
should say it needs TLS 1.2 rather than inherit it.
The gateway's dependency graph turns on two rustls crypto backends at once:
reqwest's rustls-tls pulls in ring, and litellm-core's bedrock-auth pulls in
aws-lc-rs through aws-config. rustls 0.23 refuses to guess between them, so
ClientConfig::builder panics, and that is exactly how tokio-tungstenite builds
its TLS config. Every outbound WebSocket dial killed its tokio worker and the
client saw the socket vanish with no close frame.
reqwest and the AWS SDK both pick a provider explicitly, so only the tungstenite
path was affected. Route all three dial sites through one helper that installs
ring once per process before connecting.
redact_credentials_in_payload only recursed into mappings, so a
credential-named key one level inside a list or tuple, the shape
extra_body and metadata routinely carry, still reached stdout under
set_verbose. Rebuild sequences element by element too, keeping the
container's own type so the printed repr is unchanged apart from the
secret.
`litellm.set_verbose = True` printed the caller's kwargs verbatim to stdout, so
`api_key` and its siblings landed in terminals and container log drains in
plaintext while the same statement's logger emission was already redacted.
Mask the kwargs at the source with a shared helper in
`litellm_core_utils/sensitive_data_masker.py`, reusing the existing
`SensitiveDataMasker` key classification and the `REDACTED` marker
`secret_redaction.py` already owns, so both debug surfaces agree.
