The org cache entries written by prefetch_auth_objects carry the 5s
DEFAULT_IN_MEMORY_TTL. The first @log_db_metrics getter lazily imports
litellm.proxy.proxy_server, which on a cold CI runner can take longer
than 5s, so the org entry expired before get_org_object read it and the
getter fell through to the MagicMock database. Inject a frozen clock
into InMemoryCache so the test asserts the join, not import latency.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Move the BaseLLMException import into _map_error_event_exception so the
module no longer imports it at load time, clearing the module-level cyclic
import CodeQL flagged. The class is used only on the cold error path.
Replace the mutable list-append test collector with aiter/anext so the
regression tests read the stream immutably.
RouterRateLimitError now carries the model group's deployment ids so it
can tell when every deployment is cooled down, and exposes that as
type=all_deployments_in_cooldown with an explicit message. A partial
cooldown keeps type=rate_limit_error. Either way the proxy no longer
reports type=internal_server_error next to code 429
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Mid-stream error events on the streaming Responses API were all raised as
APIError, so a content_policy_violation event never matched the router's
content-policy fallback dispatch and the client got the raw error instead
of the fallback model's answer. Map each error event's code and status
through the existing exception_type mapping, matching the non-streaming
path, and unwrap the typed ContentPolicyViolationError and
ContextWindowExceededError so the router routes them to the configured
content_policy_fallbacks and context_window_fallbacks.
The generic 400 branch of the OpenAI exception mapper dropped the wire body and no
branch carried the response headers, so an application calling a LiteLLM proxy through
a litellm_proxy/ model could not tell a guardrail block from any other failure without
walking __cause__. BadRequestError now takes headers, filled for a litellm_proxy
upstream, and the generic branch passes the body. The proxy edge treats the literal
"None" type and param an older proxy sends as absent and stops forwarding an upstream
proxy's date and server headers.
CodeQL flagged two module-level cyclic imports introduced by defining
CachedTokensDetails in litellm.types.llms.openai and importing it from
litellm.types.utils and litellm.cost_calculator. The class now lives in
litellm.types.llms.base, which imports nothing from litellm, and every
user imports it from there.
Also pins that combining realtime usages where only one response.done
carries cached_tokens_details keeps the earlier modality split in both
orders, and commits the regenerated dashboard API types.
A guardrail that rewrites text per chat message now returns the rewritten
rows as structured_messages instead of only texts, so the Responses and
chat handlers write the rewrite back through the structured path. The
generic guardrail API response accepts an optional structured_messages
list, Prompt Security modify builds one from modified_messages, and rows a
server echoes back exactly as shown are restored to the original row
objects because the request model drops undeclared keys. Texts-only
per-message answers keep the named rejection on both endpoints.
get_listed_tool now applies tool_name_to_description so the guardrail evaluates the description clients saw, with a regression test. The listed-tools cache and tool_input_schema fields use Mapping annotations so the change adds no LIT001 findings, and schema.d.ts is regenerated with npm run gen:api
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
JWT auth validated Entra app tokens but never carried an agent identity
into the authenticated principal, so agent policies (trace id
requirement, per-agent MCP restrictions, agent spend attribution) only
applied to virtual keys bound to an agent. A new litellm_jwtauth field,
agent_id_jwt_field, names the claim (dot notation supported) that is
matched against a registered agent's id, then name; the canonical
agent_id flows through the standard and proxy-admin JWT paths, and a
configured claim naming no registered agent fails closed with 403
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
CredentialLiteLLMParams omitted tenant_id, client_id, client_secret,
azure_scope, azure_username and azure_password, so the strict dump used
by credential reuse and Azure client init dropped them and the reused
credential ended with no auth at all
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The e2e harness exists to prove product features end to end against a live
proxy. The prior Hard Rule carved out an exception for "tests that cover the
harness itself" and pointed at coverage_registry/test_collector.py, which in
practice invited unit tests of harness helpers to be staged alongside e2e
work. That is the wrong tool: harness logic that is worth locking down does
not need a mock-driven unit test living under tests/e2e.
Drop the carve-out. The Hard Rule now reads that no unit tests of any kind
belong under tests/e2e, and the passing mention of unmarked harness coverage
in the transport section is removed so the doc no longer contradicts itself.
coverage_registry/test_collector.py still exists on disk and is left in place
for now; whether to relocate or remove it is a separate decision.
Keeps the base's rule that a non-admin id lookup matching no spend-log row answers 403, so the detail route never consults cold storage without an owner row
A guardrail that answers one rewritten text per message it saw no longer
matches the texts the Responses handler extracted once the request carries
instructions or tool items, so the rewrite was rejected with a 500. Spread
such an answer over the structured messages' text slots and write it back
through the structured path, have Prompt Security modify return
structured_messages directly, and give the chat completions pairing the same
named rejection instead of a silent misalignment when the counts differ.
