A failure the native path raises without an HTTP error response, such as an unreadable 200 body or a header value the client refuses to send, carries no wire headers, and Python logs none for it. The rebuilt response added a content-length from the error text
The proxy records a client disconnect, with its 499 error information, in the stream generator's cleanup, and the logged payload reads it from metadata. The native teardown billed the stream during cancel propagation, before that cleanup, so the logged row had empty error information
Python's httpx client refuses a non-string header value at send time, after pre_call has logged the resolved call, and the handler maps it to a provider 500. The native handler refused it before the provider request was logged and surfaced an APIConnectionError, so retries, cooldowns and the failure log differed
Python's messages handler calls pre_call with the request body as a json user message, and a stream's spend-log cache_key is computed over that input. The native adapter logged the string Messages, so a streamed call's cache_key differed from Python's
Python raises upstream error statuses as MaskedHTTPStatusError, which rebuilds the httpx response from the decoded body without content-encoding and content-length, so an empty error body logs no content-length. The native failure passed the wire headers through and logged content-length 0
Python updates model_call_details with the transformed request body before pre_call, so the logged messages are what was sent. The native adapter kept the caller's messages and compared the caller's params to the sent ones as JSON values, so an int 0 logged as 0.0
The proxy bills the estimated prompt of a failed call only when the Timeout was caused by an httpx ReadTimeout. The native timeout carried no transport cause, so its failure SpendLogs row and spend counters showed 0 where Python bills the prompt. The native timeout now chains the same httpx timeout Python's client raises.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python wraps a connect, DNS, TLS or reset failure in a synthesized 500 AnthropicError, which maps to InternalServerError with the transport's own text. The native path sent these through the generic fallback, so they surfaced as APIConnectionError with the bug-report footer, which also changes cooldown decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python's messages handler only records custom_llm_provider in model_call_details. The native adapter also wrote it into litellm_params after every provider result, and the proxy reuses one logger across router hops, so a fallback hop's pre_call log showed the provider where Python shows null.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python logs the requested optional params, taken before the provider transform, as model_parameters. The native handler logged the transformed body, so a reasoning_effort request was logged as thinking and output_config. The request context now carries the requested params.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python stores a unary messages response from a background cache write task that runs after the router stamps _hidden_params, so the entry carries model_id and the router headers and a later hit returns x-litellm-cache-key. The native path stored the entry inside the native call, before the router saw the response. The async unary store is now deferred until the route encodes its public response and is handed to Python's own cache write task.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python runs sync success callbacks inside the deferred logging closure, so a post-call hook that blocks the response drops them. The native adapter ran them right away, so a guardrail block still fired sync loggers. A parked native success now carries its sync callbacks and runs them on release.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python's messages handler yields to the event loop before it builds the provider body, so a during_call guardrail started next to the call masks the request first. The native path built its request without yielding, so masked text never reached the provider. The async native dispatch now yields once before calling the native hook.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
When the client disconnects mid-stream, the native adapter now logs success for the chunks it delivered before teardown, so spend, SpendLogs and the key's budget and tpm reservations settle the same way they do on the Python path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtkaNTifWAP4W7JZavE
Python's async client wrapper stamps the effective timeout and num_retries on every failure, which the proxy reads for x-litellm-timeout and the router reads for retries. The native async path skipped that wrapper, so error responses from Rust-served /v1/messages calls had no x-litellm-timeout header
A Messages stream whose upstream body ends mid-frame reached proxy callers as a mapped APIConnectionError with status 500, a stray NoneType line and a bug-report notice. Python's transport raises an unmapped httpx error at that point. The native body read now records a body cut (received, expected, reset) and the host raises the same exception the active transport would, logs the chunk-processor line and leaves cache_hit unset
The proxy defers non-stream success logging past post-call guardrails, and the native host parks its log on _native_pending_logging until the proxy releases it. The release only ran for OCR, so a unary /v1/messages call served by Rust logged no success event, no SpendLogs row and no key, team, user or end-user spend
The proxy attaches its shared aiohttp session to every request and a default stream_timeout to every stream, and the messages detectors treated both as Python-only features, so no proxy request reached Rust. The native client keeps its own pool and already applies stream_timeout as the stream's timeout, so only an explicit client, request_timeout or httpx.Timeout keeps a messages call on Python
* test(e2e): add a Codex row that calls a local MCP tool on a first and a resumed turn
* test(e2e): assert the codex mcp row got each value back from the mcp tool call itself
* test(e2e): check the codex mcp row exits cleanly on both turns
* test(e2e): drop the unused content type from the codex mcp result model
* feat(team): let default_team_params attach access_group_ids to every new team
* fix(types): keep default access_group_ids as Sequence for the type discipline gate
* fix(settings): chunk the default access group existence lookup
* test(team): prove default access groups reach the saved team row and type the settings fixtures
* fix(team): check default access groups on the writer so a fresh group is not rejected behind a read replica
* feat(spend-logs): opt-in latest_turn mode that stores only the current turn of stateless requests
* fix(spend-logs): keep bridged Responses rows and classifier audit in latest_turn mode, reject unknown values at boot
* fix(spend-logs): drop the recursive call in the latest_turn trim and type the stateless test cases
* test(spend-logs): format the stateless latest_turn cases
* test(spend-logs): wrap the classifier audit fixture
* fix(spend-logs): type the latest_turn trim inputs
* fix(spend-logs): leave the raw request body unvalidated before the latest_turn trim
* fix(spend-logs): trim classifier rows and ignore continuation keys off /v1/responses in latest_turn mode
* fix(spend-logs): take the classifier's originating request from the logging payload and redact a copy of vector store results
* fix(spend-logs): redact the body selected for storage, after the latest_turn trim
* feat(proxy): give Codex accurate model entries for every model in /v1/models
* fix(router): read a deployment's upstream model without a not-required key access
* fix(proxy): keep Codex tiers for unpriced deployments and match stock entries by declared provider
A deployment with no token price no longer votes a Codex service tier off,
an undescribed model keeps Codex's default reasoning summary parameter,
a Bedrock deployment naming its provider in custom_llm_provider gets Codex's
stock entry, and Bedrock provider namespaces follow the AWS model id format.
* fix(proxy): import cost tracking and provider helpers lazily in the Codex catalog
lite codex imports the catalog before the SDK is initialized, so importing
cost tracking, the Bedrock helpers or provider resolution at module level hit
a circular import and broke every lite command.
* fix(proxy): drop Codex's fast speed tier wherever its priority tier is dropped
Codex shows /fast for a leftover fast speed tier even when service_tiers
no longer offers priority, so an alias, an empty configured tier list or
a configured list without priority showed a /fast toggle that sends nothing.
One helper now sets service tiers, speed tiers and the default together.
* fix(proxy): give an older Codex a translated entry for models past its version
Codex's own entries name the oldest release that can read them in minimal_client_version. A release older than that can fail to decode the entry, such as a reasoning effort it has no name for, which fails the whole catalog, so it now gets the translated entry, whose levels are trimmed to that release
Reworks the redis-client, connect, resolve and testing crates, adds python-compat json loads and
TTL seconds, replaces the redis-store memory tier with a DualCache plus a shared failure log, and
gives the Python side the Redis timeout log throttle and a request OTEL parent span for service events
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(proxy): let admins allow specific S3 buckets and endpoints in request bodies, block bucket settings otherwise
Client requests can no longer pick s3_bucket_name, s3_output_bucket_name or allow_legacy_cloud_file_ids unless the
value is on general_settings.allowed_client_side_s3_buckets (buckets) or the admin opted in the usual way.
s3_endpoint_url, already blocked, can now be allowed per value through general_settings.allowed_client_side_s3_endpoint_urls
* refactor(proxy): drive the client-side S3 allowlist from one typed field-to-setting map
* fix(proxy): treat a null client-side S3 allowlist in config as empty instead of failing the request
* fix(gemini): omit raw response body from audio transcription error messages
* fix(gemini): raise incomplete transcription interactions as non-retryable 400 and drop response bodies from Vertex transcription errors
* fix(audio_transcription): keep response bodies out of transcription errors
Keep the upstream status on a Gemini interaction that did not complete, so
the router still retries it as it does on main. Gemini stops the same long
clip on some tries and not on others, so a retry can recover it.
Stop quoting the reply in the Deepgram, ElevenLabs, Watsonx, Mistral,
OVHcloud and Scaleway parse errors, and hide pydantic input values on the
Gemini and Vertex response models.
* fix(audio-transcription): keep reply bodies out of watsonx, whisper, xai and validation errors
Watsonx and whisper no longer print the parsed reply after Got: when it has no text field. TranscriptionResponse and the xAI response model hide input in validation errors, so a malformed field cannot quote transcript fragments. Deepgram and ElevenLabs keep the original error text, which is now safe.
* fix(audio-transcription): keep transcripts out of the shared converter's parse errors
* test(audio-transcription): compare wrapped lookup errors instead of pinning upstream wording
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): drop supports_web_search false deployments for Anthropic web_search_* and Gemini googleSearch tools
* test(integration): build the web search filter proxy config without mutating the shared config
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): probe scrypt instead of MD5 in the fips_mode OpenSSL check
* test(proxy): fake a working scrypt so the fips_mode check tests pass on FIPS hosts
* feat(bedrock): enable native structured output for Opus 5 models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): keep govcloud and bare model rows off native output
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): retry rejected native schemas with the JSON tool on default-native models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): keep the rejection retry on the unforceable-tool native path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): retry rejected schemas on the Messages path for default-native models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): keep JSON-tool streaming coverage on unchanged route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): format Messages native output retry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): allow profile-only native output metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): only disable default-native output on schema-rejection retries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): type fixture parameters in native output retry tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): retry with the JSON tool when Bedrock rejects a schema's grammar size
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): drop strict tools after a native-output retry hits the grammar limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): format native output fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(packaging): ship litellm-proxy wheels with the proxy extra
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(packaging): keep litellm-proxy out of the proxy extra until PyPI has it
The current release pipeline publishes litellm with whatever pyproject.toml lists, so pinning an unpublished package would break pip install litellm[proxy]. Images and CI still install the locally built gateway wheel
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(types): define the responses compact call types so during_call guardrails and deployment hooks see them
* test: regenerate dashboard api types and skip compact in the azure sdk client sweep
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The primary stream's mid-stream stall reaches failure_exits as a mapped litellm.Timeout (408), which cools the deployment on the Rust core. Keep that behavior and update the integration test that expected the Python core's exemption. Fallback hops stay exempt through is_stream_stall.
