Same bug as team budget: _assemble_user_object fetched user info from DB
but only used budget_reset_at, discarding max_budget. When the key cache
has a stale None for user_max_budget, _safe_get_remaining_budget returns
+Inf. Now falls back to DB max_budget when metadata value is None.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The file was at the repo root and excluded from pip distributions. Moving it to litellm/proxy/public_endpoints/ alongside the other provider JSON files ensures it is packaged correctly. Updates all references in the endpoint handler, coverage tests, and release notes instructions.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Return all supported endpoints and which providers support them. Includes endpoint display names, URL paths, and per-provider support lists. Results are cached for the process lifetime.
Also adds comprehensive test coverage for the new endpoint.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Updated MidStreamFallbackError to retrieve and maintain the original status code from the wrapped exception.
- Ensured that message, request, and response fields remain consistent after calling the parent constructor.
- Added unit tests to verify the correct propagation of status codes and attributes in various scenarios.
- compute image_generation cost from usage token metadata for vertex/gemini\n- map ImageUsage to Usage and reuse generic_cost_per_token\n- fallback to output_cost_per_image when usage metadata missing\n- add tests for token-based path and fallback path
A database timeout (httpcore.ReadTimeout) during get_config() in
_update_llm_router would propagate and prevent ALL DB models from
loading into the router.
Now get_config() failures are caught separately so model add/delete
operations still proceed. Similarly, _delete_deployment catches
get_config failures and safely skips cleanup rather than crashing the
entire sync cycle.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- add gemini-3.1-flash-image-preview + vertex_ai alias entries\n- set pricing to Gemini 3.1 Flash Image Preview rates\n- mirror updates in packaged backup model map\n- update llm cost calc regression test to cover new model
Claude 4.6 models use output_config as a stable API feature. This commit:
- Maps reasoning_effort to output_config for 4.6 models (minimal → low)
- Restricts effort="max" to Opus 4.6 only
- Skips beta header injection for 4.6 models
- Updates docs for Claude 4.6 effort support
Phase 2 (per-worker mark_process_dead on shutdown) only ever fired
when all workers shut down together, making it redundant — Phase 1
wipes everything on next startup anyway. This aligns with the
prometheus_client docs: just wipe the directory between runs.
LiteLLM was adding a `duration` field to audio transcription responses
for internal cost tracking. The OpenAI Python SDK uses "best match
deserialization" to determine the response type from present fields —
seeing `duration` caused it to incorrectly match plain Transcription
responses as TranscriptionVerbose/TranscriptionDiarized types.
Move the internally-calculated duration to `_hidden_params` so it
remains available for cost calculation without polluting the response
body. Provider-returned duration (e.g. from verbose_json format) is
still preserved in the response as expected.
Adds TestProxyMcpStatelessBehavior to test_proxy_mcp_e2e.py with a test
that verifies two independent MCP clients can connect, initialize, and
call tools without sharing session state. This catches the regression
from PR #19809 where stateless=False broke clients that don't manage
mcp-session-id headers.
Regression test for #20242
The tests were mocking `filter_server_ids_by_ip` but the production
code in server.py now calls `filter_server_ids_by_ip_with_info` which
returns a (server_ids, blocked_count) tuple. Update all 8 mock sites
to use the correct method name and return signature.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add unit test that scans Python source for Base64 Basic Auth patterns
that would be flagged by secret scanners like GitGuardian/ggshield
- Add secret-scan job to the linting CI workflow that runs the test on
every PR and optionally runs ggshield if GITGUARDIAN_API_KEY is set
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old test assumed ArizePhoenixLogger reused the global TracerProvider.
With the nested traces fix, Phoenix now creates its own dedicated provider
and produces litellm_proxy_request + litellm_request + raw_gen_ai_request
spans independently.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- ArizePhoenixLogger now creates spans on its own dedicated TracerProvider
instead of trying to reuse parent spans from the global otel TracerProvider
(which were invisible in Phoenix since they go to a different exporter)
- Auto-initialize ArizePhoenixLogger when otel callback is configured and
Phoenix env vars (PHOENIX_API_KEY, PHOENIX_COLLECTOR_*) are detected
- Use exact type check in get_custom_logger_compatible_class to prevent
ArizePhoenixLogger (subclass) from being returned when looking up otel
- Fix tool_permission guardrail to check non-function tools like
code_interpreter (previously skipped with `type != "function"`)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(adapter): populate cache_read_input_tokens from prompt_tokens_details
The Anthropic adapter's translate_openai_response_to_anthropic checked
only the private _cache_read_input_tokens attr (set by Anthropic/DeepSeek)
but not prompt_tokens_details.cached_tokens (set by OpenAI/Azure).
Use prompt_tokens_details.cached_tokens directly — it is already extracted
and is the standard field populated by all providers.
Fixes#22089
* fix(adapter): apply same cache_read_input_tokens fix to streaming path
The streaming path in translate_streaming_openai_response_to_anthropic
had the same bug — relying on _cache_read_input_tokens instead of
prompt_tokens_details.cached_tokens.
* fix(proxy): improve auth exception logging levels and add structured context
Downgrade expected auth failures (ProxyException, HTTPException < 500,
BudgetExceededError) from ERROR to WARNING log level to reduce noise from
routine rejected requests (e.g. missing/invalid API keys on polled endpoints
like /schedule/model_cost_map_reload/status).
Unexpected exceptions and HTTPException with status >= 500 still log at
ERROR with full traceback.
Enrich log messages with structured context: route, HTTP method, masked
API key (using existing abbreviate_api_key), error type, and error code.
All fields also passed via log extra dict for log aggregation tools.
Fixes#21293
* Update tests/test_litellm/proxy/auth/test_auth_exception_handler.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
