AsyncHTTPHandler.__del__ was closing httpx clients still in use by
AsyncOpenAI/AsyncAzureOpenAI due to independent cache lifecycles.
Restores standalone httpx client creation for OpenAI/Azure providers.
The no-args hot path (called on every proxy request) was rebuilding
a defaultdict by iterating the full model list each time. Cache the
result and invalidate at all 5 model_list mutation sites following
the _invalidate_model_cost_lowercase_map() pattern.
Line profile: 31.8µs/call → 1.1µs/call (29x improvement).
* fix fail-open for grayswan; pass metadata to cygnal api endpoint; update docs
* pass litellm_metadata to cygnal in payload
* switch error msg to const, and clean exception handling.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 4eece154d0.
* fix: check for model_response_choices before guardrail input
* test: add tests for responses api translation
* fix: protect other guardrail translations
* refactor: remove type ignores
* anthropic request body got mutated fix
* add warning when extra_body is provided but user is non premium
* fix: resolve mypy union-attr errors in anthropic guardrail handler
Cast choices[0] to Choices type before accessing .message attribute
to satisfy mypy's union type checking for Choices | StreamingChoices.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* add logger when model response has no choices for streaming /response and /messages
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 541a2b075a.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 716ea0caa1.
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ui): Add team-alias column to Models Health Status UI
- Added Team Alias column to the Models Health Status table
- Updated HealthCheckComponent to accept teams prop
- Updated health_check_columns to display team alias based on team_id
- Falls back to team_id if team alias not found, or shows '-' if no team
- Updated parent components to pass teams data to HealthCheckComponent
Co-authored-by: ishaan <ishaan@berri.ai>
* Update ui/litellm-dashboard/src/components/model_dashboard/health_check_columns.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
67 vercel_ai_gateway models were missing capability flags (supports_vision,
supports_function_calling, supports_tool_choice, supports_response_schema).
These capabilities were inferred from the corresponding direct provider entries
for the same models (e.g., vercel_ai_gateway/anthropic/claude-3.5-sonnet now has
the same capabilities as anthropic/claude-3.5-sonnet).
Models fixed include:
- Claude 3/3.5/3.7 (Anthropic)
- GPT-4/5 variants (OpenAI)
- Gemini 2.0/2.5 (Google)
- Grok 3/4 (xAI)
- Mistral/Mixtral variants
- Qwen models
- DeepSeek models
- And more
This ensures consistent capability reporting across providers for the same
underlying models.
Co-authored-by: krauckbot <krauckbot123@gmail.com>
Adds litellm.proxy_auth to automatically obtain and refresh OAuth2/JWT
tokens when connecting to LiteLLM Proxy or any OAuth2-protected endpoint.
- Add ProxyAuthHandler for token lifecycle (obtain, cache, refresh)
- Add AzureADCredential wrapper for azure-identity credentials
- Add GenericOAuth2Credential for any OAuth2 provider (Okta, Auth0, etc)
- Auto-inject Authorization headers in completion() and embedding()
Closes#19834
