- Add /v1/agents/{agent_id} to agent_routes so internal users can
access GET-by-ID (previously returned 403 due to missing route pattern)
- Add _check_agent_management_permission() guard to POST, PUT, PATCH,
DELETE agent endpoints — only PROXY_ADMIN may mutate agents
- Add user_api_key_dict param to delete_agent so the role check works
- Add comprehensive unit tests for RBAC enforcement across all roles
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Add new /public/endpoints endpoint that returns which providers support each LiteLLM
endpoint (e.g., chat_completions, embeddings). The endpoint reads from a local backup
JSON file bundled with the package, caches the result in-process, and transforms the
raw provider-centric data into an endpoint-centric response format.
Changes:
- Add litellm/provider_endpoints_support_backup.json (copy of root source file)
- Add Pydantic response models (EndpointProvider, SupportedEndpoint, SupportedEndpointsResponse)
- Add /public/endpoints route with transformation and caching logic
- Add 16 comprehensive tests covering HTTP layer and transformation functions
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix(gemini): enable inputAudioTranscription and handle transcription events for realtime guardrails
Gemini sends inputTranscription/outputTranscription inside serverContent separately from modelTurn/turnComplete. This adds handling to convert them into OpenAI-compatible events so the guardrail pipeline can inspect voice input, and enables inputAudioTranscription in the session setup config.
Made-with: Cursor
* fix(vertex_ai): enable inputAudioTranscription in realtime session config
Add inputAudioTranscription to the Vertex AI realtime setup so the backend returns transcripts of user speech, allowing guardrails to inspect voice input.
Made-with: Cursor
* fix(realtime): pass user_api_key_dict and guardrail metadata through async_realtime handler
The base LLM HTTP handler's async_realtime method was not accepting or forwarding user_api_key_dict and litellm_metadata to RealTimeStreaming. This meant guardrails configured with default_on=false were silently skipped for all provider_config-based realtime connections (Gemini, Vertex AI, etc). Also fixes wss:// connections when SSL_VERIFY=False by overriding ssl=False for secure WebSocket URLs.
Made-with: Cursor
* fix(realtime): forward guardrail metadata for generic provider_config and vertex_ai paths
The _arealtime function was not passing user_api_key_dict or litellm_metadata to base_llm_http_handler.async_realtime() for the generic provider_config path and the vertex_ai-specific path. This broke guardrail resolution since RealTimeStreaming.request_data was empty, causing should_run_guardrail to return False.
Made-with: Cursor
* fix(realtime): voice guardrail responses and block duplicate response.create on text input
When a guardrail blocks voice input, send a conversation.item.create + response.create to the backend so the LLM voices the guardrail message as audio instead of only returning text. Also adds pending_guardrail_message tracking to suppress the automatic response.create the client sends after a blocked text message, and broadens _has_audio_transcription_guardrails to match pre_call/post_call modes.
Made-with: Cursor
* test(realtime): update guardrail tests for broadened audio transcription check and add integration tests
Update existing tests to reflect that pre_call guardrails now correctly trigger the audio/VAD session.update injection. Add integration test file for live OpenAI realtime guardrail testing.
Made-with: Cursor
* fix(realtime): instruct LLM to say exact guardrail message verbatim
The previous prompt gave the LLM creative freedom to paraphrase the guardrail violation message. Now it instructs the LLM to repeat the exact configured message word for word.
Made-with: Cursor
* fix(realtime): preserve wss ssl semantics and move live guardrail test
Keep TLS enabled for wss realtime sessions while honoring SSL_VERIFY=False via a no-verify SSLContext, move the OpenAI live guardrail test into llm_translation, and dedupe duplicated guardrail-detection helpers to prevent drift.
Made-with: Cursor
Same bug as team budget: _assemble_user_object fetched user info from DB
but only used budget_reset_at, discarding max_budget. When the key cache
has a stale None for user_max_budget, _safe_get_remaining_budget returns
+Inf. Now falls back to DB max_budget when metadata value is None.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Return all supported endpoints and which providers support them. Includes endpoint display names, URL paths, and per-provider support lists. Results are cached for the process lifetime.
Also adds comprehensive test coverage for the new endpoint.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- compute image_generation cost from usage token metadata for vertex/gemini\n- map ImageUsage to Usage and reuse generic_cost_per_token\n- fallback to output_cost_per_image when usage metadata missing\n- add tests for token-based path and fallback path
- add gemini-3.1-flash-image-preview + vertex_ai alias entries\n- set pricing to Gemini 3.1 Flash Image Preview rates\n- mirror updates in packaged backup model map\n- update llm cost calc regression test to cover new model
Phase 2 (per-worker mark_process_dead on shutdown) only ever fired
when all workers shut down together, making it redundant — Phase 1
wipes everything on next startup anyway. This aligns with the
prometheus_client docs: just wipe the directory between runs.
The tests were mocking `filter_server_ids_by_ip` but the production
code in server.py now calls `filter_server_ids_by_ip_with_info` which
returns a (server_ids, blocked_count) tuple. Update all 8 mock sites
to use the correct method name and return signature.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- ArizePhoenixLogger now creates spans on its own dedicated TracerProvider
instead of trying to reuse parent spans from the global otel TracerProvider
(which were invisible in Phoenix since they go to a different exporter)
- Auto-initialize ArizePhoenixLogger when otel callback is configured and
Phoenix env vars (PHOENIX_API_KEY, PHOENIX_COLLECTOR_*) are detected
- Use exact type check in get_custom_logger_compatible_class to prevent
ArizePhoenixLogger (subclass) from being returned when looking up otel
- Fix tool_permission guardrail to check non-function tools like
code_interpreter (previously skipped with `type != "function"`)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(adapter): populate cache_read_input_tokens from prompt_tokens_details
The Anthropic adapter's translate_openai_response_to_anthropic checked
only the private _cache_read_input_tokens attr (set by Anthropic/DeepSeek)
but not prompt_tokens_details.cached_tokens (set by OpenAI/Azure).
Use prompt_tokens_details.cached_tokens directly — it is already extracted
and is the standard field populated by all providers.
Fixes#22089
* fix(adapter): apply same cache_read_input_tokens fix to streaming path
The streaming path in translate_streaming_openai_response_to_anthropic
had the same bug — relying on _cache_read_input_tokens instead of
prompt_tokens_details.cached_tokens.
* fix(proxy): improve auth exception logging levels and add structured context
Downgrade expected auth failures (ProxyException, HTTPException < 500,
BudgetExceededError) from ERROR to WARNING log level to reduce noise from
routine rejected requests (e.g. missing/invalid API keys on polled endpoints
like /schedule/model_cost_map_reload/status).
Unexpected exceptions and HTTPException with status >= 500 still log at
ERROR with full traceback.
Enrich log messages with structured context: route, HTTP method, masked
API key (using existing abbreviate_api_key), error type, and error code.
All fields also passed via log extra dict for log aggregation tools.
Fixes#21293
* Update tests/test_litellm/proxy/auth/test_auth_exception_handler.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
