Azure audio models were charging audio output tokens at the text token
rate instead of the correct audio token rate. This resulted in costs
being ~6.65x lower than expected.
The fix replaces Azure's custom cost calculation logic with the generic
cost calculator that properly handles text, audio, cached, reasoning,
and image tokens.
Fixes#19764
When using the responses API with provider-specific params (aws_*, vertex_*)
without explicitly passing custom_llm_provider, the code crashed with:
AttributeError: 'NoneType' object has no attribute 'startswith'
Root cause: local_vars was captured via locals() before get_llm_provider()
detected the provider from the model string (e.g., "bedrock/..."), so
custom_llm_provider remained None when processing provider-specific params.
Fix: Update local_vars["custom_llm_provider"] after get_llm_provider() call
so the detected provider is available for param processing.
Affected provider-specific params:
- aws_* (aws_region_name, aws_access_key_id, etc.) for Bedrock/SageMaker
- vertex_* (vertex_project, vertex_location, etc.) for Vertex AI
Fixes#19754
The gpt-5.2-codex model only supports the responses API, not chat completions.
Updated azure/gpt-5.2-codex and openrouter/openai/gpt-5.2-codex entries to use
mode: "responses" and supported_endpoints: ["/v1/responses"].
* Fix broken mocks in 6 flaky tests to prevent real API calls
Added network-level HTTP blocking using respx to prevent tests from making real API calls when Python-level mocks fail. This makes tests more reliable and retryable in CI.
Changes:
- Azure OIDC test: Added Azure Identity SDK mock to prevent real Azure calls
- Vector store test: Added @respx.mock decorator to block HTTP requests
- Resend email tests (3): Added @respx.mock decorator for all 3 test functions
- SendGrid email test: Added @respx.mock decorator
All test assertions and verification logic remain unchanged - only added safety nets to catch leaked API calls.
* Fix failing OIDC secret manager tests
Fixed two test failures in test_secret_managers_main.py:
1. test_oidc_azure_ad_token_success: Corrected the patch path for get_bearer_token_provider from 'litellm.secret_managers.get_azure_ad_token_provider.get_bearer_token_provider' to 'azure.identity.get_bearer_token_provider' since the function is imported from azure.identity.
2. test_oidc_google_success: Added @patch('httpx.Client') decorator to prevent any real HTTP connections during test execution, resolving httpx.ConnectError issues.
Both tests now pass successfully.
* Fix PLR0915: Extract system message handling to reduce statement count
* fix mypy
* fix: add host_progress_callback parameter to mock_call_tool in test
The test_call_tool_without_broken_pipe_error was failing because the mock function did not accept the host_progress_callback keyword argument that the actual implementation passes to client.call_tool(). Updated the mock to accept this parameter to match the real implementation signature.
* fixing flaky tests around oidc and email
* Add documentation comment to test file
* add retry
* add dependency
* increase retry
---------
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
**Root Cause:**
When API keys or teams have router_settings configured, the proxy creates
per-request Router instances from user_config. These new routers were missing
search_tools from the main router, causing "search tool not found" errors despite
search_tools being configured.
**The Fix:**
1. **common_request_processing.py (lines 554-556):** Pass search_tools from main
router to user_config so per-request routers inherit them
2. **proxy_server.py (line 3171):** Remove `if len(_model_list) > 0` check to
allow router creation with empty model list (needed for search-tools-only use case)
3. **proxy_server.py (line 746):** Remove redundant search_tools loading code
(already handled by _init_search_tools_in_db() called during startup)
* cache control for user messages and system messages
* add cache createion tokens in reponse
* cache controls in tool calls and assistant turns
* refactor with _should_preserve_cache_control
* add cache control unit tests
* use simpler cache creation token count logic
* use helper function
* remove unused function
* fix unit tests
