* fix(anthropic): handle empty streaming tool calls (#28549)
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* [Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing (#28490)
* feat(azure): decouple deployment ID from model name via base_model
Azure OpenAI deployments have arbitrary names (deployment IDs) that may
not match the underlying model. Previously, model-type detection
(o-series, gpt-5, etc.) relied on substring matching against the
deployment name, causing misrouted configs and rejected params when
deployment names were non-standard (e.g. 'my-deployment-id' for gpt-5.2).
This change extends the existing base_model field to drive model-type
detection, config selection, supported param resolution, and param
mapping throughout the Azure call path:
- _get_azure_config() uses base_model for is_o_series/is_gpt_5 checks
- get_provider_chat_config() threads base_model for Azure
- get_supported_openai_params() accepts and uses base_model
- get_optional_params() accepts base_model and passes it to all Azure
config method calls (get_supported_openai_params, map_openai_params)
- azure.py completion handler uses base_model for GPT-5 detection
- Config internal methods (e.g. is_model_gpt_5_2_model) now receive
base_model so features like logprobs are correctly enabled
Fully backward compatible - when base_model is unset, behavior is
identical. Existing o_series/ and gpt5_series/ prefix workarounds
continue to work.
Usage in proxy config:
model_list:
- model_name: my-gpt5
litellm_params:
model: azure/my-deployment-id
model_info:
base_model: azure/gpt-5.2
Fixes: non-standard deployment names like 'prefix-gpt-5.2' rejecting
logprobs/top_logprobs despite the underlying model supporting them.
* Addressing Greptile comments.
* gemini-3.1-flash-lite pricing (#27933)
* feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers
* fix pricing
* add service tier
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
* fix(openai-responses): strip Anthropic cache_control from Responses API requests (#28431)
Squash-merged by litellm-agent from cwang-otto's PR.
* Treat None litellm_provider as wildcard in _check_provider_match (#28523)
Squash-merged by litellm-agent from adityasingh2400's PR.
* fix greptile
* fix: use _azure_detection_model in default Azure branch of get_supported_openai_params
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(openai-responses): strip cache_control on compact endpoint as well
Co-authored-by: Yassin Kortam <yassin@berri.ai>
---------
Co-authored-by: Felipe Garé <90070734+FelipeRodriguesGare@users.noreply.github.com>
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: withomasmicrosoft <withomas@microsoft.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: cwang-otto <chengxuan.wang@ottotheagent.com>
Co-authored-by: Aditya Singh <60082699+adityasingh2400@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Azure GPT-5.4+ models now get the same auto-routing treatment as OpenAI
when both `reasoning_effort` and `tools` are used in `litellm.completion()`.
Previously, `reasoning_effort` was silently dropped for Azure; now the
request is bridged to the Responses API which supports both parameters.
Fixes#23914
- Add _get_effort_level() to extract effective effort from string or dict
- Use effective_effort for xhigh validation, tool-drop, sampling, temperature guards
- Preserve dict format when it has summary/generate_summary for Responses API
- Add tests: xhigh-dict validation, none-dict for tools/sampling/temperature
- Update tests: dict-with-summary now preserved (not normalized)
Made-with: Cursor
* fix:fix: prompt_cache_key OAI + Azure OpenAI
* test_prompt_cache_key_supported
* test_azure_openai_with_prompt_cache_key
* fix: remove unnecessary async from test_azure_openai_with_prompt_cache_key
Addresses Greptile feedback: litellm.completion() is synchronous, so
async def is unnecessary and would silently pass without running.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove unused filter_and_transform_beta_headers imports
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test_azure_openai_with_prompt_cache_key
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(azure): add logprobs support for Azure OpenAI GPT-5 models
Azure OpenAI GPT-5 models (including gpt-5.2) support logprobs
parameters, unlike OpenAI's GPT-5 reasoning models. This fix
overrides the parent class restriction to enable logprobs for Azure.
Changes:
- Override get_supported_openai_params() in AzureOpenAIGPT5Config
- Add "logprobs" and "top_logprobs" to supported params
- Add comprehensive tests for logprobs functionality
Testing:
- Verified with direct Azure API calls to gpt-5.2
- API version: 2025-01-01-preview
- Successfully returns logprobs data
Related: #7974, #4022
* refactor: restrict logprobs support to gpt-5.2 only
Only gpt-5.2 has been verified to support logprobs on Azure.
Other gpt-5 variants (gpt-5, gpt-5.1) have not been tested.
Changes:
- Add conditional check for is_model_gpt_5_2_model()
- Update tests to be specific to gpt-5.2
- Add negative tests for gpt-5 and gpt-5.1
- Update documentation to reflect gpt-5.2 specificity
* fix(azure/chat/gpt_transformation.py): support api_version="preview"
Fixes https://github.com/BerriAI/litellm/issues/12945
* Fix anthropic passthrough logging handler model fallback for streaming requests (#13022)
* fix: anthropic passthrough logging handler model fallback for streaming requests
- Add fallback logic to retrieve model from logging_obj.model_call_details when request_body.model is empty
- Fixes issue #12933 where streaming requests to anthropic passthrough endpoints would crash due to missing model field
- Ensures downstream logging and cost calculation work correctly for all streaming scenarios
- Maintains backwards compatibility with existing non-streaming requests
* test: add minimal tests for anthropic passthrough logging handler model fallback
- Add unit tests for the model fallback logic in _handle_logging_anthropic_collected_chunks
- Test existing behavior when request_body.model is present
- Test fallback logic when request_body.model is empty but logging_obj.model_call_details has model
- Test edge cases where both sources are empty or missing
- Ensure backwards compatibility and graceful degradation
* fix(anthropic_passthrough_logging_handler.py): add provider to model name (accurate cost tracking)
* fix(anthropic_passthrough_logging_handler.py): don't reset custom llm provider, if already set
* fix: fix check
---------
Co-authored-by: Haggai Shachar <haggai.shachar@backline.ai>
* fix(utils.py): support non default params for audio transcription
allows passing provider specific params straight through on transcription calls
* fix(gpt_transformation.py): fix o_series model routing
call _transform_request on async event
* refactor: refactor tests
* test(test_azure_chat_o_series_transformation.py): add unit test for azure o series error
* test: update test
* test: update json
* fix: fix mutiple keyword error
