This change ensures that sensitive message content within the `proxy_server_request` field of spend logs is redacted when message logging is turned off or when the redaction header is used. This enhances privacy and security by preventing the logging of potentially sensitive user inputs in the request details.
Co-authored-by: ishaan <ishaan@berri.ai>
* fix: align max_tokens with max_output_tokens for consistency
Fixed inconsistent max_tokens definitions in model_prices_and_context_window.json.
According to LiteLLM convention, max_tokens should equal max_output_tokens when available.
Models fixed:
- deepseek-chat: 131072 → 8192 (now equals max_output_tokens)
- dashscope/qwen-flash: 1000000 → 32768 (now equals max_output_tokens)
- databricks/databricks-gemma-3-12b: 128000 → 32000 (now equals max_output_tokens)
This ensures consistency across all providers where max_tokens represents
the maximum number of tokens that can be generated in the output.
* fix: align max_tokens with max_output_tokens for 244 models
- Fix 244 models where max_tokens != max_output_tokens
- Add test to validate max_tokens consistency and prevent regressions
According to model_prices_and_context_window.json spec:
- max_tokens is a LEGACY parameter
- Should always equal max_output_tokens when both are present
This ensures consistency across all model definitions.
Fix router embedding methods to properly propagate proxy model
configuration headers to LLM API calls by calling
_update_kwargs_before_fallbacks() just like completion() does.
Previously, router.embedding() and router.aembedding() manually
set num_retries and metadata but didn't call
_update_kwargs_before_fallbacks(), which meant default_litellm_params
(including headers) were not propagated correctly.
Changes:
- Replace manual kwargs setup with _update_kwargs_before_fallbacks()
in _embedding method (litellm/router.py:3318)
- Apply Black formatting to router.py for consistency
- Add comprehensive unit tests for header propagation
- Add integration tests for various router configurations
Tests verify:
- Headers from default_litellm_params are included in embedding calls
- Metadata (model_group) is properly set
- Consistency between completion() and embedding() behavior
- Support for deployment-specific headers, fallbacks, and retries
* fix(azure): add logprobs support for Azure OpenAI GPT-5 models
Azure OpenAI GPT-5 models (including gpt-5.2) support logprobs
parameters, unlike OpenAI's GPT-5 reasoning models. This fix
overrides the parent class restriction to enable logprobs for Azure.
Changes:
- Override get_supported_openai_params() in AzureOpenAIGPT5Config
- Add "logprobs" and "top_logprobs" to supported params
- Add comprehensive tests for logprobs functionality
Testing:
- Verified with direct Azure API calls to gpt-5.2
- API version: 2025-01-01-preview
- Successfully returns logprobs data
Related: #7974, #4022
* refactor: restrict logprobs support to gpt-5.2 only
Only gpt-5.2 has been verified to support logprobs on Azure.
Other gpt-5 variants (gpt-5, gpt-5.1) have not been tested.
Changes:
- Add conditional check for is_model_gpt_5_2_model()
- Update tests to be specific to gpt-5.2
- Add negative tests for gpt-5 and gpt-5.1
- Update documentation to reflect gpt-5.2 specificity
- Add repository check to only run workflow in BerriAI/litellm
- Prevents workflow failures and wasted resources in forked repositories
- Avoids confusion for external contributors
