Add detection for Cerebras's context window exceeded error format:
"Current length is X while limit is Y"
This ensures LiteLLM raises ContextWindowExceededError instead of
generic BadRequestError when Cerebras API calls exceed the model's
context limit, enabling downstream libraries like DSPy to properly
catch and handle these errors for automatic context management.
The 'user' parameter was being ignored when using responses API models
(e.g., model="openai/responses/gpt-4.1") because the model name check
in get_supported_openai_params() didn't account for the "responses/" prefix.
Fix: Normalize the model name by stripping "responses/" prefix before
checking if the model is in the list of supported OpenAI models.
This is a minimal, non-breaking change that:
- Adds 2 lines of code in gpt_transformation.py
- Only affects the parameter support check, not the model variable itself
- Includes unit and integration tests
When OpenAI Responses API returns both text AND tool_calls, the bridge
transformation was emitting is_finished=True after the text message completed,
causing subsequent tool_call chunks to be dropped.
The fix:
- response.output_item.done for messages no longer emits is_finished=True
- Added handler for response.completed to properly signal stream end
* refactor: remove api-key conversion logic for Azure Anthropic
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
* fix(passthrough): pass custom_llm_provider to completion_cost for Azure AI Anthropic
The passthrough logging for Anthropic was failing when using Azure AI Anthropic
because the completion_cost function was not receiving the custom_llm_provider
parameter, causing it to fail with "LLM Provider NOT provided" error.
This fix:
- Retrieves custom_llm_provider from logging_obj.model_call_details
- Prepends provider prefix to model name for cost calculation
- Passes both formatted model and custom_llm_provider to completion_cost
- Centralizes provider prefix logic in _create_anthropic_response_logging_payload
This ensures cost calculation works correctly for Azure AI Anthropic requests
with models like azure_ai/claude-sonnet-4-5_gb_20250929.
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
* test: add unit tests for Azure AI Anthropic fixes
- Add tests for custom_llm_provider cost calculation in passthrough logging
- Add tests for ProviderConfigManager returning AzureAnthropicMessagesConfig
- Update existing tests to reflect removal of api-key to x-api-key conversion
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
---------
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
Fixes#17473 - Anthropic streaming fails with JSONDecodeError when
network fragmentation causes SSE data to arrive in partial chunks.
Changes:
- Add accumulated_json buffer and chunk_type to ModelResponseIterator
- Add _handle_accumulated_json_chunk() to accumulate partial JSON
- Add _parse_sse_data() to handle both complete and partial chunks
- Modify __next__ and __anext__ to use accumulation logic
- Add unit tests for partial chunk handling
The async_post_call_streaming_iterator_hook function was broken:
1. Was a sync function (def) not async generator
2. Returned AsyncGenerator without iterating it
3. Callback generators were chained but never consumed
This fix:
1. Makes the function an async generator (async def + yield)
2. Actually iterates through the chained callbacks with 'async for'
3. Properly yields chunks to the caller
Fixes#9639
* Add Amazon Nova as a first party provider
* Added new provider folder under llms/ to outline the openai supported params
* Updated supported endpoints on the documnetation
- Reset mock call counts at start of test to ensure clean state
- Add span method to mock trace to handle log_provider_specific_information_as_span calls
- Re-establish mock chain before test call to ensure fresh state
- Add exception handling to catch and report errors during test execution
- Add verification that trace was called before checking generation
This should fix the flaky test that was failing intermittently with
'Expected generation to have been called once. Called 0 times.'
- Remove network dependency by mocking HuggingFace template fetch
- Use mock template that produces correct format for test validation
- Test now focuses on transformation logic, not network calls
- Fixes flaky test failures due to network timeouts/rate limits
The test verifies that prompt transformation occurs (not simple
concatenation), which doesn't require the actual HuggingFace template.
Mocking makes the test deterministic and faster while still validating
the core behavior.
