The `if hasattr(...)` guards in test_acompletion_with_mcp_adds_metadata_to_streaming
and test_acompletion_with_mcp_streaming_metadata_in_correct_chunks could silently skip
the provider_specific_fields assertions if chunks lacked choices/delta. Replace with
unconditional `assert hasattr(...)` so failures surface immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes test failures that occur during parallel test execution (pytest -n 4)
due to module reloading issues with conftest.py reloading litellm.
Changes:
- Add module reload fixtures to ensure fresh references after conftest reloads
- Use patch.object and string-based patches instead of direct attribute assignment
- Use class name comparison instead of isinstance for reloaded modules
- Handle case where litellm is missing from sys.modules during parallel runs
- Move stream consumption inside patch contexts to avoid real API calls
- Mock litellm.acompletion instead of low-level HTTP handlers
- Add skipif decorator for enterprise-only test classes
Affected test files:
- test_container_integration.py
- test_responses_background_cost.py
- test_huggingface_embedding_handler.py
- test_vertex_ai_rerank_integration.py
- test_volcengine_responses_transformation.py
- test_pillar_guardrails.py
- test_litellm_pre_call_utils.py
- test_proxy_server.py
- test_converse_transformation.py
- test_chat_completions_handler.py
- test_aresponses_api_with_mcp.py
- test_anthropic_experimental_pass_through_messages_handler.py
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve metadata for custom callbacks on codex/responses path (#21204)
- Use metadata or litellm_metadata when calling update_environment_variables
in responses/main.py so metadata is not overwritten by None on the
bridge path (completion -> responses API).
- Add tests for metadata in custom callback for codex models and for
litellm_metadata in aresponses().
Co-authored-by: Cursor <cursoragent@cursor.com>
* Update tests/test_litellm/responses/test_metadata_codex_callback.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
1. test_acompletion_with_mcp_streaming_metadata_in_correct_chunks:
- Moved stream consumption inside patch context to avoid real API calls
- The previous implementation had assertions outside the `with patch(...)`
block, causing real OpenAI API calls when consuming the stream
2. TestCheckResponsesCost tests:
- Added skip condition when litellm_enterprise module is not available
- These tests import from litellm_enterprise.proxy.common_utils.check_responses_cost
which is only available in the enterprise version
* fix(vertex_ai): forward extra_body to completion transformation handler
The responses() function accepted extra_body as a named parameter but
did not pass it to response_api_handler when responses_api_provider_config
was None (completion transformation path), silently dropping it.
Also adds deep-merge support for extra_body in Vertex AI Gemini
transformation, so dict values like generationConfig are merged rather
than replaced.
* refactor(vertex_ai): extract _merge_extra_body to fix PLR0915 lint
Move the extra_body merge loop into a helper function to keep
_transform_request_body under the 50-statement limit.
When transforming chat completion responses to Responses API format,
image_tokens from completion_tokens_details was not being included
in output_tokens_details. This affected Vertex AI/Gemini models that
return image token counts in candidatesTokensDetails with modality="IMAGE".
The fix adds image_tokens handling alongside existing reasoning_tokens
and text_tokens transformation.
When using the responses API with provider-specific params (aws_*, vertex_*)
without explicitly passing custom_llm_provider, the code crashed with:
AttributeError: 'NoneType' object has no attribute 'startswith'
Root cause: local_vars was captured via locals() before get_llm_provider()
detected the provider from the model string (e.g., "bedrock/..."), so
custom_llm_provider remained None when processing provider-specific params.
Fix: Update local_vars["custom_llm_provider"] after get_llm_provider() call
so the detected provider is available for param processing.
Affected provider-specific params:
- aws_* (aws_region_name, aws_access_key_id, etc.) for Bedrock/SageMaker
- vertex_* (vertex_project, vertex_location, etc.) for Vertex AI
* fix: preserve tool output ordering for gemini in responses bridge
- Keep function_call_output adjacent to its function_call when building chat messages
- Normalize function_call_output.output lists (input_* parts) into tool message content
* fix test
* small improvements
Emit Responses API streaming events for tool calls when the underlying chat stream contains tool_call deltas, and recover tool calls into the stream when they only appear in the final response.
Calculate `total_tokens` in usage data in Response manually if:
- `total_tokens` is missing
- `total_tokens` can be calculated from input and output tokens
Run the test for this feature with:
`poetry run pytest tests/test_litellm/responses/test_responses_utils.py -k "test_transform_response_api_usage_calculates_total_from_input_and_output_tokens_if_available" -v`
Fixed three flaky tests that were intermittently failing in CI:
1. test_no_duplicate_spend_logs (test_litellm/responses/test_no_duplicate_spend_logs.py)
Problem: Used await asyncio.sleep(1) to wait for async logging completion,
which created race conditions. The async logging worker queues tasks
in the background, and sleep() doesn't guarantee completion.
Fix: Replaced sleep() with GLOBAL_LOGGING_WORKER.flush() which properly waits
for the logging queue to empty, ensuring all async logging tasks complete
before assertions run.
2. test_log_langfuse_v2_handles_null_usage_values (test_litellm/integrations/test_langfuse.py)
Problem: Used datetime.datetime.now() twice for start_time and end_time, which
could cause timing inconsistencies between test runs, especially in
CI environments with variable execution speeds.
Fix: Use fixed timestamps instead of datetime.now() to ensure consistent timing
across all test runs, eliminating timing-related flakiness.
3. test_watsonx_gpt_oss_prompt_transformation (test_litellm/llms/watsonx/test_watsonx.py)
Problem: Directly accessed mock_post.call_args without checking if it exists,
which could be None if the mock wasn't called or if an exception
occurred before the POST request. The test catches exceptions and
continues, making this a potential failure point.
Fix: Added proper assertions and use call_args_list[0] for safer access:
- Assert that call_args_list has at least one call
- Assert that call_args is not None
- Assert that 'data' key exists in kwargs
This ensures the test fails with clear error messages rather than
intermittent AttributeError exceptions.
All fixes maintain the original test intent while making them deterministic
and reliable in CI environments.
* fix(responses): Add image generation support for Responses API
Fixes#16227
## Problem
When using Gemini 2.5 Flash Image with /responses endpoint, image generation
outputs were not being returned correctly. The response contained only text
with empty content instead of the generated images.
## Solution
1. Created new `OutputImageGenerationCall` type for image generation outputs
2. Modified `_extract_message_output_items()` to detect images in completion responses
3. Added `_extract_image_generation_output_items()` to transform images from
completion format (data URL) to responses format (pure base64)
4. Added `_extract_base64_from_data_url()` helper to extract base64 from data URLs
5. Updated `ResponsesAPIResponse.output` type to include `OutputImageGenerationCall`
## Changes
- litellm/types/responses/main.py: Added OutputImageGenerationCall type
- litellm/types/llms/openai.py: Updated ResponsesAPIResponse.output type
- litellm/responses/litellm_completion_transformation/transformation.py:
Added image detection and extraction logic
- tests/test_litellm/responses/litellm_completion_transformation/test_image_generation_output.py:
Added comprehensive unit tests (16 tests, all passing)
## Result
/responses endpoint now correctly returns:
```json
{
"output": [{
"type": "image_generation_call",
"id": "..._img_0",
"status": "completed",
"result": "iVBORw0KGgo..." // Pure base64, no data: prefix
}]
}
```
This matches OpenAI Responses API specification where image generation
outputs have type "image_generation_call" with base64 data in "result" field.
* docs(responses): Add image generation documentation and tests
- Add comprehensive image generation documentation to response_api.md
- Include examples for Gemini (no tools param) and OpenAI (with tools param)
- Document response format and base64 handling
- Add supported models table with provider-specific requirements
- Add unit tests for image generation output transformation
- Test base64 extraction from data URLs
- Test image generation output item creation
- Test status mapping and integration scenarios
- Verify proper transformation from completions to responses format
Related to #16227
* fix(responses): Correct status type for image generation output
- Add _map_finish_reason_to_image_generation_status() helper function
- Fix MyPy type error: OutputImageGenerationCall.status only accepts
['in_progress', 'completed', 'incomplete', 'failed'], not the full
ResponsesAPIStatus union which includes 'cancelled' and 'queued'
Fixes MyPy error in transformation.py:838
* fix: prevent duplicate spend logs in Responses API for non-OpenAI providers
Fixes#15740
This fixes a logging duplication bug where using kwargs.pop() removed
the litellm_logging_obj before passing kwargs to internal acompletion()
calls, causing duplicate spend log entries for providers without native
Responses API support (Anthropic, Gemini, etc).
By changing from pop() to get(), the logging object is preserved and
reused across the internal completion call, preventing duplicate entries
and maintaining correct cost tracking.
* test: add test for logging object preservation in responses API
Verify that litellm_logging_obj is preserved in kwargs when calling
responses(), ensuring no duplicate spend log entries are created.
