scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-03 02:22:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 81
litellm/tests/test_litellm
History
Cesar Garcia 87f94172a9
fix(responses): Add image generation support for Responses API (#16586) 
* fix(responses): Add image generation support for Responses API

Fixes #16227

## Problem
When using Gemini 2.5 Flash Image with /responses endpoint, image generation
outputs were not being returned correctly. The response contained only text
with empty content instead of the generated images.

## Solution
1. Created new `OutputImageGenerationCall` type for image generation outputs
2. Modified `_extract_message_output_items()` to detect images in completion responses
3. Added `_extract_image_generation_output_items()` to transform images from
   completion format (data URL) to responses format (pure base64)
4. Added `_extract_base64_from_data_url()` helper to extract base64 from data URLs
5. Updated `ResponsesAPIResponse.output` type to include `OutputImageGenerationCall`

## Changes
- litellm/types/responses/main.py: Added OutputImageGenerationCall type
- litellm/types/llms/openai.py: Updated ResponsesAPIResponse.output type
- litellm/responses/litellm_completion_transformation/transformation.py:
  Added image detection and extraction logic
- tests/test_litellm/responses/litellm_completion_transformation/test_image_generation_output.py:
  Added comprehensive unit tests (16 tests, all passing)

## Result
/responses endpoint now correctly returns:
```json
{
  "output": [{
    "type": "image_generation_call",
    "id": "..._img_0",
    "status": "completed",
    "result": "iVBORw0KGgo..."  // Pure base64, no data: prefix
  }]
}
```

This matches OpenAI Responses API specification where image generation
outputs have type "image_generation_call" with base64 data in "result" field.

* docs(responses): Add image generation documentation and tests

- Add comprehensive image generation documentation to response_api.md
  - Include examples for Gemini (no tools param) and OpenAI (with tools param)
  - Document response format and base64 handling
  - Add supported models table with provider-specific requirements

- Add unit tests for image generation output transformation
  - Test base64 extraction from data URLs
  - Test image generation output item creation
  - Test status mapping and integration scenarios
  - Verify proper transformation from completions to responses format

Related to #16227

* fix(responses): Correct status type for image generation output

- Add _map_finish_reason_to_image_generation_status() helper function
- Fix MyPy type error: OutputImageGenerationCall.status only accepts
  ['in_progress', 'completed', 'incomplete', 'failed'], not the full
  ResponsesAPIStatus union which includes 'cancelled' and 'queued'

Fixes MyPy error in transformation.py:838
2025-12-05 15:56:26 -08:00
..
caching fix(redis): handle float redis_version from AWS ElastiCache Valkey (#16207) 2025-11-04 19:20:00 -08:00
completion_extras Merge pull request #16844 from Chesars/fix/response-format-to-text-format-bridge-conversion 2025-11-26 08:51:09 +05:30
containers Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
enterprise [Bug Fix] - Fix litellm_enterprise ensure imported routes exist (#17337) 2025-12-01 19:14:12 -08:00
experimental_mcp_client refactor: drop MCPClient.connect and use run_with_session lifecycle (#16696) 2025-11-15 17:54:27 -08:00
google_genai [Oct Staging Branch] (#15460) 2025-10-17 17:52:25 -07:00
integrations Fix: MLflow streaming spans for Anthropic passthrough (#17288) 2025-12-05 14:59:36 -08:00
litellm_core_utils Fix: Gemini image_tokens incorrectly treated as text tokens in cost calculation (#17554) 2025-12-05 15:55:38 -08:00
llms Fixes #13652 - auth not working with ollama.com (#17191) 2025-12-05 15:52:54 -08:00
passthrough fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
proxy fix(presidio): handle empty content and error dict responses (#17489) 2025-12-05 15:45:19 -08:00
responses fix(responses): Add image generation support for Responses API (#16586) 2025-12-05 15:56:26 -08:00
router_strategy fix: Prevent AttributeError for _get_tags_from_request_kwargs (#14735) 2025-09-19 15:21:02 -07:00
router_utils 🆕 feat: support routing to only websearch supported deployments (#17500) 2025-12-04 14:18:20 -08:00
secret_managers [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297) 2025-11-05 17:20:26 -08:00
types Gemini models: capture image_tokens and support cost_per_output_image_token in costs calculations (#16912) 2025-11-21 19:59:24 -08:00
vector_stores Add vector store support for ragflow 2025-12-03 15:29:47 +05:30
__init__.py [Feat] Add github co-pilot as a new LLM API provider (#12325) 2025-07-04 13:12:16 -07:00
conftest.py Perf speed up pytest (#15951) 2025-10-27 19:43:40 -07:00
log.txt fix: fix linting errors 2025-09-27 14:25:05 -07:00
readme.md Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
test_acompletion_session_reuse_e2e.py [Fix] - Sessions not being shared (#15388) 2025-10-09 16:57:40 -07:00
test_add_deployment_no_master_key.py fix: remove strict master_key check in add_deployment (#16453) 2025-11-10 19:23:13 -08:00
test_aembedding_session_reuse_e2e.py [Fix] - shared session parsing and usage issue (#15440) 2025-10-10 19:26:53 -07:00
test_azure_video_router.py Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
test_claude_haiku_4_5_config.py fix haiku-4-5 bedrock configs (#16732) 2025-11-17 19:52:01 -08:00
test_constants.py tes numeric constants 2025-10-29 18:23:38 -07:00
test_container_router.py Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
test_cost_calculation_log_level.py Enhance logging in cost calculation tests to ensure DEBUG level captures are accurate. Updated tests to set logger level before assertions and restored original logger level after execution. This improves reliability of log level checks in both cost and batch cost calculation tests. 2025-08-14 14:21:22 -07:00
test_cost_calculator.py Fix audio transcription cost tracking (#16478) 2025-11-19 20:29:39 -08:00
test_exception_mapping_request_attribute.py Added test 2025-09-29 21:43:31 +05:30
test_filter_out_litellm_params.py [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
test_groq_streaming_encoding.py [Bug] Fix: Vertex Mistral not working for streaming (#13952) 2025-08-25 17:39:40 -07:00
test_logging.py [Bug Fix] StandardLoggingPayload on cache_hits should track custom llm provider + DD LLM Obs span type (#12652) 2025-07-16 15:43:15 -07:00
test_lowest_latency_zero_tokens.py fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 2025-09-29 15:49:46 -07:00
test_main.py fix(ocr): add missing ocr and aocr to CallTypes enum (#17435) 2025-12-03 21:28:13 -08:00
test_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
test_responses_id_security.py test_encrypt_response_id_success 2025-11-14 17:28:15 -08:00
test_router.py Add aws_bedrock_runtime_endpoint into Credential Types (#17053) 2025-11-24 20:48:51 -08:00
test_router_google_genai.py test(gemini): Add unit tests for Google GenAI adapter 2025-09-29 18:51:35 +08:00
test_shared_session_integration.py feat: Add shared_session parameter for aiohttp ClientSession reuse 2025-09-19 01:46:53 -07:00
test_system_message_format_bug.py Ensure that function_call_prompt extends system messages following its current schema 2025-08-03 17:47:13 +02:00
test_utils.py Gemini models: capture image_tokens and support cost_per_output_image_token in costs calculations (#16912) 2025-11-21 19:59:24 -08:00
test_uuid_helper.py Fix: revert fastuuid optional dependency, always use fastuuid in .__uid helper (#14941) 2025-09-26 09:14:20 -07:00
test_video_generation.py fix(gemini-video): inherit BaseVideoConfig to enable async content response (#16875) 2025-11-21 16:01:21 -08:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py