scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36
litellm/tests/test_litellm
History
Exact
Claude c69f3dd531
Fix structured outputs support in /v1/messages endpoint 
This commit fixes the issue where the output_format parameter was not
properly handled in the /v1/messages endpoint for Claude models on
Azure Foundry, Amazon Bedrock, and other providers.

Changes:
1. Added output_format field to AnthropicMessagesRequestOptionalParams
   TypedDict to prevent it from being stripped from requests
2. Added "output_format" to the list of supported parameters in
   get_supported_anthropic_messages_params()
3. Updated _update_headers_with_anthropic_beta() to automatically
   inject the structured-outputs-2025-11-13 beta header when
   output_format is present
4. Added comprehensive test suite to verify structured outputs
   functionality

The fix applies to all providers using the /v1/messages endpoint:
- Anthropic (direct)
- Amazon Bedrock
- Azure Foundry (Azure AI)
- Vertex AI

All these implementations inherit from AnthropicMessagesConfig, so
the fix automatically propagates to all of them.

Fixes issue where structured outputs returned Markdown text instead
of JSON when using /v1/messages endpoint, even though direct provider
API calls worked correctly.
2026-01-21 05:40:15 +00:00
..
a2a_protocol [Feat] A2A Gateway - allow adding Azure Foundry Agents on UI (#17909) 2025-12-12 16:38:04 -08:00
anthropic_interface/exceptions [bug fix] do not fallback to token counter if disable_token_counter is enabled (#19041) 2026-01-13 16:53:38 -08:00
caching fix(redis): handle float redis_version from AWS ElastiCache Valkey (#16207) 2025-11-04 19:20:00 -08:00
completion_extras Add annotations to completions responses API bridge 2026-01-07 14:30:31 +05:30
containers Make keepalive_timeout parameter work for Gunicorn (#19087) 2026-01-16 03:32:59 +05:30
enterprise Revert "Stabilise mock tests" 2026-01-17 06:26:18 +09:00
experimental_mcp_client tests: patch MCP client mocks via module alias to avoid real network calls 2026-01-20 12:31:27 +09:00
google_genai Fix : test_streaming_multiple_partial_tool_calls 2026-01-16 09:29:54 +05:30
images fix(image_edit): add drop_params support and fix Vertex AI config (#18077) 2025-12-17 11:28:34 +05:30
integrations Fix for Prometheus Metric Cardinality Issue with /responses Endpoint 2026-01-20 15:28:09 +05:30
interactions fix: test_create_simple_string_input 2026-01-05 17:15:12 +05:30
litellm_core_utils Merge branch 'main' into litellm_staging_01_16_2026 2026-01-19 18:11:21 +05:30
llms Fix structured outputs support in /v1/messages endpoint 2026-01-21 05:40:15 +00:00
passthrough fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
proxy attempt test_route_checks fix 2026-01-20 15:55:44 -08:00
responses Merge branch 'main' into litellm_consistent_id_streaming_responses 2026-01-20 19:02:23 +05:30
router_strategy feat(tag-routing): support toggling tag matching between ANY and ALL (#18776) 2026-01-08 23:39:03 +05:30
router_utils [Feat] New Logging Integration - Azure Sentinel Logger (#18146) 2025-12-18 02:04:49 +05:30
secret_managers [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297) 2025-11-05 17:20:26 -08:00
types fix: extend case normalization to ALL guardrail types (not just tool_permission) (#18664) 2026-01-05 11:59:00 -08:00
vector_stores Add vector store support for ragflow 2025-12-03 15:29:47 +05:30
__init__.py [Feat] Add github co-pilot as a new LLM API provider (#12325) 2025-07-04 13:12:16 -07:00
conftest.py Perf speed up pytest (#15951) 2025-10-27 19:43:40 -07:00
log.txt fix: fix linting errors 2025-09-27 14:25:05 -07:00
readme.md
test_acompletion_session_reuse_e2e.py [Fix] - Sessions not being shared (#15388) 2025-10-09 16:57:40 -07:00
test_add_deployment_no_master_key.py fix: remove strict master_key check in add_deployment (#16453) 2025-11-10 19:23:13 -08:00
test_aembedding_session_reuse_e2e.py [Fix] - shared session parsing and usage issue (#15440) 2025-10-10 19:26:53 -07:00
test_azure_video_router.py Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
test_claude_haiku_4_5_config.py fix haiku-4-5 bedrock configs (#16732) 2025-11-17 19:52:01 -08:00
test_constants.py tes numeric constants 2025-10-29 18:23:38 -07:00
test_container_router.py Add E2E Container API Support (#16136) 2025-11-01 14:03:51 -07:00
test_cost_calculation_log_level.py Enhance logging in cost calculation tests to ensure DEBUG level captures are accurate. Updated tests to set logger level before assertions and restored original logger level after execution. This improves reliability of log level checks in both cost and batch cost calculation tests. 2025-08-14 14:21:22 -07:00
test_cost_calculator.py Revert "Fix audio cost per second override (#19158)" 2026-01-19 18:51:08 +05:30
test_eager_tiktoken_load.py Add LITELLM_DISABLE_LAZY_LOADING env var to fix VCR cassette creation issue (#18725) 2026-01-06 14:49:11 -08:00
test_exception_header_preservation.py fix: preserve llm_provider-* headers in error responses (#19020) 2026-01-14 22:49:39 +05:30
test_exception_mapping_request_attribute.py Added test 2025-09-29 21:43:31 +05:30
test_filter_out_litellm_params.py [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855) 2025-10-23 11:56:30 -07:00
test_gpt_image_cost_calculator.py fix(cost_calculator): correct gpt-image-1 cost calculation using token-based pricing (#17906) 2026-01-02 23:08:52 +05:30
test_groq_streaming_encoding.py [Bug] Fix: Vertex Mistral not working for streaming (#13952) 2025-08-25 17:39:40 -07:00
test_lazy_imports.py Fix: test_token_counter_lazy_imports 2026-01-08 16:44:35 +05:30
test_logging.py [Bug Fix] StandardLoggingPayload on cache_hits should track custom llm provider + DD LLM Obs span type (#12652) 2025-07-16 15:43:15 -07:00
test_lowest_latency_zero_tokens.py fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 2025-09-29 15:49:46 -07:00
test_main.py fix(ocr): add missing ocr and aocr to CallTypes enum (#17435) 2025-12-03 21:28:13 -08:00
test_model_response_normalization.py Normalize OpenAI SDK BaseModel choices/messages to avoid Pydantic serializer warnings (#18972) 2026-01-14 03:40:11 +05:30
test_nested_drop_params.py feat: Replace jsonpath-ng with custom minimal parser for additional_drop_params 2025-12-09 17:30:30 +05:30
test_redis.py fix: Apply max_connections configuration to Redis async client (#15797) 2025-10-22 09:19:08 -07:00
test_responses_api_bridge_non_stream.py Add ChatGPT subscription support and responses bridge (#19030) 2026-01-19 05:37:45 -08:00
test_responses_id_security.py fix(ui): use non-streaming method for endpoint v1/a2a/message/send in… (#19025) 2026-01-14 03:29:10 +05:30
test_router.py Manual revert #19078 2026-01-17 06:54:08 +09:00
test_router_google_genai.py test(gemini): Add unit tests for Google GenAI adapter 2025-09-29 18:51:35 +08:00
test_router_per_deployment_num_retries.py test: fix missing test 2026-01-16 10:33:53 +09:00
test_shared_session_integration.py feat: Add shared_session parameter for aiohttp ClientSession reuse 2025-09-19 01:46:53 -07:00
test_system_message_format_bug.py Ensure that function_call_prompt extends system messages following its current schema 2025-08-03 17:47:13 +02:00
test_utils.py fix: drop_params not dropping prompt_cache_key for non-OpenAI providers 2026-01-19 08:49:03 -03:00
test_uuid_helper.py Fix: revert fastuuid optional dependency, always use fastuuid in .__uid helper (#14941) 2025-09-26 09:14:20 -07:00
test_video_generation.py fix: test_video_status_basic 2026-01-08 17:30:58 +05:30

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py