scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
litellm/tests/test_litellm
History
Exact
weiguang li 3d027c0f7a
fix(bedrock): filter out custom field from tools to prevent 400 errors (#22861) 
Claude Code v2.1.69+ sends `custom: {defer_loading: true}` on tool
definitions. Anthropic's API accepts this field, but Bedrock rejects it
with "Extra inputs are not permitted", causing ~90% of requests to fail.

Strip the `custom` field from each tool in the request body before
sending to Bedrock, in both the Messages API and Chat API invoke paths.

Fixes #22847

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2026-03-05 13:54:23 -08:00
..
a2a_protocol [Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604) 2026-02-06 15:02:34 -08:00
anthropic_interface/exceptions [bug fix] do not fallback to token counter if disable_token_counter is enabled (#19041) 2026-01-13 16:53:38 -08:00
caching fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22925) 2026-03-05 12:00:38 -08:00
completion_extras test(streaming): add comprehensive parallel tool call integration test 2026-03-04 10:17:20 +00:00
containers fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
enterprise Fixes based on greptile reviews 2026-02-18 12:19:11 +05:30
expected_responses_api_request [Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 2026-02-12 10:00:30 -08:00
experimental_mcp_client fix: FLAKY tests 2026-01-24 11:13:44 -08:00
google_genai litellm_fix_mapped_tests_core: fix test isolation and mock injection issues (#20209) 2026-01-31 17:53:54 -08:00
images Merge pull request #22307 from Chesars/fix/22244-image-edit-custom-pricing 2026-02-27 16:38:34 -03:00
integrations fix: replace sk-fake with safe test key to avoid secret scanner 2026-03-05 18:29:28 +01:00
interactions fix(test): Update status enum values to match Google Interactions OpenAPI spec (#22061) 2026-02-24 20:26:11 -08:00
litellm_core_utils fix(streaming): prevent Vertex AI Claude content truncation when finish_reason races content 2026-03-04 22:44:48 +01:00
llms fix(bedrock): filter out custom field from tools to prevent 400 errors (#22861) 2026-03-05 13:54:23 -08:00
ocr Enable local file support for OCR (#22133) 2026-02-27 10:50:02 -08:00
passthrough fix(passthrough): propagate Azure 429/5xx errors in async streaming instead of silent HTTP 200 (#22913) 2026-03-05 10:12:43 -08:00
proxy Prisma DB Failure Detection and Self-Healing (#21059) 2026-03-05 13:44:49 -08:00
responses Add tests for resp 2026-03-04 18:27:06 +05:30
router_strategy fix: complexity_router crashes on list-format message content (OpenAI multi-part messages) (#22761) 2026-03-04 16:18:49 -08:00
router_utils Fix encrypted content streaming affinity issue 2026-03-03 18:37:22 +05:30
secret_managers fix(tests): isolate flaky files endpoint tests from global proxy state (#21788) 2026-02-21 11:20:32 -08:00
test_router fix: use atomic increment-first pattern for model RPM rate limiting 2026-02-24 09:55:07 -03:00
types fix: add video_tokens to expected completion_tokens_details in test 2026-03-03 19:46:20 -03:00
vector_stores litellm_fix_mapped_tests_core: fix test isolation and mock injection issues (#20209) 2026-01-31 17:53:54 -08:00
__init__.py
conftest.py fix(tests): restore disable_aiohttp_transport and force_ipv4 in isolate_litellm_state 2026-02-17 21:18:49 -03:00
log.txt
readme.md
test_a2a_registry_lookup.py [Feat] Use A2A registered agents with /chat/completions (#20362) 2026-02-03 15:25:38 -08:00
test_acompletion_session_reuse_e2e.py
test_add_deployment_no_master_key.py
test_aembedding_session_reuse_e2e.py
test_anthropic_beta_headers_filtering.py Make tests run with local beta header mapping json 2026-02-13 22:31:42 +05:30
test_azure_video_router.py
test_claude_haiku_4_5_config.py
test_claude_opus_4_6_config.py Fix au.anthropic.claude opus 4 6 v1 (#20731) 2026-02-16 14:15:37 -08:00
test_constants.py added configurable env for mcp timeouts (#22287) 2026-03-02 13:13:41 -08:00
test_container_router.py
test_cost_calculation_log_level.py fix(tests): use record.getMessage() instead of record.message for LogRecord 2026-02-18 11:46:32 -03:00
test_cost_calculator.py perf: optimize completion_cost() — eliminate enum overhead, reduce function call indirection 2026-02-21 12:14:55 -08:00
test_deepseek_model_metadata.py fix(model-info): sync DeepSeek model metadata and add bare-name fallback (#20885) 2026-02-11 12:48:10 +05:30
test_eager_tiktoken_load.py fix(main): use local tiktoken cache in lazy loading (#19774) 2026-01-27 18:16:58 -08:00
test_exception_exports.py fix: export PermissionDeniedError from litellm.__init__ 2026-02-11 13:39:19 +01:00
test_exception_header_preservation.py Update test to righ place 2026-02-26 13:26:51 -08:00
test_exception_mapping_request_attribute.py
test_filter_out_litellm_params.py
test_get_blog_posts.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_gpt_image_cost_calculator.py Fix gpt-image-1.5 cost calculation not including output image tokens (#19515) 2026-01-22 19:42:15 -08:00
test_groq_streaming_encoding.py
test_lazy_imports.py Fix: test_token_counter_lazy_imports 2026-01-08 16:44:35 +05:30
test_logging.py fix:Parse embedded JSON in the message field of logs (#20366) 2026-02-10 16:13:33 +05:30
test_lowest_latency_zero_tokens.py
test_main.py Fix : test_video_content_handler_uses_get_for_openai 2026-02-17 20:06:08 +05:30
test_model_param_helper.py perf: cache _get_relevant_args_to_use_for_logging() at module level (#20077) 2026-02-02 10:54:49 -08:00
test_model_response_normalization.py fix(types): remove StreamingChoices from ModelResponse, use ModelResponseStream 2026-02-20 17:47:42 -03:00
test_nested_drop_params.py
test_project_tags_pydantic.py fix: req changes 2026-02-27 13:33:34 +05:30
test_redis.py
test_register_model_custom_pricing.py test: fix misleading precedence test per review feedback 2026-03-02 08:31:33 +00:00
test_responses_api_bridge_non_stream.py fix: Pydantic will fail to parse it because cached_tokens is required but not provided 2026-01-28 11:51:26 +05:30
test_responses_id_security.py Fix responses ID security test for new request_cache parameter 2026-03-04 11:29:51 -03:00
test_router.py fix: add sync streaming fallback + fix 429 for all streaming paths (#22375) 2026-02-28 15:55:05 -08:00
test_router_google_genai.py
test_router_model_cost_isolation.py [Fix] prevent shared backend model key from being polluted by per-deployment custom pricing (#20679) 2026-02-09 19:38:44 -08:00
test_router_per_deployment_num_retries.py Bugfix/19481 num retries env var type (#19507) 2026-01-22 19:39:58 -08:00
test_router_redis_init.py fix: handle deprecated 'redis_db' arg to prevent crash (#19808) 2026-02-02 18:18:05 +05:30
test_router_silent_experiment.py litellm_fix(test): fix router silent experiment tests to properly mock async functions (#20140) 2026-01-31 07:39:05 -08:00
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_shared_session_integration.py
test_ssl_verify_unit.py BUMP Enterprise PIP 2026-02-14 13:40:48 -08:00
test_streaming_connection_cleanup.py fix: add debug logging to stream cleanup, improve tests 2026-02-14 17:31:39 -08:00
test_system_message_format_bug.py
test_utils.py fix(test): add 'realtime' to model mode enum in schema validation 2026-03-05 06:41:51 -03:00
test_uuid_helper.py
test_video_generation.py fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
test_xai_responses_auto_routing.py Add routing of xai chat completions to responses when web search options is present 2026-01-30 14:15:35 +05:30

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py