scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-23 00:41:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 81
litellm/tests/test_litellm
History
Tai An 6bfb0360d6 fix(anthropic/chat): place tool 'strict' at top level (#27490) 
Anthropic only enforces tool-call 'strict' when set as a sibling of
name/input_schema on the tool object; nested inside input_schema it is
silently ignored.

_map_tool_helper was forwarding the OpenAI 'strict' field through the
input_schema filter (because AnthropicInputSchema's annotation included
'strict'), so callers got Anthropic API responses behaving as if strict
was disabled while believing it was engaged.

Extract 'strict' from the canonical OpenAI location
(tool.function.strict) and as a fallback from the legacy nested location
(tool.function.parameters.strict), then place it at the tool top level
(_tool['strict']). Drop 'strict' from AnthropicInputSchema's TypedDict
keys so the input_schema filter strips it from the schema.

Adds three regression tests covering: canonical-location lift,
legacy-nested lift, and absence-by-default (no strict key injected when
the caller did not opt in).

Fixes #27490
2026-05-08 15:17:18 -07:00
..
a2a_protocol style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
anthropic_interface/exceptions style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
caching chore(caching): remove allow_legacy_unscoped_cache_hits opt-in 2026-05-04 22:16:30 +00:00
completion_extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
containers chore(container): use delete_cache, json-encode scope key, clean test 2026-05-05 00:43:47 +00:00
enterprise fix(proxy): normalize managed resource team owner field 2026-05-04 17:05:50 -07:00
expected_fine_tuning_api refactor: refactor testing 2026-03-28 18:39:32 -07:00
expected_responses_api_request [Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 2026-02-12 10:00:30 -08:00
experimental_mcp_client style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
google_genai style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
images fix(image_edit): read vertex_project/location from litellm_params in Imagen get_complete_url 2026-04-21 15:03:40 +05:30
integrations Merge pull request #27222 from BerriAI/litellm_s3AuditParams 2026-05-07 12:49:03 -07:00
interactions test(interactions): align openapi compliance with upstream rename outputs->steps 2026-05-07 17:09:25 -07:00
litellm_core_utils fix(chat-completions): decode unified file_id when model_file_id_mapping is unavailable (#27406) 2026-05-07 13:17:04 -07:00
llms fix(anthropic/chat): place tool 'strict' at top level (#27490) 2026-05-08 15:17:18 -07:00
ocr style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
passthrough refactor: trim explanatory comments from streaming-flush fix 2026-04-30 02:39:28 +00:00
proxy fix: remove separate health app 2026-05-07 16:04:56 -07:00
responses Revert "Merge pull request #24417 from Chesars/refactor/shared-format-mapping" 2026-04-25 15:03:24 -03:00
router_strategy feat: selectively apply routing strategy according to model name 2026-05-04 13:27:32 -07:00
router_utils Merge branch 'litellm_internal_staging' into litellm_access-group-routing-fix 2026-05-01 23:07:37 +00:00
secret_managers Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
test_router fix: use atomic increment-first pattern for model RPM rate limiting 2026-02-24 09:55:07 -03:00
types Merge pull request #25256 from BerriAI/litellm_ishaan_april6 2026-04-17 16:26:45 -07:00
vector_stores style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py
conftest.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_acompletion_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_add_deployment_no_master_key.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aembedding_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_beta_headers_filtering.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_skills_transformation.py fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
test_azure_video_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_bedrock_anthropic_1hr_cache_pricing.py fix(bedrock): add 1-hour cache write tier for Claude 4.5/4.6/4.7 (Global, US) 2026-04-29 19:21:57 +00:00
test_chat_ui_responses_session.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_claude_haiku_4_5_config.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_claude_opus_4_6_config.py Revert "Merge pull request #24164 from dongyu-turo/feat/update-bedrock-claude-price-above-200k" 2026-04-25 15:04:05 -03:00
test_completion_timeout_resolution.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
test_constants.py Fix test 2026-03-27 21:21:43 +05:30
test_container_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculation_log_level.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculator.py fix(utils): allowed_openai_params must not forward unset params as None (#25777) 2026-04-16 19:04:26 +05:30
test_count_tokens_public_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_dashscope_image_generation.py fix(dashscope): fail fast on image generation API errors 2026-04-23 18:41:01 +05:30
test_deepseek_model_metadata.py Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle" 2026-04-25 17:10:41 -03:00
test_dockerfile_non_root.py fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
test_eager_tiktoken_load.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_exports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_header_preservation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_mapping_request_attribute.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_out_litellm_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_blog_posts.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_gpt_image_cost_calculator.py Add gpt-image-2 support (#26644) (#26705) 2026-04-28 20:10:42 -07:00
test_groq_streaming_encoding.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_params_reserved_keys.py fix(snowflake): transform tool_choice string to object format (#23268) 2026-03-11 01:41:24 +05:30
test_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lowest_latency_zero_tokens.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_main.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_model_cost_aliases.py [Feat] - Ishaan main merge branch (#23596) 2026-03-14 09:40:00 -07:00
test_model_param_helper.py style: black format test_model_param_helper.py 2026-04-15 18:18:52 -07:00
test_model_response_normalization.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_nested_drop_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_embedding_encoding_format_default.py fix(embeddings): allow omitting encoding_format via env sentinel none 2026-05-01 23:03:13 +05:30
test_project_alias_tracking.py feat(proxy): add project_alias tracking through callback metadata pipeline 2026-03-23 10:44:17 -07:00
test_project_tags_pydantic.py fix: req changes 2026-02-27 13:33:34 +05:30
test_redact_string_in_error_paths.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_redis.py fix(redis): cache GCP IAM token to prevent async event loop blocking (#26441) 2026-04-25 23:13:39 -07:00
test_register_model_custom_pricing.py test: fix misleading precedence test per review feedback 2026-03-02 08:31:33 +00:00
test_responses_api_bridge_non_stream.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_responses_id_security.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router.py test(router): cover _try_early_resolve_deployments_for_model_not_in_names 2026-05-02 00:18:45 +00:00
test_router_google_genai.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_model_cost_isolation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_order_fallback.py fix(router): enable order fallback for wildcard model groups 2026-04-15 17:48:25 +05:30
test_router_per_deployment_num_retries.py Bugfix/19481 num retries env var type (#19507) 2026-01-22 19:39:58 -08:00
test_router_redis_init.py fix: handle deprecated 'redis_db' arg to prevent crash (#19808) 2026-02-02 18:18:05 +05:30
test_router_retry_non_retryable_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_silent_experiment.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_secret_redaction.py Unify secret redaction patterns 2026-05-01 17:24:21 +00:00
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_setup_wizard.py test: test 2026-03-28 19:17:38 -07:00
test_shared_session_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ssl_verify_unit.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_stream_chunk_builder_annotations.py fix: merge annotations from all streaming chunks in stream_chunk_builder 2026-03-15 14:20:45 +05:30
test_streaming_connection_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_system_message_format_bug.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_utils.py refactor: remove unnecessary comments from #27074 2026-05-04 19:34:56 +00:00
test_uuid_helper.py
test_vcr_safe_body_matcher.py test: add 24hr Redis-backed VCR cache to additional test suites (#27159) 2026-05-05 15:13:31 -07:00
test_video_generation.py feat(gemini): Veo Lite pricing, size→resolution, usage video_resolution for cost tiers 2026-04-08 19:50:50 +05:30
test_xai_grok_4_3_model_metadata.py feat(xai): add grok-4.3 and grok-4.3-latest to model_prices_and_conte… (#27154) 2026-05-07 09:06:56 -07:00
test_xai_responses_auto_routing.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py