scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88
litellm/tests/test_litellm
History
mateo-berri 08697b63c5
fix(bedrock): make supports_output_config gate actually consult cost map 
The gate at three call sites was calling _supports_factory with
custom_llm_provider=None, which relies on get_llm_provider inferring the
provider from the model string. For the invoke path, the model still
carries an 'invoke/' routing prefix (e.g. 'invoke/us.anthropic.claude-opus-4-6-v1')
that is not a known provider, so inference raises BadRequestError,
_supports_factory swallows it and returns False, and the user's
output_config.effort gets silently dropped before the Bedrock request.

Strip the routing prefix with the existing strip_bedrock_routing_prefix
helper and pass custom_llm_provider='bedrock' explicitly so the
declarative 'supports_output_config' flag in
model_prices_and_context_window.json is the actual source of truth.

Also adds a regression test that exercises the full
'invoke/us.anthropic.claude-opus-4-6-v1' path and asserts output_config
survives.
2026-04-24 16:50:18 +05:30
..
a2a_protocol Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
anthropic_interface/exceptions Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
caching Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
completion_extras Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
containers Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
enterprise Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
expected_fine_tuning_api refactor: refactor testing 2026-03-28 18:39:32 -07:00
expected_responses_api_request
experimental_mcp_client Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
google_genai Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
images Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
integrations style: apply black formatting to PR files 2026-04-24 16:50:17 +05:30
interactions Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
litellm_core_utils Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
llms fix(bedrock): make supports_output_config gate actually consult cost map 2026-04-24 16:50:18 +05:30
ocr Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
passthrough Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
proxy Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
responses Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
router_strategy Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
router_utils Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
secret_managers Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_router fix: use atomic increment-first pattern for model RPM rate limiting 2026-02-24 09:55:07 -03:00
types Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
vector_stores Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
__init__.py
conftest.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
log.txt
readme.md
test_a2a_registry_lookup.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_acompletion_session_reuse_e2e.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_add_deployment_no_master_key.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_aembedding_session_reuse_e2e.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_anthropic_beta_headers_filtering.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_anthropic_skills_transformation.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_azure_video_router.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_chat_ui_responses_session.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_claude_haiku_4_5_config.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_claude_opus_4_6_config.py Fix the Pricing changes for claude models 2026-03-27 22:44:40 +05:30
test_completion_timeout_resolution.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
test_constants.py Fix test 2026-03-27 21:21:43 +05:30
test_container_router.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_cost_calculation_log_level.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_cost_calculator.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_count_tokens_public_api.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_dashscope_image_generation.py fix(dashscope): fail fast on image generation API errors 2026-04-23 18:41:01 +05:30
test_deepseek_model_metadata.py
test_dockerfile_non_root.py fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
test_eager_tiktoken_load.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_exception_exports.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_exception_header_preservation.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_exception_mapping_request_attribute.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_filter_out_litellm_params.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_get_blog_posts.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_gpt_image_cost_calculator.py
test_groq_streaming_encoding.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_lazy_imports.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_litellm_params_reserved_keys.py fix(snowflake): transform tool_choice string to object format (#23268) 2026-03-11 01:41:24 +05:30
test_logging.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_lowest_latency_zero_tokens.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_main.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_model_cost_aliases.py [Feat] - Ishaan main merge branch (#23596) 2026-03-14 09:40:00 -07:00
test_model_param_helper.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_model_response_normalization.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_nested_drop_params.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_project_alias_tracking.py feat(proxy): add project_alias tracking through callback metadata pipeline 2026-03-23 10:44:17 -07:00
test_project_tags_pydantic.py fix: req changes 2026-02-27 13:33:34 +05:30
test_redact_string_in_error_paths.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_redis.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_register_model_custom_pricing.py test: fix misleading precedence test per review feedback 2026-03-02 08:31:33 +00:00
test_responses_api_bridge_non_stream.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_responses_id_security.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_router.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_router_google_genai.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_router_model_cost_isolation.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_router_order_fallback.py fix(router): enable order fallback for wildcard model groups 2026-04-15 17:48:25 +05:30
test_router_per_deployment_num_retries.py
test_router_redis_init.py
test_router_retry_non_retryable_errors.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_router_silent_experiment.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_secret_redaction.py Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_setup_wizard.py test: test 2026-03-28 19:17:38 -07:00
test_shared_session_integration.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_ssl_verify_unit.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_stream_chunk_builder_annotations.py fix: merge annotations from all streaming chunks in stream_chunk_builder 2026-03-15 14:20:45 +05:30
test_streaming_connection_cleanup.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_system_message_format_bug.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_utils.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_uuid_helper.py
test_video_generation.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30
test_xai_responses_auto_routing.py Reapply "Merge remote-tracking branch 'origin/litellm_oss_staging_04_21_2026' into fix/bedrock-invoke-output-config-effort-4-6" 2026-04-24 16:49:48 +05:30

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py