scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-29 01:42:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 79
litellm/tests/test_litellm
History
mateo-berri 758f72e41f tests(e2e-cassette-proxy): add mitm.it to passthrough hosts 
mitmproxy serves its current-instance CA certificate at the magic
host ``mitm.it/cert/pem``. The CI pipeline downloads that CA in the
``Fetch CA from cassette-proxy`` step and trusts it (certifi append +
system trust store) so subsequent in-process tests can validate
mitmproxy's MITM leaf certs.

Once the previous commit fixed the replay path so cache hits actually
serve, the very first ``mitm.it/cert/pem`` lookup hit a stale entry
from a prior CI run — the proxy handed back a CA generated by a *previous*
mitmdump instance with a different private key. The CI then trusted that
stale CA, mitmproxy in the *current* run signed leaf certs with its fresh
CA, and every TLS handshake failed:

  SSL: CERTIFICATE_VERIFY_FAILED:
    certificate verify failed: authority and subject key identifier mismatch

This took down langfuse_logging_unit_tests (at ``test_embedding.py``
collection), search_testing, and image_gen_testing.

Adding ``mitm.it`` to the passthrough host list short-circuits in
``_should_skip`` before any Redis lookup, so the CA is always served
fresh from the running mitmdump instance. The poisoned key in the
shared dev Redis was deleted manually before pushing this commit.

Adds a regression test that exercises both hooks: ``request()`` must
return without setting ``flow.response``, and ``response()`` must not
persist anything to Redis.
2026-05-01 10:12:50 -07:00
..
a2a_protocol style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
anthropic_interface/exceptions style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
caching Refresh Redis TTL on counter writes and skip stale in-memory on Redis miss 2026-04-30 17:50:58 -07:00
completion_extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
containers style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
e2e_cassette_proxy tests(e2e-cassette-proxy): add mitm.it to passthrough hosts 2026-05-01 10:12:50 -07:00
enterprise feat: add configurable multi-threshold budget alerts for virtual keys 2026-04-17 17:12:35 -07:00
expected_fine_tuning_api refactor: refactor testing 2026-03-28 18:39:32 -07:00
expected_responses_api_request [Feat] Adds support for server-side compaction on the OpenAI Responses API context_management (#21058) 2026-02-12 10:00:30 -08:00
experimental_mcp_client style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
google_genai style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
images fix(image_edit): read vertex_project/location from litellm_params in Imagen get_complete_url 2026-04-21 15:03:40 +05:30
integrations fix(tests): replace deprecated Bedrock Claude 3.7 Sonnet model ID 2026-04-28 14:24:19 -07:00
interactions style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils fix(logging): redact standard logging choice metadata 2026-04-30 00:35:29 -07:00
llms Merge pull request #26222 from BerriAI/litellm_anthropic-json-mode-nonstreaming-mixed-tools 2026-05-01 08:24:38 +05:30
ocr style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
passthrough refactor: trim explanatory comments from streaming-flush fix 2026-04-30 02:39:28 +00:00
proxy Merge pull request #26829 from BerriAI/litellm_budgetEnforcementMultiPod 2026-04-30 18:14:41 -07:00
responses Revert "Merge pull request #24417 from Chesars/refactor/shared-format-mapping" 2026-04-25 15:03:24 -03:00
router_strategy add test(tag-routing): prevent header regex bypass for strict plain tags. 2026-04-29 20:14:30 +00:00
router_utils style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
secret_managers style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router fix: use atomic increment-first pattern for model RPM rate limiting 2026-02-24 09:55:07 -03:00
types Merge pull request #25256 from BerriAI/litellm_ishaan_april6 2026-04-17 16:26:45 -07:00
vector_stores style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py
conftest.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
log.txt
readme.md
test_a2a_registry_lookup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_acompletion_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_add_deployment_no_master_key.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aembedding_session_reuse_e2e.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_beta_headers_filtering.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_skills_transformation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_azure_video_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_bedrock_anthropic_1hr_cache_pricing.py fix(bedrock): add 1-hour cache write tier for Claude 4.5/4.6/4.7 (Global, US) 2026-04-29 19:21:57 +00:00
test_chat_ui_responses_session.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_claude_haiku_4_5_config.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_claude_opus_4_6_config.py Revert "Merge pull request #24164 from dongyu-turo/feat/update-bedrock-claude-price-above-200k" 2026-04-25 15:04:05 -03:00
test_completion_timeout_resolution.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
test_constants.py Fix test 2026-03-27 21:21:43 +05:30
test_container_router.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculation_log_level.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cost_calculator.py fix(utils): allowed_openai_params must not forward unset params as None (#25777) 2026-04-16 19:04:26 +05:30
test_count_tokens_public_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_dashscope_image_generation.py fix(dashscope): fail fast on image generation API errors 2026-04-23 18:41:01 +05:30
test_deepseek_model_metadata.py fix(model-info): sync DeepSeek model metadata and add bare-name fallback (#20885) 2026-02-11 12:48:10 +05:30
test_dockerfile_non_root.py fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
test_eager_tiktoken_load.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_exports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_header_preservation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_exception_mapping_request_attribute.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_out_litellm_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_get_blog_posts.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_gpt_image_cost_calculator.py Add gpt-image-2 support (#26644) (#26705) 2026-04-28 20:10:42 -07:00
test_groq_streaming_encoding.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_params_reserved_keys.py fix(snowflake): transform tool_choice string to object format (#23268) 2026-03-11 01:41:24 +05:30
test_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_lowest_latency_zero_tokens.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_main.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_model_cost_aliases.py [Feat] - Ishaan main merge branch (#23596) 2026-03-14 09:40:00 -07:00
test_model_param_helper.py style: black format test_model_param_helper.py 2026-04-15 18:18:52 -07:00
test_model_response_normalization.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_nested_drop_params.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_project_alias_tracking.py feat(proxy): add project_alias tracking through callback metadata pipeline 2026-03-23 10:44:17 -07:00
test_project_tags_pydantic.py fix: req changes 2026-02-27 13:33:34 +05:30
test_redact_string_in_error_paths.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_redis.py fix(redis): cache GCP IAM token to prevent async event loop blocking (#26441) 2026-04-25 23:13:39 -07:00
test_register_model_custom_pricing.py test: fix misleading precedence test per review feedback 2026-03-02 08:31:33 +00:00
test_responses_api_bridge_non_stream.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_responses_id_security.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router.py fix(router): propagate custom cost_per_token from db model_info in fallback path (#25888) 2026-04-27 08:58:41 +05:30
test_router_google_genai.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_model_cost_isolation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_order_fallback.py fix(router): enable order fallback for wildcard model groups 2026-04-15 17:48:25 +05:30
test_router_per_deployment_num_retries.py
test_router_redis_init.py
test_router_retry_non_retryable_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_router_silent_experiment.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_secret_redaction.py Litellm ishaan march30 (#24887) (#25151) 2026-04-04 14:44:07 -07:00
test_service_logger.py fix(proxy): fix master key rotation Prisma validation errors (#21330) 2026-02-16 15:13:05 -08:00
test_setup_wizard.py test: test 2026-03-28 19:17:38 -07:00
test_shared_session_integration.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ssl_verify_unit.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_stream_chunk_builder_annotations.py fix: merge annotations from all streaming chunks in stream_chunk_builder 2026-03-15 14:20:45 +05:30
test_streaming_connection_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_system_message_format_bug.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_utils.py Add gpt-image-2 support (#26644) (#26705) 2026-04-28 20:10:42 -07:00
test_uuid_helper.py
test_video_generation.py feat(gemini): Veo Lite pricing, size→resolution, usage video_resolution for cost tiers 2026-04-08 19:50:50 +05:30
test_xai_responses_auto_routing.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00

readme.md

Testing for litellm/

This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.

The point of this is to:

  1. Increase test coverage of litellm/
  2. Make it easy for contributors to add tests for the litellm/ package and easily run tests without needing LLM API keys.

File name conventions

  • litellm/proxy/test_caching_routes.py maps to litellm/proxy/caching_routes.py
  • test_<filename>.py maps to litellm/<filename>.py