scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-29 01:42:19 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 100
litellm/tests
History
mateo-berri 758f72e41f tests(e2e-cassette-proxy): add mitm.it to passthrough hosts 
mitmproxy serves its current-instance CA certificate at the magic
host ``mitm.it/cert/pem``. The CI pipeline downloads that CA in the
``Fetch CA from cassette-proxy`` step and trusts it (certifi append +
system trust store) so subsequent in-process tests can validate
mitmproxy's MITM leaf certs.

Once the previous commit fixed the replay path so cache hits actually
serve, the very first ``mitm.it/cert/pem`` lookup hit a stale entry
from a prior CI run — the proxy handed back a CA generated by a *previous*
mitmdump instance with a different private key. The CI then trusted that
stale CA, mitmproxy in the *current* run signed leaf certs with its fresh
CA, and every TLS handshake failed:

  SSL: CERTIFICATE_VERIFY_FAILED:
    certificate verify failed: authority and subject key identifier mismatch

This took down langfuse_logging_unit_tests (at ``test_embedding.py``
collection), search_testing, and image_gen_testing.

Adding ``mitm.it`` to the passthrough host list short-circuits in
``_should_skip`` before any Redis lookup, so the CA is always served
fresh from the running mitmdump instance. The poisoned key in the
shared dev Redis was deleted manually before pushing this commit.

Adds a regression test that exercises both hooks: ``request()`` must
return without setting ``flow.response``, and ``response()`` must not
persist anything to Redis.
2026-05-01 10:12:50 -07:00
..
agent_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
audio_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests test(vertex-batches): set is_redirect=False on mocked retrieve response 2026-04-29 19:01:49 -07:00
benchmarks Add CodSpeed performance benchmarks (#23676) 2026-03-14 18:44:36 -07:00
code_coverage_tests fix(vector-stores): re-raise HTTPException from get_vector_store_info; allowlist recursion 2026-04-29 18:56:55 +00:00
documentation_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
e2e_cassette_proxy tests(e2e-cassette-proxy): add mitm.it to passthrough hosts 2026-05-01 10:12:50 -07:00
enterprise Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_migration_projects 2026-04-24 12:52:10 -07:00
guardrails_tests fix(proxy): guardrail header dedupe, mypy during_call, test mock kwargs 2026-04-22 23:22:35 +03:00
image_gen_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_dbReconnectNonBlocking 2026-04-28 23:29:36 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests fix(proxy): sanitize redaction controls at ingress 2026-04-29 22:52:31 -07:00
llm_responses_api_testing style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
llm_translation Merge pull request #26222 from BerriAI/litellm_anthropic-json-mode-nonstreaming-mixed-tools 2026-05-01 08:24:38 +05:30
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing fix(test): scope ERROR log assertion to LiteLLM logger in test_model_alias_map 2026-04-29 03:48:41 +00:00
logging_callback_tests fix(proxy): sanitize redaction controls at ingress 2026-04-29 22:52:31 -07:00
mcp_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr17 2026-04-17 17:36:40 -07:00
multi_instance_e2e_tests
ocr_tests Merge branch 'litellm_internal_staging' into litellm_pages_support_for_ocr 2026-04-17 19:29:45 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
otel_tests [Test] Proxy E2E: Opt In To Client Mock Response For Model Access Tests 2026-04-30 17:05:31 -07:00
pass_through_tests [Infra] CCI: pin Ruby and Node.js installs in proxy_pass_through_endpoint_tests 2026-04-22 21:28:42 -07:00
pass_through_unit_tests fix(tests): use Sonnet 4.5 for Bedrock invoke prompt-caching tests 2026-04-28 14:51:47 -07:00
proxy_admin_ui_tests [Fix] CI/Tooling: Correct min-release-age value in .npmrc files 2026-04-29 19:49:27 -07:00
proxy_e2e_anthropic_messages_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_security_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_unit_tests Merge pull request #26815 from stuxf/fix/get-image-lfi-ssrf 2026-04-30 17:10:15 -07:00
router_unit_tests Merge branch 'litellm_internal_staging' into litellm_adaptive_routing 2026-04-20 15:28:08 -07:00
scim_tests
search_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
spend_tracking_tests [Fix] Harden spend accuracy test against transient aiohttp connection errors 2026-04-22 17:40:42 -07:00
store_model_in_db_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm tests(e2e-cassette-proxy): add mitm.it to passthrough hosts 2026-05-01 10:12:50 -07:00
unified_google_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
vector_store_tests fix: drop milvus dbName and partitionNames from MILVUS_OPTIONAL_PARAMS 2026-04-30 11:51:32 -07:00
windows_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py
test_entrypoint.py
test_fallbacks.py Revert "fix: prevent error when max_fallbacks exceeds available models (#20071)" 2026-02-03 15:15:30 +05:30
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py
test_keys.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm_proxy_responses_config.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py [Fix] test_ratelimit: skip over-limit cases that race with background RPM tracking 2026-04-11 13:08:59 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py fix(langfuse_otel): prevent empty proxy request spans from being sent to Langfuse 2026-01-28 15:35:35 +01:00
test_spend_logs.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py
test_users.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.