litellm/tests
Yuneng Jiang c3048dcd30
test(http): move the outbound HTTP/2 check into a new integration sdk suite
The check spins up a hypercorn TLS peer and drives the SDK's own httpx
handlers at it, so it needs litellm importable, hypercorn installed and a
loopback socket. It lived under tests/e2e, whose Buildkite runner image
installs neither litellm nor hypercorn by design (the suite drives a
remote proxy over HTTP), so every scheduled e2e build since #230 failed
to import the module and pytest reported it as a collection error. The
unit tree bans sockets, so it does not belong there either

tests/integration is the CircleCI tier built for real TCP against local
protocol peers. This adds an sdk shard to it for cases that exercise the
SDK's clients with no gateway in the path, registers the two HTTP/2
nodes in the contracts manifest, and adds the shard to the CircleCI
matrix. The test now flips the feature through LITELLM_HTTP2 (the user
surface) instead of patching module attributes, and asserts the version
the peer observed on the wire next to the one the client reports
2026-09-17 10:37:07 -07:00
..
agent_tests
audio_tests
base_sdk_tests
basic_proxy_startup_tests
batches_tests feat(batches): enrich batch cost rows with breakdown, identity, session, and org spend 2026-09-03 17:23:53 -04:00
benchmarks
code_coverage_tests fix(e2e): own a shared fixture's deployment by the fixture's node, not the first test 2026-09-16 17:35:05 -07:00
documentation_tests Merge remote-tracking branch 'origin/main' into litellm_bedrock_messages_disconnect_billing 2026-08-31 08:58:37 -07:00
e2e test(http): move the outbound HTTP/2 check into a new integration sdk suite 2026-09-17 10:37:07 -07:00
enterprise fix(projects): persist explicit budget cap clears 2026-09-12 13:43:55 -07:00
guardrails_tests fix(guardrails): give post-call scans the scoped request conversation and tools 2026-09-15 09:58:53 +00:00
image_gen_tests
integration test(http): move the outbound HTTP/2 check into a new integration sdk suite 2026-09-17 10:37:07 -07:00
litellm-proxy-extras test(proxy-extras): move the LITELLM_LOG logger test into the CI-run proxy-extras shard 2026-09-15 22:57:23 +00:00
litellm_utils_tests test(proxy): assert budget resets decrement the cleared spend 2026-09-15 19:48:25 -07:00
llm_responses_api_testing test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
llm_translation refactor(test): validate cost-map entries into a typed model 2026-09-16 14:32:08 -07:00
load_tests feat(proxy): per-worker admission control that rejects excess requests with 503 (#39352) 2026-09-03 18:19:04 -07:00
local_testing test: pin the post-#41289 cooldown contract and scroll the auto-router select spec 2026-09-16 20:52:19 -07:00
logging_callback_tests Merge pull request #41487 from BerriAI/litellm_isolate_generic_api_ndjson_test 2026-09-16 16:06:47 -07:00
mcp_tests fix(mcp): reject initialize with 403 when the key grants no MCP servers (#40616) 2026-09-10 14:25:30 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): exempt native parity requests from cassette replay 2026-09-11 22:50:39 -07:00
openai_endpoints_tests test(responses): fix stale Anthropic smoke request 2026-09-11 13:56:27 -07:00
otel_tests refactor(proxy): replace configurable model access denied message with a fixed clean client message 2026-09-16 01:44:36 +00:00
pass_through_tests fix(logging): key bridged /v1/messages rows on the id the caller received 2026-09-03 01:15:06 -07:00
pass_through_unit_tests Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts 2026-09-15 13:38:00 -07:00
proxy_admin_ui_tests refactor(tests): assign the streamed id and lock poll once instead of rebinding 2026-09-03 13:49:43 -07:00
proxy_behavior fix(proxy): enforce org budget ceilings on /team/update 2026-09-16 15:38:30 -07:00
proxy_e2e_anthropic_messages_tests
proxy_migration_tests fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH 2026-09-09 18:17:12 -07:00
proxy_security_tests
proxy_unit_tests Merge pull request #40596 from BerriAI/litellm_lit_7470_rate_limit_fallback_pristine_data 2026-09-16 16:47:35 -07:00
router_unit_tests test(router): force the callback to observe the usage stamp before the pre-header increment fails 2026-09-16 20:10:22 +00:00
rust-python-harness refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 2026-09-16 16:00:07 +00:00
search_tests
spend_tracking_tests
store_model_in_db_tests test(proxy): expect the sanitized unknown-model message in the spend-log error test 2026-09-11 19:18:10 -07:00
test_gateway feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545) 2026-09-10 17:14:13 -07:00
test_litellm Merge pull request #41220 from BerriAI/litellm_post_call_guardrail_context 2026-09-17 00:31:46 -07:00
test_litellm_rust refactor(ocr): move file preparation from the python bridge into litellm-core 2026-09-16 21:02:30 +00:00
unified_google_tests
vector_store_tests fix(vector-store): carry request metadata into the Router executor built from the router kwarg 2026-09-02 16:53:05 -07:00
windows_tests
__init__.py
_fake_openai_endpoint_server.py test(timeout): time out against the local fake endpoint instead of api.openai.com 2026-09-03 09:53:30 -07:00
_flush_vcr_cache.py
_live_test_helpers.py refactor(test): validate cost-map entries into a typed model 2026-09-16 14:32:08 -07:00
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_new_vector_store_endpoints.py
test_openai_endpoints.py test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_rust_python_harness.py refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 2026-09-16 16:00:07 +00:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py
test_team_logging.py
test_team_members.py
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.