litellm/tests
Yuneng Jiang 593fa5921a
test(e2e/ui): wait for the filtered budget list before clicking a row action
All three budget specs searched by typing into the search box and moved on
immediately. The search is debounced 300ms, and while the filtered query is in
flight react-query serves the previous page as placeholder data, which the list
hook reports as isLoading, which makes the table swap its whole body for
skeleton rows.

So the row assertion passed against the pre-search rows, and roughly 300ms later
the skeleton swap unmounted the row the spec had just opened the action menu on.
Playwright logged "element is not stable" twice and then "element was detached
from the DOM", and since the menu never reopened the click burned the full 15s
action timeout on all three attempts. Losing that race was pure timing: build
386 and build 387 of the UI suite ran the same commit 4b368bf066, and 386
passed where 387 failed on this spec plus "Delete a budget"

searchForBudget now waits for the GET that carries q=<budget id>, matching what
projectDetachment.spec.ts already does for a key search. That also gives the row
assertion something real to assert, since until now it could pass without the
search having filtered anything
2026-09-17 21:39:22 -07:00
..
agent_tests
audio_tests
base_sdk_tests
basic_proxy_startup_tests
batches_tests
benchmarks
code_coverage_tests ci: drop aws partition hardcode gate 2026-09-17 20:16:07 +00:00
documentation_tests
e2e test(e2e/ui): wait for the filtered budget list before clicking a row action 2026-09-17 21:39:22 -07:00
enterprise fix(projects): persist explicit budget cap clears 2026-09-12 13:43:55 -07:00
guardrails_tests fix(guardrails): give post-call scans the scoped request conversation and tools 2026-09-15 09:58:53 +00:00
image_gen_tests test: remove fully commented-out test files that collect no tests 2026-09-17 20:05:27 +00:00
integration Merge pull request #41373 from BerriAI/litellm_fix_integration_conftest_import 2026-09-17 13:34:05 -07:00
litellm-proxy-extras test(proxy-extras): move the LITELLM_LOG logger test into the CI-run proxy-extras shard 2026-09-15 22:57:23 +00:00
litellm_utils_tests fix(proxy): forward every method on the typesafe pass-through route 2026-09-17 18:04:58 -07:00
llm_responses_api_testing test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
llm_translation refactor(test): validate cost-map entries into a typed model 2026-09-16 14:32:08 -07:00
load_tests
local_testing Merge pull request #41660 from BerriAI/litellm_remove_commented_out_proxy_tests 2026-09-17 17:56:02 -07:00
logging_callback_tests test(logging): add azure_spillover to the GCS pub/sub spend-log golden 2026-09-17 18:33:42 -07:00
mcp_tests fix(mcp): reject initialize with 403 when the key grants no MCP servers (#40616) 2026-09-10 14:25:30 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): exempt native parity requests from cassette replay 2026-09-11 22:50:39 -07:00
openai_endpoints_tests test(responses): fix stale Anthropic smoke request 2026-09-11 13:56:27 -07:00
otel_tests feat(cli): deprecate the litellm-proxy entrypoint in favour of lite 2026-09-17 14:05:31 -07:00
pass_through_tests
pass_through_unit_tests Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts 2026-09-15 13:38:00 -07:00
proxy_admin_ui_tests
proxy_behavior fix(proxy): enforce org budget ceilings on /team/update 2026-09-16 15:38:30 -07:00
proxy_e2e_anthropic_messages_tests
proxy_migration_tests fix(bedrock): support aws-sdk-bedrock-runtime 0.10 and 0.11 in the realtime handler 2026-09-17 01:40:52 +00:00
proxy_security_tests
proxy_unit_tests Merge pull request #41660 from BerriAI/litellm_remove_commented_out_proxy_tests 2026-09-17 17:56:02 -07:00
router_unit_tests test(router): force the callback to observe the usage stamp before the pre-header increment fails 2026-09-16 20:10:22 +00:00
rust-python-harness refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 2026-09-16 16:00:07 +00:00
search_tests test: remove fully commented-out test files that collect no tests 2026-09-17 20:05:27 +00:00
spend_tracking_tests
store_model_in_db_tests test(proxy): expect the sanitized unknown-model message in the spend-log error test 2026-09-11 19:18:10 -07:00
test_gateway feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545) 2026-09-10 17:14:13 -07:00
test_litellm Merge pull request #41736 from BerriAI/litellm_mistral_cache_read_pricing 2026-09-17 19:35:26 -07:00
test_litellm_rust merge: port OCR request validation and upstream error mapping onto main's dispatch layout 2026-09-17 18:33:20 +00:00
unified_google_tests
vector_store_tests
windows_tests
__init__.py
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py refactor(test): validate cost-map entries into a typed model 2026-09-16 14:32:08 -07:00
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_new_vector_store_endpoints.py
test_openai_endpoints.py test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_rust_python_harness.py refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 2026-09-16 16:00:07 +00:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py
test_team_logging.py
test_team_members.py
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.