litellm/tests
Cursor Agent a35ffd420b
Some checks are pending
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
fix(claude_code): bugbot — make bare-Bash security pin actually fail on residual
The second assertion in test_bash_allow_rule_is_pinned_to_exact_echo_pong
was dead code: '"Bash"' not in text or '"Bash(echo pong)"' in text
short-circuits to True any time the allow rule is present, which is
guaranteed by the first assertion. A test file containing both the
unrestricted "Bash" pattern AND the restricted "Bash(echo pong)"
pattern would have passed this security check undetected, defeating the
exact-match permissions pin that protects the PR-gate machine executor
from arbitrary host command execution.

Strip the allowed pattern out of the file text before scanning, so the
residual check is independent of the first assertion. The pure helper
_has_bare_bash_token() is exercised directly by three new unit tests
covering both the positive (bare "Bash" → flagged) and negative
(only "Bash(echo pong)" → accepted; unrelated 'Bashing' substrings →
ignored) paths so this regression cannot recur silently.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-18 05:07:49 +00:00
..
agent_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
audio_tests test(vcr): classify cache verdicts, detect live calls, surface cost leaks 2026-05-13 00:31:47 +00:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
benchmarks Add CodSpeed performance benchmarks (#23676) 2026-03-14 18:44:36 -07:00
claude_code fix(claude_code): bugbot — make bare-Bash security pin actually fail on residual 2026-05-18 05:07:49 +00:00
code_coverage_tests feat: add componentized proxy deployment with gateway, backend, ui, and migrations (#27557) 2026-05-16 09:25:17 -07:00
documentation_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
enterprise chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
guardrails_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
image_gen_tests Merge pull request #27795 from BerriAI/litellm_vcr-cache-observability-and-fixes-c5bc 2026-05-14 13:51:16 -07:00
litellm Add new chat model metadata (#27313) 2026-05-06 15:15:21 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
llm_responses_api_testing chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
llm_translation Merge branch 'litellm_internal_staging' into litellm_grid-v4-e2e-tests-cZRwz 2026-05-16 16:19:38 +00:00
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing test(ci): skip Fireworks tests on 404 + Gemini image-size test on 429 2026-05-16 07:47:25 +00:00
logging_callback_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
mcp_tests feat: litellm shin agent oss staging 05 10 2026 (#27631) 2026-05-11 20:31:43 -07:00
multi_instance_e2e_tests
ocr_tests test(vcr): classify cache verdicts, detect live calls, surface cost leaks 2026-05-13 00:31:47 +00:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
pass_through_tests [Infra] Promote internal staging to main (#27245) 2026-05-05 16:15:03 -07:00
pass_through_unit_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
proxy_admin_ui_tests [Infra] Promote internal staging to main (#27245) 2026-05-05 16:15:03 -07:00
proxy_e2e_anthropic_messages_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
proxy_security_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_unit_tests fix(managed_batches): convert raw output_file_id to managed ID in CheckBatchCost poller (#27984) 2026-05-15 04:41:38 -07:00
router_unit_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
scim_tests
search_tests test(vcr): classify cache verdicts, detect live calls, surface cost leaks 2026-05-13 00:31:47 +00:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm fix(mcp-oauth): PROXY_BASE_URL escape hatch + diagnostic logging for {"detail":"invalid_request"} (#28086) 2026-05-16 17:48:03 -07:00
unified_google_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
vector_store_tests fix: drop milvus dbName and partitionNames from MILVUS_OPTIONAL_PARAMS 2026-04-30 11:51:32 -07:00
windows_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py
_flush_vcr_cache.py [Infra] Promote internal staging to main (#27245) 2026-05-05 16:15:03 -07:00
_vcr_conftest_common.py fix(vcr): aggregate worker stats on the controller so the session summary actually renders under xdist 2026-05-13 07:24:32 +00:00
_vcr_redis_persister.py [Infra] Promote internal staging to main (#27245) 2026-05-05 16:15:03 -07:00
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.