litellm/tests
mateo-berri c73480c653 fix(proxy): block every unpriced model a request names
A request can name more than one model, through a comma-separated model or target_model_names on
the batch and fine-tuning routes, and the gate only looked at the string case, so an unpriced model
riding alongside a priced one went through and billed. Check every candidate and name the unpriced
ones in the 403

Aliases had the same problem on the other side: a group that prices itself through its model_info
block lands in the cost map under its deployment id, and the explicit-cost check walked the raw
model list by group name, so an alias pointing at that group read as unpriced. Resolve the group
through the router the way the pricing check already does

Also correct the 403 copy. Providers that return their own usage cost still bill for these models,
so the accurate claim is that litellm has no pricing of its own for them
2026-08-20 16:37:51 -07:00
..
agent_tests test: repair stale CircleCI contracts 2026-08-08 12:19:29 -07:00
audio_tests test: remove tests that never execute 2026-08-12 10:45:38 -07:00
base_sdk_tests test(cli): cover the keyless token record and keep keyring to the cli extra 2026-08-20 03:45:00 -07:00
basic_proxy_startup_tests
batches_tests test: build redaction and batch limiter fixtures the way production does (#37416) 2026-08-18 19:37:50 -07:00
benchmarks
code_coverage_tests test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
documentation_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
e2e test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
enterprise Merge pull request #37198 from BerriAI/litellm_lit5660_batches_limit_400 2026-08-17 15:53:46 -07:00
guardrails_tests fix(guardrails): honor configured timeout in Zscaler AI Guard (#36110) 2026-08-07 00:25:52 +00:00
image_gen_tests test: point the live gemini and groq conformance suites at models that still exist (#37422) 2026-08-19 02:50:46 +00:00
integration
litellm/proxy/_experimental/mcp_server test: merge three stranded twins into the files that shadow them (#37600) 2026-08-20 14:25:23 -07:00
litellm-proxy-extras test: remove tests that never execute 2026-08-12 10:45:38 -07:00
litellm_utils_tests test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
llm_responses_api_testing test: refresh three suites that drifted from the code they cover 2026-08-15 15:37:09 -07:00
llm_translation test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
load_tests
local_testing test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
logging_callback_tests test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
mcp_tests test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): update Azure DI supported-params assertion for req_format 2026-08-18 19:21:46 -07:00
openai_endpoints_tests fix(batches): register managed output files on batch cancel 2026-08-05 18:28:52 -07:00
otel_tests
pass_through_tests
pass_through_unit_tests test: allow protocol-constrained pass-through routes to declare fewer methods (#37415) 2026-08-18 19:38:05 -07:00
proxy_admin_ui_tests fix(access groups): sync assigned_team_ids from the team write paths (#36825) 2026-08-14 04:45:36 +00:00
proxy_behavior feat(proxy): add /team/daily/activity/aggregated and switch the Usage team tab to it (#36562) 2026-08-18 11:29:57 -07:00
proxy_e2e_anthropic_messages_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
proxy_migration_tests test(docker): gate the componentized gateway and backend images on an arbitrary-uid offline boot (#36136) 2026-08-07 09:57:37 -07:00
proxy_security_tests
proxy_unit_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
router_unit_tests feat(complexity_router): custom classifier plugins via classifier_type 'custom' (#37249) 2026-08-18 14:09:19 -07:00
search_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr32448_tinyfish_headers 2026-08-18 13:03:33 -07:00
spend_tracking_tests
store_model_in_db_tests test: refresh three suites that drifted from the code they cover 2026-08-15 15:37:09 -07:00
test_litellm fix(proxy): block every unpriced model a request names 2026-08-20 16:37:51 -07:00
unified_google_tests
vector_store_tests
windows_tests
__init__.py
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py test: replace blind sleeps with deadline waits in callback and caching tests (#37660) 2026-08-20 18:48:43 +00:00
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py test: refresh three suites that drifted from the code they cover 2026-08-15 15:37:09 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_team_logging.py
test_team_members.py
test_users.py

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.