scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 84
litellm/tests
History
yucheng-berri 8122cfc1ec
fix(ptu): require an operator-declared id on a config.yaml reservation (#37794) 
A config deployment is otherwise keyed by a hash of its resolved
litellm_params, so rotating a credential or editing an endpoint mints a
second identity and the catch-up bills the reservation again under it. Flat
cost is keyed by that id and a written charge is never retracted, so the
duplicate is permanent.

The id is read before set_model_list mints one, or the rule would inspect
the value it is meant to reject. Duplicates are counted once per config
entry across the whole file, so the check is order-independent and an
organization fan-out cannot collide with itself. The refusal names the id
the deployment already uses, since inventing a fresh one starts exactly the
second identity this prevents
2026-08-21 11:20:47 -07:00
..
agent_tests test: repair stale CircleCI contracts 2026-08-08 12:19:29 -07:00
audio_tests test: remove tests that never execute 2026-08-12 10:45:38 -07:00
base_sdk_tests test(cli): cover the keyless token record and keep keyring to the cli extra 2026-08-20 03:45:00 -07:00
basic_proxy_startup_tests
batches_tests test: build redaction and batch limiter fixtures the way production does (#37416) 2026-08-18 19:37:50 -07:00
benchmarks
code_coverage_tests test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
documentation_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
e2e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit_5870_passthrough_e2e_pins_wt 2026-08-20 17:33:35 -07:00
enterprise test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
guardrails_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
image_gen_tests test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
integration
litellm/proxy/_experimental/mcp_server test: merge three stranded twins into the files that shadow them (#37600) 2026-08-20 14:25:23 -07:00
litellm-proxy-extras test: remove tests that never execute 2026-08-12 10:45:38 -07:00
litellm_utils_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
llm_responses_api_testing test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
llm_translation Merge pull request #37003 from BerriAI/litellm_forward_bedrock_response_headers 2026-08-20 20:35:13 -07:00
load_tests
local_testing test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
logging_callback_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
mcp_tests test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
multi_instance_e2e_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
ocr_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
openai_endpoints_tests test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
otel_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
pass_through_tests
pass_through_unit_tests test: allow protocol-constrained pass-through routes to declare fewer methods (#37415) 2026-08-18 19:38:05 -07:00
proxy_admin_ui_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
proxy_behavior feat(proxy): add /team/daily/activity/aggregated and switch the Usage team tab to it (#36562) 2026-08-18 11:29:57 -07:00
proxy_e2e_anthropic_messages_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
proxy_migration_tests test(docker): gate the componentized gateway and backend images on an arbitrary-uid offline boot (#36136) 2026-08-07 09:57:37 -07:00
proxy_security_tests
proxy_unit_tests test(lint): clear the two PT011/PT012 violations left on the test tree (#37864) 2026-08-21 11:15:06 -07:00
router_unit_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
search_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr32448_tinyfish_headers 2026-08-18 13:03:33 -07:00
spend_tracking_tests
store_model_in_db_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_litellm fix(ptu): require an operator-declared id on a config.yaml reservation (#37794) 2026-08-21 11:20:47 -07:00
unified_google_tests
vector_store_tests test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
windows_tests
__init__.py
_fake_openai_endpoint_server.py test(ci): serve /moderations from the canned OpenAI mock (#37739) 2026-08-20 16:58:38 -07:00
_flush_vcr_cache.py
_live_test_helpers.py
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py test: replace blind sleeps with deadline waits in callback and caching tests (#37660) 2026-08-20 18:48:43 +00:00
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py test: point the live web search, groq and vertex image suites at models that still exist (#37733) 2026-08-20 17:03:35 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748) 2026-08-20 19:36:26 -07:00
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_team_logging.py
test_team_members.py
test_users.py

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.