scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-15 23:31:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 165
litellm/tests
History
ryan-crabbe-berri 760043b533 fix(reset_budget_job): reset end users by budget link, not by user id 
The cascade zeroed end-user spend with a single update_many whose where
clause enumerated every dependent user id. Prisma compiles that IN-list
into one prepared statement carrying one bind variable per customer, and
PostgreSQL caps a statement at 32,767 of them. Once a shared budget had
more dependents than that the statement could not be parsed at all, so
the atomic cascade rolled back, budget_reset_at never advanced, and the
tier stayed due on every later tick forever. Customers sitting at their
cap were blocked indefinitely with only a recurring log line to show for
it.

End users now match on budget_id like every other gated table, plus a
NULL-budget_id branch for the implicitly created rows that carry no link
and ride the default tier. The statement's bind count now tracks the
number of expiring tiers rather than the customer population, so a reset
costs the same whether a budget has ten dependents or a million.

Fixes #40564

Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW
2026-09-10 16:57:53 -07:00
..
agent_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
audio_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
base_sdk_tests test(cli): cover the keyless token record and keep keyring to the cli extra 2026-08-20 03:45:00 -07:00
basic_proxy_startup_tests
batches_tests feat(batches): enrich batch cost rows with breakdown, identity, session, and org spend 2026-09-03 17:23:53 -04:00
benchmarks
code_coverage_tests feat(router): support percentile-based TTFT routing (#40352) 2026-09-09 10:47:34 -07:00
documentation_tests Merge remote-tracking branch 'origin/main' into litellm_bedrock_messages_disconnect_billing 2026-08-31 08:58:37 -07:00
e2e test(router): keep classifier E2E verification local 2026-09-10 12:46:50 -07:00
enterprise fix(batches): register ownership for every batch create path (#39810) 2026-09-04 23:59:51 -07:00
guardrails_tests fix(logging): blocked requests no longer report guardrail_status=success in multi-guardrail configs (#39596) 2026-09-03 17:33:31 -07:00
image_gen_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
integration test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
litellm-proxy-extras fix(jwt): cascade-delete JWT key mappings when their virtual key is deleted 2026-09-09 15:39:21 -07:00
litellm_utils_tests fix(reset_budget_job): reset end users by budget link, not by user id 2026-09-10 16:57:53 -07:00
llm_responses_api_testing fix(guardrails): deliver modify_response block as valid SSE on streaming chat and Responses 2026-08-31 16:01:39 -07:00
llm_translation Merge remote-tracking branch 'origin/litellm_internal_staging' into HEAD 2026-09-10 13:55:11 -07:00
load_tests feat(proxy): per-worker admission control that rejects excess requests with 503 (#39352) 2026-09-03 18:19:04 -07:00
local_testing chore: merge litellm_internal_staging into litellm_lit_7039_least_busy_shared_counts 2026-09-08 17:11:05 +00:00
logging_callback_tests Merge branch 'litellm_internal_staging' into litellm_fix_failing_request_slowdown 2026-09-03 00:20:51 -07:00
mcp_tests fix(mcp): reject initialize with 403 when the key grants no MCP servers (#40616) 2026-09-10 14:25:30 -07:00
multi_instance_e2e_tests test: say whether a match= pattern is a regex or a literal (ruff RUF043) 2026-08-21 16:25:33 -07:00
ocr_tests fix(vertex): avoid duplicate DeepSeek OCR model namespace 2026-09-01 14:39:15 -07:00
openai_endpoints_tests refactor(tests): assign the streamed id and lock poll once instead of rebinding 2026-09-03 13:49:43 -07:00
otel_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
pass_through_tests fix(logging): key bridged /v1/messages rows on the id the caller received 2026-09-03 01:15:06 -07:00
pass_through_unit_tests feat(proxy): granular key/team access control for Claude Code marketplace plugins (#40518) 2026-09-10 14:26:09 -07:00
proxy_admin_ui_tests refactor(tests): assign the streamed id and lock poll once instead of rebinding 2026-09-03 13:49:43 -07:00
proxy_behavior feat(ui): itemize auto-router classification spend (#40168) 2026-09-07 16:29:19 -07:00
proxy_e2e_anthropic_messages_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
proxy_migration_tests chore: merge litellm_internal_staging into litellm_fix_invalid_index_after_migration_deadlock 2026-09-08 17:07:34 +00:00
proxy_security_tests
proxy_unit_tests fix(proxy): surface runtime-registered callbacks in UI Logging page (#38974) 2026-09-09 22:03:16 -07:00
router_unit_tests fix(router): count allowed_fails in the shared router cache so multi-worker proxies bench a deployment fleet-wide (#40224) 2026-09-08 13:54:48 -07:00
rust-python-harness test(ocr): add SDK callback E2E parity (#40061) 2026-09-07 11:23:12 -07:00
search_tests fix(search): harden bing_grounding auth, result cap, status, and cost 2026-08-24 12:26:07 -07:00
spend_tracking_tests
store_model_in_db_tests test(store_model_in_db): expect the placeholder model on rejected unknown-model spend logs 2026-09-10 14:42:39 -07:00
test_gateway feat(proxy): let the in-container pgbouncer follow rotating RDS IAM and Azure Entra tokens (#40623) 2026-09-10 16:18:00 -07:00
test_litellm fix(reset_budget_job): reset end users by budget link, not by user id 2026-09-10 16:57:53 -07:00
test_litellm_rust test(ocr): isolate native Rust bridge contracts (#40410) 2026-09-10 16:51:37 -07:00
unified_google_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
vector_store_tests fix(vector-store): carry request metadata into the Router executor built from the router kwarg 2026-09-02 16:53:05 -07:00
windows_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
__init__.py
_fake_openai_endpoint_server.py test(timeout): time out against the local fake endpoint instead of api.openai.com 2026-09-03 09:53:30 -07:00
_flush_vcr_cache.py
_live_test_helpers.py
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py test: replace blind sleeps with deadline waits in callback and caching tests (#37660) 2026-08-20 18:48:43 +00:00
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
test_fallbacks.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py test: fix staging CI regressions from #38182, #38144, #38265, #37962, and #37969 2026-08-25 23:01:20 -07:00
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_new_vector_store_endpoints.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_openai_endpoints.py test: point the live web search, groq and vertex image suites at models that still exist (#37733) 2026-08-20 17:03:35 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_resource_cleanup.py
test_rust_python_harness.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_team_logging.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_team_members.py test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
test_users.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.