scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 89
litellm/tests
History
Exact
Tin Chi Lo 39622ea18f
Some checks failed
LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled
Details
fix(spend): credit prompt caching savings only when the gateway injected the breakpoints 
prompt_caching_savings_spend was computed from the response usage alone, so
client-sent cache_control (Claude Code style harnesses) and implicit provider
caching (OpenAI, Gemini) accrued savings the gateway had no hand in. The cache
control hook records how many breakpoints litellm itself inserted as a
litellm_injected_cache_breakpoints metadata marker, the spend writer reads it
back from SpendLogsMetadata, and compute_savings_spend zeroes the caching leg
when the marker is absent. Forward-only; historical rollup rows are untouched.

Recording lives in the shared prompt-hook wrappers on the logging object, so
every surface that runs the hook marks its requests the same way: chat
completions, /v1/responses, router prompt deployments, and proxy prompt
templates. Re-entry is safe because only positive breakpoint deltas accumulate.

Resolves LIT-5927
2026-08-21 17:43:30 -07:00
..
agent_tests
audio_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
base_sdk_tests test(cli): cover the keyless token record and keep keyring to the cli extra 2026-08-20 03:45:00 -07:00
basic_proxy_startup_tests
batches_tests test: build redaction and batch limiter fixtures the way production does (#37416) 2026-08-18 19:37:50 -07:00
benchmarks
code_coverage_tests test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
documentation_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
e2e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit_5870_passthrough_e2e_pins_wt 2026-08-20 17:33:35 -07:00
enterprise test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
guardrails_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
image_gen_tests test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
integration
litellm/proxy/_experimental/mcp_server test: merge three stranded twins into the files that shadow them (#37600) 2026-08-20 14:25:23 -07:00
litellm-proxy-extras
litellm_utils_tests test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
llm_responses_api_testing test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
llm_translation test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
load_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
local_testing test: wrap the raising call, not the print that follows it 2026-08-21 13:45:40 -07:00
logging_callback_tests feat(spend): surface per-request auto-router savings to logging callbacks (#37894) 2026-08-21 15:41:35 -07:00
mcp_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
multi_instance_e2e_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
ocr_tests test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
openai_endpoints_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
otel_tests test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
pass_through_tests
pass_through_unit_tests test: keep a real assertion where the tolerance handler lost its last one 2026-08-21 14:08:19 -07:00
proxy_admin_ui_tests test: wrap the raising call, not the print that follows it 2026-08-21 13:45:40 -07:00
proxy_behavior feat(proxy): add /team/daily/activity/aggregated and switch the Usage team tab to it (#36562) 2026-08-18 11:29:57 -07:00
proxy_e2e_anthropic_messages_tests ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
proxy_migration_tests
proxy_security_tests
proxy_unit_tests test: keep a real assertion where the tolerance handler lost its last one 2026-08-21 14:08:19 -07:00
router_unit_tests test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
search_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr32448_tinyfish_headers 2026-08-18 13:03:33 -07:00
spend_tracking_tests
store_model_in_db_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_litellm fix(spend): credit prompt caching savings only when the gateway injected the breakpoints 2026-08-21 17:43:30 -07:00
unified_google_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
vector_store_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
windows_tests
__init__.py
_fake_openai_endpoint_server.py test(ci): serve /moderations from the canned OpenAI mock (#37739) 2026-08-20 16:58:38 -07:00
_flush_vcr_cache.py
_live_test_helpers.py
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py test: replace blind sleeps with deadline waits in callback and caching tests (#37660) 2026-08-20 18:48:43 +00:00
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py ci: lint the test tree for undefined names and fix all 30 (#37671) 2026-08-20 13:30:34 -07:00
test_fallbacks.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py test: point the live web search, groq and vertex image suites at models that still exist (#37733) 2026-08-20 17:03:35 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748) 2026-08-20 19:36:26 -07:00
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_team_logging.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_team_members.py test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
test_users.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.