scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36
litellm/tests/test_litellm/proxy
History
Exact
Tin Chi Lo 39622ea18f
Some checks failed
LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled
Details
fix(spend): credit prompt caching savings only when the gateway injected the breakpoints 
prompt_caching_savings_spend was computed from the response usage alone, so
client-sent cache_control (Claude Code style harnesses) and implicit provider
caching (OpenAI, Gemini) accrued savings the gateway had no hand in. The cache
control hook records how many breakpoints litellm itself inserted as a
litellm_injected_cache_breakpoints metadata marker, the spend writer reads it
back from SpendLogsMetadata, and compute_savings_spend zeroes the caching leg
when the marker is absent. Forward-only; historical rollup rows are untouched.

Recording lives in the shared prompt-hook wrappers on the logging object, so
every surface that runs the hook marks its requests the same way: chat
completions, /v1/responses, router prompt deployments, and proxy prompt
templates. Re-entry is safe because only positive breakpoint deltas accumulate.

Resolves LIT-5927
2026-08-21 17:43:30 -07:00
..
_experimental/mcp_server fix(mcp): strip root_path before matching the per-server MCP route spelling (#35576) 2026-08-21 15:40:14 -07:00
a2a fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases 2026-07-21 13:24:58 -07:00
agent_endpoints fix(a2a): return SSE (text/event-stream) for message/stream instead of NDJSON (#35037) 2026-08-20 16:19:58 -07:00
analytics_endpoints feat(ui): split failed requests into their own series on the cache dashboard (#34862) 2026-07-29 09:48:17 -07:00
anthropic_endpoints fix(proxy): estimate failed-request input tokens on /v1/messages and count system prompts 2026-08-18 14:21:47 -07:00
auth test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
batches_endpoints test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
client test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
common_utils test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
config_resolvers feat(proxy): add SAML 2.0 SSO for the admin UI (#31429) 2026-07-24 12:51:28 -07:00
credential_endpoints fix(proxy): return the real status code when a credential update is rejected (#36166) 2026-08-07 08:02:52 +00:00
db fix(spend): credit prompt caching savings only when the gateway injected the breakpoints 2026-08-21 17:43:30 -07:00
discovery_endpoints test: remove tests that never execute 2026-08-12 10:45:38 -07:00
enterprise_billing feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592) 2026-07-15 12:12:52 -07:00
experimental/mcp_server
fine_tuning_endpoints fix(proxy): authenticate managed ids before routing 2026-08-08 02:42:34 -07:00
google_endpoints
guardrails test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
health_endpoints test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
hooks test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
image_endpoints fix(proxy)!: share one destination check between body and path-supplied model 2026-08-05 15:17:43 -07:00
logging_endpoints feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134) 2026-06-24 15:25:10 -07:00
management_endpoints Merge pull request #37878 from BerriAI/litellm_ruff_no_duplicate_definitions 2026-08-21 12:49:43 -07:00
management_helpers test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
memory fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
middleware feat(proxy): add Amazon Comprehend Medical passthrough provider 2026-08-17 15:44:06 -07:00
ocr_endpoints fix(ocr): validate body req_format in the proxy endpoint and run its tests in CI 2026-08-17 18:29:35 +00:00
openai_files_endpoint fix(proxy): scan batch records with the content hooks that are not guardrails (#37786) 2026-08-21 11:20:23 -07:00
pass_through_endpoints Merge pull request #37878 from BerriAI/litellm_ruff_no_duplicate_definitions 2026-08-21 12:49:43 -07:00
policy_engine test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
prompts test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
proxy_server test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
public_endpoints feat(cognition): give Cognition its own provider identity 2026-08-20 17:16:53 -07:00
rag_endpoints fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438) 2026-07-17 17:45:27 +00:00
realtime_endpoints fix(realtime): bound Vertex credential resolution and make realtime failures loud 2026-08-20 02:02:00 -07:00
response_api_endpoints fix(responses_api): map bridged chat usage on guardrail-blocked replies 2026-08-14 17:04:27 -07:00
shutdown
spend_tracking fix(spend): credit prompt caching savings only when the gateway injected the breakpoints 2026-08-21 17:43:30 -07:00
test_configs
types_utils fix(proxy): resolve router_settings.plugins dotted paths and load plugins from installed packages (#33644) 2026-07-17 11:23:18 -07:00
ui_crud_endpoints feat(ui): let admins supply a dark-mode variant of their custom logo (#37662) 2026-08-20 12:48:55 -07:00
utils fix(guardrails): run policy pipelines when the caller sends its own metadata (/v1/messages, Claude Code) (#36889) 2026-08-21 13:04:04 -07:00
vector_store_endpoints test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
vector_store_files_endpoints fix(proxy): authenticate managed ids before routing 2026-08-08 02:42:34 -07:00
video_endpoints test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py
conftest.py test(proxy): keep a leaked llm_router out of the next test in the worker 2026-08-20 23:19:17 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py
test_batch_expiry.py
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py test(proxy): give the x-litellm-model fallback test deterministic openai env creds 2026-08-17 12:42:57 -07:00
test_blocked_response_usage.py fix(responses_api): map bridged chat usage on guardrail-blocked replies 2026-08-14 17:04:27 -07:00
test_budget_reservation.py test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748) 2026-08-20 19:36:26 -07:00
test_caching_routes.py
test_chat_completion_metadata.py
test_claude_code_marketplace.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_common_request_processing.py Merge pull request #37878 from BerriAI/litellm_ruff_no_duplicate_definitions 2026-08-21 12:49:43 -07:00
test_component_allowlists.py feat(proxy): authenticate to Azure Postgres with Microsoft Entra ID tokens 2026-08-20 11:50:16 -07:00
test_conftest.py test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client 2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845) 2026-08-14 17:04:32 -07:00
test_empty_model_list.py
test_enforce_user_param.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py
test_health_check_functions.py
test_health_check_max_tokens.py fix: skip health check for semantic auto_router deployments (#31668) 2026-06-29 19:47:20 -07:00
test_init_litellm_callbacks.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py feat(proxy): add GET /v1/indexes to list vector store indexes (#36289) 2026-08-10 15:09:59 -07:00
test_litellm_pre_call_utils.py fix(spend): credit prompt caching savings only when the gateway injected the breakpoints 2026-08-21 17:43:30 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py
test_model_deprecations_endpoint.py refactor(proxy): tighten model deprecation typing and cover the endpoint 2026-08-10 22:58:35 +00:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py feat(proxy): add x-litellm-model-name response header with deployment model string (#33698) 2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py
test_model_level_guardrails.py fix(proxy): apply key/team router_settings.model_group_alias (#35486) 2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py
test_modify_response_streaming_passthrough.py fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800) 2026-06-18 23:29:08 -07:00
test_openai_ws_passthrough_routes.py fix(proxy): close websocket cleanly when OpenAI credentials are missing 2026-08-16 14:40:37 -07:00
test_openapi_schema_validation.py
test_plugin_routes.py feat: litellm plugin architecture v2 (#30688) 2026-06-20 20:37:22 -07:00
test_pricing_field_strip.py fix(guardrails): bill all chunks on mid-chunking block, strip client guardrail cost metadata, add cost map schema keys 2026-08-18 14:52:23 -07:00
test_prisma_engine_watchdog.py test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
test_prisma_migration.py fix(proxy): fail the standalone prisma migration entrypoint on migration errors (#37692) 2026-08-20 16:20:27 -07:00
test_prometheus_cleanup.py
test_provider_url_destination_guard.py fix(proxy)!: share one destination check between body and path-supplied model 2026-08-05 15:17:43 -07:00
test_proxy_cli.py fix(db): apply the configured connection params to the read replica URL (#37691) 2026-08-20 17:23:06 -07:00
test_proxy_logging_hook_detection.py test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748) 2026-08-20 19:36:26 -07:00
test_proxy_server.py Merge pull request #37878 from BerriAI/litellm_ruff_no_duplicate_definitions 2026-08-21 12:49:43 -07:00
test_proxy_types.py feat(proxy): add project-level ITPM and OTPM quotas 2026-08-05 12:53:22 +05:30
test_proxy_utils.py test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731) 2026-08-20 18:09:42 -07:00
test_pyroscope.py
test_read_model_list.py feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135) 2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_response_model_sanitization.py
test_route_a2a_models.py fix(proxy): single-row read-through resyncs and reload-race hardening 2026-08-18 21:02:12 -07:00
test_route_llm_request.py test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748) 2026-08-20 19:36:26 -07:00
test_sensitive_route_auth.py
test_shared_health_check.py
test_spend_log_cleanup.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_swagger_chat_completions.py
test_team_member_update.py
test_team_org_move.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_tools_allowlist_enforcement.py fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI (#32258) 2026-07-06 17:34:27 -07:00
test_update_llm_router_resilience.py feat(complexity_router): custom classifier plugins via classifier_type 'custom' (#37249) 2026-08-18 14:09:19 -07:00