scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102
litellm/tests/test_litellm/proxy
History
Yassin Kortam 433d016f0c
fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 
A streaming request that breaks mid-flight, for example on a mid-stream read
timeout, still bills the provider for the chunks already delivered, yet the proxy
recorded that interrupted request as a zero-spend failure. An earlier revision
logged the recovered partial usage through the success path, which mislabeled a
failed request as a success and produced a misleading spend row

This recovers the partial usage where the failure is actually logged. The
streaming handler assembles the usage from the chunks seen so far and stashes it,
with its cost, on the logging object before firing the failure handlers. The
proxy failure hook lifts that usage and cost onto request_data before the
non-serialisable logging object is popped, and the spend-log writer records the
real partial spend on the failure row instead of a hardcoded zero;
get_logging_payload honors the recovered usage for the token columns and
_failure_handler_helper_fn preserves the recovered cost so the non-DB failure
loggers stay consistent

A request that recovers via a successful fallback is unaffected: the failure hook
only fires when the whole request fails, so the fallback's combined-usage success
row stays the single source of truth and there is no double counting

Resolves LIT-3825

Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
(cherry picked from commit 4847fa5dd5)
2026-06-23 21:00:05 -07:00
..
_experimental/mcp_server feat(mcp): scope a key to zero MCP servers with no-mcp-servers sentinel (#31029) 2026-06-23 18:55:18 -07:00
agent_endpoints style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
anthropic_endpoints test: isolate proxy master_key/prisma_client module globals between tests 2026-04-23 15:31:16 -07:00
auth fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures (#29986) 2026-06-13 16:21:02 -07:00
client fix: scope CLI stored token to base_url to prevent cross-domain credential leakage (#26945) 2026-05-01 12:11:32 -07:00
common_utils fix(reset_budget): write only {spend, budget_reset_at} and stop pre-zeroing counter (#29358) (#29364) 2026-05-30 18:20:13 -07:00
db fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures (#29986) 2026-06-13 16:21:02 -07:00
discovery_endpoints style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
experimental/mcp_server
google_endpoints Gemini managed agents support (#28270) 2026-05-19 16:02:03 -07:00
guardrails fix(guardrails): stop re-initializing DB guardrails on every poll (#30542) 2026-06-20 11:57:13 -07:00
health_endpoints fix(proxy): expose db status on public /health/readiness 2026-05-13 13:02:38 -07:00
hooks fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 2026-06-23 21:00:05 -07:00
image_endpoints style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
management_endpoints fix(key_generate): harden GHSA-q775 session-token exemption against default_key_generate_params 2026-06-04 01:16:13 +00:00
management_helpers feat(mcp): scope a key to zero MCP servers with no-mcp-servers sentinel (#31029) 2026-06-23 18:55:18 -07:00
memory Litellm memory improvements v2 (#26541) 2026-04-25 19:03:43 -07:00
middleware fix(proxy): point /metrics 401 at the opt-out flag 2026-05-08 18:09:14 -07:00
openai_files_endpoint Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr17 2026-04-17 17:36:40 -07:00
pass_through_endpoints fix(passthrough): recover output tokens for interrupted anthropic streams (#30787) 2026-06-23 20:44:26 -07:00
policy_engine style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
prompts style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
public_endpoints chore(proxy): keep public AI hub unauthenticated 2026-05-01 19:07:21 -07:00
rag_endpoints fix: harden /key/update authorization checks (#27878) 2026-05-14 04:16:04 +00:00
realtime_endpoints test: isolate proxy master_key/prisma_client module globals between tests 2026-04-23 15:31:16 -07:00
response_api_endpoints style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
spend_tracking fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 2026-06-23 21:00:05 -07:00
test_configs
types_utils chore(proxy): also scrub guardrail callbacks / module paths from DB overlay 2026-05-14 01:24:51 +00:00
ui_crud_endpoints fix(proxy): preserve public ai hub ui setting 2026-05-01 19:46:27 -07:00
vector_store_endpoints fix(vector_store): cache use-time embedding-config resolution 2026-05-03 10:27:53 +00:00
__init__.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
conftest.py test: isolate proxy master_key/prisma_client module globals between tests 2026-04-23 15:31:16 -07:00
test_aiohttp_cleanup_closed.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aiohttp_session_recovery.py fix: use AsyncMock for concurrent test consistency 2026-03-18 00:54:23 +00:00
test_api_key_masking_in_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_audio_speech_prometheus_hooks.py fix req changes 2026-02-28 21:32:57 +05:30
test_batch_expiry.py Patches for v1.87.0-rc.1 (#28915) 2026-05-26 17:34:19 -07:00
test_batch_metadata_none_fix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_batch_retrieve_bedrock.py fix(proxy/batches): forward model to retrieve_batch for bedrock 2026-04-29 22:48:03 +02:00
test_budget_reservation.py fix(proxy): gate image-gen reservation strictly on model mode 2026-05-09 09:16:27 -07:00
test_caching_routes.py
test_chat_completion_metadata.py
test_common_request_processing.py fix(guardrails): return 400 not 500 when AIM blocks a request (#30573) 2026-06-20 12:13:02 -07:00
test_component_allowlists.py feat: add componentized proxy deployment with gateway, backend, ui, and migrations (#27557) 2026-05-16 09:25:17 -07:00
test_cors_config.py refactor: extract _get_cors_config() for testability, fix no-op CORS tests 2026-04-11 22:24:04 +05:30
test_custom_proxy.py
test_dynamic_mcp_route.py feat(mcp): support MCP access group names in URL-based namespacing (#27726) 2026-05-13 20:20:38 -07:00
test_empty_model_list.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_enforce_user_param.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_fallback_management_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_fastapi_offline_routes.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_models_by_team_access_group.py fix: resolve access group names in _filter_models_by_team_id (#25224) 2026-05-01 11:55:36 +05:30
test_health_check_functions.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_health_check_max_tokens.py Litellm agent oss staging 05 11 2026 (#27733) 2026-05-13 14:09:12 -07:00
test_langfuse_passthrough_security.py chore(callbacks): guard dynamic integration hosts 2026-04-30 14:27:19 -07:00
test_lazy_openapi_snapshot.py [Fix] Tests: Move Misplaced Import in Lazy OpenAPI Snapshot Test 2026-05-01 14:24:53 -07:00
test_litellm_pre_call_utils.py encrypt callback_vars in key/team metadata at rest (#27141) 2026-05-23 12:15:44 -07:00
test_max_budget_env_var.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_mcp_asgi_response.py fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847) 2026-05-13 12:03:36 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_model_info_default_limits.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_model_level_guardrails.py fix(guardrails): run pre_call hook once for model-level guardrails (#30543) 2026-06-20 11:57:05 -07:00
test_openapi_schema_validation.py fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema 2026-05-06 00:28:42 +02:00
test_pricing_field_strip.py fix(proxy): move pricing strip below the litellm_metadata JSON-string parse 2026-05-03 02:04:05 +00:00
test_prometheus_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_provider_url_destination_guard.py chore(proxy): move URL-valued model/file_id guard from SDK to proxy 2026-05-02 02:17:46 +00:00
test_proxy_cli.py feat(proxy): add option to disable server-side prepared statements for DB lookups (#29984) 2026-06-13 16:08:56 -07:00
test_proxy_logging_hook_detection.py perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths (#28289) 2026-05-23 12:15:59 -07:00
test_proxy_server.py fix(spend_counter): seed Redis counter via SET NX to prevent cross-pod double-seed (#27854) 2026-05-20 10:57:08 -07:00
test_proxy_types.py refactor(proxy/auth): normalize Bearer prefix in safe-hash helper (#29343) (#29365) 2026-05-30 18:20:40 -07:00
test_proxy_utils.py fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 2026-06-23 21:00:05 -07:00
test_pyroscope.py Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
test_redis_auth_cache_flag.py refactor(proxy): replace DualCache with UserApiKeyCache for user API key management 2026-04-28 19:15:03 +00:00
test_response_model_sanitization.py perf: eliminate per-request callback scanning on proxy hot path (#27858) 2026-05-14 09:28:31 -07:00
test_route_a2a_models.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_route_llm_request.py feat(proxy): fix vector store retrieve/list/update/delete without model (#27929) 2026-05-15 04:37:59 -07:00
test_sensitive_route_auth.py chore(proxy): guard sensitive public endpoints 2026-04-30 11:52:47 -07:00
test_shared_health_check.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_spend_log_cleanup.py fix(proxy): keep spend log cleanup running after batch failures and surface DB errors (#27303) 2026-05-06 18:39:15 +00:00
test_swagger_chat_completions.py fix(proxy): avoid misleading multi-method operation ids 2026-04-30 20:44:14 -07:00
test_team_member_update.py
test_team_org_move.py fix(team_endpoints): auto-add SSO team members to org on move (proxy admin only) (#26377) 2026-04-24 08:36:25 -07:00
test_tools_allowlist_enforcement.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_update_llm_router_resilience.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_utils.py add NO_OPENAPI env var to disable /openapi.json endpoint (#25547) 2026-04-14 23:37:49 +05:30