scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 122
litellm/litellm
History
Yassin Kortam 433d016f0c
fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 
A streaming request that breaks mid-flight, for example on a mid-stream read
timeout, still bills the provider for the chunks already delivered, yet the proxy
recorded that interrupted request as a zero-spend failure. An earlier revision
logged the recovered partial usage through the success path, which mislabeled a
failed request as a success and produced a misleading spend row

This recovers the partial usage where the failure is actually logged. The
streaming handler assembles the usage from the chunks seen so far and stashes it,
with its cost, on the logging object before firing the failure handlers. The
proxy failure hook lifts that usage and cost onto request_data before the
non-serialisable logging object is popped, and the spend-log writer records the
real partial spend on the failure row instead of a hardcoded zero;
get_logging_payload honors the recovered usage for the token columns and
_failure_handler_helper_fn preserves the recovered cost so the non-DB failure
loggers stay consistent

A request that recovers via a successful fallback is unaffected: the failure hook
only fires when the whole request fails, so the fallback's combined-usage success
row stays the single source of truth and there is no double counting

Resolves LIT-3825

Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
(cherry picked from commit 4847fa5dd5)
2026-06-23 21:00:05 -07:00
..
a2a_protocol feat(guardrails): LLM-as-a-Judge guardrail (#26360) 2026-04-24 17:15:32 -07:00
anthropic_interface build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
assistants
batch_completion
batches fix(vertex-ai): fix zero cost/usage on completed Vertex AI batch jobs (#27912) 2026-05-15 04:47:02 -07:00
caching fix(caching): replay openai/responses bridge cache hits as chat streams (#28158) 2026-05-18 16:27:06 -07:00
completion_extras Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
compression build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
containers style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
endpoints/speech/speech_to_completion_bridge
evals style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
experimental_mcp_client feat(mcp): add OBO MCP Auth (#27421) 2026-05-07 15:35:21 -07:00
files build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
fine_tuning address greptile review feedback (greploop iteration 1) 2026-03-27 20:04:41 +05:30
google_genai build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
images Feat(dashscope): add image generation support for qwen-image-2.0 and qwen-image-2.0-pro (#25672) 2026-04-22 20:03:46 -07:00
integrations fix(guardrails): run pre_call hook once for model-level guardrails (#30543) 2026-06-20 11:57:05 -07:00
interactions build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
litellm_core_utils fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 2026-06-23 21:00:05 -07:00
llms feat(bedrock_mantle): add SigV4/IAM auth to Responses API route (#29788) 2026-06-11 03:03:39 +00:00
ocr chore: reject bare str at file-input sinks to prevent local-file read (#27762) 2026-05-12 16:40:07 -07:00
passthrough fix(passthrough): log when streaming spend-tracking flush fails to schedule 2026-04-30 02:39:29 +00:00
proxy fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 2026-06-23 21:00:05 -07:00
proxy_auth
rag style: black formatting 2026-04-25 14:47:54 -07:00
realtime_api Root cause fix - migrate all logging update to use 1 function - for centralized kwarg updates (#23659) 2026-03-15 23:21:01 -07:00
rerank_api Fix review 2026-04-30 09:10:24 +05:30
responses fix(responses): use OpenAI SSEDecoder for Responses API streaming (#28566) 2026-05-22 10:03:36 -07:00
router_strategy build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
router_utils build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
search style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
secret_managers build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
skills chore(proxy): scope skills and container resources 2026-04-30 18:23:58 -07:00
types fix: stop use_chat_completions_api flag from leaking into provider request body (#29447) 2026-06-20 11:53:49 -07:00
vector_store_files build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
vector_stores chore(vector stores): tighten managed store access 2026-04-30 15:04:25 -07:00
videos style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py feat(bedrock_mantle): route Responses API to native OpenAI endpoint (#29490) 2026-06-11 03:03:22 +00:00
_internal_context.py fix(proxy): harden request parameter handling 2026-04-16 01:38:12 +00:00
_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
_lazy_imports_registry.py feat(bedrock_mantle): route Responses API to native OpenAI endpoint (#29490) 2026-06-11 03:03:22 +00:00
_logging.py fix(bedrock-mantle): use /anthropic/v1/messages path for Mantle endpo… (#27976) 2026-05-15 13:31:59 -07:00
_redis.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
_redis_credential_provider.py feat: add ability to auth to azure with token (#27556) 2026-05-09 22:34:09 +00:00
_service_logger.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
_uuid.py build(deps-dev): bump black to 26.3.1 and apply formatting (#28525) 2026-05-21 17:24:18 -07:00
_version.py
anthropic_beta_headers_config.json fix(anthropic,bedrock,databricks): four reasoning_effort follow-ups 2026-05-03 10:03:53 -07:00
anthropic_beta_headers_manager.py Refactor: Filtering beta header after transformation 2026-03-16 10:47:15 +05:30
blog_posts.json docs(blog): add WebRTC blog post link 2026-03-13 17:54:33 +05:30
budget_manager.py docs(budget_manager): add docstring to BudgetManager.reset_cost (#27867) 2026-05-13 13:28:22 -07:00
constants.py fix(guardrails): run pre_call hook once for model-level guardrails (#30543) 2026-06-20 11:57:05 -07:00
cost.json
cost_calculator.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
exceptions.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
main.py feat(azure): add speech transcription config support (#27482) 2026-05-23 12:16:01 -07:00
model_prices_and_context_window_backup.json feat(bedrock_mantle): route Responses API to native OpenAI endpoint (#29490) 2026-06-11 03:03:22 +00:00
mypy.ini
policy_templates_backup.json feat(add-new-block_code_execution-guardrail): prevent agent from executing code (#22154) 2026-02-25 22:02:14 -08:00
provider_endpoints_support_backup.json Feature/add audio support for scaleway (#26110) 2026-04-20 14:49:41 -07:00
py.typed
router.py CI: copy of #25177 (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) (#28223) 2026-05-23 12:15:41 -07:00
scheduler.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
setup_wizard.py Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 18:20:42 +00:00
timeout.py docs: add class docstring to _LoopWrapper (#27870) 2026-05-13 13:54:00 -07:00
utils.py feat(bedrock_mantle): route Responses API to native OpenAI endpoint (#29490) 2026-06-11 03:03:22 +00:00