scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88
litellm/litellm
History
Exact
Yassin Kortam 80be8ea44d
fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams (#31035) 
Streaming and pass-through requests could be logged with $0 cost or dropped from
SpendLogs entirely while the upstream provider still billed every token. This
closes the leak paths not already covered by #30160, #30787 and #30788.

- Catch a stream_chunk_builder raise in the core CustomStreamWrapper (sync and
  async). Large agentic tool-use / thinking streams can make assembly re-raise
  as APIError from inside the except-StopIteration handler, where the sibling
  except does not catch it, so it escaped __next__/__anext__ and dropped the
  request; recover best-effort usage from the raw chunks instead
- Add a usage-only fallback for Anthropic streaming pass-through: when
  stream_chunk_builder returns None or raises, rebuild usage from the
  message_start / message_delta SSE events via AnthropicConfig.calculate_usage so
  cache, web-search and geo tokens are priced instead of left at $0
- Decode buffered pass-through bytes with errors="replace" so a stream cut
  mid-multibyte-sequence still logs the usage events already received
- Record response_cost into model_call_details on the pass-through success path
  (it is read from there, not from kwargs), matching the gemini/cohere/openai
  handlers
- Name the key (alias + masked key) in the virtual-key BudgetExceededError so
  operators don't have to reverse-map spend back to a key

(cherry picked from commit b24b964e04)
2026-06-23 21:36:34 -07:00
..
a2a_protocol feat(guardrails): LLM-as-a-Judge guardrail (#26360) 2026-04-24 17:15:32 -07:00
anthropic_interface style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
assistants
batch_completion
batches feat(batch-job): bedrock batch model invocation job retrieval (#26834) 2026-05-11 13:22:26 -07:00
caching chore(caching): remove allow_legacy_unscoped_cache_hits opt-in 2026-05-04 22:16:30 +00:00
completion_extras fix(responses): normalize chat tool_choice for completions→responses bridge (#27634) 2026-05-11 10:24:34 -07:00
compression Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
containers style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
endpoints/speech/speech_to_completion_bridge
evals style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
experimental_mcp_client feat(mcp): add OBO MCP Auth (#27421) 2026-05-07 15:35:21 -07:00
files thread trusted params through vertex file content 2026-05-01 18:24:22 -07:00
fine_tuning address greptile review feedback (greploop iteration 1) 2026-03-27 20:04:41 +05:30
google_genai feat(proxy): LiteLLM headers on Google native generateContent routes (#25500) 2026-04-29 12:34:14 -07:00
images Feat(dashscope): add image generation support for qwen-image-2.0 and qwen-image-2.0-pro (#25672) 2026-04-22 20:03:46 -07:00
integrations fix(integrations): cap Anthropic cache_control injection at 4 blocks (#30480) 2026-06-23 19:34:30 -07:00
interactions style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams (#31035) 2026-06-23 21:36:34 -07:00
llms fix(router): use forwarded model_id for native Azure container IDs (#27921) 2026-06-13 16:02:29 -07:00
ocr chore: reject bare str at file-input sinks to prevent local-file read (#27762) 2026-05-12 16:40:07 -07:00
passthrough fix(passthrough): log when streaming spend-tracking flush fails to schedule 2026-04-30 02:39:29 +00:00
proxy fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams (#31035) 2026-06-23 21:36:34 -07:00
proxy_auth
rag style: black formatting 2026-04-25 14:47:54 -07:00
realtime_api Root cause fix - migrate all logging update to use 1 function - for centralized kwarg updates (#23659) 2026-03-15 23:21:01 -07:00
rerank_api Fix review 2026-04-30 09:10:24 +05:30
responses fix(responses): preserve cache_control in Responses API -> Chat Completion transformation (#27727) 2026-05-13 12:17:06 -07:00
router_strategy add test(tag-routing): prevent header regex bypass for strict plain tags. 2026-04-29 20:14:30 +00:00
router_utils fix(responses): register cooldowns on failure + fail fast on stale encrypted_content (#27820) 2026-05-13 09:03:13 -07:00
search style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
secret_managers Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
skills chore(proxy): scope skills and container resources 2026-04-30 18:23:58 -07:00
types fix(types): coerce dict server_tool_use to ServerToolUse in Usage init 2026-06-23 21:36:34 -07:00
vector_store_files Root cause fix - migrate all logging update to use 1 function - for centralized kwarg updates (#23659) 2026-03-15 23:21:01 -07:00
vector_stores chore(vector stores): tighten managed store access 2026-04-30 15:04:25 -07:00
videos style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
__init__.py Add Bedrock Claude Platform route (#27678) 2026-05-11 15:50:54 -07:00
_internal_context.py fix(proxy): harden request parameter handling 2026-04-16 01:38:12 +00:00
_lazy_imports.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
_lazy_imports_registry.py Add Bedrock Claude Platform route (#27678) 2026-05-11 15:50:54 -07:00
_logging.py Unify secret redaction patterns 2026-05-01 17:24:21 +00:00
_redis.py fix: Fix Redis Sentinel client handling to solve authentication error… (#26302) 2026-05-13 14:06:58 -07:00
_redis_credential_provider.py feat: add ability to auth to azure with token (#27556) 2026-05-09 22:34:09 +00:00
_service_logger.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
_uuid.py
_version.py
anthropic_beta_headers_config.json fix(anthropic,bedrock,databricks): four reasoning_effort follow-ups 2026-05-03 10:03:53 -07:00
anthropic_beta_headers_manager.py Refactor: Filtering beta header after transformation 2026-03-16 10:47:15 +05:30
blog_posts.json docs(blog): add WebRTC blog post link 2026-03-13 17:54:33 +05:30
budget_manager.py docs(budget_manager): add docstring to BudgetManager.reset_cost (#27867) 2026-05-13 13:28:22 -07:00
constants.py Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 18:25:33 +00:00
cost.json
cost_calculator.py Fix black 2026-05-01 17:33:50 +05:30
exceptions.py fix(proxy): invoke post-call guardrails on pass-through endpoint responses (#20270) (#26262) 2026-04-27 08:58:22 +05:30
main.py Match litellm.completion supported model parameters with proxy model info (#27720) 2026-05-12 08:25:01 -07:00
model_prices_and_context_window_backup.json Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 18:25:33 +00:00
mypy.ini
policy_templates_backup.json
provider_endpoints_support_backup.json Feature/add audio support for scaleway (#26110) 2026-04-20 14:49:41 -07:00
py.typed
router.py fix(router): use forwarded model_id for native Azure container IDs (#27921) 2026-06-13 16:02:29 -07:00
scheduler.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
setup_wizard.py Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI (#30064) 2026-06-10 18:25:33 +00:00
timeout.py docs: add class docstring to _LoopWrapper (#27870) 2026-05-13 13:54:00 -07:00
utils.py Add Bedrock Claude Platform route (#27678) 2026-05-11 15:50:54 -07:00