scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-20 00:11:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 77
49874 commits 2552 branches 1411 tags 3.1 GiB
Commit graph

49874 commits

This branch
This branch
All branches
Author SHA1 Message Date
mateo-berri
eaeeb3a9bc Merge remote-tracking branch 'origin/main' into litellm_dashscope_reasoning_effort 2026-09-16 14:41:08 -07:00
ryan-crabbe-berri
4439631e05
Merge pull request #41379 from runjivu/fix/fallback-budget-check 
fix!: re-check budget on router fallback targets
 2026-09-16 14:39:56 -07:00
Yassin Kortam
cd08c65002
Merge pull request #41425 from BerriAI/litellm_streaming_buffer_release_on_scan 
feat(guardrails): release buffered stream chunks after each passing scan
 2026-09-16 14:37:56 -07:00
Yassin Kortam
16bbff6643
Merge pull request #41474 from BerriAI/litellm_router_tpm_rpm_count_before_headers_v2 
fix(router): count TPM/RPM usage before building rate-limit headers
 2026-09-16 14:37:04 -07:00
Mateo Wang
6edb549dbd
Merge pull request #41112 from BerriAI/litellm_registry_audit_2026_09_14 
fix(models): rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching
 2026-09-16 14:31:50 -07:00
ryan-crabbe-berri
17844cf14b Merge remote-tracking branch 'origin/main' into pr41379 2026-09-16 14:17:23 -07:00
ryan-crabbe-berri
cfe65f7b55 fix(proxy)!: enforce fallback budget by default 
A budget bypass that ships off by default stays open for every deployment
that does not know to look for the flag, so `enforce_fallback_budget` now
defaults to true and `general_settings.enforce_fallback_budget: false` is
the opt-out for anyone who wants the old unguarded behaviour back.

BREAKING CHANGE: a paid fallback target is now refused for callers who are
over their key or user `max_budget`. Deployments relying on fallbacks to
keep serving over-budget callers must set enforce_fallback_budget: false.
 2026-09-16 14:15:27 -07:00
kerry-berri
930ec9643a
Merge pull request #41446 from BerriAI/litellm_fix_anthropic_stream_served_model 
fix(anthropic): carry the served model from message_start onto stream chunks
 2026-09-16 14:14:54 -07:00
Mateo Wang
b3898dfd85
Merge pull request #36815 from BerriAI/litellm_otel_gen_ai_system_none 
fix(otel): drop None metric and event attributes before OTLP export
 2026-09-16 13:56:05 -07:00
Yassin Kortam
4bf04b22b0
Merge pull request #41472 from BerriAI/litellm_customer_budget_prometheus_metrics 
feat(prometheus): add customer (end_user) budget gauges
 2026-09-16 13:46:57 -07:00
Yassin Kortam
8cdb275ec0
Merge pull request #41460 from BerriAI/litellm_pg10_migration_default_guard 
ci(migrations): flag defaulted ADD COLUMN on request-log tables
 2026-09-16 13:45:48 -07:00
Mateo Wang
8e524370e1
Merge pull request #41343 from BerriAI/litellm_lit5030_bedrock_invoke_nova_prompt_caching 
fix(bedrock): make prompt caching work on the Nova InvokeModel route
 2026-09-16 13:42:21 -07:00
Yassin Kortam
c4a9341ef6
Merge pull request #34829 from max-sixty/bugfix/http-handler-del-closes-streaming-client 
fix(http_handler): keep a handler alive while a response it issued is still reading
 2026-09-16 13:39:04 -07:00
mateo-berri
4c79abfa0c Merge commit '79fc5153d3' into litellm_otel_gen_ai_system_none 
# Conflicts:
#	litellm/integrations/opentelemetry.py
 2026-09-16 13:38:40 -07:00
yassin
857301444e Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 20:34:36 +00:00
yassin
8acd2477a6 fix(guardrails): hold legacy function_call stream windows until the end-of-stream scan 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:34:20 +00:00
Yassin Kortam
c2cf6e191e
Merge pull request #41458 from BerriAI/litellm_regenerate_no_body_secret_sync 
fix(proxy): sync AWS Secrets Manager on body-less key regenerate
 2026-09-16 13:31:01 -07:00
yassin
81d57cbad7 fix(prometheus): skip customer budget cache read when end_user tracking is off 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:23:59 +00:00
Yassin Kortam
79fc5153d3
Merge pull request #41462 from BerriAI/litellm_otel_promote_nested_request_metadata_keys 
feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes
 2026-09-16 13:23:05 -07:00
Yassin Kortam
dbecd11d99
Merge pull request #41468 from BerriAI/litellm_key_alias_update_secret_sync 
fix(proxy): rename AWS Secrets Manager secret when key alias changes
 2026-09-16 13:20:53 -07:00
mateo-berri
6015437d67 test(bedrock): assert the Nova cache-read rate as a discount instead of pinning the vendor ratio 2026-09-16 13:19:58 -07:00
yassin
e7d537442d fix(prometheus): read the cached customer row for request-time budget gauges 
The request path used get_end_user_object, which falls back to a database
lookup on a cache miss. Read the LiteLLM_EndUserTable row auth already cached
instead, with the default budget already attached, and leave misses to the
periodic refresh

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:13:52 +00:00
Mateo Wang
365c6de875
Merge pull request #41340 from BerriAI/litellm_anthropic_passthrough_strip_virtual_key 
fix(proxy): never forward the LiteLLM virtual key to Anthropic on the /anthropic passthrough
 2026-09-16 13:12:42 -07:00
Mateo Wang
737929f338
Merge pull request #41335 from BerriAI/litellm_fireworks_dict_reasoning_effort 
fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string
 2026-09-16 13:12:37 -07:00
yassin
1979533901 test(router): force the callback to observe the usage stamp before the pre-header increment fails 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:10:22 +00:00
yujonglee
560065df16
Merge pull request #41480 from BerriAI/litellm_rust_ci_nextest_rust_cache 
ci(rust): split rust jobs, use nextest and Swatinem/rust-cache
 2026-09-16 13:08:20 -07:00
Mateo Wang
2b33201a09
Merge pull request #41475 from BerriAI/litellm_bedrock_kb_user_context 
fix(bedrock): forward userContext in Knowledge Base Retrieve requests
 2026-09-16 13:07:03 -07:00
Devin AI
a229f99ac8 test(models): drive Grok prompt caching coverage through litellm APIs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:59:07 +00:00
yassin
9ebd55e53e test(router): cover success callback recovering the count when the pre-header increment fails 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:57:55 +00:00
mateo-berri
884087f01c test(bedrock): type the vector store search test helper 2026-09-16 12:55:29 -07:00
Yujong Lee
48df3d5a48 ci(rust): install nextest via pinned taiki-e/install-action 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:54:46 +00:00
mateo-berri
043c954aa9 fix(proxy): keep a caller's own Anthropic key when the proxy has no master key 
Without a master key the auth layer echoes whatever key the caller presented as the authenticated key, so the passthrough's strip-by-value matched the caller's own Anthropic key and dropped it: a bring-your-own-key request that returned 200 on main answered 401 telling the caller to send the key they had just sent. Only the auth module's own no-auth dev-mode definition, shared through is_no_auth_dev_mode, decides that nothing was authenticated, and only when no custom auth is installed; JWTs, OAuth2 tokens, and custom-auth credentials are still stripped there. The sk- prefix heuristic goes with it.

The Vertex credential-less test now sets a master key, since a virtual key can only authenticate under one: the auth layer returns before any key lookup when the master key is unset.
 2026-09-16 12:52:37 -07:00
mateo-berri
a2724e7f15 fix(otel): drop None attributes before they reach the OTLP encoder 
The metric attribute filter now removes every attribute whose value is
None, and the content and inference-details events pass their attributes
through drop_none before emitting, so a call with no provider label or no
model name never hands the OTLP exporter a NoneType attribute. This closes
the gen_ai.request.model report on #36759 the same way the gen_ai.system
one was closed, and the regression tests cover both keys.
 2026-09-16 12:52:14 -07:00
Devin AI
ba03f60f71 fix(models): add batch and flex tier prices on Azure dated and Gemini latest alias rows 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:50:28 +00:00
Yujong Lee
3f15dcd96b ci(rust): fold fmt into the clippy job 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:47:48 +00:00
yassin
7ff8945da4 fix(router): sync both usage keys from Redis even when one increment is zero 
A stream counted before its usage is known increments TPM by zero, so the
worker that served it never refreshed its local TPM value from Redis and
the first byte headers reported the token count another worker had already
consumed. Both pipeline operations now always run, matching the pre-change
callback, so the returned values refresh both worker local keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:45:48 +00:00
Yujong Lee
e850232f02 test(rust): assert merge cost scales linearly instead of a wall-clock bound 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:43:42 +00:00
yassin
0d605b7b45 Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 19:42:36 +00:00
mateo-berri
0dff64ce1a fix(bedrock): move a Nova invoke cache point behind an image or tool result to the last text block 2026-09-16 12:42:15 -07:00
kerry-berri
b6143b3711
Merge pull request #41457 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Google Gemini prices: 22 models
 2026-09-16 12:37:16 -07:00
yassin
6faeb19c0c Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 19:33:16 +00:00
yassin
1e169bb912 refactor(guardrails): mark release_on_scan flag as Final 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:33:07 +00:00
mateo-berri
c052d7816e fix(bedrock): rename the cache point inliner so the recursion detector stops flagging it 2026-09-16 12:30:33 -07:00
mateo-berri
aa6d053a37 Merge remote-tracking branch 'origin/main' into litellm_otel_gen_ai_system_none 2026-09-16 12:30:20 -07:00
yassin
cb26454034 test(router): cover success callback racing the pre-header count and drop redundant docstrings 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:28:50 +00:00
Mateo Wang
a1b1f1ef9d
Merge pull request #34455 from BerriAI/litellm_lit4767_empty_choices_streaming_guard 
fix(responses): guard empty-choices chunks in the Responses API streaming bridge
 2026-09-16 12:28:46 -07:00
yassin
8de51dfaab test(otel): describe which request metadata the pre-call seed reads 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:25:40 +00:00
yassin
938b782b8a Merge remote-tracking branch 'origin/main' into litellm_customer_budget_prometheus_metrics 2026-09-16 19:25:23 +00:00
yassin
d5837ab97a test(guardrails): pin tool-call-only scan keys as non-empty 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:24:14 +00:00
yassin
c22916377f test(prometheus): cover customer budget series expiry by end_user ttl 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:22:30 +00:00