scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-20 00:11:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 76
49926 commits 2552 branches 1411 tags 3.1 GiB
Commit graph

49926 commits

This branch
This branch
All branches
Author SHA1 Message Date
Moe Khalil
b95dbb41ae fix(router): scope discovered limits to active deployments 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:37:28 +00:00
Moe Khalil
1c9525fdab test(router): cover deployment replacement during metadata discovery 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:27:31 +00:00
Moe Khalil
29d1a191a1 refactor(router): isolate per-deployment metadata refresh 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:24:03 +00:00
Moe Khalil
b7c6befb37 feat(router): discover token limits for hosted OpenAI-compatible models 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:19:54 +00:00
yuneng-jiang
fddf83a2ac
Merge pull request #41487 from BerriAI/litellm_isolate_generic_api_ndjson_test 
test(logging): pick this test's own records out of the shared log batch
 2026-09-16 16:06:47 -07:00
ryan-crabbe-berri
be6e7ef173
Merge pull request #41345 from BerriAI/litellm_remove_duplicate_user_budget_hook 
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
 2026-09-16 16:06:41 -07:00
yucheng-berri
659c4ce5a1
Merge pull request #40669 from BerriAI/litellm_otel_passthrough_trace_propagation 
fix(otel): propagate W3C trace context on HTTP and WebSocket passthrough
 2026-09-16 16:02:46 -07:00
yuneng-jiang
7f83650a41
Merge branch 'main' into litellm_isolate_generic_api_ndjson_test 2026-09-16 15:56:53 -07:00
ryan
789b8285bc Merge remote-tracking branch 'origin/main' into litellm_remove_duplicate_user_budget_hook 2026-09-16 22:56:19 +00:00
yuneng-jiang
680233fa8b
Merge pull request #41496 from BerriAI/litellm_/litellm-ci-failure-be9267 
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge
 2026-09-16 15:56:07 -07:00
ryan
dae16264c1 test(auth): cover over-budget user on zero-cost vs paid model in common_checks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 15:42:15 -07:00
ryan
287bbaa6c1 fix(proxy): remove duplicate user budget hook that 429'd zero-cost models 
_PROXY_MaxBudgetLimiter re-checked spend:user:{id} against user_max_budget in
async_pre_call_hook without the zero-cost model exemption that
_user_max_budget_check applies in auth, so free models were rejected with
"Max budget limit reached." once a user was over budget. Auth already owns
this check, so the hook is deleted rather than taught the exemption again

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 15:42:15 -07:00
ryan-crabbe-berri
9357491f91
Merge pull request #35418 from BerriAI/litellm_skills_ssh_sources 
feat(ui): accept ssh clone urls when registering a skill
 2026-09-16 15:36:18 -07:00
Yassin Kortam
a6e87526f8
Merge pull request #40843 from elifozdamar/litellm_fix_nonstream_parallel_slot_release 
fix(proxy): release completed max-parallel slots promptly
 2026-09-16 15:13:59 -07:00
Yuneng Jiang
a6b10ad654
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge 
PR #41343 and PR #41112 both added cache_read_input_token_cost to the six amazon.nova-{micro,lite,pro}-v1:0 and us.amazon.nova-* entries, one at the top of each entry and one at the bottom. The merge kept both, so every PR now fails test_price_map_has_no_duplicate_keys. Both copies carried the same value, so this only removes the trailing duplicate in both price files
 2026-09-16 14:58:57 -07:00
Yassin Kortam
f62f140f75
Merge pull request #41403 from BerriAI/litellm_key_total_spend 
feat(proxy): expose lifetime total_spend on virtual keys
 2026-09-16 14:49:02 -07:00
Yassin Kortam
2bfa10b6d9
Merge pull request #41311 from BerriAI/litellm_key_status_filter_deleted_lookup 
feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
 2026-09-16 14:48:34 -07:00
Yassin Kortam
95abc9fb0b
Merge pull request #41330 from BerriAI/litellm_team_model_max_budget_v2 
feat(team): team-level model_max_budget with key-level overrides
 2026-09-16 14:48:29 -07:00
Yassin Kortam
314c0d71a5
Merge pull request #41327 from BerriAI/litellm_s3_log_prompts_only 
feat(s3): add s3_log_prompts_only option to log prompts without responses
 2026-09-16 14:48:18 -07:00
yujonglee
cc99ad00dd
Merge pull request #41489 from BerriAI/litellm_ocr_fileio_core 
refactor(ocr): move file preparation from the python bridge into litellm-core
 2026-09-16 14:45:09 -07:00
ryan-crabbe-berri
4439631e05
Merge pull request #41379 from runjivu/fix/fallback-budget-check 
fix!: re-check budget on router fallback targets
 2026-09-16 14:39:56 -07:00
Yassin Kortam
cd08c65002
Merge pull request #41425 from BerriAI/litellm_streaming_buffer_release_on_scan 
feat(guardrails): release buffered stream chunks after each passing scan
 2026-09-16 14:37:56 -07:00
Yassin Kortam
16bbff6643
Merge pull request #41474 from BerriAI/litellm_router_tpm_rpm_count_before_headers_v2 
fix(router): count TPM/RPM usage before building rate-limit headers
 2026-09-16 14:37:04 -07:00
Yuneng Jiang
0de187e76c style(test): annotate the new locals as Final 2026-09-16 14:34:13 -07:00
Mateo Wang
6edb549dbd
Merge pull request #41112 from BerriAI/litellm_registry_audit_2026_09_14 
fix(models): rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching
 2026-09-16 14:31:50 -07:00
ryan-crabbe-berri
17844cf14b Merge remote-tracking branch 'origin/main' into pr41379 2026-09-16 14:17:23 -07:00
ryan-crabbe-berri
cfe65f7b55 fix(proxy)!: enforce fallback budget by default 
A budget bypass that ships off by default stays open for every deployment
that does not know to look for the flag, so `enforce_fallback_budget` now
defaults to true and `general_settings.enforce_fallback_budget: false` is
the opt-out for anyone who wants the old unguarded behaviour back.

BREAKING CHANGE: a paid fallback target is now refused for callers who are
over their key or user `max_budget`. Deployments relying on fallbacks to
keep serving over-budget callers must set enforce_fallback_budget: false.
 2026-09-16 14:15:27 -07:00
kerry-berri
930ec9643a
Merge pull request #41446 from BerriAI/litellm_fix_anthropic_stream_served_model 
fix(anthropic): carry the served model from message_start onto stream chunks
 2026-09-16 14:14:54 -07:00
Yujong Lee
c621435ef7 refactor(ocr): move file preparation from the python bridge into litellm-core 
Delete litellm/ocr/input.py and the native _ocr_file_document, _ocr_upload_document
and _ocr_mime_type helpers. File documents now project to a typed OcrDocumentInput
and the core lifecycle reads local paths, encodes bytes and asks the host to read
file-like objects through a ReadDocument operation before the provider request

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 21:02:30 +00:00
Mateo Wang
b3898dfd85
Merge pull request #36815 from BerriAI/litellm_otel_gen_ai_system_none 
fix(otel): drop None metric and event attributes before OTLP export
 2026-09-16 13:56:05 -07:00
Yuneng Jiang
baca62df13
test(logging): pick this test's own records out of the shared log batch 
The generic API logger batches whatever is queued when it flushes, so records
from tests in other files in the same job land in the same request. Two tests
assumed otherwise: one read actual_request[0], the other counted NDJSON lines,
and both broke whenever another file logged first. Select by the messages each
test sent instead, which keeps the format assertions and stops the order from
deciding the outcome.
 2026-09-16 13:53:28 -07:00
Yassin Kortam
4bf04b22b0
Merge pull request #41472 from BerriAI/litellm_customer_budget_prometheus_metrics 
feat(prometheus): add customer (end_user) budget gauges
 2026-09-16 13:46:57 -07:00
Yassin Kortam
8cdb275ec0
Merge pull request #41460 from BerriAI/litellm_pg10_migration_default_guard 
ci(migrations): flag defaulted ADD COLUMN on request-log tables
 2026-09-16 13:45:48 -07:00
Mateo Wang
8e524370e1
Merge pull request #41343 from BerriAI/litellm_lit5030_bedrock_invoke_nova_prompt_caching 
fix(bedrock): make prompt caching work on the Nova InvokeModel route
 2026-09-16 13:42:21 -07:00
Yassin Kortam
c4a9341ef6
Merge pull request #34829 from max-sixty/bugfix/http-handler-del-closes-streaming-client 
fix(http_handler): keep a handler alive while a response it issued is still reading
 2026-09-16 13:39:04 -07:00
mateo-berri
4c79abfa0c Merge commit '79fc5153d3' into litellm_otel_gen_ai_system_none 
# Conflicts:
#	litellm/integrations/opentelemetry.py
 2026-09-16 13:38:40 -07:00
yassin
857301444e Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 20:34:36 +00:00
yassin
8acd2477a6 fix(guardrails): hold legacy function_call stream windows until the end-of-stream scan 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:34:20 +00:00
Yassin Kortam
c2cf6e191e
Merge pull request #41458 from BerriAI/litellm_regenerate_no_body_secret_sync 
fix(proxy): sync AWS Secrets Manager on body-less key regenerate
 2026-09-16 13:31:01 -07:00
yucheng
df9a87f44a fix(otel): keep caller tracestate on the legacy request span 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:28:03 +00:00
yassin
81d57cbad7 fix(prometheus): skip customer budget cache read when end_user tracking is off 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:23:59 +00:00
Yassin Kortam
79fc5153d3
Merge pull request #41462 from BerriAI/litellm_otel_promote_nested_request_metadata_keys 
feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes
 2026-09-16 13:23:05 -07:00
Yassin Kortam
dbecd11d99
Merge pull request #41468 from BerriAI/litellm_key_alias_update_secret_sync 
fix(proxy): rename AWS Secrets Manager secret when key alias changes
 2026-09-16 13:20:53 -07:00
mateo-berri
6015437d67 test(bedrock): assert the Nova cache-read rate as a discount instead of pinning the vendor ratio 2026-09-16 13:19:58 -07:00
yassin
e7d537442d fix(prometheus): read the cached customer row for request-time budget gauges 
The request path used get_end_user_object, which falls back to a database
lookup on a cache miss. Read the LiteLLM_EndUserTable row auth already cached
instead, with the default budget already attached, and leave misses to the
periodic refresh

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:13:52 +00:00
Mateo Wang
365c6de875
Merge pull request #41340 from BerriAI/litellm_anthropic_passthrough_strip_virtual_key 
fix(proxy): never forward the LiteLLM virtual key to Anthropic on the /anthropic passthrough
 2026-09-16 13:12:42 -07:00
Mateo Wang
737929f338
Merge pull request #41335 from BerriAI/litellm_fireworks_dict_reasoning_effort 
fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string
 2026-09-16 13:12:37 -07:00
yassin
1979533901 test(router): force the callback to observe the usage stamp before the pre-header increment fails 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:10:22 +00:00
yujonglee
560065df16
Merge pull request #41480 from BerriAI/litellm_rust_ci_nextest_rust_cache 
ci(rust): split rust jobs, use nextest and Swatinem/rust-cache
 2026-09-16 13:08:20 -07:00
Mateo Wang
2b33201a09
Merge pull request #41475 from BerriAI/litellm_bedrock_kb_user_context 
fix(bedrock): forward userContext in Knowledge Base Retrieve requests
 2026-09-16 13:07:03 -07:00