scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-23 00:41:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 99
49917 commits 2669 branches 1413 tags 3.3 GiB
Commit graph

49917 commits

This branch
This branch
All branches
Author SHA1 Message Date
Yujong Lee
1f0cf4bf42 fix(rust_bridge): bind Python fallbacks at import so module patches do not leak into public entrypoints 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:23:36 +00:00
Yujong Lee
13cb739089 fix(rust_bridge): qualify runtime calls in dispatch and drop OCR transport rows from wheel matrix 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:16:31 +00:00
Yujong Lee
3de23e7f18 test(rust_bridge): drop generated OCR route assertions from bridge_route tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:10:10 +00:00
Yujong Lee
23c059faba ci: assign chat_completions and messages test dirs to the misc shard 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:05:55 +00:00
Yujong Lee
c85acc8d28 test(rust_bridge): cover binding validation, async upstream errors, and OCR preparation failures 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 23:04:38 +00:00
Yujong Lee
c635399f6e merge: origin/main into litellm_rust_bridge_declarative_route_catalog 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 22:54:51 +00:00
Yujong Lee
9617312ab2 add PublicDispatch 2026-09-16 15:44:46 -07:00
ryan-crabbe-berri
9357491f91
Merge pull request #35418 from BerriAI/litellm_skills_ssh_sources 
feat(ui): accept ssh clone urls when registering a skill
 2026-09-16 15:36:18 -07:00
Yassin Kortam
a6e87526f8
Merge pull request #40843 from elifozdamar/litellm_fix_nonstream_parallel_slot_release 
fix(proxy): release completed max-parallel slots promptly
 2026-09-16 15:13:59 -07:00
Yujong Lee
a84f68b6e3 refactor(rust_bridge): give chat completions, messages and responses the ocr dispatch shape 
Each route now has litellm/rust_bridge/<route>/{entrypoints,callbacks}.py and a
public dispatch module (litellm/chat_completions/dispatch.py,
litellm/responses/dispatch.py, litellm/messages/dispatch.py) that binds the
public call to the legacy Python signature, builds a frozen request, and asks
the runtime to pick Rust or Python from the catalog. The legacy implementations
stay in litellm/main.py, litellm/responses/main.py and the anthropic messages
handler, and litellm/__init__.py re-exports the dispatch names over them the
same way it already does for ocr

The per-handler shims in rust_bridge/chat_completions/native.py and
rust_bridge/messages/native.py are removed along with their call sites in the
anthropic and bedrock chat handlers and the http handler. The exception
mapping that every callbacks module repeated moves to rust_bridge/failures.py
and the signature binding helpers to rust_bridge/public_call.py
 2026-09-16 15:02:12 -07:00
Yassin Kortam
f62f140f75
Merge pull request #41403 from BerriAI/litellm_key_total_spend 
feat(proxy): expose lifetime total_spend on virtual keys
 2026-09-16 14:49:02 -07:00
Yassin Kortam
2bfa10b6d9
Merge pull request #41311 from BerriAI/litellm_key_status_filter_deleted_lookup 
feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
 2026-09-16 14:48:34 -07:00
Yassin Kortam
95abc9fb0b
Merge pull request #41330 from BerriAI/litellm_team_model_max_budget_v2 
feat(team): team-level model_max_budget with key-level overrides
 2026-09-16 14:48:29 -07:00
Yassin Kortam
314c0d71a5
Merge pull request #41327 from BerriAI/litellm_s3_log_prompts_only 
feat(s3): add s3_log_prompts_only option to log prompts without responses
 2026-09-16 14:48:18 -07:00
yujonglee
cc99ad00dd
Merge pull request #41489 from BerriAI/litellm_ocr_fileio_core 
refactor(ocr): move file preparation from the python bridge into litellm-core
 2026-09-16 14:45:09 -07:00
ryan-crabbe-berri
4439631e05
Merge pull request #41379 from runjivu/fix/fallback-budget-check 
fix!: re-check budget on router fallback targets
 2026-09-16 14:39:56 -07:00
Yujong Lee
62c862796a cleanup 2026-09-16 14:38:59 -07:00
Yassin Kortam
cd08c65002
Merge pull request #41425 from BerriAI/litellm_streaming_buffer_release_on_scan 
feat(guardrails): release buffered stream chunks after each passing scan
 2026-09-16 14:37:56 -07:00
Yassin Kortam
16bbff6643
Merge pull request #41474 from BerriAI/litellm_router_tpm_rpm_count_before_headers_v2 
fix(router): count TPM/RPM usage before building rate-limit headers
 2026-09-16 14:37:04 -07:00
Mateo Wang
6edb549dbd
Merge pull request #41112 from BerriAI/litellm_registry_audit_2026_09_14 
fix(models): rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching
 2026-09-16 14:31:50 -07:00
ryan-crabbe-berri
17844cf14b Merge remote-tracking branch 'origin/main' into pr41379 2026-09-16 14:17:23 -07:00
Yujong Lee
64cd6538a6 cleanup 2026-09-16 14:16:15 -07:00
ryan-crabbe-berri
cfe65f7b55 fix(proxy)!: enforce fallback budget by default 
A budget bypass that ships off by default stays open for every deployment
that does not know to look for the flag, so `enforce_fallback_budget` now
defaults to true and `general_settings.enforce_fallback_budget: false` is
the opt-out for anyone who wants the old unguarded behaviour back.

BREAKING CHANGE: a paid fallback target is now refused for callers who are
over their key or user `max_budget`. Deployments relying on fallbacks to
keep serving over-budget callers must set enforce_fallback_budget: false.
 2026-09-16 14:15:27 -07:00
kerry-berri
930ec9643a
Merge pull request #41446 from BerriAI/litellm_fix_anthropic_stream_served_model 
fix(anthropic): carry the served model from message_start onto stream chunks
 2026-09-16 14:14:54 -07:00
Yujong Lee
c621435ef7 refactor(ocr): move file preparation from the python bridge into litellm-core 
Delete litellm/ocr/input.py and the native _ocr_file_document, _ocr_upload_document
and _ocr_mime_type helpers. File documents now project to a typed OcrDocumentInput
and the core lifecycle reads local paths, encodes bytes and asks the host to read
file-like objects through a ReadDocument operation before the provider request

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 21:02:30 +00:00
Mateo Wang
b3898dfd85
Merge pull request #36815 from BerriAI/litellm_otel_gen_ai_system_none 
fix(otel): drop None metric and event attributes before OTLP export
 2026-09-16 13:56:05 -07:00
Yassin Kortam
4bf04b22b0
Merge pull request #41472 from BerriAI/litellm_customer_budget_prometheus_metrics 
feat(prometheus): add customer (end_user) budget gauges
 2026-09-16 13:46:57 -07:00
Yassin Kortam
8cdb275ec0
Merge pull request #41460 from BerriAI/litellm_pg10_migration_default_guard 
ci(migrations): flag defaulted ADD COLUMN on request-log tables
 2026-09-16 13:45:48 -07:00
Mateo Wang
8e524370e1
Merge pull request #41343 from BerriAI/litellm_lit5030_bedrock_invoke_nova_prompt_caching 
fix(bedrock): make prompt caching work on the Nova InvokeModel route
 2026-09-16 13:42:21 -07:00
Yassin Kortam
c4a9341ef6
Merge pull request #34829 from max-sixty/bugfix/http-handler-del-closes-streaming-client 
fix(http_handler): keep a handler alive while a response it issued is still reading
 2026-09-16 13:39:04 -07:00
mateo-berri
4c79abfa0c Merge commit '79fc5153d3' into litellm_otel_gen_ai_system_none 
# Conflicts:
#	litellm/integrations/opentelemetry.py
 2026-09-16 13:38:40 -07:00
Yujong Lee
64f2a3d098 refactor(rust_bridge): group route modules into packages and split ocr into main and rust 
Move each route's bridge module under litellm/rust_bridge/<route>/ so a folder
means a Rust implementation exists while the catalog row says whether it is
used. OCR now keeps the Python implementation in litellm/ocr/main.py and the
Rust selection in litellm/ocr/rust.py, removing litellm/ocr/legacy.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:34:51 +00:00
yassin
857301444e Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 20:34:36 +00:00
yassin
8acd2477a6 fix(guardrails): hold legacy function_call stream windows until the end-of-stream scan 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:34:20 +00:00
Yassin Kortam
c2cf6e191e
Merge pull request #41458 from BerriAI/litellm_regenerate_no_body_secret_sync 
fix(proxy): sync AWS Secrets Manager on body-less key regenerate
 2026-09-16 13:31:01 -07:00
yassin
81d57cbad7 fix(prometheus): skip customer budget cache read when end_user tracking is off 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:23:59 +00:00
Yassin Kortam
79fc5153d3
Merge pull request #41462 from BerriAI/litellm_otel_promote_nested_request_metadata_keys 
feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes
 2026-09-16 13:23:05 -07:00
Yassin Kortam
dbecd11d99
Merge pull request #41468 from BerriAI/litellm_key_alias_update_secret_sync 
fix(proxy): rename AWS Secrets Manager secret when key alias changes
 2026-09-16 13:20:53 -07:00
mateo-berri
6015437d67 test(bedrock): assert the Nova cache-read rate as a discount instead of pinning the vendor ratio 2026-09-16 13:19:58 -07:00
Yujong Lee
9484595fa2 test(rust_bridge): drop the responses websocket opt-in assertion the catalog no longer allows 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:16:15 +00:00
yassin
e7d537442d fix(prometheus): read the cached customer row for request-time budget gauges 
The request path used get_end_user_object, which falls back to a database
lookup on a cache miss. Read the LiteLLM_EndUserTable row auth already cached
instead, with the default budget already attached, and leave misses to the
periodic refresh

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:13:52 +00:00
Mateo Wang
365c6de875
Merge pull request #41340 from BerriAI/litellm_anthropic_passthrough_strip_virtual_key 
fix(proxy): never forward the LiteLLM virtual key to Anthropic on the /anthropic passthrough
 2026-09-16 13:12:42 -07:00
Mateo Wang
737929f338
Merge pull request #41335 from BerriAI/litellm_fireworks_dict_reasoning_effort 
fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string
 2026-09-16 13:12:37 -07:00
Yujong Lee
803baead7a refactor(rust_bridge): keep every route but OCR and Bedrock transcription on Python 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:12:04 +00:00
yassin
1979533901 test(router): force the callback to observe the usage stamp before the pre-header increment fails 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 20:10:22 +00:00
yujonglee
560065df16
Merge pull request #41480 from BerriAI/litellm_rust_ci_nextest_rust_cache 
ci(rust): split rust jobs, use nextest and Swatinem/rust-cache
 2026-09-16 13:08:20 -07:00
Mateo Wang
2b33201a09
Merge pull request #41475 from BerriAI/litellm_bedrock_kb_user_context 
fix(bedrock): forward userContext in Knowledge Base Retrieve requests
 2026-09-16 13:07:03 -07:00
Devin AI
a229f99ac8 test(models): drive Grok prompt caching coverage through litellm APIs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:59:07 +00:00
yassin
9ebd55e53e test(router): cover success callback recovering the count when the pre-header increment fails 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:57:55 +00:00
mateo-berri
884087f01c test(bedrock): type the vector store search test helper 2026-09-16 12:55:29 -07:00