scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
49857 commits 3198 branches 1436 tags 4.5 GiB
Commit graph

49857 commits

This branch
This branch
All branches
Author SHA1 Message Date
berriai-litellm-provider-info-sync[bot]
b893e6b926
chore(prices): sync Google Gemini prices: 22 models 
gemini/gemini-2.5-flash: max_tokens, max_output_tokens, supports_audio_input
gemini/gemini-2.5-flash-image: max_input_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-lite: max_tokens, max_output_tokens, supports_audio_input
gemini-2.5-flash-native-audio-preview-12-2025: supports_vision, max_input_tokens, supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-native-audio-preview-12-2025: supports_vision, max_input_tokens, supports_web_search, supports_response_schema, supports_function_calling
gemini-2.5-flash-preview-tts: max_tokens, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-preview-tts: max_tokens, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-pro: max_tokens, max_output_tokens
gemini/gemini-2.5-pro-preview-tts: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-3-flash-preview: max_tokens, max_output_tokens, supports_audio_input
gemini/gemini-3-pro-image: supports_response_schema
gemini/gemini-3.1-flash-image: max_input_tokens, supports_response_schema
gemini/gemini-3.1-flash-lite-image: supports_web_search, supports_function_calling
gemini-3.1-flash-live-preview: supports_response_schema
gemini/gemini-3.1-flash-live-preview: supports_response_schema
gemini/gemini-3.1-flash-tts-preview: supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-3.5-flash: max_tokens, max_output_tokens
gemini/gemini-3.5-live-translate-preview: supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-3.5-transcribe: supports_function_calling
gemini/gemini-3.5-transcribe-live: supports_function_calling
gemini/gemini-embedding-2: supports_vision, supports_audio_input
gemini/gemini-omni-1.1-flash: max_input_tokens
 2026-09-16 19:06:19 +00:00
Devin AI
03931180d7 merge main into litellm_registry_audit_2026_09_14 2026-09-16 19:03:36 +00:00
yassin
e105cde56a fix(router): sync worker-local usage cache to shared post-increment counters 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 19:02:15 +00:00
yassin
5e2d9e1d5c refactor(otel): build promoted baggage without local dict mutation 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:59:21 +00:00
yucheng-berri
b04d530ecf
Merge pull request #41386 from BerriAI/litellm_otel_trace_correlation 
fix(proxy): default litellm_trace_id to the OTel server span trace id
 2026-09-16 11:58:45 -07:00
mateo-berri
878fe17735 chore(proxy): restore the lazy OpenAPI snapshot main renders under Python 3.12 2026-09-16 11:56:18 -07:00
yassin
a2108f02fb test(router): cover _increment_deployment_usage delta and unlimited deployment behavior 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:51:40 +00:00
mateo-berri
033aa8ba6d fix(bedrock): forward userContext in Knowledge Base Retrieve requests 
The Bedrock vector store search only lifted retrievalConfiguration out of extra_body, so the caller's userContext (the Retrieve API's ACL identity) never reached Bedrock and ACL-enabled data sources answered with zero results. The transform now forwards userContext, taken from extra_body first and then from the top-level params where the OpenAI SDK's extra_body merge lands, as the caller sent it.
 2026-09-16 11:50:36 -07:00
yujonglee
560200da21
Merge pull request #41464 from BerriAI/litellm_rust_extract_auth_cache_crates 
refactor(rust): extract auth and cache crates
 2026-09-16 11:45:04 -07:00
yassin
6e5cdd8f12 fix(router): count TPM/RPM usage before building rate-limit headers 
Router.make_call now increments the deployment TPM/RPM counter before set_response_headers reads remaining usage, so the headers carry post-increment values directly and the in-flight subtraction workaround from LIT-2719 is removed. deployment_callback_on_success adds only the tokens not yet counted (streams) and never a second request. The counter key uses the resolved deployment name so wildcard routes are read back correctly, and the proxy strips the router-owned counted-tokens marker from client metadata

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:45:01 +00:00
yassin
8c89cff0e0 feat(prometheus): add customer (end_user) budget gauges 
Mirror the key, team, user and org budget gauges for customer objects with
litellm_remaining_customer_budget_metric, litellm_customer_max_budget_metric
and litellm_customer_budget_remaining_hours_metric. The gauges carry only the
end_user label, are emitted after each request and from the startup budget
refresh for every customer with a budget attached, and reuse the
enable_end_user_cost_tracking_prometheus_only opt-in and end_user series caps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:39:56 +00:00
yassin
8a059cd4b4 fix(otel): promote nested metadata keys under the caller's dotted path 
Strip only the proxy's requester_metadata. wrapper from an allowlisted key so
requester_metadata.trace_id lands as litellm.metadata.trace_id while other
dotted keys keep their full path and cannot collide on a shared leaf name

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:39:55 +00:00
mateo-berri
774fc6021b chore(proxy): drop restating docstrings on the passthrough header helpers and refresh the lazy OpenAPI snapshot 2026-09-16 11:35:43 -07:00
Mateo Wang
78848d01a3
Merge pull request #34427 from BerriAI/litellm_bedrock_rag_retrieval_filter 
fix(rag): forward retrieval_filter from retrieval_config to vector store search
 2026-09-16 11:34:44 -07:00
yassin
81799a131b Merge branch 'litellm_regenerate_no_body_secret_sync' into litellm_key_alias_update_secret_sync 2026-09-16 18:34:37 +00:00
yassin
bb380fec28 test(proxy): drop redundant docstrings from alias rename secret sync tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:34:37 +00:00
yassin
c438c3b2f8 test(proxy): drop redundant docstring from regenerate secret sync test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:33:50 +00:00
Yujong Lee
44a518ea25 fix(rust): evict in-memory cache entries at the expiry boundary 
Matches Python evict_cache, which uses expiration <= now

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:30:45 +00:00
yassin
be1664a485 fix(proxy): rename AWS Secrets Manager secret when key alias changes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:29:29 +00:00
mateo-berri
938702a3a7 Merge branch 'main' of https://github.com/BerriAI/litellm into litellm_anthropic_passthrough_strip_virtual_key 2026-09-16 11:28:09 -07:00
Yujong Lee
fc93c9ccba ci(rust): drop removed bedrock-auth feature steps 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:20:33 +00:00
yassin
8cab3a7846 refactor(otel): walk nested metadata iteratively instead of recursively 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:20:10 +00:00
yassin
4513f78df5 fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:14:36 +00:00
Yujong Lee
23633445d7 refactor(rust): extract auth and cache crates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:12:11 +00:00
yassin
17059564a8 feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes 
baggage_metadata_keys entries such as requester_metadata.trace_id now resolve the caller's nested metadata.trace_id and stamp it on the LLM-call span as litellm.metadata.trace_id, in both the OTEL v2 logger and the legacy OpenTelemetry callback. Nested metadata mappings are flattened to dotted paths, only allowlisted leaves are promoted, and the requester_metadata blob itself is never promoted

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:08:14 +00:00
yucheng
1b5dacc717 fix(proxy): tolerate malformed auth spans and read the OTel span from request state for custom auth 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:05:38 +00:00
yassin
bf39aebcf1 ci(migrations): flag defaulted ADD COLUMN on request-log tables 
Postgres 10 has no fast default path, so ADD COLUMN ... DEFAULT on
LiteLLM_SpendLogs rewrites the heap and every index under an ACCESS
EXCLUSIVE lock inside the boot-time migrate deploy. The checker now
reports it on LiteLLM_SpendLogs and LiteLLM_ErrorLogs; the two shipped
migrations that already do it are grandfathered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:02:00 +00:00
yassin
ed18626291 fix(proxy): sync AWS Secrets Manager on body-less key regenerate 
POST /key/{key}/regenerate with no request body reaches async_key_rotated_hook with data=None, and the secret manager sync was gated on data being present, so the rotated key never reached AWS Secrets Manager and the revoked key stayed stored. Gate on response.token_id only and read the requested alias null-safely

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:56:39 +00:00
berriai-litellm-provider-info-sync[bot]
f8fb31db3a
chore(prices): sync Azure prices: 2 models 
azure_ai/grok-4.3: input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens
azure_ai/grok-4.6: input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens
 2026-09-16 17:51:09 +00:00
kerry
686556fffc test(anthropic): build served-model stream chunks immutably 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:49:15 +00:00
kerry
07d4936428 fix(anthropic): satisfy strict lint and update tests pinned to the dropped served model 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:44:59 +00:00
kerry-berri
4e996400e2
Merge pull request #41154 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated
 2026-09-16 10:37:21 -07:00
kerry
847172d311 fix(anthropic): carry the served model from message_start onto stream chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:31:04 +00:00
Yassin Kortam
8aebd4ff63
Merge pull request #40571 from BerriAI/litellm_presidio_mcp_mode_no_post_call_scan 
fix(guardrails): don't add post_call output scan for MCP-only Presidio modes
 2026-09-16 10:09:12 -07:00
yujonglee
a7cfdd4cd5
Merge pull request #41432 from BerriAI/litellm_remove_rust_gateway_router_realtime 
refactor(rust): remove gateway, config, router, realtime, and Rust trace-parity instrumentation
 2026-09-16 10:06:31 -07:00
Yassin Kortam
8491d01668
Merge pull request #41268 from BerriAI/litellm_outbound_http2_opt_in 
feat(http): opt-in outbound HTTP/2 for httpx clients
 2026-09-16 10:05:37 -07:00
kerry
b9ab36279b fix(prices): add tpm and rpm to gemini 3.8 live rows 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:17 +00:00
kerry
eed760a37d test: drop azure gpt-5.6 rate tests pinned to a dated price page 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:15 +00:00
yassin
8a43fed20c fix(guardrails): use explicit returns in _configured_event_hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:48:15 +00:00
Yassin Kortam
f0474bb70e
Merge pull request #41310 from BerriAI/litellm_model_access_denied_message 
fix(proxy): hide model allowlist from client-facing model access denied errors
 2026-09-16 09:38:29 -07:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
7b3582aa66 fix(guardrails): treat tag-based Mode as MCP-only when all hooks are MCP hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:31:44 +00:00
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
 2026-09-16 16:27:18 +00:00
berriai-litellm-provider-info-sync[bot]
809685603a
chore(prices): sync Azure prices: 247 models 
azure_ai/Codestral-2501: 
azure_ai/cohere-command-a: 
azure_ai/deepseek-r1: 
azure_ai/deepseek-v3: 
azure_ai/deepseek-v3-0324: 
azure_ai/deepseek-v3.1: 
azure_ai/deepseek-v3.2: 
azure_ai/deepseek-v3.2-speciale: 
azure_ai/deepseek-v4-flash: 
azure_ai/DeepSeek-V4-Flash-0731: 
azure_ai/deepseek-v4-pro: 
azure_ai/embed-v-4-0: 
azure_ai/FW-DeepSeek-V3.2: 
azure_ai/FW-DeepSeek-V4-Pro: 
azure_ai/FW-GLM-5: 
azure_ai/FW-GLM-5.1: 
azure_ai/FW-GLM-5.2: 
azure_ai/FW-GLM-5.2-Fast: 
azure_ai/FW-Inkling: 
azure_ai/FW-Kimi-K2.5: 
azure_ai/FW-Kimi-K2.6: 
azure_ai/FW-Kimi-K2.7-Code: 
azure_ai/FW-Kimi-K3: 
azure_ai/FW-MiniMax-M2.5: 
azure_ai/FW-MiniMax-M3: 
azure_ai/FW-Nemotron-3-Ultra-NVFP4: 
azure_ai/FW-Nemotron-Lightning-3.5-30B-A3B: 
azure_ai/gpt-oss-120b: 
azure_ai/grok-3: 
azure_ai/global/grok-3: 
azure_ai/grok-3-mini: 
azure_ai/global/grok-3-mini: 
azure_ai/grok-4: 
azure_ai/grok-4-1-fast-non-reasoning: 
azure_ai/grok-4-1-fast-reasoning: 
azure_ai/grok-4-20-non-reasoning: 
azure_ai/grok-4-20-reasoning: 
azure_ai/grok-4-fast-non-reasoning: 
azure_ai/grok-4-fast-reasoning: 
azure_ai/grok-4.3: 
azure_ai/grok-4.6: 
azure_ai/grok-code-fast-1: 
azure_ai/kimi-k2.5: 
azure_ai/kimi-k2.6: 
azure_ai/kimi-k2.7-code: 
azure_ai/Llama-3.3-70B-Instruct: 
azure_ai/Llama-4-Maverick-17B-128E-Instruct-FP8: 
azure_ai/MAI-DS-R1: 
azure_ai/MAI-Image-2.5: 
azure_ai/MAI-Image-2.5-Flash: 
azure_ai/MAI-Image-2e: 
azure_ai/MAI-Thinking-1: 
azure_ai/mistral-large-3: 
azure_ai/Phi-3-medium-128k-instruct: 
azure_ai/Phi-3-medium-4k-instruct: 
azure_ai/Phi-3-mini-128k-instruct: 
azure_ai/Phi-3-mini-4k-instruct: 
azure_ai/Phi-3-small-128k-instruct: 
azure_ai/Phi-3-small-8k-instruct: 
azure_ai/Phi-3.5-mini-instruct:
 2026-09-16 16:26:23 +00:00
Yassin Kortam
e941edd08a
Merge pull request #41407 from BerriAI/litellm_content_filter_stream_bounded_scan 
perf(content_filter): scan a bounded window per streamed chunk
 2026-09-16 09:25:23 -07:00
yassin
23e775796c Merge remote-tracking branch 'origin/main' into pr34829 2026-09-16 16:23:53 +00:00
kerry
d398c9199c Merge remote-tracking branch 'origin/main' into HEAD 2026-09-16 16:18:47 +00:00
jesus
47117d880c fix(guardrails): don't add post_call output scan for MCP-only Presidio modes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:49 +00:00
Yujong Lee
6330d80efa refactor(rust): glob workspace members 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:46 +00:00