Commit graph

50180 commits

Author SHA1 Message Date
kerry-berri
b6143b3711
Merge pull request #41457 from BerriAI/litellm-providers/price-sync
chore(prices): sync Google Gemini prices: 22 models
2026-09-16 12:37:16 -07:00
Yujong Lee
d2433ec063 fix(ocr): resolve decline types through native_exception_types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:35:28 +00:00
yassin
6faeb19c0c Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 19:33:16 +00:00
yassin
1e169bb912 refactor(guardrails): mark release_on_scan flag as Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:33:07 +00:00
mateo-berri
c052d7816e fix(bedrock): rename the cache point inliner so the recursion detector stops flagging it 2026-09-16 12:30:33 -07:00
mateo-berri
aa6d053a37 Merge remote-tracking branch 'origin/main' into litellm_otel_gen_ai_system_none 2026-09-16 12:30:20 -07:00
yassin
cb26454034 test(router): cover success callback racing the pre-header count and drop redundant docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:28:50 +00:00
Mateo Wang
a1b1f1ef9d
Merge pull request #34455 from BerriAI/litellm_lit4767_empty_choices_streaming_guard
fix(responses): guard empty-choices chunks in the Responses API streaming bridge
2026-09-16 12:28:46 -07:00
yassin
8de51dfaab test(otel): describe which request metadata the pre-call seed reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:25:40 +00:00
yassin
938b782b8a Merge remote-tracking branch 'origin/main' into litellm_customer_budget_prometheus_metrics 2026-09-16 19:25:23 +00:00
Yujong Lee
e60298bb2d fix(rust-bridge): align _native stub with stubtest conventions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:24:18 +00:00
yassin
d5837ab97a test(guardrails): pin tool-call-only scan keys as non-empty
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:24:14 +00:00
yassin
c22916377f test(prometheus): cover customer budget series expiry by end_user ttl
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:22:30 +00:00
Yujong Lee
e8b5632c20 ci(rust): run the token counter timing test alone under nextest
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:22:02 +00:00
kerry
e46106e20b test: drop gemini-3.1-flash-lite-image capability pins
The per-route capability test hardcoded vendor facts, including function calling support on the gemini route, which the live model card says is not supported. Keep the backup-matches-main invariant and the routing tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:21:50 +00:00
Devin AI
93eefc5922 fix(models): dedupe Fireworks deprecation keys and sync Azure dated snapshot service tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:20:20 +00:00
Devin AI
8caf2cb61f fix(models): flag prompt caching on Vertex and Azure AI Grok rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:17:45 +00:00
Mateo Wang
734038a83c
Merge pull request #41333 from BerriAI/litellm_cost_json_codeowners
chore(codeowners): add ryan and kerry as owners of the cost map
2026-09-16 12:17:13 -07:00
Devin AI
7734e3e186 fix(models): sonnet 4.5 1M input, daybreak alias, Mistral GLM 5.3, Azure dated snapshots, Together/OpenRouter sync
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:17:13 +00:00
Yujong Lee
61fc005fd5 fix(ci): install native wheel before route smoke test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:14:57 +00:00
Yujong Lee
b70ddc2fd8 ci(rust): split rust jobs, use nextest and Swatinem/rust-cache
Split the Rust workflow into fmt, clippy, nextest and wheel jobs so they run in parallel, replace manual actions/cache with Swatinem/rust-cache, and install a pinned checksum-verified cargo-nextest. Make two python-bridge tests self-contained so they pass when nextest runs each test in its own process.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:14:32 +00:00
yassin
30f02aa6da fix(otel): read only the caller's requester_metadata snapshot in the v2 pre-call hook
The pre-call hook passed the proxy's whole per-request metadata dict into the
request identity, so proxy-owned siblings such as requester_ip_address were
promoted alongside the caller's keys. Only the requester_metadata mapping is
read now, keyed under its wrapper, which keeps the default allowlist behaviour
unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:11:25 +00:00
yassin
73a0c3bb8a Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 19:09:19 +00:00
yassin
2821ba9667 fix(prometheus): refresh default-budget customers and honor independent customer gauges
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:08:42 +00:00
berriai-litellm-provider-info-sync[bot]
b893e6b926
chore(prices): sync Google Gemini prices: 22 models
gemini/gemini-2.5-flash: max_tokens, max_output_tokens, supports_audio_input
gemini/gemini-2.5-flash-image: max_input_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-lite: max_tokens, max_output_tokens, supports_audio_input
gemini-2.5-flash-native-audio-preview-12-2025: supports_vision, max_input_tokens, supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-native-audio-preview-12-2025: supports_vision, max_input_tokens, supports_web_search, supports_response_schema, supports_function_calling
gemini-2.5-flash-preview-tts: max_tokens, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-flash-preview-tts: max_tokens, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-2.5-pro: max_tokens, max_output_tokens
gemini/gemini-2.5-pro-preview-tts: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_web_search, supports_audio_input, supports_response_schema, supports_function_calling
gemini/gemini-3-flash-preview: max_tokens, max_output_tokens, supports_audio_input
gemini/gemini-3-pro-image: supports_response_schema
gemini/gemini-3.1-flash-image: max_input_tokens, supports_response_schema
gemini/gemini-3.1-flash-lite-image: supports_web_search, supports_function_calling
gemini-3.1-flash-live-preview: supports_response_schema
gemini/gemini-3.1-flash-live-preview: supports_response_schema
gemini/gemini-3.1-flash-tts-preview: supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-3.5-flash: max_tokens, max_output_tokens
gemini/gemini-3.5-live-translate-preview: supports_web_search, supports_response_schema, supports_function_calling
gemini/gemini-3.5-transcribe: supports_function_calling
gemini/gemini-3.5-transcribe-live: supports_function_calling
gemini/gemini-embedding-2: supports_vision, supports_audio_input
gemini/gemini-omni-1.1-flash: max_input_tokens
2026-09-16 19:06:19 +00:00
Yujong Lee
e09da4d485 Merge remote-tracking branch 'origin/main' into litellm-ocr-new-mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm-rust/crates/core/src/ocr/adapters/azure/cohere.rs
#	litellm-rust/crates/core/src/ocr/adapters/azure/document_intelligence/mod.rs
#	litellm-rust/crates/core/src/ocr/adapters/azure/document_intelligence/polling.rs
#	litellm-rust/crates/core/src/ocr/adapters/azure/mistral.rs
#	litellm-rust/crates/core/src/ocr/adapters/cohere.rs
#	litellm-rust/crates/core/src/ocr/adapters/mistral.rs
#	litellm-rust/crates/core/src/ocr/adapters/reducto/mod.rs
#	litellm-rust/crates/core/src/ocr/adapters/vertex/deepseek.rs
#	litellm-rust/crates/core/src/ocr/adapters/vertex/mistral.rs
#	litellm-rust/crates/core/src/ocr/adapters/vertex/mod.rs
#	litellm-rust/crates/core/src/ocr/registry.rs
#	litellm-rust/crates/core/src/ocr/wire.rs
#	litellm-rust/crates/core/tests/host_lifecycle.rs
#	litellm-rust/crates/core/tests/ocr.rs
#	litellm-rust/crates/core/tests/vertex_ai_deepseek_ocr.rs
#	litellm-rust/crates/core/tests/vertex_ai_ocr.rs
#	litellm-rust/crates/python-bridge/src/routes/definition.rs
#	litellm-rust/crates/python-bridge/src/routes/ocr/lifecycle.rs
#	litellm-rust/crates/python-bridge/src/routes/ocr/value.rs
2026-09-16 19:06:00 +00:00
Devin AI
03931180d7 merge main into litellm_registry_audit_2026_09_14 2026-09-16 19:03:36 +00:00
yassin
e105cde56a fix(router): sync worker-local usage cache to shared post-increment counters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:02:15 +00:00
Yujong Lee
e8d22b9d67 chore(rust_bridge): suppress LIT002 on dicts handed to logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 19:00:59 +00:00
yassin
5e2d9e1d5c refactor(otel): build promoted baggage without local dict mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:59:21 +00:00
yucheng-berri
b04d530ecf
Merge pull request #41386 from BerriAI/litellm_otel_trace_correlation
fix(proxy): default litellm_trace_id to the OTel server span trace id
2026-09-16 11:58:45 -07:00
yassin
ad25fe1886 fix(guardrails): hold unscannable Responses windows and key terminal envelopes by output items
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:56:56 +00:00
mateo-berri
878fe17735 chore(proxy): restore the lazy OpenAPI snapshot main renders under Python 3.12 2026-09-16 11:56:18 -07:00
yassin
a2108f02fb test(router): cover _increment_deployment_usage delta and unlimited deployment behavior
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:51:40 +00:00
yucheng
99aa9f76c8 fix(proxy): build failure headers immutably to keep LIT002 within budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:51:22 +00:00
mateo-berri
033aa8ba6d fix(bedrock): forward userContext in Knowledge Base Retrieve requests
The Bedrock vector store search only lifted retrievalConfiguration out of extra_body, so the caller's userContext (the Retrieve API's ACL identity) never reached Bedrock and ACL-enabled data sources answered with zero results. The transform now forwards userContext, taken from extra_body first and then from the top-level params where the OpenAI SDK's extra_body merge lands, as the caller sent it.
2026-09-16 11:50:36 -07:00
ryan
2bb478fb59 fix(ui): keep http and upper-case https skill sources clickable on the detail page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:45:11 +00:00
yujonglee
560200da21
Merge pull request #41464 from BerriAI/litellm_rust_extract_auth_cache_crates
refactor(rust): extract auth and cache crates
2026-09-16 11:45:04 -07:00
yassin
6e5cdd8f12 fix(router): count TPM/RPM usage before building rate-limit headers
Router.make_call now increments the deployment TPM/RPM counter before set_response_headers reads remaining usage, so the headers carry post-increment values directly and the in-flight subtraction workaround from LIT-2719 is removed. deployment_callback_on_success adds only the tokens not yet counted (streams) and never a second request. The counter key uses the resolved deployment name so wildcard routes are read back correctly, and the proxy strips the router-owned counted-tokens marker from client metadata

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:45:01 +00:00
ryan
7cd6869cfa test(ui): match skill source links by exact name so codeql stops flagging the host regexes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:40:10 +00:00
yassin
8c89cff0e0 feat(prometheus): add customer (end_user) budget gauges
Mirror the key, team, user and org budget gauges for customer objects with
litellm_remaining_customer_budget_metric, litellm_customer_max_budget_metric
and litellm_customer_budget_remaining_hours_metric. The gauges carry only the
end_user label, are emitted after each request and from the startup budget
refresh for every customer with a budget attached, and reuse the
enable_end_user_cost_tracking_prometheus_only opt-in and end_user series caps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:39:56 +00:00
yassin
8a059cd4b4 fix(otel): promote nested metadata keys under the caller's dotted path
Strip only the proxy's requester_metadata. wrapper from an allowlisted key so
requester_metadata.trace_id lands as litellm.metadata.trace_id while other
dotted keys keep their full path and cannot collide on a shared leaf name

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:39:55 +00:00
yucheng
74d8328ad0 Merge remote-tracking branch 'origin/main' into litellm_lit7836_call_id_endpoint_logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/test_proxy_server.py
2026-09-16 18:39:55 +00:00
mateo-berri
774fc6021b chore(proxy): drop restating docstrings on the passthrough header helpers and refresh the lazy OpenAPI snapshot 2026-09-16 11:35:43 -07:00
Mateo Wang
78848d01a3
Merge pull request #34427 from BerriAI/litellm_bedrock_rag_retrieval_filter
fix(rag): forward retrieval_filter from retrieval_config to vector store search
2026-09-16 11:34:44 -07:00
yassin
81799a131b Merge branch 'litellm_regenerate_no_body_secret_sync' into litellm_key_alias_update_secret_sync 2026-09-16 18:34:37 +00:00
yassin
bb380fec28 test(proxy): drop redundant docstrings from alias rename secret sync tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:34:37 +00:00
yassin
c438c3b2f8 test(proxy): drop redundant docstring from regenerate secret sync test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:33:50 +00:00
Yujong Lee
44a518ea25 fix(rust): evict in-memory cache entries at the expiry boundary
Matches Python evict_cache, which uses expiration <= now

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:30:45 +00:00
yassin
0143fe5583 fix(guardrails): hold tool-call windows until the final scan and expose Bedrock streaming flags to the UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 18:30:12 +00:00