scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 84
53282 commits 3393 branches 1449 tags 5.1 GiB
Commit graph

19683 commits

Author SHA1 Message Date
yassin
6e5cdd8f12 fix(router): count TPM/RPM usage before building rate-limit headers 
Router.make_call now increments the deployment TPM/RPM counter before set_response_headers reads remaining usage, so the headers carry post-increment values directly and the in-flight subtraction workaround from LIT-2719 is removed. deployment_callback_on_success adds only the tokens not yet counted (streams) and never a second request. The counter key uses the resolved deployment name so wildcard routes are read back correctly, and the proxy strips the router-owned counted-tokens marker from client metadata

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:45:01 +00:00
yassin
8c89cff0e0 feat(prometheus): add customer (end_user) budget gauges 
Mirror the key, team, user and org budget gauges for customer objects with
litellm_remaining_customer_budget_metric, litellm_customer_max_budget_metric
and litellm_customer_budget_remaining_hours_metric. The gauges carry only the
end_user label, are emitted after each request and from the startup budget
refresh for every customer with a budget attached, and reuse the
enable_end_user_cost_tracking_prometheus_only opt-in and end_user series caps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:39:56 +00:00
yassin
8a059cd4b4 fix(otel): promote nested metadata keys under the caller's dotted path 
Strip only the proxy's requester_metadata. wrapper from an allowlisted key so
requester_metadata.trace_id lands as litellm.metadata.trace_id while other
dotted keys keep their full path and cannot collide on a shared leaf name

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:39:55 +00:00
yucheng
74d8328ad0 Merge remote-tracking branch 'origin/main' into litellm_lit7836_call_id_endpoint_logs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/test_proxy_server.py
 2026-09-16 18:39:55 +00:00
Devin AI
4f585d3931 fix(responses): drop top_p for gpt-5 reasoning models when drop_params is set 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:35:49 +00:00
mateo-berri
774fc6021b chore(proxy): drop restating docstrings on the passthrough header helpers and refresh the lazy OpenAPI snapshot 2026-09-16 11:35:43 -07:00
Mateo Wang
78848d01a3
Merge pull request #34427 from BerriAI/litellm_bedrock_rag_retrieval_filter 
fix(rag): forward retrieval_filter from retrieval_config to vector store search
 2026-09-16 11:34:44 -07:00
yassin
81799a131b Merge branch 'litellm_regenerate_no_body_secret_sync' into litellm_key_alias_update_secret_sync 2026-09-16 18:34:37 +00:00
yassin
bb380fec28 test(proxy): drop redundant docstrings from alias rename secret sync tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:34:37 +00:00
yassin
c438c3b2f8 test(proxy): drop redundant docstring from regenerate secret sync test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:33:50 +00:00
yassin
0143fe5583 fix(guardrails): hold tool-call windows until the final scan and expose Bedrock streaming flags to the UI 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:30:12 +00:00
yassin
be1664a485 fix(proxy): rename AWS Secrets Manager secret when key alias changes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:29:29 +00:00
mateo-berri
938702a3a7 Merge branch 'main' of https://github.com/BerriAI/litellm into litellm_anthropic_passthrough_strip_virtual_key 2026-09-16 11:28:09 -07:00
yassin
4513f78df5 fix(migrations-check): read the table name past comments, ignore referential SET DEFAULT 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:14:36 +00:00
yassin
17059564a8 feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes 
baggage_metadata_keys entries such as requester_metadata.trace_id now resolve the caller's nested metadata.trace_id and stamp it on the LLM-call span as litellm.metadata.trace_id, in both the OTEL v2 logger and the legacy OpenTelemetry callback. Nested metadata mappings are flattened to dotted paths, only allowlisted leaves are promoted, and the requester_metadata blob itself is never promoted

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:08:14 +00:00
yucheng
1b5dacc717 fix(proxy): tolerate malformed auth spans and read the OTel span from request state for custom auth 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:05:38 +00:00
yassin
bf39aebcf1 ci(migrations): flag defaulted ADD COLUMN on request-log tables 
Postgres 10 has no fast default path, so ADD COLUMN ... DEFAULT on
LiteLLM_SpendLogs rewrites the heap and every index under an ACCESS
EXCLUSIVE lock inside the boot-time migrate deploy. The checker now
reports it on LiteLLM_SpendLogs and LiteLLM_ErrorLogs; the two shipped
migrations that already do it are grandfathered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 18:02:00 +00:00
Devin AI
732ac614cc test(passthrough): expect absent query params 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:58:50 +00:00
yassin
ed18626291 fix(proxy): sync AWS Secrets Manager on body-less key regenerate 
POST /key/{key}/regenerate with no request body reaches async_key_rotated_hook with data=None, and the secret manager sync was gated on data being present, so the rotated key never reached AWS Secrets Manager and the revoked key stayed stored. Gate on response.token_id only and read the requested alias null-safely

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:56:39 +00:00
Zach Bernstein
242bff782f
fix(scim): clamp collection page size 2026-09-16 12:52:22 -05:00
kerry
686556fffc test(anthropic): build served-model stream chunks immutably 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:49:15 +00:00
kerry
9301aaf95d test: add cost map price relationship invariants 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:47:32 +00:00
kerry
07d4936428 fix(anthropic): satisfy strict lint and update tests pinned to the dropped served model 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:44:59 +00:00
Joshua Valluru
70ef8b24b6 fix(mcp): reject bare schemes in raw authorization 2026-09-16 10:44:00 -07:00
Devin AI
94771abd84 fix(passthrough): only fall back to url query when client sends none 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:38:40 +00:00
kerry
df41f67399 test: assert cost-map schema instead of tautological rate lookups 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:37:48 +00:00
kerry-berri
4e996400e2
Merge pull request #41154 from BerriAI/litellm-providers/price-sync 
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated
 2026-09-16 10:37:21 -07:00
Devin AI
b6f97a51d2 fix(passthrough): keep target URL query when client sends no query params 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:35:20 +00:00
kerry
847172d311 fix(anthropic): carry the served model from message_start onto stream chunks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:31:04 +00:00
kerry
0e8aa60b41 test: tidy price-derivation cleanup 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:24:53 +00:00
yassin
64ce436036 Merge remote-tracking branch 'origin/main' into litellm_streaming_buffer_release_on_scan 2026-09-16 17:22:01 +00:00
kerry
82289529c7 test: derive expected prices from the cost map instead of pinning vendor values 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 17:21:40 +00:00
Yassin Kortam
8aebd4ff63
Merge pull request #40571 from BerriAI/litellm_presidio_mcp_mode_no_post_call_scan 
fix(guardrails): don't add post_call output scan for MCP-only Presidio modes
 2026-09-16 10:09:12 -07:00
yujonglee
a7cfdd4cd5
Merge pull request #41432 from BerriAI/litellm_remove_rust_gateway_router_realtime 
refactor(rust): remove gateway, config, router, realtime, and Rust trace-parity instrumentation
 2026-09-16 10:06:31 -07:00
Yassin Kortam
8491d01668
Merge pull request #41268 from BerriAI/litellm_outbound_http2_opt_in 
feat(http): opt-in outbound HTTP/2 for httpx clients
 2026-09-16 10:05:37 -07:00
Yuneng Jiang
7d42bc751d
docs(e2e): name all four rejection reasons in the counter note 2026-09-16 10:05:13 -07:00
kerry
eed760a37d test: drop azure gpt-5.6 rate tests pinned to a dated price page 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:49:15 +00:00
kerry
5507de326e test(e2e): type the wire-shape parametrize ids callback 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:45:09 +00:00
kerry
feb69c5f78 test(e2e): add tool-call, terminal, and image-input shapes to the cost matrix 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:38:15 +00:00
kerry
cab6732928 test: drop price-pinning tests that break on catalog updates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:32:37 +00:00
yassin
7b3582aa66 fix(guardrails): treat tag-based Mode as MCP-only when all hooks are MCP hooks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:31:44 +00:00
yassin
e48dde7b9b fix(tests): drop leftover merge markers in test_handle_jwt 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:28:13 +00:00
yassin
5fee1c8710 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/auth/test_handle_jwt.py
 2026-09-16 16:27:18 +00:00
Yassin Kortam
e941edd08a
Merge pull request #41407 from BerriAI/litellm_content_filter_stream_bounded_scan 
perf(content_filter): scan a bounded window per streamed chunk
 2026-09-16 09:25:23 -07:00
yassin
23e775796c Merge remote-tracking branch 'origin/main' into pr34829 2026-09-16 16:23:53 +00:00
jesus
47117d880c fix(guardrails): don't add post_call output scan for MCP-only Presidio modes 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:15:49 +00:00
Yassin Kortam
29a712b186
Merge pull request #41368 from BerriAI/litellm_shadow_streaming_multi_target 
feat(router): stream shadow traffic and fan out silent_model to multiple targets
 2026-09-16 09:08:24 -07:00
yassin
04aad317c7 Merge remote-tracking branch 'origin/main' into pr34829 2026-09-16 16:02:08 +00:00
Joshua Valluru
6c517bfc49 fix(mcp): reject scheme-only API key authorization payloads 2026-09-16 09:00:17 -07:00
Yujong Lee
96baeb8b04 refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 16:00:07 +00:00