Commit graph

52143 commits

Author SHA1 Message Date
Mateo Wang
6cdf398bea
Merge pull request #41504 from BerriAI/litellm_bedrock_agent_runtime_strip_virtual_key
fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough
2026-09-16 17:02:17 -07:00
Yujong Lee
6d30006592 refactor(rust_bridge): share call_hook instead of per-route native lambdas
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 00:01:52 +00:00
Yujong Lee
43c50325a6 refactor(rust_bridge): pass dispatch context functions directly 2026-09-16 16:59:57 -07:00
kerry
a3d3fe4ada fix(ci): pin the auto-merge request to the evaluated head sha
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:57:44 +00:00
yucheng
8d054ba230 test(otel): cover the routed tracer budget for spans opened at the pre_call boundary
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:56:57 +00:00
kerry
42bf9f0ece Merge remote-tracking branch 'origin/main' into litellm_auto_merge_price_sync 2026-09-16 23:56:48 +00:00
Devin AI
75eec8712c fix(a2a): keep the upstream status on card discovery failures and inject the card client in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:55:57 +00:00
kerry
bba15b1382 style(responses_bridge): apply ruff format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:52:53 +00:00
yassin
d6b13f938d test(proxy): cover guardrail tag budget edge cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:52:39 +00:00
yassin
488666ccae fix(proxy): enforce tag budgets for tags added by guardrails
Auth runs the tag budget check before pre_call_hook, so a tag that a custom guardrail adds is attributed spend but never budget checked. After the pre-call hook, budget check only the newly added tags with the same exemptions auth applied (budget-free routes, zero-cost models), keep the pre-guardrail tag baseline across fallback retries, and surface an over-budget tag as the same budget_exceeded 429 auth returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:52:39 +00:00
Devin AI
bc8e28cfcf test(a2a): inject a fake httpx client into card resolver tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:52:16 +00:00
Mateo Wang
319f427c40
Merge pull request #41201 from BerriAI/litellm_gemini_37_38_flash_no_minimal_thinking
fix(gemini): map minimal thinking to low for Gemini 3.7 and 3.8 Flash
2026-09-16 16:52:08 -07:00
yassin
5f1d87911a test(spend_tracking): cover unresolvable deployment leaving provider empty
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:50:22 +00:00
mateo-berri
ada0a1ad3a fix(azure_ai): strip the azure_ai/ prefix when a Responses call is remapped to azure
A catalog OpenAI name on an .openai.azure.com host (or with AZURE_AI_API_BASE set to one) is remapped from azure_ai to azure before the Responses request is built, and the azure_ai/ prefix stayed in the wire model, so Azure answered DeploymentNotFound. The Azure Responses config now strips azure_ai/ next to responses/ and o_series/.
2026-09-16 16:49:29 -07:00
yassin
8bd598f13c feat(proxy): add Amazon Transcribe SigV4 pass-through routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:48:15 +00:00
yucheng-berri
c5325b1492
Merge pull request #40596 from BerriAI/litellm_lit_7470_rate_limit_fallback_pristine_data
fix(proxy): retry rate-limit fallbacks from a pristine request snapshot
2026-09-16 16:47:35 -07:00
ryan-crabbe-berri
fd411373fc style(proxy): collapse the enduser page signature onto one line 2026-09-16 16:44:39 -07:00
yucheng-berri
672f43fd54
Merge pull request #41356 from BerriAI/litellm_lit7836_call_id_endpoint_logs
fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses
2026-09-16 16:43:12 -07:00
kerry
b7a9042f1b style(responses_bridge): suppress type-discipline flags with reasons
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:43:04 +00:00
mateo-berri
574ea15b8f fix(mcp): count admin static headers as api_key credential slots
An api_key server whose key lives in static_headers, the documented
shape for upstreams that expect a custom header name, dispatched fine
before the fail-closed check and was rejected as misconfigured after
it. The check now treats every static header the admin configured as a
credential slot for api_key mode, on both the MCP client path and the
OpenAPI tool path, with regression tests at all three layers.
2026-09-16 16:42:20 -07:00
ryan-crabbe-berri
0d8b46b88c refactor(proxy): trim the invalidation docstrings and inject the page read failure
Cuts the new docstrings back to the parts a reader cannot get from the code,
and fixes a stale reference: the walk this one is modelled on is
_reset_windows_for, not _reset_windows_for_source.

The truncation test reached in and replaced MockTable.find_many. The mock takes
a scheduled read failure instead, the way it already takes canned rows.
2026-09-16 16:41:13 -07:00
kerry
9212124266 style(responses_bridge): use dict.get in text merge helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:37:45 +00:00
kerry
255ef3bc26 refactor(bedrock_mantle): build supported params without mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:37:45 +00:00
ryan-crabbe-berri
3ee8d43fdd fix(ui): send only a changed TPM limit from the team admin settings form
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Save stays disabled until the value differs from the team's, so an unchanged form never reaches /team/update
2026-09-16 16:37:35 -07:00
mateo-berri
41737aeda8 test(proxy): drop the docstring from the agent-runtime passthrough regression class 2026-09-16 16:37:32 -07:00
Moe Khalil
b95dbb41ae fix(router): scope discovered limits to active deployments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:37:28 +00:00
yassin
9af014d75a fix(spend_tracking): keep inferred provider out of model reconstruction and OAuth provider lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:36:54 +00:00
mateo-berri
e243237a7c feat(a2a): reach Microsoft Foundry agents with Entra auth and versioned card discovery
Foundry serves its agent card only at agentCard/v1.0, accepts only an Entra ID
bearer, and defaults to a non-blocking send, so the A2A relay and the chat
completions route could not use it.

The relay gains an agent_card_path litellm_param plus agentCard/v1.0 as a third
discovery probe, mints a bearer from flat Entra fields on the agent
(tenant_id, client_id, client_secret, azure_ad_token, azure_username,
azure_password, azure_scope) for https://ai.azure.com/.default, and sends it on
the card fetch, message/send, message/stream, tasks/* and the chat bridge.

Chat completions look the registered agent up by its provider-stripped name so
its api_key and headers reach the request, tag every message with its kind, ask
for a blocking send, fall back to a blocking send when the registered card says
streaming: false, and fail the call on a JSON-RPC error inside a stream instead
of yielding an empty one. Entra fields stay out of the chat bridge's logged
parameters.

Resolves LIT-5122
2026-09-16 16:36:47 -07:00
yucheng
d3f4a8b983 fix(otel): read the attribute budget from the span's own provider limits so routed tracers fit correctly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:34:43 +00:00
kerry
438681be1a refactor(responses_bridge): share text merge helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:33:28 +00:00
kerry
e8f246bb6b fix(responses_bridge): forward verbosity as text.verbosity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:32:32 +00:00
mateo-berri
ce722ab1b3 fix(proxy): evict the member's cached user row on team member add
JWT auth caches the user row before it adds the user to the JWT's team, and admission checks the credential's team against the cached row on whichever worker takes the next request. On a two-worker gateway the credential minted for a newly joined team answered 403 "not in your team memberships" until the management-object TTL ran out, because /team/member_add only evicted the membership spend sentinel. The add now evicts the added members' cached user rows and broadcasts the eviction to the other workers, the way /team/member_delete already did

The mint test now also covers a user SCIM deactivated after the cache last saw them active: the database read refuses the mint while the cached row still says active
2026-09-16 16:31:28 -07:00
Mateo Wang
390ab45448
Merge pull request #37506 from BerriAI/litellm_dashscope_reasoning_effort
Some checks are pending
LiteLLM Rust / rust-wheel (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(dashscope): forward reasoning_effort to the provider
2026-09-16 16:29:25 -07:00
mateo-berri
2f186055e6 fix(logging): decide keep-or-scrub for a log extra by comparing it to its scrubbed copy
Some checks are pending
ai-gateway image / ai-gateway release image (push) Waiting to run
The code-quality check refuses recursive functions and the walk that inspected
extras was one, so the filter no longer walks anything itself. safe_dumps now
builds its JSON-native structure through safe_json_structure, the filter scrubs
the extra through that, and the original object is kept only when the scrubbed
copy compares equal to it. Anything the serializer skipped (non-string keys,
nests past its depth, fields a repr hides) makes the copy differ, so the copy
wins. A host object whose equality raises, as numpy arrays and torch tensors
do, counts as changed instead of breaking the caller's log call
2026-09-16 16:29:23 -07:00
yujonglee
1c5d4d0be8
Merge pull request #41501 from BerriAI/litellm_1789598099-cache-redis
feat(rust): scaffold Redis cache crate
2026-09-16 16:28:54 -07:00
yassin
bddb64ddc5 test(vertex_ai): cover unterminated last row, unparseable rows, and sync stream rejection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:28:39 +00:00
kerry
aea9678f61 refactor(streaming): compute the reasoning token cap once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:28:28 +00:00
ryan-crabbe-berri
14fbd623d7 fix(proxy): clear both cache partitions and carry the walk position as a value
UserApiKeyCache's batch delete ran the two partitions in sequence, so a Redis
failure on the hashed token partition returned before the ordinary management
keys were touched. Both partitions are attempted now and the first failure is
re-raised for the caller to report.

The customer walk kept its position in two locals it reassigned each page. It
now mirrors the window walk in the same file: a page helper returns where the
walk goes next, and the driver rebinds one value.
2026-09-16 16:28:16 -07:00
kerry
884e96958c fix(streaming): cap estimated reasoning tokens to the provider total and cover dict chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:48 +00:00
Moe Khalil
1c9525fdab test(router): cover deployment replacement during metadata discovery
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:31 +00:00
kerry
15c18ad6cd fix(bedrock_mantle): accept verbosity on gpt-5.x chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:07 +00:00
ryan-crabbe-berri
ab92a6637d test(e2e): cover team admin editable fields on /team/update
Team admins are refused until a proxy admin enables a field, then limited to the enabled fields, and resending unchanged budget settings keeps the team's budget reset times
2026-09-16 16:26:52 -07:00
joshua-berri
41410e9556
Merge pull request #41364 from BerriAI/litellm_fix_mcp_auth_fail_closed_4501
fix(mcp): fail closed on missing upstream credentials
2026-09-16 23:26:38 +00:00
Moe Khalil
29d1a191a1 refactor(router): isolate per-deployment metadata refresh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:24:03 +00:00
Yujong Lee
1f0cf4bf42 fix(rust_bridge): bind Python fallbacks at import so module patches do not leak into public entrypoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:23:36 +00:00
Moe Khalil
b7c6befb37 feat(router): discover token limits for hosted OpenAI-compatible models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:19:54 +00:00
Yujong Lee
93ba409adf fix(rust): address Redis cache review findings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:19:28 +00:00
mateo-berri
3712d8de92 Merge remote-tracking branch 'origin/main' into litellm_model_group_info_proxy_admin_all_models
Some checks failed
LiteLLM Rust / rust-wheel (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-16 16:18:45 -07:00
mateo-berri
2a9fa48730 fix: expand wildcard deployments for proxy admins on /model_group/info 2026-09-16 16:18:44 -07:00
kerry
813d96f26e fix(e2e): resolve remaining merge markers in e2e_config
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:18:26 +00:00