Commit graph

51523 commits

Author SHA1 Message Date
ryan-crabbe-berri
bdbb4e4610
Merge pull request #41681 from BerriAI/litellm_org_alias_from_team
fix(auth): inherit org alias, budget and rate limits for JWT and team-linked keys
2026-09-19 15:25:14 -07:00
ryan-crabbe-berri
370eacce12
Merge pull request #41997 from BerriAI/litellm_project_zero_budget_blocks
fix(proxy): block project requests when max_budget is 0
2026-09-19 15:24:18 -07:00
yuneng-jiang
b6dbf145cb
Merge pull request #42009 from BerriAI/litellm_config_store_qa_fixes
fix(proxy): close the config-ownership gaps QA found in the settings store
2026-09-19 15:20:13 -07:00
yuneng-jiang
974e4f109c
Merge pull request #42025 from BerriAI/litellm_/release-version-bump-906e1d
chore: bump litellm-enterprise 0.1.68 -> 0.1.69, litellm-proxy-extras 0.4.99 -> 0.4.100
2026-09-19 14:38:35 -07:00
yuneng-jiang
0b145ea149
Merge pull request #42008 from BerriAI/litellm_/litellm-e2e-buildkite-rc-694bbe
test(e2e): stop the config suite locking itself out of the shared proxy
2026-09-19 14:32:15 -07:00
Yuneng Jiang
8767f12794
bump: litellm-enterprise 0.1.68 -> 0.1.69, litellm-proxy-extras 0.4.99 -> 0.4.100 2026-09-19 14:27:21 -07:00
Yuneng Jiang
9c3a7133f1
test: cover the config-owned refusal on the email reset route 2026-09-19 14:25:52 -07:00
yucheng-berri
ac281507fd
Merge pull request #41991 from BerriAI/litellm_otel_v2_langfuse_responses_output
fix(otel v2): map Responses API output onto the Langfuse generation output
2026-09-19 14:23:59 -07:00
Yuneng Jiang
e5398e7e30
test: drop two inert type: ignore comments
pyrightconfig.json sets enableTypeIgnoreComments to false and does not
include tests/, so neither comment suppressed anything.
2026-09-19 14:22:19 -07:00
Yassin Kortam
884351168a
Merge pull request #42003 from BerriAI/litellm_fix_terraform_registry_docs_links 2026-09-19 14:02:05 -07:00
yujonglee
d675c1285b
Merge pull request #41987 from BerriAI/litellm_rust_fork_safety
fix(rust): refuse native routes in processes forked after the runtime started
2026-09-19 13:47:54 -07:00
Shivam Rawat
c02399b29d fix(terraform): unlink the registry docs entries that 404 on click
The resource and data source links on the provider's registry docs
overview page 404 when clicked. They are written as relative paths like
./resources/team, and the registry serves the overview at
.../latest/docs with no trailing slash and passes hrefs through
unrewritten, so the browser resolves them to .../latest/resources/team.

Drops the link markup and keeps both lists and their descriptions. No
relative form works in both places: only a docs/-prefixed target
resolves correctly on the registry, and that same path is wrong when
reading the file on GitHub. The registry sidebar already links every
resource and data source for the version being read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-19 13:39:47 -07:00
mateo-berri
cae6634192 fix(auth): keep the last-known org copy when the auth prefetch warmed the org row
The last-known org copy was written only on get_org_object's DB-read path. The
virtual-key auth prefetch fills the same 5s org entry directly, so with keys and
JWTs of one org on the same worker the JWT lookup always hit the cache, never
wrote the copy, and a DB outage turned that JWT traffic into 503s again.

get_org_object_for_request now writes the copy itself whenever this worker holds
none, under the management-object TTL, and get_org_object is back to its shape
on main.
2026-09-19 13:36:37 -07:00
Mateo Wang
a93bfdc749
Merge pull request #41995 from BerriAI/litellm_ci_google_genai_vertex_global_retry
test(unified_google_tests): use the Vertex global endpoint and retry 429s with backoff
2026-09-19 13:00:47 -07:00
Yuneng Jiang
3c6a2f258a
test(proxy): capture the saved config with an AsyncMock instead of a mutable list
Greptile flagged the unannotated list and append against the repository's
immutable-state and Final-local rules (LIT001/LIT010). Recording the call on an
AsyncMock removes the accumulator entirely and matches how the neighbouring
audit-log tests in this file read their captured arguments.
2026-09-19 12:56:51 -07:00
mateo-berri
854f5d959e Merge remote-tracking branch 'origin/main' into litellm_org_alias_from_team 2026-09-19 12:55:15 -07:00
mateo-berri
9075cafb98 fix(auth): serve the last-known org through a database outage
A JWT whose team sits in an org resolves the org on every request, and the
org row is cached for only DEFAULT_IN_MEMORY_TTL seconds while the team and
user rows ride the 60s management-object TTL. A few seconds into a database
outage the org lookup failed closed and that traffic got 503s while the same
request through a virtual key kept succeeding on its cached team.

get_org_object now also keeps a last-known copy of the org row under the
management-object TTL, and get_org_object_for_request serves that copy when
the database is unreachable, so JWT traffic degrades the same way the team
lookup does. A missing copy keeps the previous behaviour: fail closed unless
allow_requests_on_db_unavailable is set.
2026-09-19 12:55:14 -07:00
Yuneng Jiang
bf9c717d77
test(e2e): stop the config suite locking itself out of the shared proxy
Two tests in the config/misc management suite were failing every run against
the Buildkite e2e stack, and one of them took the rest of the build with it.

test_add_allowed_ip_does_not_store_unrelated_config_value posted 127.0.0.1 to
/add/allowed_ip. That route sets the live general_settings["allowed_ips"] that
auth_utils._check_valid_ip reads before it persists anything, and the check is
exact string membership with no CIDR support, so from the moment the POST
returns only 127.0.0.1 can reach the proxy. The runner 403s on its very next
call, and the deferred /delete/allowed_ip sits behind the same auth dependency,
so the cleanup is locked out too and every later test in the build 403s. Build
254's first attempt lost 459 of its 465 failures to that one cascade.

There is no safe way to exercise the route against a shared proxy: nothing
reports the caller's address as the proxy sees it, so a test cannot allowlist
itself first. Move the claim to the route's own TestClient suite, where the
auth dependency is overridden and general_settings is per-test, and record the
route in the module docstring beside /cache/settings and the Vault override so
it is not re-added. save_config's end of the contract was already covered by
test_ProxyConfig_save_config_merges_changed_keys_without_copying_file_settings;
the new test covers the route's end, that what it hands save_config differs
from the loaded config in allowed_ips and nothing else.

The unrelated-key probe also only ever worked on one lane: max_parallel_requests
was added to tests/e2e/gateway/stage_mirror_ci_config.yml and never to the
Buildkite stack's config, where resolve() reports it as "unset" rather than
"config". That key is now unused, so drop it again.

test_config_update_persists_router_setting_to_get wrote router_settings.
num_retries, which both lanes declare in their config file, so the config-
ownership work correctly refuses it with a 400. Switch to retry_after, which is
declared by neither lane, is accepted by /config/update, and is reported back by
GET /router/settings. Verified against a live proxy: max_fallbacks also takes
the write but never reads back, so the read-back poll is what picks the key.
2026-09-19 12:51:54 -07:00
Yuneng Jiang
5de9fc6961
test: give the new proxy_server-global patches a test-quality reason 2026-09-19 12:49:17 -07:00
yucheng
e8f2ee8200 fix(redaction): redact Responses refusal parts under turn_off_message_logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:35:08 +00:00
yucheng
7d93821e41 fix(otel v2): keep Responses refusal text on the folded assistant message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:30:57 +00:00
Yuneng Jiang
9905458e25
Merge remote-tracking branch 'origin/main' into litellm_config_store_qa_fixes 2026-09-19 12:17:27 -07:00
Yuneng Jiang
3dff41f369
fix(proxy): close the config-ownership gaps QA found in the settings store
- apply_db_row only clears runtime values for keys the row actually changed, so an env-resolved DB-owned setting survives a reload
- DELETE /config/field/delete refuses a key the config file owns instead of silently rewriting the row
- GET /config/field/info reports the declared value of a config-owned key, not the env-resolved secret
- SettingsStore gains a short-circuiting __bool__ so truthiness checks stop at the first key
- _initialize_jwt_auth resolves os.environ refs into a local mapping instead of mutating the shared general_settings dict
- rejected_writes compares against the resolved value, matching what __setitem__ accepts
- a stored value identical to the config template is no longer reported as shadowed
- the enterprise email-settings and coordination-redis writers go through reject_config_owned_writes
2026-09-19 12:17:20 -07:00
mateo-berri
a7870a902a test(unified_google_tests): import ReadOnly from typing_extensions and cover the Vertex global endpoint
The first commit imported ReadOnly from typing, which only exists on Python 3.13 and up. CircleCI runs this suite on 3.12, so the module failed at import and the job stopped at collection before any of its tests ran. ReadOnly and TypedDict now come from typing_extensions, like the rest of the repo

A new test resolves the Vertex deployment's location from the suite's config with VERTEXAI_LOCATION set to a region, and fails if the vertex_location line is removed

The expected minimum backoff is now derived from litellm's INITIAL_RETRY_DELAY and MAX_RETRY_DELAY, so the test holds when those are overridden through the environment
2026-09-19 12:15:34 -07:00
kerry-berri
b946d12ffd
Merge pull request #41996 from BerriAI/litellm-providers/price-sync-openrouter
chore(prices): sync OpenRouter prices: 2 models
2026-09-19 12:13:09 -07:00
ryan-crabbe-berri
162d6225e0 fix(proxy): block project requests when max_budget is 0
A project max_budget of 0 was treated as unbudgeted by #41354, while key budgets block at 0 and null is the unlimited value. Drop the <= 0 skip so 0 blocks and null stays unlimited
2026-09-19 12:06:35 -07:00
yucheng-berri
4301ac4940
Merge pull request #41986 from BerriAI/litellm_revert_post_call_guardrail_context
revert(guardrails): drop the scoped request conversation and tools from post-call scans (#41220)
2026-09-19 12:06:30 -07:00
berriai-litellm-provider-info-sync[bot]
38b310b751
chore(prices): sync OpenRouter prices: 2 models
openrouter/deepseek/deepseek-v4-flash: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/qwen/qwen-plus-2025-07-28: supports_prompt_caching
2026-09-19 19:00:52 +00:00
mateo-berri
47d06d9fdd test(unified_google_tests): use the Vertex global endpoint and retry 429s with backoff
The google_generate_content_endpoint_testing job went red on main when us-central1 ran out of shared gemini-2.5-flash-lite capacity for a few hours. The suite's proxy config now sends the Vertex deployment to the global endpoint and retries rate limit errors 5 times with exponential backoff, and a regression test pins that the config rides out 3 consecutive 429s
2026-09-19 11:56:02 -07:00
joshua-berri
eda1faba14
Merge pull request #41718 from BerriAI/litellm_mcp_dependency_gate_7737
refactor(mcp): upgrade SDK2 while preserving legacy gateway behavior
2026-09-19 18:53:19 +00:00
kerry-berri
c3fa53be8d
Merge pull request #41993 from BerriAI/litellm_remove_auto_merge_price_sync
ci: remove auto-merge-price-sync workflow, the Devin sync automation merges price PRs
2026-09-19 11:44:36 -07:00
yujonglee
f081e447a4
Merge pull request #41992 from BerriAI/litellm_deepseek_ocr_defaults
fix(ocr): set DeepSeek OCR sampling defaults
2026-09-19 11:42:49 -07:00
kerry-berri
b89a6fba4c
Merge pull request #41833 from BerriAI/litellm-providers/price-sync-openrouter
chore(prices): sync OpenRouter prices: 172 models, 2 new
2026-09-19 11:42:34 -07:00
Yujong Lee
d67d9984f7 test: expect TQ009 in the shipped quality budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:38:37 +00:00
kerry
987af6c66c ci: remove auto-merge-price-sync workflow, the Devin sync automation merges price PRs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:33:06 +00:00
ryan-crabbe-berri
659cef0f57
Merge pull request #40729 from BerriAI/litellm_lit7572_top_keys_user_column
fix(ui): show user attribution in Top Virtual Keys usage tables
2026-09-19 11:32:35 -07:00
Yujong Lee
8c21a988b7 fix(ocr): set DeepSeek OCR sampling defaults
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:31:49 +00:00
berriai-litellm-provider-info-sync[bot]
a04ba30f7d
chore(prices): sync OpenRouter prices: 172 models, 2 new
openrouter/~anthropic/claude-fable-latest: supports_web_search
openrouter/~anthropic/claude-haiku-latest: supports_web_search
openrouter/~anthropic/claude-opus-latest: supports_web_search
openrouter/~anthropic/claude-sonnet-latest: supports_web_search
openrouter/~deepseek/deepseek-flash-latest: max_tokens, max_output_tokens, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/~deepseek/deepseek-v4-flash-latest: max_tokens, max_output_tokens, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/~google/gemini-flash-latest: supports_web_search
openrouter/~google/gemini-pro-latest: supports_web_search
openrouter/~moonshotai/kimi-latest: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/~openai/gpt-astra-latest: supports_web_search
openrouter/~openai/gpt-luna-latest: supports_web_search
openrouter/~openai/gpt-mini-latest: supports_web_search
openrouter/~openai/gpt-sol-latest: supports_web_search
openrouter/~openai/gpt-terra-latest: supports_web_search
openrouter/~x-ai/grok-latest: supports_web_search
openrouter/~z-ai/glm-latest: max_tokens, max_output_tokens, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/anthropic/claude-3-haiku: supports_web_search
openrouter/anthropic/claude-fable-5: supports_web_search
openrouter/anthropic/claude-fable-5:batch: supports_web_search
openrouter/anthropic/claude-fable-5.1: supports_web_search
openrouter/anthropic/claude-fable-5.1:batch: supports_web_search
openrouter/anthropic/claude-haiku-4.5: supports_web_search
openrouter/anthropic/claude-haiku-4.5:batch: supports_web_search
openrouter/anthropic/claude-opus-4: supports_web_search
openrouter/anthropic/claude-opus-4.1: supports_web_search
openrouter/anthropic/claude-opus-4.1:batch: supports_web_search
openrouter/anthropic/claude-opus-4.5: supports_web_search
openrouter/anthropic/claude-opus-4.5:batch: supports_web_search
openrouter/anthropic/claude-opus-4.6: supports_web_search
openrouter/anthropic/claude-opus-4.6:batch: supports_web_search
openrouter/anthropic/claude-opus-4.7: supports_web_search
openrouter/anthropic/claude-opus-4.7:batch: supports_web_search
openrouter/anthropic/claude-opus-4.8: supports_web_search
openrouter/anthropic/claude-opus-4.8:batch: supports_web_search
openrouter/anthropic/claude-opus-5: supports_web_search
openrouter/anthropic/claude-opus-5:batch: supports_web_search
openrouter/anthropic/claude-sonnet-4: supports_web_search
openrouter/anthropic/claude-sonnet-4.5: supports_web_search
openrouter/anthropic/claude-sonnet-4.5:batch: supports_web_search
openrouter/anthropic/claude-sonnet-4.6: supports_web_search
openrouter/anthropic/claude-sonnet-4.6:batch: supports_web_search
openrouter/anthropic/claude-sonnet-5: supports_web_search
openrouter/anthropic/claude-sonnet-5:batch: supports_web_search
openrouter/deepseek/deepseek-v4-flash: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/deepseek/deepseek-v4-flash-0731: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/deepseek/deepseek-v4-flash-vision-exp: max_tokens, max_output_tokens
openrouter/deepseek/deepseek-v4-pro: max_tokens, max_output_tokens, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/deepseek/deepseek-v4.1-flash: off_peak_pricing, input_cost_per_token, output_cost_per_token, cache_read_input_token_cost
openrouter/google/gemini-2.5-flash: supports_web_search
openrouter/google/gemini-2.5-flash-image: supports_web_search
openrouter/google/gemini-2.5-flash-lite: supports_web_search
openrouter/google/gemini-2.5-flash-lite:batch: supports_web_search
openrouter/google/gemini-2.5-flash:batch: supports_web_search
openrouter/google/gemini-2.5-pro: supports_web_search
openrouter/google/gemini-2.5-pro-preview: supports_web_search
openrouter/google/gemini-2.5-pro:batch: supports_web_search
openrouter/google/gemini-3-flash-preview: supports_web_search
openrouter/google/gemini-3-flash-preview:batch: supports_web_search
openrouter/google/gemini-3-pro-image: supports_web_search
openrouter/google/gemini-3-pro-image-preview: supports_web_search
2026-09-19 18:31:02 +00:00
yucheng
364d897545 fix(otel v2): map Responses API output onto the Langfuse generation output
Responses API calls build the generation output only from response["choices"],
which Responses payloads do not carry, so Langfuse rendered a blank output.
Fold output[] into one assistant choice (output_text parts concatenated,
function_call and custom_tool_call items as tool_calls) and derive the finish
reason from status when choices are absent. Custom tool call input is now
redacted alongside function call arguments under turn_off_message_logging.

Carries the behavior of #41604 by @moshemorad (issue #41591) onto current
main with typed conversion and single-message output.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:26:59 +00:00
Yujong Lee
cc23e5781e refactor(rust): drop a comment that repeats the reserve doc
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:22:06 +00:00
ryan-crabbe-berri
82fd632153 test(ui): share one chart bar lookup across Top Virtual Keys tests
The key_exists chart test added a second direct DOM lookup for the Recharts bars, which exposes no role or label, and pushed testing-library/no-node-access over its budget (709 > 707). Both chart tests now go through one helper
2026-09-19 11:22:01 -07:00
Yujong Lee
38fa8a7f55 fix(rust): leave the fork gate untouched when a late reservation is refused
reserve() stored fork_only_pid before noticing the runtime already ran under that pid, so a refused reservation still reserved the process: the next enter() cleared the runtime claim and children forked afterwards inherited a dead runtime and hung. Undo the reservation on the error path so the gate is exactly as it was.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:21:32 +00:00
kerry-berri
fc5f6c4b50
Merge pull request #41964 from BerriAI/litellm_registry_audit_20260919
fix(model_prices): drop anthropic deprecation floors and correct azure gpt-4.1-nano retirement date
2026-09-19 11:18:32 -07:00
Yujong Lee
18a1491bd2 test(rust): pin child interpreters to the parent's litellm and lint for it
Children spawned as [sys.executable, -c, ...] put the working directory first on sys.path, so under 'make test-rust-extension' a source checkout shadows the installed wheel and the child imports a litellm with no compiled extension. A shared helper spawns them with -I and asserts the child resolved the same litellm.__file__ as the parent, and a new TQ009 rule flags un-isolated sys.executable spawns.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:17:54 +00:00
yucheng-berri
2542ea0320
Merge pull request #41787 from BerriAI/litellm_llm_guard_proxy_call_types
fix(llmguard): accept proxy async call types
2026-09-19 11:17:21 -07:00
ryan-crabbe-berri
89bf870225 fix(ui): stop Top Virtual Keys from opening keys that are not in the database
/user/daily/activity now reports key_exists on each api key's metadata, true
only when the key is in the active key table that /key/info reads. Top Virtual
Keys renders the Key ID as plain text with an explanatory tooltip and ignores
chart bar clicks when key_exists is false, so deleted keys and CLI/SSO session
keys no longer dead-end on a "Key not found in database" toast
2026-09-19 11:08:21 -07:00
Yujong Lee
1bcd8d704f test: run fork-guard contract subprocesses with python -I
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:08:19 +00:00
Devin AI
107ec2706b style(ui): format Top Virtual Keys aggregation test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 11:04:46 -07:00
Devin AI
88799f6f80 fix(ui): fall back to user id in Top Virtual Keys user column
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 11:04:46 -07:00
Devin AI
e93fe60512 fix(ui): hide Top Virtual Keys user column when rows carry no user
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 11:04:26 -07:00