Commit graph

46528 commits

Author SHA1 Message Date
devin-ai-integration[bot]
558f42e304
fix(proxy): default max_idle_connection_lifetime to 60s on DB URLs (#39134)
* fix(proxy): default max_idle_connection_lifetime to 60s on DB URLs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): regenerate schema.d.ts for database_max_idle_connection_lifetime

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep URL-pinned max_idle_connection_lifetime over config value

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 14:16:51 -07:00
Mateo Wang
deb67ce6e2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_buffered_responses_stream
# Conflicts:
#	tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py
2026-09-01 14:11:29 -07:00
mateo-berri
b0aa1506fc Merge branch 'litellm_internal_staging' into litellm_fix_supported_openai_params_router_alias 2026-09-01 14:11:17 -07:00
ryan-crabbe-berri
2de33555ea style: run ruff format on fallback_event_handlers 2026-09-01 14:08:40 -07:00
mateo-berri
b311c24253 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_azure_chat_anyof_tool_schema 2026-09-01 14:08:34 -07:00
mateo-berri
28d0ac5339 fix(router): guard the declared-provider check for requests without a model 2026-09-01 14:06:14 -07:00
mateo-berri
5192b2162c fix(proxy-extras): schema-qualify the _prisma_migrations logs lookup for non-public Prisma schemas 2026-09-01 14:04:12 -07:00
mateo-berri
0f35328a21 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_azure_chat_anyof_tool_schema
# Conflicts:
#	tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_common_utils.py
2026-09-01 13:59:59 -07:00
mateo-berri
647f32c407 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bump_tornado_658 2026-09-01 13:59:41 -07:00
yuneng-jiang
8bc862f52c
Merge pull request #39129 from BerriAI/litellm_/litellm-issue-39078-c025be
fix(ui): render the logs Tools panel with theme tokens
2026-09-01 13:58:03 -07:00
mateo-berri
848a3edbe8 merge: sync litellm_internal_staging into litellm_rerank_provider_error_body 2026-09-01 13:55:04 -07:00
mateo-berri
1eea8e2831 fix(deps): raise the tornado floor to 6.5.8 for GHSA-8423-8fgw-73vq and GHSA-wwv5-g3v4-889x 2026-09-01 13:54:34 -07:00
Mateo Wang
2c5f429ad4
Merge pull request #39185 from BerriAI/litellm_fix_embedding_encoding_format_suite_break
test: exempt MockTransport request-shape embedding tests from VCR replay
2026-09-01 13:52:51 -07:00
mateo-berri
018d8ae9eb Merge remote-tracking branch 'origin/litellm_internal_staging' into HEAD 2026-09-01 13:49:51 -07:00
Yassin Kortam
aab9abdd1d
fix: keep litellm_credential_name from LiteLLM Params JSON and gate stored credential attach to proxy admins (#39047)
* fix(ui): keep litellm_credential_name from LiteLLM Params JSON when no credential is selected

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): drop null litellm_credential_name from AddModelPanel payload fixture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): validate JSON litellm_credential_name against accessible credentials

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): enforce proxy-admin-only credential attachment on model create/update

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): raise ProxyException for unauthorized credential attach and gate /model/update

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): fold credential-change detection into can_user_attach_credential to satisfy complexity budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): decrypt stored credential name before unchanged-credential comparison

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover credential attach rejection on add_new_model and patch_model

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): annotate proxy-global patches with test-quality suppressions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 13:46:18 -07:00
mateo-berri
692f3b513c fix(proxy-extras): recover the v2 migration resolver from concurrent migrate deploy deadlocks
Two instances racing prisma migrate deploy on one database deadlock on
CREATE INDEX CONCURRENTLY: the victim gets P3018 with 40P01 and the
survivor then sees the failed ledger row as P3009. Both were treated as
unrecoverable, so neither instance came up.

Roll the deadlocked migration's ledger row back and retry the deploy on
P3018, consult the failed row's logs in _prisma_migrations to do the
same on P3009, and retry a deadlock reported without a Prisma error
code. Genuinely broken migrations still fail fast.
2026-09-01 13:43:01 -07:00
Mateo Wang
8ce02c0666
Merge pull request #39184 from BerriAI/litellm_fable_5_1_structured_output 2026-09-01 13:41:22 -07:00
ryan-crabbe-berri
6d6c9af4ab
Merge pull request #39155 from BerriAI/litellm_agent_hub_search
feat(ui): add search to the Agent Hub tab and admin agents table
2026-09-01 13:36:36 -07:00
Mateo Wang
44b595f0bb
Merge pull request #39136 from BerriAI/litellm_lit6611_requested_model_label_cap
fix(prometheus): bound requested_model label cardinality on client failure paths
2026-09-01 13:35:33 -07:00
yuneng-jiang
6028afa2ab
Merge pull request #39178 from BerriAI/litellm_revert_v2_migration_resolver_default
revert: default the proxy back to the v1 migration resolver
2026-09-01 13:32:18 -07:00
mateo-berri
95a0ca3ba5 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_gemini_thinking_content 2026-09-01 13:31:55 -07:00
yucheng-berri
cdb1245e74
fix(s3): bound s3 object keys and download filenames for long Responses API ids (#39164)
* fix(s3): bound object keys and download filenames to s3 limits

Long OpenAI-compatible Responses API ids pushed the s3 object key past s3's
1024 UTF-8 byte cap, so the PUT failed with a 400 and the log record was
dropped. Keys that still fit are unchanged, byte for byte. An oversized one
now keeps a readable head of the file name and appends the sha256 of the full
name. A configured path/alias prefix that is long enough to overflow on its
own keeps whole leading path segments, so a prefix-scoped IAM policy or
lifecycle rule still matches, and ends in a short digest of the full
configured value so two operators do not land in the same folder.

The Content-Disposition filename carried the same unbounded id and hit s3's
2048 byte metadata-header cap, so the upload still failed with
MetadataTooLarge once the key was bounded. It is bounded the same way, head
plus digest, so two records downloaded from the console stay distinct files.

The full response id stays in the uploaded JSON payload.

* fix(s3): keep the configured prefix whole and spend the whole key budget

Shorten the response id first and only trim the operator's configured prefix
when the prefix itself is what does not fit, so prefix scoped IAM policies and
lifecycle rules keep matching. Trim by bytes rather than whole segments so the
longest possible string prefix survives, and route the audit log key through
the same shared builder.

* chore(s3): trim the comments and docstrings the review flagged

Keep the two external facts that are not visible from the code, the 1024 byte
object key cap and the 2048 byte metadata header cap, and drop the rest.
2026-09-01 13:30:02 -07:00
mateo-berri
8b0441a628 fix(vector_stores): block caller-supplied embedding selection params on query surfaces 2026-09-01 13:29:25 -07:00
mateo-berri
6a9dcb5ce6 test: allow dashscope domain in qwen alias default api_base check 2026-09-01 13:29:12 -07:00
mateo-berri
f4347f25de test: exempt MockTransport request-shape embedding tests from VCR replay 2026-09-01 13:29:12 -07:00
Mateo Wang
286a754999
Merge pull request #38839 from BerriAI/litellm_fix_chat_anyof_tool_schema
fix(openai): flatten top-level tool schema combinators on chat completions
2026-09-01 13:25:03 -07:00
mateo
d568bbe58d fix(bedrock): use tool fallback without forced tool_choice for claude-fable-5-1 structured output
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 20:19:46 +00:00
Mateo Wang
c913b09e66
Merge pull request #31725 from Srivatsa03/time-based-cost-pricing
feat(cost): support time-based off-peak pricing in cost calculation
2026-09-01 13:18:30 -07:00
mateo-berri
8b5ae3da9d test(vector_stores): package the suite dir to avoid test_main basename collision 2026-09-01 13:15:51 -07:00
yuneng-jiang
6c26345ab5
Merge pull request #39175 from BerriAI/litellm_ui_select_popup_race
test(ui): pick select options by role instead of by text
2026-09-01 13:05:54 -07:00
mateo-berri
60de5468d7 fix(cost): inherit the backend's raw cost map entry for off-peak-only deployments
Filtering copied fields by name dropped companion billing rules like
web_search_billing_unit and the regional uplift multipliers, so grounding
and uplifts billed differently through the deployment entry. Copy the
backend's raw litellm.model_cost entry wholesale instead, which also
removes the synthesized-zero special case since the raw entry only holds
real values.
2026-09-01 13:05:25 -07:00
mateo-berri
ac53ea0756 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_stream_usage_cost_default 2026-09-01 13:03:18 -07:00
Yuneng Jiang
529ac12ba5
test(ui): drop the helper docblock
The repo does not take explanatory comments. The reason the helper queries by
role lives in the commit that introduced it and in the PR description.
2026-09-01 13:01:46 -07:00
mateo
3e3e4d6970 fix(anthropic): use native structured output for claude-fable-5-1 on Vertex AI and Bedrock Invoke
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 20:01:03 +00:00
Mateo Wang
19ca4cd4a9
Merge branch 'litellm_internal_staging' into litellm_fix_supported_openai_params_router_alias 2026-09-01 12:57:36 -07:00
Yuneng Jiang
fd72a39b1b
revert: default the proxy back to the v1 migration resolver
This reverts merge commit 2b1bd20834 (#31125)

Two CircleCI jobs on the staging-to-main promotion went red the moment
that PR landed. proxy_multi_instance_tests boots two proxies against one
database, and both now race the same migration:

  Error: P3018 A migration failed to apply
  Database error code: 40P01, deadlock detected
  Process 73 waits for ShareLock on virtual transaction 4/11;
  blocked by process 75. Process 75 waits for ExclusiveLock on
  advisory lock [16384,0,72707369,1]; blocked by process 73

Neither proxy comes up, so the job times out after 300s waiting on
localhost:4000. The same wait took 36.5s on the last green run

Timeline: #31125 merged at 18:46:14Z and the failing run started at
18:49:59Z. The merge commit is not an ancestor of the last green
revision (194a3cc) and is an ancestor of the first failing one
(01de2837)

The v2 resolver was meant to avoid exactly this class of contention, so
the deadlock looks like a bug in it rather than a reason to abandon it.
Putting the default back to v1 buys time to fix it without holding up
the release
2026-09-01 12:56:08 -07:00
mateo-berri
babe7816ad fix(rag): store-wins merge, single lookup, allowlisted search params
rag_query reuses the store resolved during authorization instead of a
second registry lookup, merges registry data store-wins so callers
cannot override a managed store's provider or credentials, and logs ids
instead of the merged config, which can carry resolved credentials.
aquery forwards only allowlisted retrieval_config keys to vector store
search, keeping caller-supplied connection overrides like api_base and
api_key away from the search call
2026-09-01 12:52:45 -07:00
mateo-berri
3914de24ef fix(router): route model-less sync vector store calls to the SDK
_generic_api_call_with_fallbacks requires a model, so sync
vector_store_search and vector_store_create raised a TypeError whenever
the call carried no model. Model-less calls now go directly to the SDK
function, with the router injected for search, matching the async
wrapper's behavior
2026-09-01 12:52:45 -07:00
Mateo Wang
8ae072b501
Merge pull request #39147 from BerriAI/litellm_openai_drop_toolless_tool_choice
fix(openai): drop tool_choice when request has no tools on chat completions
2026-09-01 12:50:38 -07:00
Mateo Wang
ab8b8deb39
Merge pull request #39144 from BerriAI/litellm_fix_responses_tool_call_id_shape
fix(responses): tool call id shape breaks gpt-5 -> claude fallback conversations
2026-09-01 12:50:35 -07:00
mateo-berri
0ec3e936b7 fix(cost): inherit the backend's full price structure for off-peak-only deployments
Copying only the flat token rates dropped threshold, tiered, service-tier,
cache, character, and per-second rates from peak-hour billing once cost
lookup switched to the deployment entry, and get_model_info synthesizes
zero flat rates for backends without one, which would have marked
tiered-only backends explicitly priced free. Copy every price-bearing
field instead, deep-copied, rejecting the synthesized zeros the way
_inherit_builtin_tiered_output_rate already does.
2026-09-01 12:47:26 -07:00
Devin AI
537bad67e6 Merge origin/litellm_techdebt_20260901 into litellm_techdebt_20260901
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 19:47:20 +00:00
mateo-berri
d3dab8e294 fix(rerank): map provider errors with the resolved provider on sync and async paths 2026-09-01 12:46:04 -07:00
mateo-berri
4567fc784c fix: close non-message open items as incomplete when a stream is blocked 2026-09-01 12:45:23 -07:00
Yuneng Jiang
116efee3f7
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ui_select_popup_race 2026-09-01 12:42:33 -07:00
Yuneng Jiang
eb53639ecb
test(ui): pick select options by role instead of by text
Clicking a Base UI select entry found by text or by a title attribute is a
race. The text node exists one render before the popup finishes entering,
and until then the positioner still carries pointer-events: none, so
user-event refuses the click and the test throws. Querying by role only
matches once the popup is exposed to the accessibility tree, which is after
that window closes.

Route the 37 remaining select interactions through chooseSelectOption, which
does the role query. Instrumenting the converted files shows the text query
resolving while the popup was still pointer-blocked on 6 of 41 samples; the
role query was never blocked.

Seven files kept their text queries because their popup entries carry no
accessible role, so there is nothing to query by.
2026-09-01 12:42:27 -07:00
Devin AI
818fc5b913 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_2026_09_01 2026-09-01 19:40:50 +00:00
yuneng-jiang
a3e115f4cd
fix(ui): render the guardrail garden detail page with theme tokens (#39131)
The page set its headings, table borders, sidebar labels and tag pills
inline with a fixed light palette (#202124, #5f6368, #dadce0, #f8f9fa,
#fff), so in dark mode it drew dark text on hardcoded white surfaces.

Move those to the foreground/muted/border/card/info tokens, matching
the back link and Create Guardrail button that already used them.
2026-09-01 12:39:43 -07:00
Devin AI
2a26ef0d99 Merge origin/litellm_internal_staging into litellm_techdebt_20260901
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 19:38:19 +00:00
mateo-berri
4c7dd0b522 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_stream_modify_response_chunks
# Conflicts:
#	type-discipline-budget.json
2026-09-01 12:36:48 -07:00