Reading _target_order with .get left it in the request kwargs after selection, and only
nine provider boundaries stripped it. _atext_completion and _aadapter_completion spread
the raw kwargs, so an order-2 hop on /completions sent _target_order upstream, which
real providers reject as an unknown argument. Popping at selection strips it for every
path in one place; the PR's retry-keeping test already passed with pop because each retry
hands the callee its own kwargs copy.
Claude-Session: https://claude.ai/code/session_01XKkTFa6g7Rmd6vtHL91GMn
Two P0 rows in the reliability coverage registry had no test.
reliability.retry.timeout.succeeds_within_retries gets a new file. The model
group is a pair: an always-timing-out deployment holding all of the group's
shuffle weight, and a healthy backup at weight 0. The weighted pick always opens
on the timing-out one, its first Timeout benches it via an allowed_fails_policy
of TimeoutErrorAllowedFails 0, and the retry falls through to the only
deployment left, so the outcome is a completion plus a reported retry with no
random first pick in the middle.
reliability.fallback.context_window.routes_to_fallback joins the existing
fallbacks spec. It registers a genuinely small-context OpenAI deployment, sends
a prompt past its limit so the provider refuses it on length, and reroutes with
context_window_fallbacks, which is the setting that handles that refusal rather
than plain fallbacks.
Both drive real provider calls through router_settings_override, so no config
change and no second proxy is needed. Reliability & Performance goes 16/36 to
18/36.
Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
get_llm_provider runs the OAuth device flow for github_copilot and chatgpt,
so calling it on the event loop before the executor dispatch let an
authenticated caller block the loop for the length of the polling window.
Adopt the declared provider via declared_authenticating_provider, matching
the metadata callers in utils.py, and only resolve for everything else.
* fix(mcp): follow tools/list pagination from upstream servers
Adopts BerriAI/litellm#32244 by Jupiter363 onto litellm_internal_staging
with merge conflicts resolved
* fix(mcp): degrade buggy pagination to partial results and bound the preview walk
A repeated nextCursor now returns the tools collected so far instead of
discarding every page with a RuntimeError, an empty-string cursor is treated
as terminal, load_mcp_tools shares the same pagination walk instead of
returning only the first page, and the tools/list preview is bounded by the
listing timeout instead of only the per-request timeout times the page cap
* fix(mcp): annotate deliberate rebind for the preview timeout scope
* fix(mcp): bound the shared pagination walk with an overall listing deadline
The per-request session read timeout restarts on every page, so direct SDK
callers of list_tools and load_mcp_tools could run up to the page cap with
no overall bound. The walk now returns the tools collected so far when
max(MCP_CLIENT_TIMEOUT, MCP_TOOL_LISTING_TIMEOUT) expires
* fix(mcp): let a per-server timeout extend the pagination deadline
MCPClient carries a per-server timeout that can exceed the global default;
list_tools now passes max(self.timeout, MCP_TOOL_LISTING_TIMEOUT) into the
shared walk so a deliberately slow server is not silently truncated at the
global deadline
* fix(mcp): honor per-server timeouts in the preview deadline and test the walk sessionless
The preview deadline now extends with the created client's own timeout, and
the pagination walk's cap, repeated-cursor, and empty-cursor cases are tested
directly against the helper instead of through patched SDK internals
* fix(mcp): forward the preview request's per-server timeout to the temporary server model
The tools preview built its temporary MCPServer without the request's
timeout field, so the client factory always fell back to the global
default and a per-server timeout could never extend the preview's
listing deadline (or its per-request timeout).
* fix(proxy): default max_idle_connection_lifetime to 60s on DB URLs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): regenerate schema.d.ts for database_max_idle_connection_lifetime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep URL-pinned max_idle_connection_lifetime over config value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep litellm_credential_name from LiteLLM Params JSON when no credential is selected
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drop null litellm_credential_name from AddModelPanel payload fixture
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): validate JSON litellm_credential_name against accessible credentials
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): enforce proxy-admin-only credential attachment on model create/update
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): raise ProxyException for unauthorized credential attach and gate /model/update
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): fold credential-change detection into can_user_attach_credential to satisfy complexity budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): decrypt stored credential name before unchanged-credential comparison
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover credential attach rejection on add_new_model and patch_model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): annotate proxy-global patches with test-quality suppressions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Two instances racing prisma migrate deploy on one database deadlock on
CREATE INDEX CONCURRENTLY: the victim gets P3018 with 40P01 and the
survivor then sees the failed ledger row as P3009. Both were treated as
unrecoverable, so neither instance came up.
Roll the deadlocked migration's ledger row back and retry the deploy on
P3018, consult the failed row's logs in _prisma_migrations to do the
same on P3009, and retry a deadlock reported without a Prisma error
code. Genuinely broken migrations still fail fast.
* fix(s3): bound object keys and download filenames to s3 limits
Long OpenAI-compatible Responses API ids pushed the s3 object key past s3's
1024 UTF-8 byte cap, so the PUT failed with a 400 and the log record was
dropped. Keys that still fit are unchanged, byte for byte. An oversized one
now keeps a readable head of the file name and appends the sha256 of the full
name. A configured path/alias prefix that is long enough to overflow on its
own keeps whole leading path segments, so a prefix-scoped IAM policy or
lifecycle rule still matches, and ends in a short digest of the full
configured value so two operators do not land in the same folder.
The Content-Disposition filename carried the same unbounded id and hit s3's
2048 byte metadata-header cap, so the upload still failed with
MetadataTooLarge once the key was bounded. It is bounded the same way, head
plus digest, so two records downloaded from the console stay distinct files.
The full response id stays in the uploaded JSON payload.
* fix(s3): keep the configured prefix whole and spend the whole key budget
Shorten the response id first and only trim the operator's configured prefix
when the prefix itself is what does not fit, so prefix scoped IAM policies and
lifecycle rules keep matching. Trim by bytes rather than whole segments so the
longest possible string prefix survives, and route the audit log key through
the same shared builder.
* chore(s3): trim the comments and docstrings the review flagged
Keep the two external facts that are not visible from the code, the 1024 byte
object key cap and the 2048 byte metadata header cap, and drop the rest.
Filtering copied fields by name dropped companion billing rules like
web_search_billing_unit and the regional uplift multipliers, so grounding
and uplifts billed differently through the deployment entry. Copy the
backend's raw litellm.model_cost entry wholesale instead, which also
removes the synthesized-zero special case since the raw entry only holds
real values.
This reverts merge commit 2b1bd20834 (#31125)
Two CircleCI jobs on the staging-to-main promotion went red the moment
that PR landed. proxy_multi_instance_tests boots two proxies against one
database, and both now race the same migration:
Error: P3018 A migration failed to apply
Database error code: 40P01, deadlock detected
Process 73 waits for ShareLock on virtual transaction 4/11;
blocked by process 75. Process 75 waits for ExclusiveLock on
advisory lock [16384,0,72707369,1]; blocked by process 73
Neither proxy comes up, so the job times out after 300s waiting on
localhost:4000. The same wait took 36.5s on the last green run
Timeline: #31125 merged at 18:46:14Z and the failing run started at
18:49:59Z. The merge commit is not an ancestor of the last green
revision (194a3cc) and is an ancestor of the first failing one
(01de2837)
The v2 resolver was meant to avoid exactly this class of contention, so
the deadlock looks like a bug in it rather than a reason to abandon it.
Putting the default back to v1 buys time to fix it without holding up
the release
rag_query reuses the store resolved during authorization instead of a
second registry lookup, merges registry data store-wins so callers
cannot override a managed store's provider or credentials, and logs ids
instead of the merged config, which can carry resolved credentials.
aquery forwards only allowlisted retrieval_config keys to vector store
search, keeping caller-supplied connection overrides like api_base and
api_key away from the search call
