Managed batches mangled any Vertex model that is not a plain publisher
model: a fine-tuned Gemini endpoint id was filed under
publishers/google/models/gemini/<id> at upload, then the batch create
parse dropped the id and targeted the nonexistent publisher model
'publishers/google/models/gemini', which Vertex rejects.
Fine-tuned endpoints are now stored under endpoints/<id> in the GCS
object path, and batch create resolves the endpoint to its deployed
tuned model resource (projects/../models/<id>) via GET endpoints/<id>,
which is the only form the v1 batch API accepts for tuned models. The
cost poller's bare-model parse round-trips the endpoint id so unmanaged
batch spend still maps to the configured deployment.
custom_endpoint deployments have no Vertex batch surface, so batch file
uploads and batch creation against them now return a clear 400 instead
of creating a doomed job.
Resolves LIT-6899
Calling the endpoint without going through FastAPI leaves the new search param set to its Query default object, which is not None, so the grouped-session and request_id lookup tests started taking the search branch
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
A sent null for tpm_limit, rpm_limit, max_budget and the other budget fields
was dropped by a 'v is not None' filter, so update_budget was never called and
the request returned 200 without changing the budget row. Presence is now read
from model_fields_set (merge-patch semantics, matching /v2/organization) and
the nested litellm_budget_table payload no longer drops nulls either
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The search= param on /key/list, /audit, and /spend/logs/ui, plus key_hash= on /key/list, now compare the pasted value verbatim. Only a copied key ID (the hash) matches, so a raw virtual key never needs to travel in a GET query string
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
Keeps staging's response-id keying next to the pass-through failure-path helpers
and types the read-only raw_bytes parameters as Sequence[bytes] so the merged tree
stays inside the lint budgets
With every kept get_configured_mode test using mode "chat", a Router that
answered "chat" for any non-blank configured mode passed all four of them
(the deleted #39630 pair's audio_speech case was the only test catching it).
Read the mode back as audio_speech on an unmapped model so the configured
value itself is what the test checks. Six hand-applied mutations of
Router.get_configured_mode, including that hardcoded-chat one, are now all
killed by the four surviving tests.
Routes without /projects/<project>/locations/<location>/ built the upstream host from the URL's
still-empty location and 500ed even with default_vertex_config set. Build the base URL once after
the configured project and location are applied, drop the hook that re-derived it afterwards, and
answer 400 with a fix-it message when no location is available at all.
Resolves LIT-6905
PRs #39630 and #39634 both added test_get_configured_mode_reads_deployment_model_info
to tests/test_litellm/test_router.py, so the staging tip defines it twice and the
required lint check fails with F811 on every PR synced past 321636ef5d.
Keep the four tests from #39634 (mode read, None for unset or unknown, no wildcard
pattern routing, malformed values treated as absent), which subsume the #39630 pair,
and delete that pair. Five hand-applied mutations of Router.get_configured_mode are
all still killed by the surviving tests.
GET /key/list?search= matches the key hash (a raw sk- key is hashed
first) or a case-insensitive alias substring, and key_hash= now hashes a
raw sk- value too. GET /v1/memory?search= matches a key prefix or an
exact memory_id. GET /audit?search= matches id, object_id, changed_by,
or changed_by_api_key. GET /spend/logs/ui?search= matches request_id
across all time and api_key, team_id, user, end_user, session_id, or
model_id inside the date window; session grouping is skipped while a
search is active.
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
* fix(caching): don't trip redis circuit breaker on short timeout bursts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): scope timeout duration gate to timeout failures and count breaker states per label
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): reset the timeout streak on hard failures so stale timeouts cannot pre-age the duration gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(docker): match USE_DDTRACE case-insensitively and route build_from_pip through prod_entrypoint.sh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(docker): run build_from_pip ENTRYPOINT and CMD through the shipped prod_entrypoint.sh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): resolve OAuth broker endpoints by server_id with IP access checks
Resolve named OAuth lookups through server IDs while retaining client IP checks\n\nCo-authored-by: KK291860 <krishnakumar.kocherykumaran@sephora.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger e2e pipeline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The allowlist check read model from the request body first, while
bedrock_llm_proxy_route dispatches purely on the path model for invoke,
converse, and their streaming variants. A caller could put an allowed
model in the JSON body while targeting a disallowed model in the URL and
slip past the check. count_tokens keeps reading from the body since its
route has no model segment in the path.
get_model_from_request could not resolve a model for /bedrock/... routes since it only checked the JSON body's model field and a small set of URL regexes, none matching Bedrock's passthrough path. This let common_checks skip the key/project model allowlist entirely for any Bedrock passthrough action (invoke, converse, and their streaming variants), while the same model was correctly blocked on /v1/chat/completions
Extract the model from the Bedrock endpoint path using the same helper the passthrough handler itself relies on, so the existing allowlist check applies uniformly across auth methods and call paths
Listing agents (GET /v1/agents and MCP agent_search) treated the absence of any
agent grant on the key or team as permission to see every agent. Non-admin keys
now list only the union of explicit grants, and dashboard sessions resolve that
union through the user's real teams and user row instead of the shared
dashboard team. Proxy admins still see everything and direct access to a named
agent is unchanged.
Resolves LIT-6862
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
