Commit graph

39655 commits

Author SHA1 Message Date
yucheng-berri
096eee0a23
fix(docker): bump wolfi-base digest to patch openssl CVE-2026-34182 (#31133)
Re-pins LITELLM_BUILD_IMAGE and LITELLM_RUNTIME_IMAGE across all 6 Dockerfiles
from the prior digests (openssl 3.6.2-r3) to the current chainguard wolfi-base
digest c61ac691 (openssl 3.6.3-r2, >= the fixed 3.6.3-r0). The runtime stage is
the shipped image, so the runtime digest is what actually resolves the
customer-facing CVE; the build image is bumped too for hygiene. Two Dockerfiles
tracked a second equally-stale digest; both are unified onto the patched one.

(cherry picked from commit fda08dd727)
2026-06-24 17:46:02 -07:00
Yassin Kortam
79cc47d568
fix(passthrough,streaming): recover cost on interrupted and agentic Anthropic streams (#31035)
Streaming and pass-through requests could be logged with $0 cost or dropped from
SpendLogs entirely while the upstream provider still billed every token. This
closes the leak paths not already covered by #30160, #30787 and #30788.

- Catch a stream_chunk_builder raise in the core CustomStreamWrapper (sync and
  async). Large agentic tool-use / thinking streams can make assembly re-raise
  as APIError from inside the except-StopIteration handler, where the sibling
  except does not catch it, so it escaped __next__/__anext__ and dropped the
  request; recover best-effort usage from the raw chunks instead
- Add a usage-only fallback for Anthropic streaming pass-through: when
  stream_chunk_builder returns None or raises, rebuild usage from the
  message_start / message_delta SSE events via AnthropicConfig.calculate_usage so
  cache, web-search and geo tokens are priced instead of left at $0
- Decode buffered pass-through bytes with errors="replace" so a stream cut
  mid-multibyte-sequence still logs the usage events already received
- Record response_cost into model_call_details on the pass-through success path
  (it is read from there, not from kwargs), matching the gemini/cohere/openai
  handlers
- Name the key (alias + masked key) in the virtual-key BudgetExceededError so
  operators don't have to reverse-map spend back to a key

(cherry picked from commit b24b964e04)
2026-06-24 17:45:20 -07:00
Yassin Kortam
a62034d460
fix(proxy): record partial spend on the failure row for interrupted streams (#30788)
A streaming request that breaks mid-flight, for example on a mid-stream read
timeout, still bills the provider for the chunks already delivered, yet the proxy
recorded that interrupted request as a zero-spend failure. An earlier revision
logged the recovered partial usage through the success path, which mislabeled a
failed request as a success and produced a misleading spend row

This recovers the partial usage where the failure is actually logged. The
streaming handler assembles the usage from the chunks seen so far and stashes it,
with its cost, on the logging object before firing the failure handlers. The
proxy failure hook lifts that usage and cost onto request_data before the
non-serialisable logging object is popped, and the spend-log writer records the
real partial spend on the failure row instead of a hardcoded zero;
get_logging_payload honors the recovered usage for the token columns and
_failure_handler_helper_fn preserves the recovered cost so the non-DB failure
loggers stay consistent

A request that recovers via a successful fallback is unaffected: the failure hook
only fires when the whole request fails, so the fallback's combined-usage success
row stays the single source of truth and there is no double counting

Resolves LIT-3825

Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
(cherry picked from commit 4847fa5dd5)
2026-06-24 17:41:54 -07:00
Yassin Kortam
1b45b02c0e
fix(passthrough): recover output tokens for interrupted anthropic streams (#30787)
(cherry picked from commit bd74c62ff1)
2026-06-24 17:39:02 -07:00
yuneng-jiang
d114da1b48
Merge pull request #31170 from BerriAI/litellm_/silly-swirles-75f4e4
chore(ui): rebuild dashboard artifacts for stable/1.89.x
2026-06-23 20:51:19 -07:00
Yuneng Jiang
741044e933
chore: update Next.js build artifacts (2026-06-24 03:48 UTC, node v20.20.2) 2026-06-23 20:48:48 -07:00
yuneng-jiang
1798110aee
Merge pull request #31168 from BerriAI/litellm_backport_1_89_x_bp-nomcp-189-0623
chore(release): backport #31029 to stable/1.89.x and cut 1.89.4
2026-06-23 20:45:52 -07:00
Yuneng Jiang
2e72b266df
chore: refresh uv.lock for 1.89.4 2026-06-23 20:05:17 -07:00
Yuneng Jiang
2722e71072
bump: version 1.89.3 → 1.89.4 2026-06-23 20:05:08 -07:00
ryan-crabbe-berri
9b10543427
feat(mcp): scope a key to zero MCP servers with no-mcp-servers sentinel (#31029)
* feat(mcp): scope a key to zero MCP servers with no-mcp-servers sentinel

A key under a team that has MCP servers had no way to opt out of them;
an empty list has always meant "inherit the team". This adds a
no-mcp-servers sentinel (mirroring no-default-models for models) so a key
can declare an explicit zero that overrides team inheritance, additive
grants, and allow_all_keys servers, surfaced as an exclusive "No MCP
Servers" option in the key create/edit UI.

* refactor(ui): centralize no-mcp-servers sentinel in a shared constant

The sentinel string was defined under two different local names and
inlined in two more files; a single exported constant removes the drift
risk flagged in review.

* fix(mcp): enforce no-mcp-servers sentinel on toolset-scoped routes

Toolset scoping replaced a key's mcp_servers with the toolset's servers,
dropping the no-mcp-servers sentinel, so a key opted out of all MCP could
still execute a granted toolset's tools via /toolset/{name}/mcp. Deny
toolset access when the key carries the sentinel, checked before the admin
branch to match get_allowed_mcp_servers.

(cherry picked from commit 19a29e0579)
2026-06-23 20:00:04 -07:00
yuneng-jiang
33df5891a9
Merge pull request #30888 from BerriAI/litellm_backport_1_89_x_0620
chore(release): backport #30480, #30543, #30542, #30573 to stable/1.89.x and cut 1.89.3
2026-06-20 14:45:08 -07:00
Yuneng Jiang
b4bf258bff
chore: refresh uv.lock for 1.89.3 2026-06-20 12:05:34 -07:00
Yuneng Jiang
d596376311
bump: version 1.89.2 → 1.89.3 2026-06-20 12:05:10 -07:00
ryan-crabbe-berri
ccf892d339
fix(guardrails): return 400 not 500 when AIM blocks a request (#30573)
* fix(guardrails): return 400 not 500 when AIM blocks a request

AIM guardrail blocks raised a bare HTTPException whose type and param
serialized as the literal string "None", which broke OpenAI-SDK error
parsing for downstream consumers. Switching AIM to raise a ProxyException
surfaced a second bug: the shared error funnel re-derived the HTTP status
from a nonexistent status_code attribute and downgraded the 400 to a 500.
The funnel now honors an already-normalized ProxyException rather than
rebuilding it, and ProxyException is excluded from llm_exceptions alerting
so a content-policy block no longer pages on-call as an LLM API failure

Resolves LIT-3751

* fix(guardrails): route all AIM rejection paths through ProxyException

The block-action fix left two AIM rejection paths raising a bare
HTTPException: the multimodal anonymize rejection and the output-side
block. Both serialized type and param as the literal string "None", the
same malformed shape the block fix removed. Funnel all three through a
shared _rejection helper so they return a conformant OpenAI error body.
The output block carries content_policy_violation; the multimodal
rejection stays a plain invalid_request_error because it is a usage
error, not a policy violation

Resolves LIT-3751

* fix(guardrails): record AIM ProxyException blocks in failure logs

Switching AIM blocks from HTTPException to ProxyException made
_is_proxy_only_llm_api_error return False for them, so
_handle_logging_proxy_only_error was skipped and the blocked prompt was
dropped from the configured failure loggers. Classify ProxyException as a
proxy-only error alongside HTTPException so guardrail blocks are recorded
again, matching the prior behavior. The llm_exceptions alert suppression
is a separate check and stays in place

Resolves LIT-3751

* style(guardrails): use str | None over Optional[str] in AIM _rejection

* style(guardrails): collapse AIM _rejection signature per black

(cherry picked from commit b5fcd859be)
2026-06-20 11:58:38 -07:00
Yassin Kortam
9683808366
fix(guardrails): stop re-initializing DB guardrails on every poll (#30542)
* fix(guardrails): stop re-initializing DB guardrails on every poll

InMemoryGuardrailHandler._has_guardrail_params_changed compared the
in-memory LitellmParams against the raw dict loaded from the DB. The
in-memory side carries every field default and coerces enums via
model_dump(), while the DB side only holds the keys originally stored,
so the two shapes never compared equal and the guardrail was rebuilt on
every poll cycle.

Each rebuild created a fresh instance, but delete_in_memory_guardrail
only removed the old callback from litellm.callbacks. Request handling
promotes guardrail callbacks into the success/failure/async lists, so
the previous instance stayed referenced there and instances accumulated.

Normalize both sides through LitellmParams(...).model_dump() before
diffing, and purge the callback from every callback list on delete.

* refactor(guardrails): narrow params-normalization fallback to ValidationError

The comparison normalizer caught a bare Exception and silently fell back
to the raw dict, which hid the cause and quietly degraded the affected
guardrail back to re-initializing on every poll. Catch only the
ValidationError that LitellmParams construction can raise, log a warning
so the offending row is diagnosable, and let any other error surface
instead of being swallowed.

* refactor(callbacks): add remove_callback_from_all_lists helper to manager

Move the knowledge of which callback lists a callback can be promoted
into out of the guardrail registry and into LoggingCallbackManager, where
the rest of the callback-list bookkeeping already lives. delete_in_memory_guardrail
now delegates to the new helper instead of iterating the lists itself.

(cherry picked from commit 9fa74ad8b4)
2026-06-20 11:49:30 -07:00
Yassin Kortam
3ab500c79e
fix(guardrails): run pre_call hook once for model-level guardrails (#30543)
* fix(guardrails): run pre_call hook once for model-level guardrails

A CustomGuardrail attached to a deployment via litellm_params.guardrails
gets its async_pre_call_hook invoked twice per request: once by the proxy
pre-call loop and again by async_pre_call_deployment_hook after the router
spreads the model-level guardrails into the top-level request kwargs.

Record in request metadata that the proxy pre-call loop already ran a given
guardrail, and have the deployment hook skip it when the marker is present.
Direct-SDK usage never runs the proxy loop, so the deployment hook stays the
sole invocation there and still fires exactly once.

The marker key is stripped from untrusted caller metadata so a request body
cannot suppress a model-only guardrail by pre-seeding it.

* fix(guardrails): mark pre_call dedup on the post-hook request data

Record the exactly-once marker after async_pre_call_hook runs, on the data
object that flows downstream, rather than before it. A guardrail whose hook
returns a brand-new request dict (instead of mutating or spreading the one it
received) would otherwise discard the marker, letting the deployment hook
re-run the guardrail a second time.

(cherry picked from commit 4faeabc254)
2026-06-20 11:49:30 -07:00
Shivam Rawat
311be7f2cd
fix(integrations): cap Anthropic cache_control injection at 4 blocks (#30480)
* fix(integrations): cap Anthropic cache_control injection at 4 blocks

Respect Anthropic's 4 cache_control breakpoint limit by counting client-supplied blocks, skipping messages that already carry cache_control, and stopping further auto-injection once the limit is reached.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(integrations): reserve cache slot for tool_config and short-circuit cap

Address review feedback on the cache_control cap: break out of the injection loop before resolving target indices once the limit is reached, and reserve one of the four breakpoint slots when a tool_config injection point is present so the cachePoint appended by the Bedrock transform does not push the total past Anthropic's limit.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit fc9d789d24)
2026-06-20 11:49:18 -07:00
yuneng-jiang
94dae27b0c
Merge pull request #30681 from BerriAI/litellm_backport_1_89_x_0617
chore(release): backport #30380, #30503, #30558, #30130, #30588, #30495, #30690 to stable/1.89.x and cut 1.89.2
2026-06-17 19:22:38 -07:00
Yuneng Jiang
ad758c9788
chore: refresh uv.lock for 1.89.2 2026-06-17 19:02:33 -07:00
Yuneng Jiang
01d3593cbd
bump: version 1.89.1 → 1.89.2 2026-06-17 19:02:32 -07:00
Mateo Wang
0dc6951af6
fix(cost): stop non-string service_tier from silently dropping cost tracking (#30690)
completion_cost read service_tier straight from the request optional_params
and called service_tier.lower() on it, so a non-string value (dict/int/list,
reachable via allowed_openai_params/drop_params) raised AttributeError.
_response_cost_calculator swallowed that and returned response_cost=None, so
the request's cost was silently lost.

The isinstance guard alone is not enough: a surviving dict would crash again
downstream in _get_service_tier_cost_key, which also calls .lower(). A
request-level service_tier is only meaningful for pricing when it is a concrete
billable tier string, so coerce any non-string value to None and defer to the
tier the provider reports on the response usage, the same way "auto" already
does.

Adds a regression test driving a dict service_tier through completion_cost; it
raises AttributeError before the fix and prices at the served tier after.

(cherry picked from commit 43dadc5138)
2026-06-17 18:34:37 -07:00
Shivam Rawat
9e30985290
fix(proxy): resolve list files credentials from team BYOK deployments (#30495)
* fix(proxy): resolve list files credentials from team BYOK deployments

GET /v1/files without target_model_names now prefers the team's own
deployment (model_info.team_id) over shared global provider keys, so JWT
team auth lists files against the correct upstream account.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): scope list files credential lookup to team allowlist

Remove the unrestricted deployment scan that could leak global provider
keys to teams without access, normalize all-proxy-models to the team-scoped
model list, and fix TID251 violations by using dict instead of Dict/Any.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit 6c8b60d50d)
2026-06-17 18:34:37 -07:00
ishaan-berri
5b2477bca1
fix(proxy): list public team model name in /v1/models (#30588)
* fix(proxy): optionally surface public team model name in /v1/models

Behind general_settings.use_team_public_model_name (default False). When
enabled, /v1/models and /models surface the public team_public_model_name
for team-scoped (BYOK) models instead of the internal routing key
model_name_{team_id}_{uuid} -- consistent with /v1/model/info and
OpenAI-compatible. Off by default so the listing's model ids stay
backward-compatible for callers that scripted against the internal name;
routing by the internal name is unchanged regardless of the flag.

Presentation-layer only: access-group, auth, and routing semantics are
unchanged; non-team models are pass-through.

* fix(proxy): default team model listings to public names

* test(proxy): cover team model listing metadata

* test(proxy): cover empty team listing deployments

* refactor(proxy): simplify team model listing translation

* fix(proxy): resolve public team model name on GET /v1/models/{id}

The listing endpoints advertise team_public_model_name, but the retrieve
endpoint validated and looked up by the raw id, so a public name 404'd.
Resolve the public name back to the internal routing key (scoped to the
caller's accessible models so colliding names never cross teams), look up
by it, and echo the public name back as the response id.

* test(proxy): cover public-name resolution on model retrieve

* refactor(proxy): extract team model-name translation into TeamModelNameTranslator

Move the team-scoped (BYOK) listing/retrieve name translation out of
proxy_server.py into a dedicated common_utils module. Static methods with
general_settings injected so the logic is unit-testable without globals and
proxy_server.py stays thin.

* refactor(proxy): use TeamModelNameTranslator in model_list and model_info

* test(proxy): target TeamModelNameTranslator for model-name translation

* fix(proxy): type create_model_info_response return as dict[str, object]

* fix(proxy): keep internal routing key for team model listing metadata lookup

Add listing_entries returning (public response id, internal lookup id) so
include_metadata=true resolves fallbacks against the routing key the router
indexes by, instead of the translated public name (which never matches).

* fix(proxy): build /v1/models metadata from internal key, show public id

* test(proxy): cover team listing fallback metadata via internal key

* fix(proxy): use builtin dict generics in create_model_info_response (UP006)

---------

Co-authored-by: Tushar More <tusharmore8408@gmail.com>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
(cherry picked from commit 60f4c01b74)
2026-06-17 18:34:07 -07:00
T. Kobayashi
98636c7d6e
feat: add opt-in healthy_only filter to GET /v1/models (#30130)
* feat: add opt-in healthy_only filter to GET /v1/models

Adds an opt-in `healthy_only=true` query parameter to GET /v1/models and
GET /models that hides models whose backing deployments are all marked
unhealthy by background health checks.

- Add Router.async_get_fully_unhealthy_model_names(), mirroring the
  semantics of get_fully_blocked_model_names(): a model is hidden only
  when every backing deployment is unhealthy and the health state is
  not stale (fail open otherwise).
- Reuses the existing DeploymentHealthCache populated by
  _run_background_health_check(), so no new health state is introduced.
- No-op when allowed_fails_policy is set, mirroring
  _async_filter_health_check_unhealthy_deployments semantics.
- team_public_model_name aliases are aggregated alongside model_name.
- Hiding is presentation-only; default behavior is unchanged.

Fixes #30128

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: address Greptile review notes

- Note team-alias asymmetry vs get_fully_blocked_model_names
- Debug-log when healthy_only is set but no health state is available

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 9dd9d2322a)
2026-06-17 18:33:56 -07:00
Mateo Wang
0c40c1e572
fix(anthropic): price and surface response service_tier in cost tracking (#30558)
(cherry picked from commit b638bc2248)
2026-06-17 18:33:56 -07:00
Shivam Rawat
f8831f3720
fix(proxy): allow internal roles to access vector store CRUD routes (#30503)
Add bare /v1/vector_stores/{vector_store_id} to openai_routes so retrieve, update, and delete classify as LLM API routes for internal user and internal viewer roles.

Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit 902122a06b)
2026-06-17 18:33:24 -07:00
Yassin Kortam
ca7d360d87
fix(otel): record full error message on standard exception event in otel v2 (#30380)
The v2 span engine only stamped error.type and stuffed the message into the
span status description; it never recorded the standard OTel exception event.
Backends that dynamic-map unknown string fields (e.g. Elasticsearch) index the
message as a keyword capped at ignore_above:1024, truncating it. Emit the full
message under the recognized exception.message semconv field via a span event so
it is mapped as full text instead.

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit 3b84150137)
2026-06-17 18:33:24 -07:00
yuneng-jiang
ba5e46b709
Merge pull request #30703 from BerriAI/litellm_189x_ui_build
chore(ui): rebuild ui
2026-06-17 17:32:59 -07:00
Yuneng Jiang
44931e81dc
chore: update Next.js build artifacts (2026-06-18 00:28 UTC, node v20.20.2) 2026-06-17 17:28:27 -07:00
yuneng-jiang
39832310a7
Merge pull request #30502 from BerriAI/litellm_backport_1_89_x_bp_1_89x_multi
chore(release): backport 1.84.8 patch set + MCP/model-info/DB fixes to stable/1.89.x and cut 1.89.1
2026-06-15 20:08:03 -07:00
Yuneng Jiang
24e30b551f
fix(types): coerce dict server_tool_use to ServerToolUse in Usage init
Prerequisite for #27346 (completion_cost AttributeError on streaming
Anthropic web_search). #27346's per-chunk coercion is undone by the
existing Usage(**returned_usage.model_dump()) reconstruction in
calculate_usage, which round-trips server_tool_use back to a plain dict;
without this Usage.__init__ coercion the cost path still does attribute
access on a dict and raises. The same prerequisite was bundled into the
stable/1.88.x backport of #27346 (24b9655cd4).

Content-verified present on litellm_internal_staging via aggregator
32c88ca74f (Litellm oss staging 080626, #29932); this restores only the
two-line coercion, not the rest of that aggregator.

(cherry picked from commit 32c88ca74f)
2026-06-15 19:03:23 -07:00
Yuneng Jiang
9bd10221bc
bump: version 1.89.0 -> 1.89.1
Regenerate uv.lock (pypdf 6.13.2, tornado/aiohttp constraints from #30220)
and ui/litellm-dashboard/package-lock.json (vitest 3.2.6). Pin
brace-expansion to 5.0.6 via an explicit package.json override so the
line's lockfile resolves the patched transitive version that #30220
ships on staging
2026-06-15 18:45:46 -07:00
yuneng-jiang
9dcd406769
chore(deps): bump vitest, brace-expansion, pypdf and tornado (#30220)
(cherry picked from commit d96ab467f1)

Backport adaptation: applied only the pyproject.toml and
ui/litellm-dashboard/package.json manifest hunks; uv.lock and
package-lock.json are regenerated on this line in a separate commit
rather than cherry-picked, and the package.json devDependencies are
unioned with this line's @types/uuid and vite entries
2026-06-15 18:40:21 -07:00
ryan-crabbe-berri
7771ec96ee
fix(mcp): drop phantom 401 span on delegated OAuth2 tool calls (#30494)
The OAuth2 passthrough ran user_api_key_auth on the client's upstream bearer
first and only recovered after the failed validation had already logged a 401
auth event to the tracer, so successful tool calls to a delegated server each
carried a phantom 401 span. Check delegate_auth_to_upstream before validating:
a delegated server skips the doomed call entirely so nothing is logged, and a
non-delegated server validates normally and surfaces a real 401 rather than
being exchanged for an anonymous upstream-passthrough session.

(cherry picked from commit 039a2d8bf5)
2026-06-15 18:37:37 -07:00
yuneng-jiang
bcfed3e75d
fix(proxy): return deprecated-key lookup result directly in get_data combined view (#30327)
The grace-period branch assigned the recursive get_data result (a
finished LiteLLM_VerificationTokenView) back into the variable that the
combined-view dict normalization then subscripts, raising TypeError on
every request made with a rotated key inside its grace window; auth
surfaced that as a 401. Return the recursive result directly instead.

Regression test drives the full get_data flow: old hash misses the view,
deprecated table resolves to the active token, and the call must return
the view object

(cherry picked from commit 5047eaf7f0)
2026-06-15 18:37:37 -07:00
Sameer Kankute
bf6f134c24
fix(passthrough): skip [DONE] sentinels and non-JSON SSE frames in Anthropic streaming logging
Targeted subset of staging commit cfcdf8714a (#30202): only the
anthropic_passthrough_logging_handler.py hardening hunks and their four
tests are taken; the rest of that staging batch is intentionally excluded.

(cherry picked from commit cfcdf8714a)
(cherry picked from commit 973c7eb8d6)
2026-06-15 18:37:37 -07:00
Sameer Kankute
929f047c83
fix(proxy): populate access_via_team_ids on /v1/model/info (#30274)
* fix(proxy): populate access_via_team_ids on /v1/model/info

Team metadata enrichment previously only ran on /v2/model/info with
include_team_models=true, leaving /v1/model/info without
access_via_team_ids for project model-picker flows.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(dashboard): sync OpenAPI schema for /v1/model/info query params

Add include_team_models and teamId to the generated schema for /model/info
and /v1/model/info after the proxy endpoint gained team-access filtering.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): always return direct_access on /v1/model/info

Set direct_access to true or false on every enriched model so clients
can filter without treating a missing field as ambiguous.

Co-authored-by: Cursor <cursoragent@cursor.com>

* perf(proxy): fail fast when teamId is set without a connected DB on /v1/model/info

Raise the db_not_connected error before building, enriching, and translating the model list instead of after, so a teamId query against a proxy with no database no longer wastes the full enrichment pipeline.

* fix(proxy): fail fast when include_team_models is set without a database

include_team_models=True relies on _populate_team_access_on_models to set
direct_access/access_via_team_ids, which only runs when a database is connected.
Without one, _filter_models_to_user_accessible discarded every model and the
endpoint returned an empty list with HTTP 200. Mirror the teamId guard so the
request fails fast with a clear db_not_connected error before any model-list work.

* fix(proxy): populate direct_access on single-model /model/info lookup

The /v1/model/info list path populates model_info.direct_access (and
access_via_team_ids) when a database is connected, but the
litellm_model_id single-model lookup returned early without it. This
made the two endpoints disagree, breaking the parity assertion in
test_get_specific_model. Run the same population on the single-model
path so both responses match.

* fix(proxy): apply no-DB fast-fail before litellm_model_id branch

The teamId/include_team_models no-DB guard sat after the litellm_model_id
early return, so ?litellm_model_id=X&teamId=Y with no DB returned 200 with
unpopulated access fields instead of the 500 raised on every other path.
Move the guard ahead of the branch so the fast-fail is uniform.

* fix(proxy): apply teamId/include_team_models filters on single-model lookup

The litellm_model_id early-return branch in model_info_v1 populated the
team access fields but returned before the teamId and include_team_models
filters ran, so a single-model lookup surfaced the deployment regardless
of team access when the DB was connected. Run both filters on the
single-model list before returning so the documented query params behave
the same with and without litellm_model_id.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 7d1f68e72a)
2026-06-15 18:33:45 -07:00
Yassin Kortam
ddc4da55ff
fix(passthrough): resolve costing model when body model is unknown (#30160)
(cherry picked from commit 1828a7c6f0)
2026-06-15 18:33:44 -07:00
Aanchal Khandelwal
7a72b1062d
Merge pull request #29528 from aanchal22/litellm_byok-alias-merge
fix(proxy): atomic merge for team model aliases and team.models on BYOK create

(cherry picked from commit ac7c2dc0d7)
2026-06-15 18:32:16 -07:00
Yassin Kortam
b337b63472
fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures (#29986)
(cherry picked from commit da9d64b4de)
2026-06-15 18:22:38 -07:00
ishaan-berri
dae738876f
fix: completion_cost AttributeError on streaming Anthropic web_search responses (#26153) (#27346)
* fix: coerce server_tool_use dict to ServerToolUse in Usage.__init__ (#26153)

* fix: coerce server_tool_use to ServerToolUse in stream_chunk_builder (#26153)

* fix: dict/pydantic-tolerant access in tool_call_cost_tracking (#26153)

* fix: dict/pydantic-tolerant access in anthropic cost_calculation (#26153)

* test: assert ServerToolUse type in existing stream_chunk_builder anthropic web search test

* test: regression test for #26153 (stream_chunk_builder server_tool_use type)

* test: dict/pydantic safety for tool_call_cost_tracking helper

* test: dict/pydantic safety for anthropic web_search cost

* refactor: consolidate _get_web_search_requests into shared cost-calc utils

* test(realtime): use gpt-realtime; openai retired gpt-4o-realtime-preview

OpenAI shut down the gpt-4o-realtime-preview family (incl. the undated
alias) on 2026-05-07, causing the live realtime test to fail with a
4000 invalid_request_error.invalid_model close. gpt-realtime is the GA
successor; switch the live-call tests to it, matching the base branch.

* refactor(types): drop redundant server_tool_use coercion in Usage.__init__

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 4a3860df1f)
2026-06-15 18:22:38 -07:00
Aanchal Khandelwal
8f4b57c824
feat(datadog): add team-scoped Datadog callback support (#29947)
Enable teams to configure their own Datadog credentials via
POST /team/{team_id}/callback, following the same pattern as Langfuse.

(cherry picked from commit f5e6012ab0)
2026-06-15 18:22:38 -07:00
Sameer Kankute
e2ab316a84
fix(proxy): align /v1/model/info with router deployments (#30025)
* fix(proxy): align /v1/model/info with router deployments

Return router model_list entries (including team-scoped models) with team
access metadata instead of wildcard-expanded names from get_complete_model_list.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): gate v1 team filter and honor key allowlists

Only apply get_all_team_and_direct_access_models for admin or user-bound
keys, then intersect with key/team model restrictions to avoid empty lists
for service tokens and metadata leaks for restricted keys.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): skip v1 team filter when user row is missing

Require a DB-backed user before applying team-access filtering on
/v1/model/info, and skip the trailing filter in get_all_team_and_direct_access_models
when user context cannot be resolved.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Revert "fix(proxy): skip v1 team filter when user row is missing"

This reverts commit 74e1fbd77a.

* fix(proxy): restore legacy v1 model access filtering

Keep /v1/model/info on key/team allowlists instead of DB team-membership
filtering, while still listing router deployments for team-scoped models.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): drop A2A agent entries from public /v1/model/info list

* fix(proxy): scope team BYOK rows on /v1/model/info to caller's teams

Listing the full router model_list let any authenticated key without
explicit model restrictions enumerate other teams' BYOK deployments
(public name, team_id, api_base) via /v1/model/info. Reuse the existing
_get_caller_byok_team_scope check so non-admin callers only see global
deployments plus their own team's BYOK rows; admins keep the full view.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 6068bb7781)
2026-06-15 18:22:23 -07:00
Yassin Kortam
38019d87ba
feat(proxy): add option to disable server-side prepared statements for DB lookups (#29984)
(cherry picked from commit dff25fef44)
2026-06-15 18:22:23 -07:00
Yassin Kortam
4605d14473
fix(proxy): recover from cached-plan errors by reconnecting the Prisma client (#29983)
(cherry picked from commit 3bd3951e37)
2026-06-15 18:21:46 -07:00
tin-berri
03e501df71
fix(mcp): drop orphaned per-user credential rows when an MCP server is deleted (#30141)
(cherry picked from commit 1436ee9092)
2026-06-15 18:21:45 -07:00
Sameer Kankute
322c589a04
fix(proxy): authorize batch files using upload target_model_names (LIT-3593) (#30009)
* fix(proxy): authorize batch files using upload target_model_names (LIT-3593)

After replace_model_in_jsonl, body.model is a stripped provider id. Reverse-mapping it via resolve_model_name_from_model_id is first-match on model_list and caused false 403s when multiple deployments share the same stripped name. Use target_model_names from the unified file id instead.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): restore resolve_model_name_from_model_id for JSONL fallback path (LIT-3593)

Restores the reverse-lookup for the JSONL body.model fallback path so that
legacy/pre-target_model_names managed files still map stripped provider IDs
back to proxy aliases before auth. Also cleans up redundant `or None`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Revert "fix(proxy): restore resolve_model_name_from_model_id for JSONL fallback path (LIT-3593)"

This reverts commit 30d2e96f77.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
(cherry picked from commit 2cd7e87485)
2026-06-15 18:21:45 -07:00
tin-berri
5204ab5639
fix(mcp): allow team access-group grants in OAuth authorize/token access check (#30041)
* fix(mcp): honor team access-group grants in OAuth authorize/token access check

* test(mcp): mock build_effective_auth_contexts in non-admin authorize tests for isolation

(cherry picked from commit 5b7063d194)
2026-06-15 18:21:45 -07:00
tin-berri
acd0d42228
fix(ui/mcp): reset OAuth state on create-server modal close so a prior server's token no longer leaks into the next add-server session (#30000)
* fix(ui/mcp): reset OAuth hook state on modal close so a prior server's token no longer leaks into the next add-server session

* fix(ui/mcp): clear in-flight OAuth guard on reset and reset form/tools on modal close so nothing leaks on a parent-driven dismiss

(cherry picked from commit d8fe091938)
2026-06-15 18:21:45 -07:00
tin-berri
b417e858f9
fix(mcp): load MCP tool configuration tools via the OBO/passthrough-aware GET path (#29960)
* fix(ui): load MCP tool configuration tools via the OBO/passthrough-aware GET path

* fix(mcp): admin-only include_disabled_tools so the settings UI shows toggled-off tools

* fix(ui): repopulate MCP server edit form when server data loads after mount (OAuth return)

* fix(ui): persist MCP OAuth token on save and return to the Settings tab after authorize

* fix(ui): scope MCP OAuth callback to the initiating form so create and edit flows don't cross-talk

* fix(ui): derive OAuth-return Settings tab via lazy state init instead of setState-in-effect

* Fix MCP OAuth edit token handling

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
(cherry picked from commit 51ba6e39cd)
2026-06-15 18:21:29 -07:00