gpt-5.4-pro and gpt-5.4-pro-2026-03-05 do not support the
/v1/chat/completions endpoint — OpenAI returns a 404 with
"This is not a chat model". These models are responses-only,
like o3-pro and o1-pro.
Changes:
- Set mode from "chat" to "responses" for both model entries
- Update supported_endpoints to ["/v1/responses", "/v1/batch"]
- Add regression test for responses API bridge routing
FixesBerriAI/litellm#23014
PR #20950 added extra_body forwarding to Vertex AI Gemini. LiteLLM-internal
keys (cache, tags) were being merged into the request body, causing Vertex AI
to reject with 400: 'Unknown name "cache": Cannot find field.'
- Add _LITELLM_INTERNAL_EXTRA_BODY_KEYS frozenset (cache, tags)
- Skip these keys in _pop_and_merge_extra_body before merging
- Add regression tests for cache and tags stripping
Fixes regression from 1.79.3 → 1.81.12 when using proxy cache with
extra_body={"cache": {"use-cache": True, "ttl": 86400}}
Made-with: Cursor
Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call
Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.
This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().
Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).
Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.
Related issues: #11804, #11029, #6836, #1782, #151
Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.
The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.
Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.
Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.
Related issues: #12020, #64, #1671
AgentsPanel no longer renders AgentCardGrid — it uses a Table directly.
The two tests that looked for agent-card-grid are updated to instead verify
the Actions column header appears for admins and is absent for non-admins.
When calling get_users() directly (not via FastAPI), Query() defaults
are not resolved. Pass organization_ids=None explicitly to avoid
'Query' object has no attribute 'split' error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The `health_check` Query() call was missing its closing parenthesis,
causing a SyntaxError that blocked all proxy_server imports.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- test_get_users_*: pass proxy admin user_api_key_dict since get_users
now calls _authorize_user_list_request which checks user_role
- test_validate_team_member_add_permissions_non_admin: set
organization_id on mock team since _is_user_org_admin_for_team
accesses it
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- _authorize_and_filter_teams now uses Prisma WHERE clause
(organization_id IN ...) instead of fetching the full team table
- Add missing Switch import in agents.tsx (pre-existing build error)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract _authorize_user_list_request and _authorize_and_filter_teams
helpers to reduce statement count in get_users and list_team. Remove
unused _is_user_team_admin import.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When using extended thinking with web search, Anthropic interleaves thinking
blocks between server_tool_use/tool_result blocks. The previous code prepended
all thinking blocks first, then appended tool calls last, breaking Anthropic's
thinking block signature verification on round-trip.
This change detects when both thinking_blocks and server tool calls (srvtoolu_*)
are present, and interleaves them in the original order: each thinking block
precedes its corresponding server tool use group. This preserves the signature
positions that Anthropic validates.
Fixes#23047
- Thread org objects {organization_id, organization_alias} instead of bare IDs from
users/page.tsx → view_users.tsx → CreateUserButton so the selector can show aliases
- Replace single-select org dropdown with multi-select; always shown when organizationIds
is non-null; disabled/pre-selected for single-org admins; displays "Alias (id)"
- handleCreate: maps organization_ids → organizations before POST, removes redundant
organizationMemberAddCall (backend _add_user_to_organizations handles it)
- _user_is_org_admin: also checks organizations list field in addition to singular
organization_id so /user/new succeeds for org admins
- Add 5 backend unit tests for _user_is_org_admin and 2 frontend tests for new form behavior
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Org admins and team admins opening the invite-user modal could not see
the 4 global proxy roles because GET /user/available_roles has no
request body, so the org-admin route check (which requires
organization_id in the payload) always returned False and blocked them.
Add /user/available_roles to self_managed_routes so the route-access
check passes for any authenticated user. The endpoint's existing
Depends(user_api_key_auth) still requires a valid API key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(bedrock): strip output_config from Bedrock Invoke requests
Bedrock Invoke API does not support the output_config parameter
(added to Anthropic Messages API). Requests with output_config cause
400 errors: 'extraneous key [output_config] is not permitted'.
Strip output_config in both Bedrock Invoke transformation layers
(messages and chat), consistent with how output_format is already
handled and how VertexAI strips both parameters.
Fixes: https://github.com/BerriAI/litellm/issues/22797
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(bedrock): add output_config test for chat/invoke path
Addresses review feedback — the chat/invoke_transformations path now has
symmetric test coverage matching the messages/invoke_transformations path.
Fixes: https://github.com/BerriAI/litellm/issues/22797
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: giulio-leone <6887247+giulio-leone@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: enforce x-litellm-trace-id in header, if required
* feat: update spend for agent
* refactor: update agent table to follow similar format as other entities - also add a spend column - allows us to see spend of an agent
* fix: cleanup ui
* feat: return spend on agent endpoints
* feat: scope pr
* feat(agents/): support budgets + rate limiting on agents + agent sessions
* fix: address PR review feedback
- Add missing tpm_limit, rpm_limit, session_tpm_limit, session_rpm_limit
columns to root schema.prisma to match proxy and extras schemas
- Add backwards-compatible fallback to key metadata for max_iterations
so existing users don't silently lose enforcement
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: qa'ed RPM limiting on agents
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add health check toggle to agents page
Backend:
- Add health_check query parameter to GET /v1/agents endpoint
- When health_check=true, performs concurrent GET requests to each agent's
URL and filters out agents with unreachable URLs (5s timeout)
- Agents returning HTTP <500 are considered healthy; 5xx and connection
errors mark agents as unhealthy
UI:
- Add Health Check toggle (Switch) to agents panel header
- Toggle triggers re-fetch with health_check=true, filtering the agent list
- Icon color changes (green/gray) to indicate toggle state
- Tooltip explains behavior: 'only agents with reachable URLs are shown'
Networking:
- Update getAgentsList to accept optional healthCheck boolean parameter
Tests:
- Backend: 9 new tests covering health check filtering, _check_agent_url_health
helper (no URL, 200, 404, 500, connection error cases)
- UI: 3 new tests verifying toggle renders, initial fetch without health check,
and fetch with health check after toggle click
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix: fix greptile comment re: security issue
* fix: fix based on greptile feedback
* fix: align health check tests with implementation
- Rename test_should_return_unhealthy_when_no_url to
test_should_return_healthy_when_no_url (implementation returns
healthy=True for agents without a URL)
- Patch get_async_httpx_client instead of httpx.AsyncClient so mocks
actually intercept the HTTP calls made by _check_agent_url_health
- Remove unnecessary __aenter__/__aexit__ context-manager mocks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: undo _experimental/out renames from cherry-pick
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update litellm/proxy/agent_endpoints/endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(docker): bump tar/minimatch/pypdf for CVE fixes + harden SBOM patching
- Bump tar 7.5.8→7.5.10, minimatch 10.2.1→10.2.4, pypdf 6.6.2→6.7.3
- Add sed-based SBOM metadata patching with properly indented find/sed
- Add npm package manager cleanup (apk del / apt-get purge) to remove
stale SBOM entries from image scanners
- Scope || true to only apk del via brace grouping { ... || true; }
- Guard npm root -g with non-empty assertion to prevent silent failures
- Scope minimatch sed regex to ^10.x to avoid matching other major versions
Addresses: CVE-2026-27903, CVE-2026-27904, GHSA-qffp-2rhf-9h96, CVE-2026-27888
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(docker): scope find to /usr/local/lib /usr/lib, drop autoremove
- Replace `find /` with `find /usr/local/lib /usr/lib` to avoid
traversing /proc, /sys, /dev during SBOM metadata patching
- Remove `apt-get autoremove -y` from Debian-based Dockerfiles to
prevent nodejs from being removed as an auto-installed dependency
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
UI sends organization_ids=org1,org2,org3 and backend parses the
comma-separated string, matching the existing pattern used elsewhere.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Backend authorization: enforce server-side org scoping on /user/list.
Non-proxy-admin callers must be org_admin; their permitted org IDs are
fetched from the DB and intersected with any requested org filter.
- Race condition: gate userListQuery on org data being loaded for
non-proxy-admin users (undefined = loading, null = no filter needed).
- Silent failure: surface organizationMemberAddCall errors via
NotificationsManager instead of only console.error.
- Multi-org: org admins managing multiple orgs can choose which org to
add the new user to via a dropdown (defaults to first org).
- Change organization_id param to List[str] (repeated query params)
instead of comma-separated string.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Updates two tests in TeamInfo.test.tsx that were asserting for "X Members" text that was removed from the TeamVirtualKeysTable component in commit ec4ef9c924.
- Changed test "should display X Members in Virtual Keys tab when navigated to" to assert for pagination controls instead
- Changed test "should show Filters and pagination controls in Virtual Keys tab" to wait for Filters button instead of member count
All 29 tests now pass.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Extract _resolve_org_filter_for_user_search and _resolve_team_org_filter
helpers from ui_view_users to bring complexity under the ruff C901 limit.
- Fix misleading error when a team admin searches but team has no org:
now says "this team is not part of an organization" instead of the
generic "only admins can search" message.
- Add distinct errors for team-not-found and not-a-team-admin cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "fix(vertex): drop bare {} schemas from anyOf before adding nullable=True (#23060)"
This reverts commit 3ad9a536d3.
* Revert "Merge pull request #22589 from Chesars/fix/vertex-preserve-any-type-schema"
This reverts commit da941e4261, reversing
changes made to f77f28a5f8.
