mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
4746 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Sameer Kankute
|
e8fcb01215
|
Litellm OSS Staging (#29161)
* Cato Networks guardrail, based on Aim (#26597) * Aim was acquired by Cato Networks, creating Cato Networks guardrail based on Aim * Add more tests * Move test so they are reached by codecov coverage * base URL trailing slashes * Support Lemonade runtime context metadata (#28135) * Support Lemonade runtime context metadata * Add provider hook for runtime model metadata * Address provider model info review feedback Keep the runtime model info hook duck-typed instead of extending the base model-info class, and avoid importing ModelInfoBase from Ollama common utilities to reduce CodeQL cyclic-import noise. Co-authored-by: openhands <openhands@all-hands.dev> * Fix CI after staging rebase Relax the Ollama runtime metadata return annotation to match the provider-hook dict response and update the Google Interactions OpenAPI status expectation for the current live spec. Co-authored-by: openhands <openhands@all-hands.dev> * Normalize Lemonade runtime model metadata * Avoid leaking Ollama metadata auth * Avoid leaking Lemonade metadata auth --------- Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com> Co-authored-by: openhands <openhands@all-hands.dev> * fix(cato): address guardrail review feedback Use proxy-authenticated user identity, forward moderation hook return values, and ensure streaming sender tasks are cancelled and awaited on exit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vertex_ai): route google/gemma-*-maas through partner-models OpenAI path - clone of #28010 (#28846) * fix(vertex_ai): route google/gemma-*-maas through partner-models OpenAI path Fixes #26083 vertex_ai/google/gemma-4-26b-a4b-it-maas previously fell through to the NON_GEMINI route. Per owtaylor's plan on #26083: add the google/gemma- prefix to PartnerModelPrefixes so is_vertex_partner_model picks it up and should_use_openai_handler routes it to the OpenAI-compatible /endpoints/openapi/chat/completions URL. No gemma-detection exclusion needed (the "gemma/" check uses a slash, which google/gemma-... doesn't match). No OpenAIGPTConfig subclass needed — works with the base handler. * fix(vertex_ai): mark gemma-4-26b-a4b-it-maas as vision-capable (empirically verified) * fix(vertex_ai): address greptile feedback — provider category, canonical URL, sync backup * test(vertex_ai): add function-calling and vision pass-through tests for Gemma MaaS Addresses oss-pr-review-agent-shin feedback on PR #28010: supports_function_calling, supports_tool_choice, and supports_vision were marked true but had no tests proving the payloads actually reached the OpenAI-compatible endpoint. Added: - test_gemma_maas_supports_function_calling — verifies the utility returns True when the model_cost entry carries supports_function_calling=true - test_gemma_maas_supports_vision — same for supports_vision - test_vertex_ai_gemma_function_calling_passthrough — verifies tools + tool_choice appear in the JSON body POSTed to /endpoints/openapi/chat/completions - test_vertex_ai_gemma_vision_passthrough — verifies image_url content parts survive transformation and reach the global endpoint URL * fix: Delete uv.lock * test(vertex_ai): add function-calling and vision pass-through tests for Gemma MaaS Addresses oss-pr-review-agent-shin feedback on PR #28010: P1 (patch target): Added a comment explaining why patching litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler is correct — get_async_httpx_client() (defined in http_handler.py) instantiates AsyncHTTPHandler within that module's scope, so the definition-site patch intercepts it. Without the mock the test raises AuthenticationError, confirming it never silently passes. P2 (partner-provider regression guard): Added test_gemma_routes_through_openai_handler() which calls VertexAIPartnerModels.should_use_openai_handler() directly, so if Gemma's routing to VertexPartnerProvider.llama ever changes the URL-shape tests below it become a real regression guard rather than an unanchored unit test. Also added: - test_gemma_maas_supports_function_calling / supports_vision — capability flag checks via patch.dict(litellm.model_cost) - test_vertex_ai_gemma_function_calling_passthrough — tools + tool_choice forwarded in the request body - test_vertex_ai_gemma_vision_passthrough — image_url part survives transformation to the global endpoint Added: - test_gemma_maas_supports_function_calling — verifies the utility returns True when the model_cost entry carries supports_function_calling=true - test_gemma_maas_supports_vision — same for supports_vision - test_vertex_ai_gemma_function_calling_passthrough — verifies tools + tool_choice appear in the JSON body POSTed to /endpoints/openapi/chat/completions - test_vertex_ai_gemma_vision_passthrough — verifies image_url content parts survive transformation and reach the global endpoint URL * fix: proper patch for unit tests --------- Co-authored-by: Iana <iana@Shivakumars-MacBook-Pro.local> * fix(cato): guardrail all completion choices on output When n > 1, only choices[0] was analyzed and redacted. Iterate every Choices entry so block and anonymize actions apply to all completions. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix review * fix(cato_networks): harden output anonymize handling and restructure nested UI routes Guard against empty redacted_output and empty all_redacted_messages from Cato. Restructure nested admin UI HTML exports to index.html so extensionless routes work. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix mypy * fix(cato): guard missing policy_drill_down and all_redacted_messages keys * fix(cato): avoid KeyError bypassing block action on missing analysis_result * fix(cato): preserve non-text message fields during anonymize Rebuild redacted messages from the original messages, overwriting only content, so tool_calls, tool_call_id, name and multimodal fields survive the anonymize action. * fix(cato): preserve trailing messages when fewer redacted messages returned Avoid silently truncating the conversation in _anonymize_request when Cato returns fewer redacted messages than were sent, and isolate the no-api-key config test from a pre-existing CATO_API_KEY environment variable. * fix(cato,model-info): preserve stream block signal on sender teardown; forward api_key in dynamic model-info lookup Suppress ConnectionClosed (alongside CancelledError) when tearing down the Cato streaming sender task so a backend ConnectionClosed cannot mask the original StreamingCallbackError (e.g. a guardrail block) raised by the receive loop. Thread api_key through get_model_info -> _get_model_info_helper so an explicit key reaches a provider's dynamic get_model_info for a caller-supplied api_base. Previously only api_base was forwarded, so authenticated Ollama and Lemonade servers at a custom base could only be queried unauthenticated. * fix(cato): surface mid-stream forwarding errors instead of blocking on recv If the upstream LLM stream errors mid-flight, the sender task dies before sending the terminal done frame, so the consumer would block on websocket.recv() until Cato closes the connection. Race recv against the sender task and raise the stored sender exception promptly as a StreamingCallbackError. * fix(cato): drop spoofable end_user_id from guardrail user identity Only the key/JWT-bound user_email is a trusted identity. end_user_id is resolved from caller-supplied request fields (OpenAI user param, headers, metadata), so an authenticated caller with no bound user_email could set it to another user's email and have LiteLLM forward x-cato-user-email for that victim, poisoning Cato audit and policy attribution. Forward only user_email and omit the header otherwise. * fix(cato): harden output anonymize path against missing content key * fix(cato): fall back to original message when redacted content key is missing * refactor(model-info): drop unused api_key from cached model-info helper _cached_get_model_info_helper is only called by the cost-tracking hot path, which never authenticates, so the api_key parameter was never populated. Keeping it in the lru_cache key offered no benefit and risked fragmenting the high-RPS cache and retaining credential strings per entry. * fix(cato): preserve None content on tool-call-only choices in output hook * fix(ollama): respect static-model guard in OllamaConfig.get_model_info Delegate to OllamaModelInfo.get_model_info so statically-priced Ollama models short-circuit before the /api/show network call instead of hitting the server unconditionally. * fix(lemonade,ollama): treat empty api_key as unset to avoid leaking server creds An empty-string api_key was treated as an explicit key, so it passed the guard meant to keep server-side credentials off caller-supplied bases and then fell back through the env/global key chain. A caller could point api_base at a server they control and send api_key="" to receive the configured provider key in the Authorization header. Gate the credential fallback on the api_key being truthy instead of merely not-None. * fix(cato): inspect and redact Responses-API input, not just messages The guardrail only read data["messages"], so /v1/responses requests, which carry their text in data["input"], reached Cato as an empty message list and bypassed inspection entirely. Send build_inspection_messages(data) so both shapes are analyzed, and write anonymized results back with apply_redacted_messages_back when the request used input. * perf(utils): keep api_key out of get_model_info lru_cache key * fix(cato): propagate ssl_verify to streaming WebSocket connection The streaming hook applied ssl_verify only to the HTTP handler; the websockets.connect() call used default verification, so a custom Cato instance behind TLS with a self-signed cert worked for non-streaming calls but failed every streaming request. Resolve the ssl_verify setting into the connect() ssl argument, mirroring the HTTP handler. * refactor(utils): rename shadowing local in _get_model_info_helper * fix(cato): flatten multimodal chat content before inspection Chat Completions requests whose message content is a multimodal parts array were posted to Cato as the raw OpenAI parts, so text inside content: [{"type":"text", ...}] reached the model without Cato ever inspecting the string. Flatten each message's list content to plain text while keeping the list 1:1 with the request so the index-based redaction write-back stays valid; Responses-API input requests still go through build_inspection_messages. * test(lemonade): clear get_model_info cache around api_base test * fix(cato): inspect and redact Responses-API input even when messages present _inspection_messages returned early once messages was non-empty, so a /v1/responses caller could place benign text in messages and disallowed text in input and have only messages reach Cato while the model used input. Inspect both fields and write anonymize redactions back to input as well as the index-aligned messages. * test(log_db_metrics): assert table_name event_metadata contract log_db_metrics now emits minimal event_metadata via _safe_db_event_metadata (table_name only, function_name/function_kwargs/function_args dropped as redundant with call_type and unsafe to stamp on a span). The success-path test still asserted function_name membership and crashed with TypeError on the None metadata returned when no table_name is passed. Pass a table_name and assert the surfaced contract instead. * fix(cato): inspect and redact completion prompt and Responses-API instructions The Cato guardrail only inspected chat messages and the Responses-API input field, so blocked text placed in the legacy /v1/completions prompt or the /v1/responses instructions field reached the model without ever being sent to Cato. Both fields are now appended as synthetic inspection messages, and the anonymize path slices Cato's redactions back to the field they came from. * fix(cato): serialize non-str/bytes websocket chunks before forwarding * fix(cato): inspect tool descriptions and tool-call arguments * fix(cato): map redacted output by assistant index; restore get_model_info.cache_info * fix(cato): block output even when detection_message is null/empty A block_action returned by Cato on the output hook whose detection_message was null or empty was let through to the caller: the truthiness guard on detection_message skipped the HTTPException and the unblocked response was returned. Raise the HTTPException directly in _handle_block_action_on_output so the output path blocks unconditionally, mirroring the input path. * fix(cato): inspect and redact nested tool param and legacy function descriptions Tool/function parameter descriptions and the legacy functions[] array are forwarded to the model but were not seen by Cato, so blocked text hidden there bypassed inspection and anonymization. Recursively walk every description string in tools[].function and functions[] schemas for both the analyze payload and the anonymize write-back. * fix(cato): traverse schema descriptions iteratively to satisfy recursive detector The nested walk() generator recursed over tool/function JSON schemas with no depth bound, which the recursive_detector code-quality gate rejects. Replace it with an explicit-stack DFS that yields the same (container, key) refs in the same pre-order, so schema description redaction is unchanged. * fix(cato): inspect and redact response_format JSON schema descriptions response_format json_schema descriptions are forwarded to the model, so blocked text hidden in nested schema descriptions could bypass Cato inspection and redaction. Extend the schema-description walk to cover response_format alongside tools and legacy functions. * fix(cato): skip output rewrite when Cato returns no redaction Return None from call_cato_guardrail_on_output on monitor/no-action so the post-call hook only mutates the message when there is an actual redaction, instead of redundantly re-writing the original content. * refactor(utils): resolve explicit api_key model info without the cache Move the model-info build into a non-cached _build_model_info helper and drop api_key from the lru-cached _cached_get_model_info signature. Both cached helpers now take the same (model, provider, api_base) key and never forward api_key, while explicit per-caller keys are resolved through the builder directly instead of reaching into the cache wrapper's __wrapped__. * fix(cato): inspect and redact non-description schema string values Tool, function and response_format JSON schemas forward more than just description text to the model. enum, const, default, examples and title values are sent verbatim, so blocked content hidden in any of them bypassed Cato inspection and redaction. Walk those schema string values alongside descriptions on both the inspection and anonymize paths. * fix(model-info): surface swallowed dynamic model-info errors The provider-specific get_model_info dispatch falls back to the static cost map when a provider's dynamic lookup raises, which is intentional graceful degradation. Previously the exception was discarded with a bare debug line, so a real failure (e.g. a provider whose get_model_info signature does not accept api_key) was invisible. Log the exception at warning level with the model and provider context so the fallback is diagnosable. * fix(cato): inspect and redact Responses API output in post-call hook The post-call success hook only handled ModelResponse, so /v1/responses (which returns a ResponsesAPIResponse) bypassed the Cato output guardrail. Extract and inspect/redact every output_text content block and function-call arguments string, blocking on a block action, so generated text cannot escape inspection by using the Responses API. * chore: reset _experimental/out folder * chore(ui): remove orphaned prebuilt dashboard chunk files The _experimental/out manifests are byte-identical to the base branch, so the served dashboard already matches base. 436 unreferenced Next.js chunk files had accumulated in the directory and are not loaded by any manifest; removing them restores the committed UI artifacts to the base build and drops the artifact churn from this PR's diff. * fix(guardrails,ollama): forward ssl_verify to Cato init and raise_for_status on /api/show --------- Co-authored-by: Alex Yaroslavsky <trexinc@gmail.com> Co-authored-by: Graham Neubig <neubig@gmail.com> Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com> Co-authored-by: openhands <openhands@all-hands.dev> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Piotr Placzko <piotr@icep-design.com> Co-authored-by: Iana <iana@Shivakumars-MacBook-Pro.local> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
ryan-crabbe-berri
|
609e1e9763
|
fix(ui): render caller-supplied filter options in caller order (LIT-3151) (#29462)
FilterComponent iterated a hardcoded orderedFilters whitelist instead of the options prop, so any consumer whose filter names were not on that list rendered nothing. The Tool Policies page passes "Input Policy", "Output Policy", "Team Name" and "Key Name", none of which were whitelisted, so its Filters panel opened to an empty area. Drop the whitelist and render the options the caller passes, in the order they pass them, so each page owns its own filter set and ordering. The Logs page array is reordered to match its prior on-screen order; VirtualKeys and TeamVirtualKeys already matched the old whitelist order and are unaffected. |
||
|
ryan-crabbe-berri
|
1cce49b9d0
|
fix(vector-stores): support engines URL for Vertex AI Search (#27885)
Adds optional vertex_engine_id field to vertex_ai/search_api so users can route through a Discovery Engine search app instead of the data store directly. Required for website, healthcare, and connector-based data stores that return FAILED_PRECONDITION on the existing dataStores URL. Existing data-store-direct callers are unaffected. Resolves LIT-3036 |
||
|
yuneng-jiang
|
54ed5a4eb5
|
fix(e2e): tolerate trailing slash in SERVER_ROOT_PATH login redirect (#29369)
The Next.js admin UI is exported with trailingSlash: true, so the proxy
serves /ui/login at /ui/login/index.html and 308s /ui/login → /ui/login/.
The waitForURL predicate used endsWith("/ui/login"), which never matched
the canonicalized URL and timed out after 15s.
This was masked until the build artifacts were regenerated against the
AuthContext fix: the prior bundles still hit the racy redirect path that
fired before proxyBaseUrl was populated, producing /ui/login (no prefix,
no proxy round-trip, no trailing slash) which fortuitously satisfied the
predicate. The first PR to ship the corrected bundle exposed the
assertion bug.
Switch the predicate to includes("/ui/login"); the prefix assertion below
still validates the SERVER_ROOT_PATH preservation that is the actual
contract under test.
|
||
|
yuneng-jiang
|
f0ebfb2a1b
|
fix(ui): break logout redirect loop across origins (#29360)
When the user has visited both the dev UI (e.g. localhost:3000) and the
proxy UI (e.g. localhost:4000) in the same tab, logging out from the dev
origin produced an infinite logout/login redirect.
The proxy-side LoginPage's "is the user still authenticated?" check
was reading getCookie("token"), which falls back to sessionStorage when
document.cookie has no token. The cross-origin clearTokenCookies() call
from the dev origin can clear cookies on the shared hostname, but cannot
reach sessionStorage on the proxy origin (sessionStorage is per-origin),
so the fallback returned a stale token and LoginPage interpreted the
user as logged in, redirecting back to the dev origin. Dev origin then
saw no cookie and redirected to LoginPage, repeating ~20x per second.
This change introduces getCookieFromDocument(), a cookie-only read with
no sessionStorage fallback, and uses it in LoginPage's already-logged-in
check. The HttpOnly-reverse-proxy defense from PR #23532 is unaffected:
storeLoginToken still writes both the JS cookie at /ui and the
sessionStorage backup, and getCookie still falls back for callers that
want the full read path.
|
||
|
ryan-crabbe-berri
|
a9cc6ed68c
|
test(e2e): cover PROXY_LOGOUT_URL redirect on Logout (#29080)
* test(e2e): cover PROXY_LOGOUT_URL redirect on Logout Env-gated spec mirroring the existing serverRootPathRedirect pattern: when the proxy is booted with PROXY_LOGOUT_URL set, clicking Logout in the navbar must navigate to that external URL. The standard run_e2e.sh exports an empty value so the rest of the suite is unaffected; this spec self-skips unless the env var is populated. * test(e2e): run PROXY_LOGOUT_URL spec in the suite + harden logout assertions Boot the e2e proxy with PROXY_LOGOUT_URL set (job-level env in CircleCI and run_e2e.sh) so proxyLogoutUrl.spec.ts actually runs instead of self-skipping. Nothing else in the suite performs a logout, so this only affects the behavior under test. Harden the spec to verify the logout flow rather than a URL substring: - wait for /sso/get/ui_settings before clicking so logoutUrl is populated (otherwise window.location.href = "" silently reloads same-origin) - assert a token cookie exists first, and is cleared after logout - locate the dropdown via getByRole instead of internal antd CSS classes - stub the external destination and assert on URL origin + path prefix * test(e2e): assert exact PROXY_LOGOUT_URL on logout redirect Replace the origin + startsWith(pathname) checks with a single normalized href comparison. With PROXY_LOGOUT_URL=https://www.example.com the path was "/", so startsWith("/") matched any path and left path/query/hash unchecked. Comparing normalized hrefs pins scheme, host, port, path, query and hash while still tolerating the browser's trailing-slash/default-port normalization. |
||
|
ryan-crabbe-berri
|
90b5104475
|
feat(mcp/auth): additive key access-group grants + opt-in member assignment (#29313)
* fix(mcp): make key.access_group_ids grants additive over team ceiling A key whose unified access_group_ids grant a private MCP server was having that grant intersected against its team's MCP ceiling, so a key in a team scoped to other servers (or with no own scope) lost the granted server entirely. Resolve access_group_ids once as ungated additive grants and union them on top of the key/team ceiling instead of folding them into the key scope that gets intersected. * test(mcp): align key access-group tests with additive-grant model The previous commit moved key.access_group_ids resolution out of the intersected key ceiling (_get_allowed_mcp_servers_for_key) and into the ungated additive grant path (_get_key_access_group_mcp_server_extras), unioned on top of the team ceiling. Five tests from #28890/#29195 still asserted the old gated / in-key-scope contract and failed: - _get_allowed_mcp_servers_for_key now returns the object_permission ceiling only and never resolves access_group_ids; two tests now assert the group resolver is not called from that path (with and without an object_permission present). - The extras path is ungated, so a group whose assigned_team_ids / assigned_key_ids exclude the caller still contributes its servers. - The end-to-end test asserts the grant surfaces via the extras path rather than the base key path. - Dropped test_key_access_group_ids_empty_returns_no_extras; the empty case is already covered by the extras family's no-groups test. * feat(auth): gate member access-group assignment on keys behind opt-in Non-admin team members could attach access_group_ids to keys they create or update, letting them self-grant resources (MCP servers/models) the team admin never intended. Add an opt-in KEY_ACCESS_GROUP_ASSIGNMENT team-member permission (default-deny) enforced at /key/generate and /key/update; proxy and team admins bypass. Surfaces automatically as a checkbox in the team Member Permissions UI. * fix(auth): gate access-group assignment on /key/regenerate too RegenerateKeyRequest inherits access_group_ids and prepare_key_update_data persists it, so a non-admin key owner could self-grant access groups by regenerating. Apply the same opt-in member gate using the existing key's team. * test(auth): cover member access-group gate and additive MCP grants Add unit tests for enforce_member_can_assign_access_groups (deny without opt-in, allow with opt-in, and proxy-admin / team-admin / non-team-key bypasses) and for _get_key_access_group_mcp_server_extras (no-auth and no-resolved-servers return empty, resolved ids are expanded, errors degrade to no grants). |
||
|
ryan-crabbe-berri
|
37e6e2da1c
|
test(e2e): assert internal-user navbar identity is scoped to that user (#29077)
* test(e2e): assert internal-user navbar identity is scoped to that user The existing login.spec.ts only checks the admin's navbar identity. This adds the symmetric check for the internal user — verifying the account button + dropdown surface the internal user's email, id, and role, and that no admin-scoped values leak through. * test(e2e): harden navbar identity test per review feedback Locate the user dropdown panel by a data-testid on the popupRender div instead of Ant Design internal + Tailwind class names, so styling refactors no longer risk breaking the identity-scoping assertions. Source the seeded user emails/ids from shared constants (match seed.sql) instead of hardcoding them inline. |
||
|
ryan-crabbe-berri
|
892838963c
|
test(e2e): cover Internal User create-key flow when in no teams (#29083)
* test(e2e): cover Internal User create-key flow when in no teams The seeded e2e-internal-user is in two teams, so the "no team" branch of the Create Key modal — where the team dropdown must render empty — was unreachable. Seeds a noteam@test.local user and adds a spec that logs in fresh, opens the modal, and asserts the dropdown has zero options. * test(e2e): harden no-team dropdown assertion + add with-teams counterpart Replace the one-shot count() check with a settled-empty assertion: wait for the dropdown's loaded "No teams found" state before asserting zero options, so the test can't pass on a transient empty frame while the team-options request is still in flight. Add internalUserWithTeams.spec.ts as the differential partner; it logs in as the seeded e2e-internal-user (two team memberships) and asserts the dropdown lists exactly those teams. Without it, the no-team spec's zero-options assertion would still pass against a regression that empties the dropdown for every user. |
||
|
Sameer Kankute
|
af17400c38
|
feat(a2a): well-known agent-card discovery + LangGraph Platform mode (#28860)
* feat(a2a): well-known agent-card discovery + LangGraph Platform mode Adds a registration-time discovery flow so admins can paste an upstream agent URL, see its skills/capabilities, pick what to expose, and have the proxy front it with a LiteLLM-shaped agent card. Backend (new litellm/proxy/a2a/ module): - fetch_well_known_card walks /.well-known/agent-card.json, /.well-known/agent.json, /agent.json by default. langgraph_platform mode hits the canonical path with ?assistant_id=<id> (LangGraph serves one shared endpoint per deployment). - merge_agent_card overlays LiteLLM overrides on the upstream card: drops upstream url, forces protocolVersion=1.0, replaces securitySchemes with LiteLLMKey bearer, emits supportedInterfaces pointing at the proxy, filters capabilities to a small allowlist, strips non-v1.0 fields. - POST /v1/a2a/discover returns the raw upstream card (admin-only) so the UI can render skills/capabilities for selection. - create/update/patch agent endpoints pre-generate the agent_id and run merge_agent_card before storing, so DB.agent_card_params already embeds the proxy-fronted URL. UI (ui/litellm-dashboard): - New AgentCardDiscovery component with a parent-driven plan: discovery_mode + params + display URL. For LangGraph the parent composes (api_base, assistant_id); for pure A2A it uses the url field. Component hides the manual URL input when the parent drives. - add_agent_form wires discovery for every non-custom agent type and overlays the user's selections onto agent_card_params at submit, fixing the bug where dynamic agent forms ignored discovery picks. Completion-bridge fixes (paired): - Add kind: "message" to A2A response messages and unwrap result so it's a Message directly per spec (matches a2a SDK SendMessageResponse validation). - Forward A2A metadata to LangGraph runs via extra_body.metadata. * fix(a2a): preserve agent url, fix streaming chunk envelope, and protect forwarded metadata - Streaming chunk: move final out of the message object into the result envelope per the A2A spec. - Agent card merge: keep upstream url on the stored card so the runtime invocation path can locate the upstream backend; the public well-known endpoint already rewrites this field to the proxy URL before exposing it to clients. - Completion bridge: apply A2A forward metadata after merging litellm_params so an agent-configured extra_body cannot overwrite the forwarded metadata. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): fix legacy streaming chunk, agent card test, and metadata merge - providers/litellm_completion: move 'final' out of the message object into the result envelope per the A2A spec (matches the bridge fix). - agent endpoints test: the runtime invocation path now preserves the top-level 'url' on the stored card, so update the assertion to match. - completion bridge metadata: when forwarding A2A metadata via extra_body.metadata, merge into any existing extra_body.metadata instead of replacing it, so an agent-configured metadata block is preserved (forward metadata still wins on key conflicts). Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): remove dead duplicate transformation dir; drop SSRF-prone headers field from /v1/a2a/discover Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): revert accidental html→index.html rename from |
||
|
ryan-crabbe-berri
|
5eafe1c1fc
|
test(e2e): cover navbar Logout flow as proxy admin (#29076)
* test(e2e): cover navbar Logout flow as proxy admin
The Logout button under the navbar User dropdown was an uncovered
manual-QA step. This test signs in as admin, opens the dropdown,
clicks Logout, then navigates to a protected page and asserts the
redirect to /ui/login — proving the session was cleared.
* test(e2e): fix logout dropdown trigger and account-menu selector
The button never rendered the literal text "User" (it shows initials +
display name), and the antd Dropdown uses trigger={["click"]}, so the
synthetic mouseover/mouseenter never opened the popup. Open it with a real
click on the button's aria-label ("Account menu — ...").
|
||
|
ryan-crabbe-berri
|
fcd5760891
|
test(e2e): cover Internal User key modal, team info, key page (#29074)
* test(e2e): cover Internal User key modal, team info, key page Three previously-uncovered manual-QA paths for the Internal User role: - Create Key modal — confirm the team dropdown is populated with the user's teams (verifying the role-scoped UI flow exists). - Team info page — confirm the Settings/Members tabs are hidden for a regular team member; only the read-only tabs render. - Virtual Keys page — confirm the proxy's internal litellm-dashboard team keys never leak into an internal user's table. * test(e2e): share clickTeamId helper, strengthen key-filter assertion Address review feedback on the Internal User e2e spec: - Extract clickTeamId into helpers/navigation.ts; import in both internalUser and teams specs instead of duplicating it. - Anchor the litellm-dashboard absence check on the user's own seeded key so it cannot pass vacuously against an empty table. - Drop redundant dismissFeedbackPopup calls (navigateToPage already dismisses internally). |
||
|
ryan-crabbe-berri
|
10bda4456a
|
test(e2e): cover Internal Viewer nav, key, and team-info gating (#29075)
* test(e2e): cover Internal Viewer nav, key, and team-info gating Three previously-uncovered manual-QA paths for the Internal Viewer role: - Nav only renders the read-only sections; admin-only items (Internal Users, Organizations, Models + Endpoints) stay hidden. - Virtual Keys page hides Create New Key, and the key detail view hides Regenerate / Reset Spend / Delete actions. - Team info page hides Members and Settings tabs for the viewer. * test(e2e): scope viewer nav to sidebar, strengthen tab assertions Address review feedback on the Internal Viewer e2e spec: - Scope the nav test to the sidebar complementary landmark and match items by link role + accessible name. The prior CSS nav, aside selector grabbed the top bar (the sidebar is a complementary landmark, not a <nav> tag), so the assertions never hit the real nav links. - Land via navigateToPage so the networkidle wait settles the role-gated nav before asserting. - Assert the Virtual Keys tab is visible (was only commented). - Use toHaveCount(0) for hidden team tabs to match the nav block; tabs are conditionally rendered, not CSS-hidden. - Drop redundant dismissFeedbackPopup calls (navigateToPage already dismisses internally). |
||
|
michelligabriele
|
68852ef165
|
fix(teams): expose keys_count on /v2/team/list and wire UI Resources badge (#28502) | ||
|
Mateo Wang
|
f27df8d516
|
docs: hand-written CLAUDE.md; point GEMINI.md and AGENTS.md at it (#29252)
Some checks are pending
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
* docs: replace generated CLAUDE.md with hand-written guidance, remove AGENTS.md Swap the auto-generated CLAUDE.md for a concise hand-written version that captures how we actually want agents to work in this repo: minimal comments, simplicity first, meaningful tests with a high mutation kill rate, PRs based off litellm_internal_staging rather than main, and curl against a live proxy as proof of fix instead of pasted pytest output. Remove AGENTS.md so there is one source of truth for agent guidance. The customer and company name confidentiality policy, along with the MCP available_on_public_internet note, are carried over from the previous CLAUDE.md. * fix: further clarify communication guidelines * docs: point GEMINI.md at CLAUDE.md instead of duplicating guidance Replace the standalone GEMINI.md copy, which had already drifted from the new CLAUDE.md, with a one-line pointer so Gemini reads the same single source of truth. * docs: simplify PR template test checklist item Replace the rigid "at least 1 test is a hard requirement" checklist line with "I have added meaningful tests", which matches the testing guidance in CLAUDE.md, and tidy a comma into a semicolon in the scope-isolation item. * docs: point AGENTS.md at CLAUDE.md instead of deleting it Keep AGENTS.md so tools that read it still resolve guidance, but collapse it to the same one-line pointer to CLAUDE.md used by GEMINI.md, keeping a single source of truth. * fix: make AI-generated rules more concise * fix: spelling Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: make the .env usage more careful * docs: restore MCP available_on_public_internet note to CLAUDE.md The PR description states this note was carried over verbatim from the previous CLAUDE.md, but it was dropped in the rewrite. Restore it so the file matches the description and the team guidance is not lost. * docs: restore browser storage and CI supply-chain safety notes to CLAUDE.md These security-relevant rules were dropped in the rewrite. Restore the sessionStorage-over-localStorage (XSS) guidance and the CI supply-chain rules (no curl|bash, pin versions, verify checksums) so agents editing UI or CI code are still steered away from those pitfalls. * docs: move area-specific guidance into nested CLAUDE.md files The MCP, browser-storage, and CI supply-chain notes are scoped to particular parts of the tree, so move each into a nested CLAUDE.md that Claude Code loads on demand when those files are touched: the MCP note under the mcp_server gateway, the browser-storage rule under the UI dashboard, and the CI supply-chain rules under .circleci. Keeps the root CLAUDE.md focused on general guidance while the area notes surface where they are relevant. * docs: keep CI supply-chain note in root CLAUDE.md CI guidance applies beyond .circleci (it also covers downloads in GitHub workflows and any CI script), and CI work does not reliably touch a single subtree, so a nested file under .circleci would not surface it dependably. Keep it in the always-loaded root instead. The MCP and browser-storage notes stay nested where they map cleanly to one area of the tree. * fix: make it clear we prefer httpOnly * chore: make ci rule more concise * chore: make concise Fix formatting and punctuation in MCP note. * fix: don't include Claude attribution --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
ryan-crabbe-berri
|
2bfbf14882
|
test(e2e): cover Team Admin view + member + key flows (#29072)
* test(e2e): cover Team Admin view + member + key flows
Adds a new spec exercising the previously-uncovered team-admin manual-QA
items: viewing all team keys (including other members'), adding a member,
removing a member, and creating a team key with All Team Models. Also
seeds a dedicated invitee user so the add-member test can run in parallel
with the proxy-admin invite test without colliding on the team roster.
* test(e2e): harden team-admin member specs per review feedback
Address Greptile feedback on the Team Admin spec:
- locate the delete action via getByTestId("delete-member") instead of
the fragile svg/img .last() selector
- match the seeded removable member by user_id (members_with_roles stores
no email, so the roster renders user_id)
- assert exact success-toast strings rather than broad regexes that could
match unrelated "success" text
|
||
|
ryan-crabbe-berri
|
9918a9c78c
|
fix(guardrails): persist disable_global_guardrails on keys (#29233)
* fix(guardrails): restore disable_global_guardrails persistence for keys The per-key/team "Disable Global Guardrails" toggle silently stopped working after #17042, which removed `disable_global_guardrails` from the key/team request models and from the premium metadata allowlist. Without those, the UI's top-level field was dropped by pydantic and never folded into key `metadata`, so the runtime gate always read False and global default_on guardrails kept running. Restore the request-model fields (KeyRequestBase, NewTeamRequest, UpdateTeamRequest) and the `LiteLLM_ManagementEndpoint_MetadataFields_Premium` entry so the flag is promoted into metadata again. Because the key edit form always submits the flag (false by default), guard the UI so it is only sent when it actually changed (edit) or is enabled (create) — this keeps the premium gate on enabling intact while not 403-ing non-premium users who edit unrelated key fields, mirroring how guardrails/tags are already stripped. * test(guardrails): cover disable_global_guardrails toggle-off + clarify premium field comment Add a prepare_metadata_fields case asserting `disable_global_guardrails: False` overwrites an existing `True`, and rewrite the PREMIUM_METADATA_FIELDS comment to explain why boolean premium fields are excluded from the empty-value strip loop. |
||
|
ryan-crabbe-berri
|
5699a06413
|
test(e2e): cover AI Hub make-public flow and public model_hub_table (#29071)
Some checks are pending
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
* test(e2e): cover AI Hub make-public flow and public model_hub_table Three previously-uncovered manual-QA paths land in one spec: - Admin opens "Select Models to Make Public", advances through the multi-step modal, and verifies the success toast. - AI Hub tab strip exposes Model Hub / Agent Hub / MCP Hub / Skill Hub — note the manual-QA "Claude Code Plugin Marketplace" label was renamed to Skill Hub; the test pins the current name. - Anonymous /ui/model_hub_table loads with the master key as `?key=` and renders the Model Hub tab. Agent Hub / MCP Hub tabs are conditional on public data and are not asserted here. * test(e2e): harden AI Hub make-public + public hub assertions Address Greptile review: - Make-public test now asserts "Select All (N)" with N>=1 before clicking, so a missing-seed-data run surfaces immediately instead of timing out on the disabled Next button or the success toast. - Public model_hub_table test dismisses the feedback popup before the tab visibility assertion, matching the ordering used by navigateToPage so a popup race can't mask the tab mid-evaluation. * docs(e2e): explain admin vs public AI Hub tab asymmetry Greptile flagged the all-4-tabs assertion as a potential CI flake, inferring from the public-page comment that Agent Hub / MCP Hub might be data-conditional in the admin view too. They aren't — ModelHubTable renders all four tabs unconditionally for admins. Document the asymmetry inline so future readers (and future review passes) don't re-derive it. |
||
|
ryan-crabbe-berri
|
bc31c570f0
|
test(e2e): cover add-MCP-server flow via discovery → custom form (#29070)
* test(e2e): cover add-MCP-server flow via discovery → custom form
The "Add MCP server" manual-QA step was uncovered. This adds a test
that opens the discovery modal, jumps into the custom-server form,
fills name + Streamable HTTP transport + a placeholder URL + None
auth, submits, and verifies both the success toast and the new row.
* test(e2e): apply greptile fixes to MCP add-server test
- Anchor the auth-type Select via its enclosing Collapse panel
("Authentication") instead of the placeholder text. The Form.Item has
no label prop, so the previous `hasText: /auth type/i` filter was
matching via "Select auth type" placeholder copy — fragile.
- Document the intentional lack of teardown, matching the pattern used
in addModel.spec.ts: the e2e runner discards the DB per invocation.
Addresses Greptile P2s on PR #29070.
* test(e2e): scope MCP row assertion to the servers table
Scope the post-create row lookup to `table tbody` so the form modal's
`server_name` input — which still holds the timestamped value during
its close animation — can't satisfy the assertion before the server
actually lands in the list.
* docs(e2e): note MCP coverage scope and link to tracker
This spec only smoke-tests the happy-path Streamable HTTP + None auth
flow. Add a top-of-file comment pointing at E2E_COVERAGE.md so future
contributors can see what's still uncovered (other transports, all
auth types, edit/delete, BYOK, tool list/call, access groups).
|
||
|
ryan-crabbe-berri
|
9cac0471ae
|
test(e2e): cover Team-BYOK add-model flow as proxy admin (#29068)
* test(e2e): cover Team-BYOK add-model flow as proxy admin The team-only model + team assignment was an uncovered manual-QA path. This adds a premium-gated test that toggles Team-BYOK, picks the seeded E2E Team CRUD, submits, and verifies the model lands in All Models with the team alias attached. * test(e2e): apply greptile fixes to Team-BYOK test - Add the 2s networkidle settle that the sibling addModel tests use — networkidle fires before the All Models table finishes re-rendering, so the search input was racing with the render. - Assert on `models-results-count` before inspecting the table body so an empty search result fails with a clear "expected results count" message instead of timing out on a missing row. Addresses Greptile P2s on PR #29068. * test(e2e): harden Team-BYOK test against flake and stale state - Add before/after cleanup that deletes any Cohere model already scoped to e2e-team-crud via /v2/model/info + /model/delete, so Playwright retries and local reruns don't accumulate rows. - Pick the team from the dropdown by role/option name instead of a global getByText match — avoids matching a previously-rendered tag elsewhere in the form. - Scope the "created successfully" assertion to .ant-notification so a stale toast from an earlier test in the same browser context can't vacuously satisfy it. - Tighten the All Models assertion: require a single row that contains BOTH the cohere model name AND the e2e-team-crud alias, so the team-less wildcard from the sibling "Add wildcard route" test can't satisfy the check. |
||
|
ryan-crabbe-berri
|
b0ea013042
|
test(e2e): cover add-fallback flow in Router Settings (#29069)
* test(e2e): cover add-fallback flow in Router Settings as proxy admin The Router Settings → Fallbacks → Add Fallbacks flow was an uncovered manual-QA path. This adds a test that opens the modal, picks a primary + fallback from the seeded mock models, saves, and verifies both render in the fallback table. * fix(e2e): make router-fallback test idempotent and pick antd options by text - Match `.ant-select-item-option` by text instead of `getByTitle(...)` — FallbackGroupConfig uses `options=` (not <Select.Option> children), so no `title` attribute is emitted and the title-based selector hangs. - Add before/after hooks that wipe any fallback for fake-openai-gpt-4 via /config/update so retries and local reruns don't trip on leftover state. - Tighten the success assertion to a single tbody row containing BOTH the primary and the fallback names — pre-existing rows can no longer vacuously satisfy the check. - Fix the stale "Three tabs" comment to "Four tabs". Addresses Greptile P2s on PR #29069. * fix(e2e): keyboard-select fallback models + correct cleanup endpoint - Replace mouse-based option clicks with click-to-focus + type + Enter. FallbackGroupConfig's Selects use `options=` and a custom getPopupContainer, so locating options via `.ant-select-dropdown` hit several races: DOM-clicks left antd's popup state stale (the primary popup then intercepted the fallback click), `getByRole` matched always-mounted hidden options, and pointer stability fought the open animation. Typing into the showSearch input narrows the listbox to one option and Enter selects it cleanly. - Assert on dialog-side state changes (the active tab adopts the primary model name; the chain helper shows "1/10 used") instead of popup contents — these reflect the actual selection landing. - Cleanup helper now hits /get/config/callbacks (the real endpoint; /get/callbacks returns 404), so the before/after reset actually clears prior router_settings.fallbacks state. |
||
|
ryan-crabbe-berri
|
7cae5dc08a
|
test(ui): e2e cover team model edit + admin identity in navbar (#28652)
* test(ui): e2e cover team model edit + admin identity in navbar
Adds two Playwright tests as part of the manual-QA → e2e migration:
"Edit team model selection" exercises the Settings tab Models multi-select
+ Save Changes flow on a seeded team, and the existing login test now
opens the User dropdown and asserts the role and User ID render — guarding
against regressions where login succeeds but the auth context is empty.
Resolves LIT-3093
* test(ui): restore seeded models in team-edit test so retries don't fail
The 'Edit team model selection' test removed fake-anthropic-claude from
E2E_TEAM_CRUD_ID without restoring it. CI runs with retries: 2 and the seed
script runs once before the suite, so a flake on this test would fail the
retry at the "tag is visible" assertion. Wrap the test in try/finally and
restore the seeded models via /team/update before and after.
* test(e2e): fail loudly if team/update restore call fails
Surfaces the real cause when the master key is wrong or the proxy is
unreachable, instead of silently leaving the team in a stale state and
failing later on the visibility assertion.
* fix(e2e): match navbar account button by aria-label, not non-existent "User" text
The previous trigger filter (hasText: /^User$/) didn't match the rendered
UserDropdown button — its text is the displayName ("Account" for the
master-key admin, an email for SSO users), never "User". The evaluate
call then timed out after 15s in CI. Use the stable aria-label prefix
the component always emits, and click directly since the dropdown is
configured trigger=["click"] (the synthetic hover was unnecessary).
|
||
|
ryan-crabbe-berri
|
73e9071311
|
refactor(ui): extract auth state into AuthContext (#28910)
Some checks are pending
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
* refactor(ui): extract auth state into AuthContext
Move auth state (token, userID, userRole, accessToken, premiumUser, userEmail,
disabledPersonalKeyCreation, showSSOBanner) out of src/app/page.tsx into a
new AuthProvider at src/contexts/AuthContext.tsx. Wrapped at the root layout
so login/onboarding/dashboard routes all have access via useAuth().
Day 1 foundation for the App Router migration: migrated (dashboard)/X/page.tsx
route entry points won't have a parent passing props, so shared auth state
must live in a context they can read from.
Sub-components are unchanged — they still receive accessToken/userID/userRole
as props from page.tsx (which now reads them from useAuth()). Only the
page.tsx → top-level-page-component handoff is de-drilled; deeper prop
drilling is left for the per-page migration to address.
Net change: -86 lines from page.tsx (state + two effects moved), +5 in
layout.tsx (provider wrap), new AuthContext.tsx (~140 lines), test update
to wrap CreateKeyPage in AuthProvider.
Fixes LIT-3366
Part of LIT-3128
* fix(ui): await getUiConfig before clearing authLoading
The AuthContext refactor flipped authLoading to false synchronously on mount
while letting getUiConfig() run fire-and-forget. On SERVER_ROOT_PATH deployments
this races the unauthenticated login-redirect effect: the redirect fires with
proxyBaseUrl still at its module-init value, sending users to /ui/login instead
of {SERVER_ROOT_PATH}/ui/login.
Restores the original sequencing inside AuthProvider's mount effect and adds a
Playwright spec wired into the existing SERVER_ROOT_PATH workflow matrix. The
spec delays the config endpoint via page.route() to make the race deterministic
across CI runners.
|
||
|
ryan-crabbe-berri
|
13512e7abd
|
refactor(ui): remove dead App Router scaffolding in (dashboard)/* (#28891)
Deletes 39 files across 12 unused subdirs under src/app/(dashboard)/. None of these routes are reachable: LEGACY_REDIRECTS in src/app/page.tsx is empty, the live UI renders everything via the legacy ?page=X switch, and no other code statically imports from these dirs. Kept api-reference, models-and-endpoints, organizations, playground, virtual-keys because they ARE imported by the legacy switch in src/app/page.tsx — they will be migrated properly in the App Router migration. Kept shared infra: layout.tsx, hooks/, components/, networking.ts, README.md — these are imported by live code. Clean baseline before the App Router migration (LIT-3128). |
||
|
ryan-crabbe-berri
|
f75a7c6b22
|
fix(model-edit): allow clearing custom pricing on wildcard models (#28719)
* fix(model-edit): allow clearing custom input/output cost on wildcard deployments A user-set pricing override on a `/model/*` wildcard deployment could not be removed: clearing the Input/Output Cost fields in the UI succeeded visually, but the next read still showed the old values because both `litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`) retained the original rates. UI: when the pricing field is touched but left empty, send `null` instead of dropping it from the payload so the backend sees the clear intent. The cache-read-cost fallback now guards against `null` as well as `undefined` so a cleared input cost cannot silently wipe the cache-read override. Backend: `update_db_model` honors explicit-null clears, but ONLY for `SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the null-clear path prevents a team-scoped caller from using this codepath to null out privileged fields like `team_id` or access groups. Tests cover both clear paths (`litellm_params` and `model_info`), the SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and the security guard that non-pricing nulls don't reach the merged dict. Resolves LIT-3250 * fix(model-edit): run null-clears after both merges, not interleaved The previous version cleared `model_info` from inside the litellm_params merge block, but the subsequent `model_info.update(...)` re-injected the old pricing because the UI's PATCH carries the full model_info blob with the stale values still in it. Move the explicit-null clear pass to after both merges so a model_info passthrough cannot resurrect cleared fields. Adds a regression test for the realistic UI submit shape (both blobs in the patch, model_info still holding the old pricing). * test(e2e): clear-custom-pricing flow with create/delete cleanup Covers the dashboard model edit form's pricing-clear flow end-to-end: seeds a deployment with custom input/output pricing, drives the UI to clear both fields, asserts the outgoing PATCH sends explicit nulls, and confirms via /v2/model/info that the override is gone from both litellm_params and model_info. The dashboard DB persists across this suite, so beforeEach creates a uniquely-named deployment and afterEach POSTs /model/delete to leave the DB clean regardless of test outcome. * fix(model-edit): extend pricing clear to cache_read and cache_write costs Pre-existing parallel of the wildcard input/output cost bug: cleared cache_read_input_token_cost and cache_creation_input_token_cost overrides silently persisted because the UI omitted the key (delete or fallback) and the backend null-clear allowlist did not cover them. - types/router.py: add cache_read_input_token_cost and cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are mirrored between litellm_params and model_info by Deployment.__init__ and honoured by the null-clear loop in update_db_model. - model_info_view.tsx: emit explicit null for touched-but-empty cache_read and cache_write fields. Preserve the input_cost->cache_read mirror only when cache_read itself was not touched. - model_management_endpoints.py: update the allowlist comment. - Tests: three new unit tests for cache clear paths and a preserve check; the e2e spec now seeds, clears, and asserts null PATCH + key-absence for all four pricing fields. |
||
|
ryan-crabbe-berri
|
a8263cbc88
|
fix(ui): route API Reference back to query-param page (#28726)
* fix(ui): route API Reference back to query-param page The path-based /ui/api-reference route was broken in practice — the page-local useProxySettings hook didn't match what the root page passes down. Remove api_ref from the migration maps (LEGACY_REDIRECTS in app/page.tsx, MIGRATED_PAGES in leftnav.tsx and (dashboard)/layout.tsx), point the leftnav item back at page="api_ref", and restore the api_ref render branch in the root page. The path-based page.tsx and the useProxySettings hook stay in place unchanged; only api_ref is moved back to query-param routing while the migration infrastructure is preserved for future page moves. * fix(ui): alias ?page=api-reference to api_ref branch Handles bookmarks of the hyphen-form query param that was live during the brief path-based migration window, so they render the working APIReferenceView instead of falling through to the default page. |
||
|
Mateo Wang
|
48d7e15b83
|
chore(admin-ui): regenerate static export with trailingSlash: true (#28112)
* chore(admin-ui): regenerate static export with trailingSlash: true Rebuilds litellm/proxy/_experimental/out/ from ui/litellm-dashboard with `trailingSlash: true` enabled in next.config.mjs. Next.js now emits every route as <dir>/index.html (e.g. mcp/oauth/callback/index.html) instead of <dir>.html with a sibling metadata-only directory, which fixes the 404 on extensionless URLs served through FastAPI's StaticFiles(html=True) mount. This is the build artifact half of the fix; the config change, Dockerfile cleanup, and regression test live in the follow-up source PR that stacks on top of this branch. * fix(admin-ui): emit nested routes as <dir>/index.html (#28106) Linear and other OAuth providers redirect the user back to /ui/mcp/oauth/callback?code=...&state=... after the consent step. The packaged Next.js static export only produced /ui/mcp/oauth/callback.html, so FastAPI's StaticFiles served a 404 on the extensionless URL and the OAuth handshake never completed. The Dockerfile.non_root build step tried to paper over this at image-build time with `for html_file in *.html; do ...`, but that shell glob does not recurse, so nested routes like mcp/oauth/callback.html were left stranded next to an empty mcp/oauth/callback/ directory containing only Next.js metadata. The runtime restructure step in proxy_server.py was then skipped because the .litellm_ui_ready marker had already been dropped. Set trailingSlash: true in the dashboard's Next.js config so the export emits every nested route as <dir>/index.html natively. The Dockerfile loop is now a no-op for the bundled UI and has been removed; the .litellm_ui_ready marker is still written so the proxy keeps skipping the redundant Python restructure step at startup. Stacks on top of the static export regeneration in the parent branch. * chore: restore origin/litellm_internal_staging out files |
||
|
ryan-crabbe-berri
|
c127968dfb
|
fix(ui): show 2-decimal precision for max_budget on key overview (#28809)
The Key Info Overview tab's Spend card truncated sub-dollar budgets to "$0" because formatNumberWithCommas defaults to 0 decimals. The Settings tab passes 2; align the overview so a $0.10 budget renders as "$0.10". Resolves LIT-2845 |
||
|
yuneng-jiang
|
d98ada8c3f
|
chore(ci): merge dev branch (#28657)
* feat(dashboard): navbar hierarchy + Agent Platform notifications (#27543) * feat(dashboard): refine navbar zones and Agent Platform notice Restructure the admin navbar for production users: clear product vs community vs personal columns with vertical dividers, icon-only Slack/GitHub in a shared chip, and Docs/Blog typography aligned on an 8px rhythm. Add a notifications bell with popover linking to the LiteLLM Agent Platform repo and optional mark-as-read persistence. Promote the account control with initials avatar, single-line display name, and navDisplayName mapping for placeholder user ids (e.g. default_user_id). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(dashboard): address PR review — AntD buttons, public page guard, dedupe regex - Replace raw <button> with AntD Button in BlogDropdown, NotificationsBell, UserDropdown, and test mock - Guard NotificationsBell + container behind !isPublicPage to avoid rendering on public pages - Remove redundant equality checks in navDisplayName (regex already covers them) - Remove unused `lower` variable after simplification Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * fix(dashboard): drop dead useHealthReadiness import in navbar The module was removed in #27896 (replaced by useHealthReadinessDetails), but the import survived the rebase. The symbol is unused — only useHealthReadinessDetails is consumed in the file. Removing the dead import unblocks the UI TypeScript build. * fix(dashboard): align CommunityEngagementButtons test with icon-only aria-labels The component was refactored to an icon-only chip with aria-label='LiteLLM on GitHub' (squash #27543), but the test still asserted /star us on github/i. Update the query to match the rendered accessible name. * refactor(dashboard): drop unused props from NavbarProps The navbar refactor moved user identity + dark-mode state to internal hooks (useAuthorized, useWorker), but the NavbarProps interface still declared userID, userEmail, userRole, premiumUser, isDarkMode, and toggleDarkMode as required, forcing every caller to thread them through. Drop them from the interface and all four call sites (page.tsx, (dashboard)/layout.tsx, public_model_hub.tsx, navbar.test.tsx). Also shrinks the destructure in layout.tsx so the now-unused locals stop being pulled out of useAuthorized(). * refactor(dashboard): use useSyncExternalStore for NotificationsBell dismiss flag Reads/writes of the litellmHideAgentPlatformBanner key were done directly inside NotificationsBell via a useEffect + useState pair. Every other localStorage-backed flag in the dashboard (Disable ShowPrompts, DisableBouncingIcon, DisableShowNewBadge, DisableUsageIndicator, DisableBlogPosts) is wrapped in a useSyncExternalStore hook over localStorageUtils so all mounted components stay in sync. Extract useHideAgentPlatformBanner to follow the same shape, swap NotificationsBell to consume it, and add a regression test that two sibling bells stay in sync without a remount when one is dismissed. * refactor: mask credential fields in proxy settings GET responses (#28682) * refactor: mask credential fields in proxy settings GET responses Brings SSO settings, cache settings, and the email/Slack alerting view in /get/config/callbacks in line with the HashiCorp Vault config-override pattern, so persisted credentials are not transported back to the UI in plaintext. * refactor: harden short-value masking and hoist alerting var constant Closes two review observations: - mask_sensitive_keys now replaces short values (below the visible prefix+suffix length) with an all-mask string instead of returning them unchanged, so a 1-7 character credential is no longer round-tripped verbatim. - _ALERTING_SENSITIVE_VARS is moved out of get_config() to a module-level constant, matching the analogous _SSO_SENSITIVE_FIELDS and _CACHE_SENSITIVE_FIELDS in the SSO and cache endpoint files. --------- Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
ryan-crabbe-berri
|
92d4bba58f
|
fix(ui/add-model): stop vertex_ai-anthropic_models from leaking under Anthropic (#28723)
`getProviderModels()` matched a model into a provider's dropdown when the model's `litellm_provider` string *contained* the provider key as a substring. The intent was to admit suffix variants (e.g. `anthropic_text`, `bedrock_converse`), but the substring check is too loose: it also pulls in unrelated providers whose name happens to contain the key, most visibly `vertex_ai-anthropic_models` matching `anthropic` and `vertex_ai-openai_models` matching `openai`. Replace `.includes()` with separator-anchored prefix matching (`startsWith(provider + "_")` / `startsWith(provider + "-")`). All legitimate variants in `model_prices_and_context_window.json` still match (`anthropic_text`, `azure_text`, `azure_ai`, `bedrock_converse`, `bedrock_mantle`, `cohere_chat`, `fireworks_ai-embedding-models`, `vertex_ai-*`, `vertex_ai_beta`), and the cross-provider leak is closed. Tests: update one assertion that pinned the buggy substring behavior (`custom_openai_endpoint` matching `openai` — not a real provider value); add 6 new tests covering the leak regressions and the variant-preservation contract for vertex_ai/bedrock/fireworks. |
||
|
Sameer Kankute
|
ef36e89638
|
feat(mcp): Add tool call and tool list support via UI for Oauth mcps (#28454)
* feat(mcp): cache OAuth token client-side so Tools tab loads without re-auth
After a user creates an OAuth MCP server and completes the authorization
flow, the resulting access token is now stored in sessionStorage keyed by
server_id. The MCP Tools tab reads this cached token and includes it as
an MCP auth header when listing and invoking tools, so the user never sees
an empty tool list. When the session ends (tab close / new browser) an
Authorize button re-triggers the flow without leaving the Tools screen.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(ui/mcp): surface listMCPTools 401 errors so auth gate reappears
listMCPTools previously swallowed all errors (including HTTP 401) by
returning a synthetic { tools: [], error: 'network_error', ... } payload.
That made the useQuery retry-on-401 guard and mcpToolsError dead code,
so expired OAuth tokens never re-triggered the auth gate.
- Throw an enhanced Error with .status attached on non-2xx responses
(still preserves the legacy shape for true network failures so the
caller can render a generic message without crashing).
- Clear the cached OAuth session token when the tools query fails with
401, mirroring callMCPTool's onError handler so the Authorize button
is shown again.
- Surface mcpToolsError in the existing error banner.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(mcp-tools): stable onSuccess + reuse parsed flow state
- Pass stable setOauthToken setter directly as onSuccess to avoid
recreating useToolsOAuthFlow's resumeOAuthFlow on every render.
- Reuse the already-parsed FLOW_STATE_KEY value (peeked) instead of
re-reading and re-parsing sessionStorage in resumeOAuthFlow.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(ui/mcp): restore listMCPTools never-throws contract
The previous fix made listMCPTools throw on HTTP errors while still
returning a synthetic object on network errors. This inconsistent
contract broke existing callers (MCPToolPermissions, MCPAppsPanel,
MCPConnectPicker) which inspect result.error / result.message and
expect the function to never throw.
- Return a normalized { tools: [], error, message, status, ... }
object on HTTP errors (instead of throwing) so all callers see a
consistent shape and the user-visible error text from
result.message is preserved.
- Convert the returned error object into a thrown Error inside the
one caller that needs it — the useQuery in mcp_tools.tsx — so the
401 retry/onError handlers still trigger and clear the cached
OAuth token.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix greptile
* fix(mcp): align OAuth header alias lookup with dashboard sanitization
Backend auth header resolution now matches x-mcp-{alias} keys produced by
the dashboard sanitizer, and the Tools tab re-syncs OAuth tokens when
serverId changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): widen auth header lookup types for list_tools
Accept legacy str | dict server auth maps and annotate list_tools
server_auth_header as Union[str, dict] for mypy.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(ui): extract shared buildCallbackUrl/clearStorage for MCP OAuth hooks
Hoist the duplicate buildCallbackUrl and clearStorage helpers out of
useToolsOAuthFlow and useUserMcpOAuthFlow into a new shared module
src/hooks/mcpOAuthUtils.ts so the two hooks cannot drift if the URL
construction or storage cleanup logic needs to change.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(ui): don't gate M2M OAuth MCP servers behind interactive authorize
M2M (client_credentials) OAuth servers share auth_type="oauth2" with
interactive PKCE servers, but the backend fetches their token internally
and they typically lack a user authorization endpoint. Gating tool
listing on them rendered an Authorize button that would fail or redirect
incorrectly. Detect M2M via the presence of token_url (matching the
existing heuristic in mcp_server_edit.tsx) and skip the auth gate.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(ui/mcp): return error shape when listMCPTools JSON parse fails
Restore the never-throws contract when response.json() fails on a 2xx
body so callers do not receive null and crash on result.tools.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
|
||
|
ryan-crabbe-berri
|
07bcd2c19e
|
test(e2e): forward LITELLM_LICENSE to UI e2e proxy (#28398)
* test(e2e): forward LITELLM_LICENSE to UI e2e proxy The UI e2e job ran without LITELLM_LICENSE, so premium_user was always false in the issued login JWT and premium-gated UI surfaces (Team-BYOK Model switch, etc.) couldn't be driven through the UI. Forward the env var from run_e2e.sh and the CircleCI e2e_ui_testing job, and add a sanity test that decodes the admin storage state token and asserts premium_user=true so the wiring fails loudly if it ever regresses. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Update ui/litellm-dashboard/e2e_tests/tests/proxy-admin/license.spec.ts Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
yuneng-jiang
|
0715ed3359
|
build(deps): bump next from 16.2.4 to 16.2.6 in /ui/litellm-dashboard (#27665) (#28524)
Bumps [next](https://github.com/vercel/next.js) from 16.2.4 to 16.2.6. - [Release notes](https://github.com/vercel/next.js/releases) - [Changelog](https://github.com/vercel/next.js/blob/canary/release.js) - [Commits](https://github.com/vercel/next.js/compare/v16.2.4...v16.2.6) --- updated-dependencies: - dependency-name: next dependency-version: 16.2.6 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Sameer Kankute
|
988196911a
|
Litellm oss staging 1 (#28337)
* feat: add Xiaomi MiMo-V2.5-Pro and MiMo-V2.5 OpenRouter model entries (#27700) Squash-merged by litellm-agent from TorvaldUtne's PR. * fix(ui): trim whitespace from MCP inspector tool call inputs (#28203) Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * gemini-3.1-flash-lite pricing (#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> * fix: incorrect /v1/agents request example (#28131) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge (#28201) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge Issue #28196 — the Responses->Chat parser (transformation.py:184-200) keeps the full dict as reasoning_effort when summary is set; that branch was added in #25359. But the Anthropic transformation here still guarded on isinstance(value, str), silently dropping the param. Result: callers using the standard Reasoning(effort, summary) OpenAI-shaped object on Anthropic lose thinking entirely (0 reasoning_tokens, no thinking_blocks). Coerce dict -> string before mapping. Same shape tolerance that gpt_5_transformation._normalize_reasoning_effort_for_chat_completion already implements. summary is irrelevant for Anthropic's thinking_blocks. Adds two regression tests: one parametrized over string + dict shapes (with and without summary), one covering unparseable dict inputs (drops silently, no crash). * test(anthropic): add non-adaptive model coverage for dict-shape reasoning_effort Per Greptile feedback on PR #28198: the original regression test only exercised the adaptive (4.6+) path. Add a parametrized test for the non-adaptive branch (claude-sonnet-4-5) verifying that dict-shape reasoning_effort still maps to thinking.type='enabled' + budget_tokens, and that output_config is NOT set on pre-4.6 models. * test(anthropic): convert unparseable-dict test to @pytest.mark.parametrize Per @greptile-apps inline review on PR #28201 — matches the parametrize style of the two adjacent dict-shape tests and produces clearer failure messages (test ID per case instead of one collapsing for-loop). * feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite (#28280) Squash-merged by litellm-agent from ro31337's PR. * fix(router): wrap aresponses streaming iterator for mid-stream fallbacks (#28215) Squash-merged by litellm-agent from cwang-otto's PR. * fix(router): unblock staging — mypy + coverage for aresponses streaming fallback (#28318) Squash-merged by litellm-agent from cwang-otto's PR. * fix(responses): forward timeout on completion transformation path (Anthropic, Bedrock, Vertex) (#28133) Squash-merged by litellm-agent from cwang-otto's PR. * feat(ui): add pause/resume Switch to the models table (#28151) Squash-merged by litellm-agent from Cyberfilo's PR. * fix(responses): merge sync completion kwargs to avoid duplicate keys Double-splatting litellm_completion_request and kwargs raised TypeError when metadata or service_tier were set. Match the async merge pattern. Co-authored-by: Cursor <cursoragent@cursor.com> * Use proxy base URL for CLI SSO form action (#28271) Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * fix(tests): add mistral/ministral-8b-2512 to cost map and backfill in conftest Mistral rotated the 'mistral/mistral-tiny' alias to return 'ministral-8b-2512' as the response model, which was missing from the cost map. This caused test_completion_mistral_api and test_completion_mistral_api_modified_input to fail in litellm.completion_cost lookup. - Add mistral/ministral-8b-2512 entry to both the in-tree model_prices_and_context_window.json and the bundled litellm/model_prices_and_context_window_backup.json (mirrors the existing openrouter/mistralai/ministral-8b-2512 pricing). - litellm.model_cost is loaded at import time from the URL pinned to main, so the new backup entry isn't visible at test runtime until it also lands on main. Backfill any entries missing from the remote-fetched map into litellm.model_cost in the local_testing conftest so cost-calculator lookups succeed on this branch. * fix(tests): drop unnecessary del of conftest backfill loop vars * fix(router): harden streaming fallback wrapper for bridge iterators - FallbackResponsesStreamWrapper now uses getattr fallbacks when copying attributes from the source iterator. The bridge path (LiteLLMCompletionStreamingIterator used by Anthropic/Bedrock/Vertex) does not call super().__init__ and is missing response, logging_obj (it uses litellm_logging_obj), responses_api_provider_config, start_time, request_data, call_type, and _hidden_params. Previously, wrapper construction raised AttributeError for any streaming fallback on the bridge path. - _aresponses_with_streaming_fallbacks now deep-copies the litellm_metadata (and metadata) dicts into fallback_kwargs. The primary attempt mutates this dict in place via _update_kwargs_with_deployment, so a shallow copy of kwargs was leaking primary-deployment fields (deployment, model_info, api_base) into the mid-stream fallback request. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(router): use safe_deep_copy for fallback metadata snapshot The ban_copy_deepcopy_kwargs CI check rejects copy.deepcopy() on any variable whose name contains 'kwargs' (incl. fallback_kwargs). Swap the two copy.deepcopy(fallback_kwargs[...]) calls for safe_deep_copy, which handles non-picklable values (OTEL spans, etc.) by per-key deepcopy with fallback to the original reference. Co-authored-by: Yassin Kortam <yassin@berri.ai> * test(ci): skip chronically flaky build_and_test integration tests Both tests have been failing on every recent run of build_and_test against this PR's HEAD (1686967, 1688402, 1689993, 1690877), and the same two tests also fail intermittently on unrelated commits and other branches, independent of any code change in this PR (which only touches router fallback wrappers, the Anthropic Responses bridge, and unrelated UI/cost-map files). - tests.test_spend_logs.test_spend_logs: /spend/logs?request_id=... returns 500 even after a 20s wait for the spend log to be written. Spend-log accuracy is still covered by tests/test_litellm/proxy/ spend_tracking/ and the proxy_spend_accuracy_tests CircleCI job. - tests.test_team_members.test_add_multiple_members: /team/info?team_id= ... intermittently returns 404/400 mid-loop after add_team_member calls in the same fixture-created team. Single-member coverage in test_add_single_member already exercises the same endpoints, and team-member CRUD has dedicated unit coverage under tests/test_litellm/proxy/management_endpoints/. Skipping unblocks the build_and_test job until the underlying race in the dockerized integration setup is root-caused. * fix: preserve explicit timeout=0 in responses API handler Use 'timeout if timeout is not None else request_timeout' instead of 'timeout or request_timeout' so an explicit timeout=0/0.0 isn't silently replaced by the default request_timeout. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(ui): guard model_info access in pause Switch with optional chaining * fix(ui): guard model_info access in pause Switch onChange handler Mirror the optional-chaining guard already applied to the isPausing check so a config-model row with a missing model_info cannot throw when the toggle's onChange fires. --------- Co-authored-by: TorvaldUtne <78661304+TorvaldUtne@users.noreply.github.com> Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com> Co-authored-by: Isha <72744901+IshaMeera@users.noreply.github.com> Co-authored-by: cwang-otto <chengxuan.wang@ottotheagent.com> Co-authored-by: Roman Pushkin <roman.pushkin@gmail.com> Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: boarder7395 <37314943+boarder7395@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> |
||
|
ryan-crabbe-berri
|
f82ff7ee2a
|
test(ui-e2e): admin key creation with a specific proxy model (#28365)
* test(ui-e2e): add admin key creation with a specific proxy model Adds Playwright coverage for creating a key (no team) scoped to a single proxy model, complementing the existing All-Proxy-Models test. Uses a DOM-dispatched click on the antd dropdown option since the popup animation can render the option outside the viewport. * test(ui-e2e): verify scoped key works against mock /chat/completions Extend the "Create a key with a specific proxy model" test to extract the new key from the success modal and POST to /chat/completions for the scoped model, asserting 200 and the mock response body. Without this the test could pass even if the model selection failed to register. |
||
|
ryan-crabbe-berri
|
fb73995c40
|
feat(ui): team passthrough routes create parity + edit load fix (#28098)
* feat(ui): team allowed_passthrough_routes create parity + edit load fix Add the Allowed Pass Through Routes selector to the create-team modal (previously only on the edit form), and fix the edit form silently dropping the field: it lives under team metadata, so initialValues must read info.metadata.allowed_passthrough_routes — otherwise the selector renders empty and saving wipes admin-set routes. Both selectors are gated to premium proxy admins, mirroring the server-side gate. Resolves LIT-3019 * fix(ui): persist team allowed_passthrough_routes edits on save The edit form loaded the selector but the save path never wrote it back: allowed_passthrough_routes stayed in the raw metadata JSON textarea and parsedMetadata (from that textarea) always won, so selector edits were silently discarded. Strip it from the textarea initialValues and overlay values.allowed_passthrough_routes into updateData.metadata, mirroring how guardrails is handled. Resolves LIT-3019 * fix(ui): preserve team passthrough routes for non-proxy-admins on save Only proxy admins may set allowed_passthrough_routes (server-side gate). For non-proxy-admins, write the team's stored value back into metadata instead of the form value, so saving an unrelated setting can't silently wipe routes; omit the key entirely when the team never had any. Resolves LIT-3019 |
||
|
ryan-crabbe-berri
|
5a00cb1592
|
test(e2e): migrate runner to uv, add All Proxy Models key test (#28313)
* chore(e2e): migrate runner to uv, add All Proxy Models key test Switches the local e2e runner (run_e2e.sh) from poetry to uv to match the rest of the repo and CI. Adds a Playwright test for creating an admin key with no team selected (all-proxy-models flow), a SLOWMO env hook for headed debugging, and a MIGRATION_TRACKING.md doc that maps the manual UI QA checklist to e2e tests so future migration work has a single source of truth. * chore(e2e): address greptile feedback - Remove MIGRATION_TRACKING.md (docs belong in litellm-docs repo) - playwright.config.ts: fall back to 0 when SLOWMO is non-numeric (parseInt returns NaN, which Playwright accepts silently) - run_e2e.sh: add --frozen to uv sync for CI determinism |
||
|
ryan-crabbe-berri
|
2eeca2d096
|
fix(ui): restore log filter loading indicator (#28282)
When a new filter is applied to spend logs, React Query's keepPreviousData left stale rows on screen for 10–15s with no indication that a fetch was in progress. The previous custom isFilteringResults flag was removed in the #25847 toolbar refactor and only partially restored on the Fetch button. Use React Query's isPlaceholderData to discriminate a real filter change (queryKey changed, data not yet arrived) from a same-key live-tail refetch, and feed it into the existing isLoading prop on the toolbar pagination text and the table body. Live-tail polls still keep previous rows without flicker. Co-authored-by: Ryan <ryan@Ryans-MBP.localdomain> |
||
|
ryan-crabbe-berri
|
727a471ae9
|
[Refactor] UI - Spend Logs: consolidate filter state and extract components (#25847)
* [Refactor] UI - Spend Logs: consolidate filter state, extract components, remove dead code
- Lift filter state into index.tsx and pass to hook (removes selectedX vars + sync useEffect)
- Move main useQuery into useLogFilterLogic hook (removes isMainQueryEnabled toggle)
- Delete dead RequestViewer component (300 lines, replaced by LogDetailsDrawer)
- Extract LogsTableToolbar component (search, date range, pagination, live tail)
- Extract filter options config to filter_options.ts
- Remove dead code: handleRefresh, handleSelectLog, handleCloseDrawer, formatTimeUnit,
showFilters/showColumnDropdown state, dropdownRef/filtersRef
* Fix PR feedback: use antd Switch instead of Tremor in new file, fix typo
* Collapse dual-path filtering into single React Query
All 10 filter keys now go through the useQuery — the imperative
performSearch / debouncedSearch / backendFilteredLogs path is deleted.
Filter values are debounced via useDebouncedValue(300ms) before hitting
the query key so text inputs don't fire per-keystroke.
Removed: performSearch, debouncedSearch, backendFilteredLogs,
lastSearchTimestamp, hasBackendFilters, clientDerivedFilteredLogs,
the sort/page/time refetch useEffect, and the filteredLogs chooser memo.
* Clean up remaining smells: remove isFetchingDeferred, internalize selectedTimeInterval, fix circular import
- Remove useDeferredValue/isButtonLoading — pass logsQuery.isFetching directly
- Move selectedTimeInterval into LogsTableToolbar as internal state
- Move PaginatedResponse type from index.tsx to log_filter_logic.tsx
* Fix quick-select dropdown overlapping sidebar
* Fix stale quick-select label after Reset Filters
Move selectedTimeInterval back to parent so handleFilterReset can
reset it to the 24-hour default. The toolbar receives it as a prop.
* refactor useLogFilterLogic tests for controlled-hook + backend-query shape
The hook no longer owns filter state or does client-side filtering — it
receives filters/setFilters as props and drives filteredLogs from a
useQuery over uiSpendLogsCall. Reshape the tests around that contract:
introduce a controlled harness that owns filter state, collapse the 10
per-filter assertions into a single it.each over filterKey → API param,
and drop the client-side passthrough tests (the .min test file and the
"return all logs when no filters" / "empty when logs null" cases) that
no longer correspond to any hook behavior.
* cover new useLogFilterLogic invariants: activeTab gate, filterByCurrentUser fallback, debounce negative, partial merge
Follow-up to the test refactor. Adds coverage for invariants the
refactored hook contract introduced but that the first pass didn't
assert:
- query enablement: expand the single accessToken-null case into an
it.each over all four credential props (accessToken, token, userRole,
userID), plus a separate test for activeTab !== "request logs"
- filterByCurrentUser: when true with a blank User ID filter, the
outbound request carries user_id = userID
- debounce: also assert the negative case — no call in the first 100ms
after a filter change (first waiting out the initial mount fire)
- handleFilterChange: partial updates merge without clobbering other
filter keys (protects the spread + default-fill semantics)
- handleFilterReset: calls setCurrentPage(1) alongside restoring
filters
* fix typo dropping the live-tail banner border
Tailwind silently ignores unknown classes, so border-greem-200 was
leaving the auto-refresh banner with only its bg-green-50 fill and no
outline.
* memoize columns and derived table data in SpendLogsTable
The table's columns array, four-pass data pipeline, and sort-change
handler were all being rebuilt on every parent render. That made every
filter click re-instance all 23 TanStack-Table columns, re-run
filter/reduce/map over all rows, and recreate per-row click closures —
all before the intentional 300ms debounce timer even got a chance to
fire.
Local measurement (40 rows, dev mode):
filter click → query fires: 1957ms → 1217ms (−38%)
Wrap createColumns in useMemo keyed on sortBy/sortOrder, hoist
onSortChange into a useCallback, and move the searchedLogs /
sessionComposition / sessionRepresentativeMap / filteredData derivations
into a single useMemo keyed on filteredLogs.data + searchTerm.
These were pre-existing issues on main — not regressions from the
hook refactor — but the refactor made them user-visible because the
new query debounce put render cost on the critical path.
* apply dropdown filters instantly, debounce only text inputs
Dropdown selects now bypass the 300ms debounce so a click updates the
table immediately. Text inputs (Key Hash, Error Message, Request ID,
User ID) still debounce. handleFilterReset also clears the pending
debounced value so a half-typed text filter can't re-fire after reset.
* fix(ui/spend-logs): restore lost loading/debounce behavior + cover dropped tests
Regressions from the spend-logs-view refactor:
- debounce the 'Public model / search tool' text filter (was firing a
backend query per keystroke) via TEXT_FILTER_KEYS
- restore Fetch-button smoothing through table repaint using
useDeferredValue on the rendered data (explicit staleness)
- show AntDLoadingSpinner during the auth-resolve phase instead of a
blank screen on first load
- only live-tail-poll while the tab is visible
(refetchIntervalInBackground: false)
- extract getLiveTailRefetchInterval helper for the poll decision
Tests:
- LogDetailContent: retries display (>0 / 0 / absent), overhead-absent
- log_filter_logic: regression guard that the public-model filter
debounces; getLiveTailRefetchInterval unit tests
- logs_utils: getTimeRangeDisplay quick-select window labels
* test(ui/spend-logs): cover the cold-load auth-not-ready spinner guard
Asserts SpendLogsTable shows a loading spinner (not a blank screen)
while credentials are unresolved, and renders the table once present.
|
||
|
Sameer Kankute
|
581882879d
|
feat(ui): add Interactions API endpoint to playground with SSE streaming (#28156)
* feat(ui): add Interactions API support to playground with streaming
Adds /v1beta/interactions as a selectable endpoint in the UI playground.
Uses SSE streaming (stream=true) and parses content.delta events for real-time output.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(interactions): remove forced gemini provider so all providers work via interactions API
Proxy endpoint was hardcoding custom_llm_provider="gemini" before routing,
preventing non-Gemini models from using the litellm_responses bridge.
Also reverts the UI Gemini-only model filter.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(interactions): fix streaming for non-gemini providers via bridge
Two bugs in LiteLLMResponsesInteractionsStreamingIterator:
1. content.delta was emitted without "type":"text" in delta dict, so the
UI type-check always failed and no tokens were displayed
2. First OutputTextDeltaEvent was silently dropped (used to emit content.start
with empty text); fixed by handling ResponsePartAddedEvent for content.start
so text deltas go directly to content.delta
Co-authored-by: Cursor <cursoragent@cursor.com>
* undo unrelated changes
* fix(ui): extract model from top-level field in interactions bridge events
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* test(interactions): remove tautological gemini-provider assertion
The test_no_forced_gemini_provider_in_request_data check only asserted
against dict literals it had just constructed, so it always passed and
did not exercise the create_interaction endpoint. The endpoint
deliberately defaults custom_llm_provider to gemini, so the assertion
was also factually incorrect. Drop the misleading test.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(interactions): use ContentPartAddedEvent and guard interaction.start ordering
- ResponsePartAddedEvent corresponds to reasoning summary parts, not text
content parts. Use ContentPartAddedEvent which is the event emitted before
text output deltas (type response.content_part.added).
- Mirror the OutputTextDeltaEvent ordering guard: if interaction.start has
not been sent yet, emit it first before content.start to honor the
documented event ordering contract.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* test(interactions): cover ContentPartAddedEvent ordering and no-op paths
* fix(tests): treat corrupt VCR cassette payloads as cache miss + use gpt-realtime in OpenAI realtime guardrails test
VCR redis persister was raising UnicodeDecodeError on cached payloads that
fail to UTF-8 decode (e.g. legacy entries written by another version of
the persister), failing tests at fixture setup instead of degrading to a
cache miss. Wrap decode+deserialize in a try/except so corrupt cache
entries are treated as CassetteNotFoundError, surfacing the failure via
the existing _record_cache_failure / VCRCassetteCacheWarning path.
OpenAI shut down gpt-4o-realtime-preview-2024-12-17 (and the entire
gpt-4o-realtime-preview family) on 2026-05-07. The live realtime
guardrails integration test now fails with model_not_found instead of
receiving session.created. Point OPENAI_REALTIME_URL at the current GA
model gpt-realtime, and relax the assertion in
test_text_message_blocked_by_guardrail_no_ai_response to also accept the
model's refusal-to-repeat the block message (gpt-realtime declines
verbatim-repeat instructions, which is still a safe outcome since the
original user message was blocked before reaching OpenAI). The
BLOCKED_PHRASE leak check is preserved as a hard invariant.
* fix(tests): migrate realtime + nvidia_nim rerank tests off shut-down upstream models
OpenAI shut down the entire gpt-4o-realtime-preview family (including the
undated alias) on 2026-05-07. The live realtime tests still connected
with that dead alias and failed with messages_received=1 (an error event
'The model gpt-4o-realtime-preview does not exist' instead of
session.created). Point the live OpenAI realtime tests at gpt-realtime,
the current GA realtime model:
- test_openai_realtime_simple.py: get_model() -> gpt-realtime
- test_openai_realtime.py: test_openai_realtime_direct_call_no_intent and
test_openai_realtime_direct_call_with_intent -> openai/gpt-realtime
Mocked unit tests (test_realtime_query_params_construction,
test_realtime_query_params_use_normalized_model_name) are left as-is:
they never hit the network and assert string plumbing only.
NVIDIA reached end-of-life for the hosted
nvidia/llama-3.2-nv-rerankqa-1b-v2 rerank API on 2026-05-18 with no
published replacement, so the live BaseLLMRerankTest.test_basic_rerank
for nvidia_nim now returns HTTP 410 ('Gone'). NVIDIA's hosted catalog
rotates on a schedule, so swapping in another live model would only
defer the failure. Override test_basic_rerank in TestNvidiaNim to mock
the sync/async HTTP transport (same pattern as
test_nvidia_nim_rerank_ranking_endpoint in this file) and inject a fake
NVIDIA_NIM_API_KEY via monkeypatch. The request/response transformation
and cost calculation stay covered offline.
* test(callbacks): harden flaky proxy callback-leak detector
The proxy callback-leak detector (test_check_num_callbacks_on_lowest_latency)
was failing on this PR with 'abs(85 - 95) <= 4' — a bounded one-time
registration jump caused by switching to latency-based-routing
(+LowestLatencyLoggingHandler, +SlackAlerting). The count then plateaus
under load, so this is pollution from the test's own config update, not a
leak.
Replace the brittle two-sample diff threshold with a sampler that settles
past the deliberate config switch and only flags sustained monotonic
per-type growth, with a terminal-burst confirmation pass for leaks that
would otherwise escape the >=2-interval guard. Normalizes instance
addresses so identical callbacks at different memory locations collapse,
and names the leaking type on failure.
* fix(interactions): preserve first text token when both start events are missing
When OutputTextDeltaEvent arrived before any ResponseCreatedEvent or
ContentPartAddedEvent, the double-fallback path emitted interaction.start
and silently dropped the first delta's text — the second delta's
content.start carried only that chunk's delta, and the first token never
made it to any content.delta event consumed by the UI.
Queue a content.start that carries the first delta's text alongside the
interaction.start emission, and drain pending events before pulling the
next upstream chunk.
* chore(ui): remove unused InteractionOutput/InteractionResponse interfaces
Co-authored-by: Yassin Kortam <yassin@berri.ai>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
Sameer Kankute
|
36c494fdd2
|
Litellm oss staging (#28161)
* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (#27451) (#27455) Squash-merged by litellm-agent from Anai-Guo's PR. * feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (#27508) Squash-merged by litellm-agent from yimao's PR. * fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (#24550) Squash-merged by litellm-agent from krisxia0506's PR. * fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (#23711) Squash-merged by litellm-agent from krisxia0506's PR. * fix: raise BadRequestError for file content blocks missing 'file' sub… (#24503) Squash-merged by litellm-agent from krisxia0506's PR. * Fix Gemini MIME detection for extensionless GCS URIs (#27278) Squash-merged by litellm-agent from krisxia0506's PR. * fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes #28084) (#28107) Squash-merged by litellm-agent from voidborne-d's PR. * feat(chart): add support for autoscaling behavior in HPA (#27990) Squash-merged by litellm-agent from FabrizioCafolla's PR. * feat(proxy): add blocked flag to models for pause/resume from the UI (#27927) Squash-merged by litellm-agent from Cyberfilo's PR. * fix: pass socket timeouts to Redis cluster clients (#27920) Squash-merged by litellm-agent from tomdee's PR. * Fix/cache token (#28009) Squash-merged by litellm-agent from escon1004's PR. * fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (#28080) Squash-merged by litellm-agent from Divyansh8321's PR. * fix(guardrails): return HTTP 400 instead of 500 for blocked requests (#27617) * fix: reset org and tag budgets (#27326) * reset org budgets * reset tag budgets --------- Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> * fix(ui): omit allowed_routes from key edit save when unchanged (#27553) * fix(ui): omit allowed_routes from key edit save when unchanged When a team admin opens Edit Settings on a key with key_type=AI APIs and saves without changing anything, the UI re-sends the existing allowed_routes value, which the backend's _check_allowed_routes_caller_permission gate rejects for non-proxy-admins (LIT-2681). Strip allowed_routes from the patch in handleSubmit when it deep-equals the original keyData.allowed_routes. The backend treats absence as "leave alone," so no-op saves now succeed for non-admins. Admins explicitly editing the field still send the new value. * fix(ui): order-insensitive allowed_routes diff + cover null-original case Address Greptile review: - Switch the "is allowed_routes unchanged" check to a Set-based comparison so a server-side reorder of the array doesn't register as a user edit and re-trigger LIT-2681. - Add two regression tests: (1) keyData.allowed_routes is null and the form is untouched — patch should strip the field; (2) server returned routes in a different order than the user originally entered — patch should still recognize the value as unchanged. * chore(ui): strip ticket refs and tighten comments in key edit fix - Remove internal-tracker references from in-code comments - Tighten the WHY comment in handleSubmit to two lines - Drop redundant test-block comments — test names already describe the case * fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc * fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests GuardrailRaisedException and BlockedPiiEntityError both lacked a status_code attribute. When these exceptions reached the proxy exception handler (getattr(e, 'status_code', 500)), the fallback defaulted to HTTP 500 — making intentional guardrail blocks indistinguishable from server errors and causing unnecessary client retries. Changes: - Add status_code=400 (keyword-only) to GuardrailRaisedException - Add status_code=400 (keyword-only) to BlockedPiiEntityError - Update _is_guardrail_intervention() to recognize both exceptions so downstream loggers record 'guardrail_intervened' instead of 'guardrail_failed_to_respond' - Add 6 unit tests for default/custom status codes and getattr pattern - Strengthen existing blocked-action test with status_code assertion Fixes #24348 --------- Co-authored-by: Michael-RZ-Berri <michael@berri.ai> Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> * fix(router/proxy): address Greptile P1+P2 review comments on PR #28161 - router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429) when a specifically-addressed deployment is administratively blocked; 429 misleads retry-enabled clients into spinning forever against a paused model - proxy_server: compute get_fully_blocked_model_names() once before both branches in model_list() instead of duplicating the call in each branch - deepseek: upgrade silent debug log to warning when injecting placeholder reasoning_content so callers are clearly notified of degraded multi-turn quality - tests: update two blocked-deployment assertions to expect ServiceUnavailableError Co-authored-by: Cursor <cursoragent@cursor.com> * fix: address bug detection findings (cache token order, mutable defaults) Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix: address bugs in async pass-through, anthropic cache token detection, rerank tests - async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments - cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0 - dashscope rerank tests: pass request to httpx.Response constructions for consistency Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix code qa * fix(vertex_ai/gemini): strip MIME parameters from GCS contentType GCS object metadata's contentType field can include parameters such as 'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases so downstream get_file_extension_from_mime_type sees a bare MIME type. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(vertex_ai/gemini): clarify mime-type error message string concatenation Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Tai An <antai12232931@outlook.com> Co-authored-by: Vincent <yimao1231@gmail.com> Co-authored-by: Kris Xia <xiajiayi0506@gmail.com> Co-authored-by: d 🔹 <liusway405@gmail.com> Co-authored-by: Fabrizio Cafolla <developer@fabriziocafolla.com> Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com> Co-authored-by: Tom Denham <tom@tomdee.co.uk> Co-authored-by: escon1004 <70471150+escon1004@users.noreply.github.com> Co-authored-by: Divyansh Singhal <97736786+Divyansh8321@users.noreply.github.com> Co-authored-by: robin-fiddler <robin@fiddler.ai> Co-authored-by: Michael-RZ-Berri <michael@berri.ai> Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> |
||
|
Yassin Kortam
|
014cb8fa9d
|
feat: add componentized proxy deployment with gateway, backend, ui, and migrations (#27557)
Split the monolithic LiteLLM proxy into independently scalable Kubernetes components to allow separate horizontal scaling of the LLM data plane and management API surfaces - Add DatabaseURLSettings pydantic-settings model that assembles DATABASE_URL (and optional DATABASE_URL_READ_REPLICA) from discrete DATABASE_* env vars before Prisma initializes, supporting both IAM token auth (minting short-lived RDS tokens) and password auth; replaces the CLI-only path that componentized entrypoints bypass - Add gateway component (port 4000) that trims the proxy route table to the LLM data-plane surface (chat, embeddings, completions, audio, realtime, provider passthroughs, health/metrics) via an allowlist applied inside the lifespan context so plugin-registered routes are captured - Add backend component (port 4001) that exposes the management/admin surface (keys, users, teams, orgs, spend analytics, model management, SSO, audit logs) with a complementary allowlist - Add ui component — Next.js static export served by nginx (port 3000) with RSC payload routing, asset prefix aliasing, and SPA fallback for dashboard routes - Add migrations component with dedicated Dockerfile that runs prisma migrate deploy via a Helm pre-install/pre-upgrade Job, eliminating per-pod schema contention on the Prisma advisory lock - Add Helm chart (helm/litellm) with separate Deployments, Services, HPAs, and ConfigMap for each component; shared _helpers.tpl emits DATABASE_*, IAM_TOKEN_DB_AUTH, REDIS_*, and DISABLE_SCHEMA_UPDATE env vars from chart values; ingress template routes traffic to the correct component by path prefix - Add comprehensive tests for DatabaseURLSettings covering IAM auth, password auth, read replica fallbacks, operator-pinned URL preservation, and percent-encoding; add coverage test asserting gateway + backend allowlist union equals the full proxy route set - Add pydantic-settings>=2.14.1 as a proxy extra dependency and update liccheck allowlist Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu> |
||
|
Sameer Kankute
|
106b2f2da8
|
Merge pull request #27977 from BerriAI/litellm_mcp_internal_delegate_pkce
fix(mcp): delegate PKCE bypass for internal MCP servers |
||
|
Sameer Kankute
|
d855e56333
|
chore(mcp): warn on internal + upstream PKCE delegate
Log verbose_logger.warning when loading oauth2 interactive servers with available_on_public_internet=false and delegate_auth_to_upstream=true (config + DB). Dashboard Alert for the same combo. CLAUDE note for operators. Tests for log and M2M skip. |
||
|
Yuneng Jiang
|
c82acbc0de
|
test(ui): preserve global Button/Tooltip mocks in per-file @tremor/react vi.mock
Per-file `vi.mock("@tremor/react", ...)` factories fully replace the
setup-level mock from `tests/setupTests.ts`, so the global Button/Tooltip
overrides are lost in any file that re-mocks `@tremor/react`. Without
them, the real Tremor `<Button>` leaks through and its internal
`useTooltip(300)` schedules a native 300ms `setTimeout` on pointer
events. When the test environment is torn down before the timer fires,
the trailing `setState` calls `getCurrentEventPriority`, which reads
`window.event` against a destroyed jsdom -> "window is not defined"
flake observed on CI.
Patches the 7 leaky test files to re-supply `Button` (bare `<button>`)
and `Tooltip` (Fragment) overrides matching `setupTests.ts`. Also drops
a dead `afterEach` workaround in `user_edit_view.test.tsx` (the
fake-timer dance it ran could not drain a real timer scheduled before
the swap) and corrects a misleading comment in `MakeMCPPublicForm.test.tsx`.
|
||
|
yuneng-jiang
|
1bea77e564
|
fix(ui): fetch version + debug flag from /health/readiness/details (#27896)
* fix(ui): fetch version + debug flag from /health/readiness/details The proxy moved `litellm_version`, `is_detailed_debug`, and other diagnostic fields off the public `/health/readiness` payload behind an auth-gated `/health/readiness/details` endpoint. The navbar version tag and the detailed-debug-mode banner stopped working because they were still reading those fields from the unauthed response, which no longer contains them. Replace `useHealthReadiness` with a `useHealthReadinessDetails` hook that takes an `accessToken` argument and sends a Bearer header to the auth-gated endpoint. The hook stays disabled while `accessToken` is falsy, so the navbar can keep rendering on the public model hub (where the token is null) without triggering an auth redirect or a 401-loop. * fix(ui): disable retries on readiness/details + cover token forwarding Two small follow-ups on the readiness/details migration: - Set `retry: false` on the query. The payload feeds a passive navbar tag and a debug banner; a 401 from an expired token shouldn't fan out into three retries against the proxy. - Add navbar specs that assert the `accessToken` prop is forwarded into the hook (matches the DebugWarningBanner spec). Without this, the navbar could silently regress to passing `undefined` and the existing tests wouldn't catch it. |
||
|
Sameer Kankute
|
18f77ff7bc
|
feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough (#27834)
* feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough Adds an opt-in per-server flag that lets clients (e.g. VS Code) complete PKCE directly with an upstream OAuth2 MCP server, instead of LiteLLM double-gating with its own API-key/SSO check. Only honored when auth_type=oauth2 and the operator explicitly sets the flag; mixed-target or non-oauth2 requests fail closed. - Adds the field to Pydantic models, Prisma schema, and a migration - New MCPRequestHandler._target_servers_delegate_auth_to_upstream gate that runs only when no x-litellm-api-key is present, so authenticated users still get user_id resolution + stored-credential lookup - Anonymous callers now see delegate servers in get_allowed_mcp_servers (scoped to delegate servers only; the upstream still enforces auth) - mcp_management_endpoints: allow anonymous /authorize and /token for delegate servers so VS Code can complete PKCE without a LiteLLM session - UI toggle (shown only for oauth2) + payload/view wiring - Tests covering: oauth2 on/off, non-oauth2 with flag, mixed targets, no resolvable target, explicit key precedence, and 401 emission Co-authored-by: Cursor <cursoragent@cursor.com> * Enforce oauth2 for delegated MCP auth bypass Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): close secondary Authorization bypass for delegate servers The delegate-auth bypass gated only on the primary `x-litellm-api-key` header, so a LiteLLM key sent via `Authorization: Bearer sk-...` (the secondary header) was silently dropped — skipping spend tracking and rate limiting. Gate on the resolved litellm_api_key (which considers both headers) so the bypass fires only when neither is present. Also update the existing "Authorization header present" test to reflect that an upstream OAuth token now flows through the existing oauth2 fallback (LiteLLM auth attempt → fail → anonymous), not via the delegate branch. Co-authored-by: Cursor <cursoragent@cursor.com> * Avoid duplicate MCP OAuth credential lookup Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): block delegate bypass for M2M and internal-only servers Two security issues flagged in code review: 1. High – client_credentials (M2M) servers must not be delegatable: LiteLLM auto-fetches the upstream token using stored credentials, so allowing anonymous bypass would let any external caller invoke tools authenticated as LiteLLM's service account. Fix: check `server.has_client_credentials` in `_target_servers_delegate_auth_to_upstream`, the anonymous allow-list in `get_allowed_mcp_servers`, and `_mcp_oauth_user_api_key_auth`. 2. Medium – internal-only servers exposed to public internet: The anonymous delegate allow-list was not filtering by `available_on_public_internet`, so external callers with an upstream OAuth token could invoke tools on servers marked internal-only. Fix: add `available_on_public_internet` guard to the anonymous delegate server list in `get_allowed_mcp_servers`. Tests added for both cases. Co-authored-by: Cursor <cursoragent@cursor.com> * Require public MCP delegate auth servers Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): align delegate auth path parsing with downstream routing `_extract_target_server_names_from_path` used a naive segments-based split while `server.py::_get_mcp_servers_in_path` uses a regex that allows server names with one embedded slash and comma-separated lists. With the old parser, a request to `/mcp/<delegated>/<garbage>` was parsed as targeting `<delegated>` by the auth gate (bypassing LiteLLM auth) while the routing layer parsed it as `<delegated>/<garbage>` — when that name did not resolve, the request fell back to the anonymous allow-list, which can include `allow_all_keys` servers that normally require a LiteLLM key. Replace the parser with the same regex logic as `_get_mcp_servers_in_path` so auth gating sees the exact target name(s) downstream routing sees. Add regression tests covering parser parity and the specific extra-path-segment bypass attempt. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * fix(mcp): close header/path TOCTOU in MCP delegate auth gate `_target_servers_delegate_auth_to_upstream` and `_target_servers_use_oauth2` trusted the `x-mcp-servers` header when present, but `server.py::extract_mcp_auth_context` overrides that header with the path-derived list for `/mcp/...` routes. An attacker could set `x-mcp-servers: <delegated>` while pointing the URL path at a non-delegate server, flipping the auth gate without changing the target downstream routing actually uses. Extract a shared `_resolve_target_server_names` helper that mirrors the downstream override (path-derived names for `/mcp/...` routes, header value otherwise). Add regression tests covering the TOCTOU attempt and the helper's path-vs-header precedence. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix delegated MCP OAuth test mock Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): drop unreachable /{server}/mcp branch in auth path parser `_extract_target_server_names_from_path` also matched the ``/{server_name}/mcp`` form, but the downstream parser ``_get_mcp_servers_in_path`` only handles ``/mcp/...`` — and ``dynamic_mcp_route`` in ``proxy_server`` rewrites ``/{name}/mcp`` to ``/mcp/{name}`` on the scope before the MCP handler runs. Parsing the un-rewritten form on the auth side was therefore unreachable in production, and contradicted the docstring's claim of mirroring the downstream parser — exactly the kind of mismatch that risks a future header/path TOCTOU if any new entry point skips the rewrite. Drop the branch; the canonical ``/mcp/...`` path matches both parsers. Update the regression test to assert the new behavior. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix MCP path auth target resolution Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): require auth for refresh_token grants on delegate-auth servers `_mcp_oauth_user_api_key_auth` gates the unauthenticated PKCE flow for ``delegate_auth_to_upstream`` servers, but the bypass applied to BOTH ``/authorize`` and ``/token`` regardless of grant type. ``mcp_token`` accepts ``grant_type=refresh_token`` as well as ``authorization_code``, and ``exchange_token_with_server`` attaches the server's stored ``client_secret`` to whatever is forwarded upstream. An unauthenticated caller holding a refresh token issued to that OAuth client could mint fresh upstream access tokens through LiteLLM. Limit the anonymous bypass on ``/token`` to ``grant_type=authorization_code`` (the only grant PKCE actually protects via ``code_verifier``); fall through to normal LiteLLM auth for ``refresh_token`` and any other grant. ``/authorize`` continues to allow anonymous PKCE redirects. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * fix(ui): clear delegate_auth_to_upstream when switching off oauth2 The ``delegate_auth_to_upstream`` form field is rendered inside an ``isOAuth2 && (...)`` conditional, so the Form.Item unmounts when the user changes ``auth_type`` away from ``oauth2``. The follow-up ``form.setFieldValue("delegate_auth_to_upstream", false)`` runs after the field has already deregistered, so ``onFinish`` receives ``undefined`` and the fallback ``?? mcpServer.delegate_auth_to_upstream`` preserved the old ``true``. The flag then persisted in the database for a non-oauth2 server and silently re-activated if ``auth_type`` was later switched back to ``oauth2``. In the edit payload, force the flag to ``false`` whenever ``auth_type !== oauth2``; only trust the form value (and the existing DB fallback) when the server is actually oauth2. Backend defense-in-depth already ignores the flag for non-oauth2 servers, but the DB state should stay clean too. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix MCP delegate auth reset on edit Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Claude <claude@anthropic.com> |
||
|
ryan-crabbe-berri
|
ddb4bd85cb
|
feat(ui): add Vertex AI Search as vector store provider (#27790)
* feat(ui): add Vertex AI Search as vector store provider Adds a "Vertex AI Search" entry to the provider dropdown (custom_llm_provider=vertex_ai/search_api) with fields for project, location (global/us/eu select), and optional collection ID. Extends VectorStoreFieldConfig with `options` so select fields can be data-driven instead of falling through to the embedding-model list. * fix(ui): clarify vertex_collection_id placeholder copy Placeholder previously displayed "default_collection" — the literal fallback value — which invited users to type it instead of leaving the field blank. Switch to an example placeholder and tighten the tooltip. |
||
|
ryan-crabbe-berri
|
9c4faeabc9
|
feat(ui): search teams by team ID alongside name (#27684)
* feat(ui): search teams by team ID alongside name The Teams page search box only matched team_alias, so pasting a team UUID returned zero results. Detect when the input is a full UUID and route it to the team_id filter instead; otherwise keep the existing alias substring search. Placeholder now reads "Search teams by name or ID...". Resolves LIT-2648 * refactor: search teams via backend OR clause, drop client-side UUID detection Adds a `search` query param to /v2/team/list that ORs across team_id (exact) and team_alias (case-insensitive contains), so the search box sends one param regardless of input format. Removes the isLikelyTeamId helper and the client-side branching it fed. |
||
|
ryan-crabbe-berri
|
b39cac9382
|
feat(ui): add Expires to key Overview header; merge User into one field (#27696)
* fix(ui): resolve created_by to human-readable name for team keys The team's Virtual Keys table rendered a raw UUID under Created By for keys created on behalf of a real user, and the key details page header showed "-" because it was reading the wrong field (user_email/user_id instead of created_by_user). - TeamVirtualKeysTable Created By column now prefers created_by_user.user_alias > user_email > UUID, with a Popover on hover that surfaces all three fields with copy icons (matches the existing AllKeys table pattern in VirtualKeysTable.tsx) - key_info_view passes the resolved created_by_user value to the details-page header so it renders the readable name Resolves LIT-2517 * fix(ui): add created_by to KeyResponse type The backend returns created_by on every key row, but the frontend type omitted it. The Created By cell already reads the field via info.row.original, which trips the production typecheck. * feat(ui): add Expires to key Overview header; merge User into one field The key details page header omitted Expires (only Settings tab had it) and showed User Email + User ID as two separate rows. This PR: - adds Expires below Created At, reusing the Settings tab's formatTimestamp(...) ?? "Never" formatting so both views agree - merges User Email / User ID into a single "User" field that displays alias / email / user_id (in that fallback order) with a Popover on hover exposing all three with copy icons — mirrors the Created By pattern in TeamVirtualKeysTable.tsx and VirtualKeysTable.tsx Refs LIT-2517 * chore(ui): User field icon + alias-primary test Address review feedback on #27696: - Swap MailOutlined → UserOutlined on the merged User cell; the envelope icon implied "this is an email" but the cell can render alias / email / user_id depending on what's available. - Add a test asserting userAlias displays as primary and overrides userEmail — closes the gap where a fallback-order regression would have silently passed. * fix(ui): truncate long User values and reshuffle header layout - Swap column groupings so Created By stays paired with Created At (matches the pre-merge layout); User and Expires now share col 1. - Ellipsis-truncate the visible User cell at maxWidth 200 so a raw UUID fallback doesn't sprawl. Full identity still revealed via the hover Popover. - Cap each Popover row at maxWidth 220 so a UUID truncates inside the panel too; antd's built-in ellipsis tooltip surfaces the full value. |