147 commits

Author	SHA1	Message	Date
yuneng-jiang	47c988e05c	refactor(ui): move the dashboard onto class-variance-authority (#38125) ... The dashboard used `cva@1.0.0-beta.4` with the object-argument API behind `@/lib/cva.config`, while shadcn emits `class-variance-authority` with the positional API. Every `shadcn add` of a cva-based primitive therefore needed a hand fix-up before it compiled, which meant `components/ui/` could never match a fresh CLI run and `shadcn add <name> --diff` reported the whole file as changed instead of showing real upstream drift. Swap the dependency, and regenerate `badge`, `button`, `button-group`, `input-group` and `tabs` straight from the base-vega registry so they are now byte-identical to the CLI output plus prettier. Two primitives could not be regenerated because they are local code rather than registry items, so they move out of `components/ui/`: `sidebar` (203 lines against upstream's 730, and only `leftnav` consumes it) and `meter` (no registry entry at all, it wraps Base UI's Meter). The customisations that were baked into the regenerated files move to wrappers, following the rule that `components/ui/` holds CLI output and anything on top of it lives outside: - badge carried info, success and warning variants that duplicated the existing `StatusBadge` tone map, so its five call sites now use `StatusBadge`, which gains an optional `className` - input-group's addon focuses `[data-slot=input-group-control]` rather than upstream's `input`, which matters because the chat composer puts a textarea there. That handler now sits at the one call site that needs it `cx` keeps its previous twMerge behaviour. It came from the old `defineConfig({hooks: {onComplete: twMerge}})`, and CVA's own `cx` is plain clsx, so pointing it at `cn` avoids silently dropping conflict resolution in the six files that use it. `Sidebar.test.tsx` covers the failure mode this migration can hide: passing the object form to the positional API is accepted by clsx and renders the literal class string "base variants defaultVariants", so the component loses every style while the type checker and the existing suite stay green.	2026-08-24 14:10:31 -07:00
ryan-crabbe-berri	933e28d900	feat(ui): add a light/dark/system theme toggle to the top bar (#37669) ... * feat(ui): add a light/dark/system theme toggle The dashboard already carried a full `.dark` palette, dark-aware surfaces and a dark logo variant, but nothing ever put the `dark` class on the document, so none of it could be reached. next-themes now owns that class: it reads the stored choice, falls back to the OS preference, and stamps the class from an inline script before first paint so there is no light flash on load. The toggle is a three-way System / Light / Dark control in the account menu, in both the sidebar menu and the older navbar dropdown, so it is reachable from the gateway dashboard, chat and the model hub alike. useIsDarkMode watched the root element with a MutationObserver purely to answer a question next-themes now answers directly, so it goes, and useSyntaxTheme reads resolvedTheme instead. The toaster follows the resolved theme too. * feat(ui): move the theme control to the top bar and default to light The toggle now lives in the header toolbar of both shells, the gateway dashboard's DashboardHeader and the older full-width Navbar, where it replaces the placeholder comment that had been holding its spot. It reads better there as a single icon button with a System / Light / Dark menu than as a segmented row buried in the account popover, so the account menus lose their theme row. Dark mode is still being rolled out, so an install that has never touched the control now stays light instead of following the OS. System is still a choice, just no longer the default. While dark is active the toolbar carries a small Experimental badge, so nobody mistakes an unstyled surface for a bug. * fix(ui): serve the dark logo in the legacy navbar too The sidebar already paired its logo with a dark variant, but the full-width navbar kept a single light-only image. That did not matter while dark mode was unreachable; now that the toggle sits in that shell's own top bar, the white JPEG slab lands on a dark bar. It gets the same two-image swap the sidebar uses, and a test that pins the pairing so the two shells cannot drift apart again.	2026-08-20 12:58:29 -07:00
ryan-crabbe-berri	7b574b9df6	chore(ui): drop the antd dependency and its leftovers (#37574) ... Nothing in the dashboard renders antd any more, so the package and the scaffolding around it can go. This removes `antd` and `@ant-design/cssinjs` from package.json, deletes the global StyleProvider the root layout wrapped every page in, drops the `antd` cascade layer and the z-index override that lifted Base UI popups over an antd Modal, and retires the lint rules that policed antd imports and antd class selectors in tests. Fifteen test files still carried `vi.mock("antd", ...)` factories for components that stopped importing antd during the migration. They were inert, and they resolve the real module, so they would have broken the moment the package left node_modules. The compatibility shims keep their behaviour and lose the antd name: `antdRules`/`antdRequired` become `validatorRules`/`requiredRule`, `isAntdUrl` becomes `isValidUrl`, and `ABOVE_ANTD_MODAL` becomes `NESTED_DIALOG_LAYER`. Comments that explain why a contract looks the way it does still name antd, because that history is the reason.	2026-08-19 20:44:27 -07:00
ryan-crabbe-berri	629d7683f2	refactor(ui): swap @ant-design/icons for lucide-react (#37553) ... The dashboard drew its icons from two libraries at once: lucide-react, which shadcn/ui ships with, and @ant-design/icons, left over from antd. This moves the last 39 files onto lucide and drops the dependency, so the icon set matches the component library everywhere. antd icons sized themselves from the inherited font-size and rendered as role="img" with an aria-label, neither of which a lucide svg does, so the swap carries explicit size classes and gives the two icon-only plugin buttons real accessible names.	2026-08-20 00:04:14 +00:00
ryan-crabbe-berri	f1e143a87c	chore(ui): upgrade the dashboard to React 19 (#37411) ... * chore(ui): upgrade the dashboard to React 19 Bumps react and react-dom from 18.3.1 to 19.2.8 with matching @types. Next 16 already required a React 19 peer, so this aligns the dashboard with what the framework expects and unblocks Base UI and shadcn work that assumes the React 19 ref model. React 19 passes ref through as a regular prop, so the setup file's forwardRef tripwire and the ref-forwarding test's forwardRef case no longer describe real behavior; both now assert the React 19 contract instead. useRef<T>(null) now yields RefObject<T | null>, which is the one prop type MessageList had to widen. * test(ui): wait for a Base UI select popup to open before clicking an option The option lands in the DOM one render before the popup finishes entering, while its positioner still carries pointer-events: none, so clicking it throws. Waiting on the option's text alone was a race that React 19's flush timing loses, which is why four ToolPolicies cases went red on the bump. chooseSelectOption in test-utils opens the trigger, finds the option by role, waits for it to stop being pointer-blocked, then clicks. It also replaces the last-match-by-text hack, which only worked because the popup happens to portal after the table.	2026-08-19 21:18:08 +00:00
ryan-crabbe-berri	bb8324c119	refactor(ui): drop @tremor/react and the theming scaffolding it needed (#37394) ... The last tremor component import left the dashboard when the primitive sweep merged, so the package, its v3 compatibility shim, its @theme token block and the palette safelist it needed at runtime all have no consumer. Removing the safelist is what shrinks the shipped stylesheet: tremor built class names at runtime, so Tailwind had to emit every bg/text/border/ring/ stroke/fill utility across 22 palettes and 11 shades in case one was used. Nothing in the app constructs a class name that way any more, so the scanner finds every utility on its own. The date-fns overrides pin also goes. It only existed because tremor and react-day-picker@8 peered on date-fns 3 while Base UI wanted 4, and the lockfile still resolves a single hoisted 4.4.0 without it.	2026-08-18 17:38:12 -07:00
ryan-crabbe-berri	68d4ba5da5	refactor(ui): move dashboard toasts from antd message/notification onto sonner (#37207) ... * refactor(ui): move dashboard toasts from antd message/notification onto sonner Add lib/toast.ts as the single toast surface (success/info/warning/error/ fromError/dismiss) backed by sonner, with a <Toaster /> in the root layout. fromError titles a toast from the proxy error type or the HTTP status instead of matching prose phrases, and shows the extracted proxy message as the description. MessageManager and NotificationManager become thin facades over lib/toast so the ~250 existing call sites keep working; the mutable antd instance setters, setMessageInstance/setNotificationInstance, and the antd App/message/ notification providers in AntdGlobalProvider are gone. Prunes the eslint suppression baseline accordingly. * test(ui): mock the MessageManager seam in the Fallbacks tests and drop toast doc comments AddFallbacks and FallbackSelectionForm asserted on a mocked antd message spy that MessageManager no longer calls; they now mock the facade the components import. Also removes the explanatory comments Greptile flagged in lib/toast.ts and both facades. * fix(ui): keep NotificationManager's antd config-object contract on the sonner facade success/info/warning/error accept the { message, description, duration } object form again (CreateMCPServer's admin-review notice uses it) and fromBackend keeps its extra.duration seconds argument, both mapped onto lib/toast. Prunes stale suppressions picked up by the rebase. * feat(ui): read the proxy error type and code out of JSON envelopes embedded in string errors Legacy networking helpers throw new Error(responseText) and callers prefix that text, so the envelope arrives as a substring. fromError now parses the first embedded JSON object for type/code and shows the unwrapped message in its place, so those toasts get a status title (Request Error, Not Found) and a readable description instead of raw JSON.	2026-08-17 18:22:41 -07:00
Yuneng Jiang	41d6dbb50c	build(ui): gate dashboard test assertions with testing-library and jest-dom ... Enables 14 rules at error with zero violations, so the vacuous-assertion class the previous commit fixed cannot come back. No budget file and no suppressions baseline: a rule is on only if it is already at zero. prefer-to-have-value stays off. It matches any attribute whose name contains "value", so it rewrites toHaveAttribute("aria-valuenow", n) into toHaveValue(n), and jest-dom's toHaveValue supports only form controls, which breaks every role="meter" element the dashboard renders. Records the enabled set, the rules left off with their measured counts, and the seven ways these plugins' autofixers produce broken output.	2026-08-17 14:46:55 -07:00
Devin AI	630a1d0b69	build(deps): bump nanoid to 3.3.18 to clear osv-scan ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-08-13 16:38:24 +00:00
mateo	2f70f4edba	build(deps): bump nanoid to 3.3.17 in the dashboard lockfile ... GHSA-2v37-7h3g-55p8 (CVE-2026-67213) rates 8.2 against nanoid 3.3.16 and reds osv-scan on every PR into staging. Custom generators can loop indefinitely when size is zero, so a caller that passes through a zero size hangs the process. nanoid is transitive through the dashboard's toolchain and 3.3.17 is a patch release that published 08-03, so it already clears the .npmrc min-release-age=3 guard. The lock diff is the version, resolved url, and integrity hash for that one package. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-08-07 21:30:09 +00:00
Yuneng Jiang	0253154780	build(deps-dev): bump js-yaml to 4.3.1 ... Closes GHSA-5p4m-2wfm-xmqj (CVSS 7.5), flagged by osv-scan against ui/litellm-dashboard/package-lock.json. js-yaml is pinned by an exact npm override, so the override and the lock move together. Dev-only dependency: js-yaml reaches the tree through eslintrc, knip and @redocly/openapi-core, none of which ship in the built dashboard. 4.3.1 published 2026-07-31, clear of the 3-day min-release-age cooldown.	2026-08-06 18:14:26 -07:00
ryan-crabbe-berri	2dc49a913c	refactor(ui): replace hand-rolled query-param routing with nuqs (#35871) ... * refactor(ui): replace hand-rolled query-param routing with nuqs The dashboard carried five copies of the same pushState-based detail routing hook plus a shared navigateWithParams helper, each with its own plumbing test and a copy-pasted reactive useSearchParams mock in component tests. nuqs provides the same shallow history-API routing behind useQueryState/useQueryStates, so the key, team and org hooks are deleted in favor of inline useQueryState at their single consumers, while the models and logs hooks keep their interfaces but drop their hand-rolled internals. Component tests now mount NuqsTestingAdapter (via renderWithProviders or locally) instead of patching window.history, and URL assertions go through onUrlUpdate spies that can additionally distinguish push from replace, which the old window.location checks could not * test(ui): assert browser back closes the log drawer after in-drawer selection Greptile flagged that the nuqs port of the switching-logs test stopped at asserting emitted push and replace modes. The test now replays those recorded modes against a history stack and performs the back step, so a regression to push-on-select or broken URL-derived drawer state fails the test instead of passing silently	2026-08-05 13:29:58 -07:00
Yuneng Jiang	64aab7be85	ci: pin Node on the Playwright UI lanes so npm ci meets the engines floor ... e2e_ui_testing and e2e_ui_testing_server_root_path run on cimg/python:3.12-browsers, the one UI executor whose image supplies Node rather than taking it from a cimg/node tag. That image ships Node 24.14.0, which bundles npm 11.9.0, so both lanes have failed EBADENGINE against the engines floor added in #35801. Every Node 24 release through 24.14.0 bundles an npm below 11.10.0, so engines.node also rises to 24.14.1 (npm 11.11.0), the first release where the two floors agree The pinned install goes into /opt/node with /opt/node/bin prepended to PATH instead of unpacking over /usr/local. On this image /usr/local already holds npm 11.9.0, and extracting the tarball on top of it merges the two trees into an npm that reports 11.17.0 and then exits 1 on npm ci printing no error text at all, which is a worse failure than the one being fixed The install moves into a reusable install_node command so the version and its checksum have one home, shared with proxy_pass_through_endpoint_tests, and the command refuses to run when it disagrees with ui/litellm-dashboard/.nvmrc. A lane drifting off the version the rest of the toolchain uses is what produced this failure, so that mismatch now stops the job instead of surfacing later as an install error The e2e node_modules cache key moves to v4 because the saved trees were built by the old npm	2026-08-04 16:31:38 -07:00
yuneng-jiang	487074f602	chore(build): move the Admin UI toolchain to Node 24 (#35801) ... * chore(build): move the Admin UI toolchain to Node 24 Node 18 and Node 20 both reached end of life (2025-04-30 and 2026-04-30), and the release images along with every CI lane were still building on them. Node 24 is the current LTS through 2028-04-30, so this moves the four UI build images, the CircleCI lanes, and the four GitHub Actions workflows onto it Node 24 also ships npm 11.17, which is the first line that implements the min-release-age setting this repo already carries in its .npmrc files. On npm 10 the key is parsed and discarded, so the release-age gate has had no effect regardless of its value. Tightening the dashboard's engines range and turning on engine-strict makes an unsupported npm fail loudly rather than skip the gate quietly, and a new step in the UI build workflow probes an impossible cooldown so an inert setting cannot pass unnoticed again Node 24's bundled undici tightened its brand check on RequestInit.signal, which rejects the AbortSignal jsdom installs and broke the two cases in src/lib/http/api.test.ts that rebase a request onto a runtime base url. Under jsdom the Request global comes from Node while AbortSignal comes from jsdom; tests/jsdomFetchEnv.ts delegates to the jsdom environment and then restores Node's native AbortController and AbortSignal so both come from one realm. Upgrading jsdom does not address this, as jsdom still does not own Request The workflows now read ui/litellm-dashboard/.nvmrc instead of repeating a literal, so the Node version has a single source of truth, and ui/Dockerfile is pinned by digest to match the other three build images. The lockfile changes are npm 11 normalising the engines range and dropping optional peer entries it no longer records * fix(build): point every Admin UI build script at .nvmrc The enterprise Docker path was left on Node 18. docker/build_admin_ui.sh runs only when enterprise/enterprise_ui/enterprise_colors.json is present, which it never is in the OSS tree, so neither CI nor a default image build reaches it; it pinned nvm to v18.17.0 and then built the dashboard, which now requires Node 24, so a customized enterprise image would have failed EBADENGINE All three UI build scripts now resolve the version from ui/litellm-dashboard/.nvmrc rather than carrying their own pin, so the Node version has a single home across Docker, CI, and local builds. build_ui.sh was on v20 and build_ui_custom_path.sh on v18.17.0 Also drops the dependency-cooldown probe from the UI build workflow. The engines floor plus engine-strict already fails an unsupported npm loudly at install time, so the probe was redundant, and treating any nonzero exit from a live registry call as proof of enforcement made it unsound besides	2026-08-04 12:36:07 -07:00
Yuneng Jiang	3bc4989ce4	chore(ui): update brace-expansion and postcss to current patch releases ... The dashboard pins both packages exactly in `overrides`, so the lockfile stays on whatever those pins say. Move brace-expansion from 5.0.8 to 5.0.9 and postcss from 8.5.22 to 8.5.23, both upstream patch releases, and regenerate the lockfile. `npm ci`, `next build`, and the 5888-test vitest suite all pass on the updated lockfile.	2026-08-03 13:15:32 -07:00
Yuneng Jiang	c9d067fccc	chore(deps): bump gitpython to 3.1.55 and brace-expansion to 5.0.8 ... gitpython arrives transitively through mlflow-skinny; re-resolved with uv so the lock moves that one package only. brace-expansion is a dev-only transitive dep already pinned in the dashboard 'overrides' block, so the pin is bumped alongside the lockfile to keep the change durable across reinstalls. 5.0.8 narrows its engines range from '18 || 20 || >=22' to '20 || >=22'; the dashboard already requires node >=20.9.0 and every CI job pins node 20, so nothing loses support.	2026-07-27 10:06:50 -07:00
Yuneng Jiang	3467871007	chore(deps): bump gitpython and postcss to advisory-clear versions ... Clears five OSV findings the scanner flags on every PR: four gitpython advisories fixed in 3.1.54, and one postcss advisory fixed in 8.5.18. gitpython 3.1.55 and brace-expansion 5.0.8 are left for a follow-up; both were published less than three days ago and are still inside the dependency cooldown window.	2026-07-25 10:29:06 -07:00
yuneng-jiang	301a02b7be	chore(ui): bump next to 16.2.11 (#34329) ... Moves the dashboard's next pin from 16.2.6 to the latest 16.2.x patch and bumps eslint-config-next to match. Regenerating the lock also healed in explicit bundled-dependency records under @tailwindcss/oxide-wasm32-wasi	2026-07-22 17:29:29 -07:00
ryan-crabbe-berri	0fcaadf11c	test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196) ... Relocates ui/litellm-dashboard/e2e_tests to tests/e2e/ui so all end to end suites live under tests/e2e. The suite stays in TypeScript and becomes a self-contained npm package with its own package.json, lockfile and tsconfig instead of leaning on the dashboard's toolchain; the dashboard drops its @playwright/test dependency, e2e scripts and knip/vitest/tsconfig carve-outs. CI paths follow the move: both CircleCI jobs (main e2e and the SERVER_ROOT_PATH migration smoke) and the test_server_root_path workflow now install and run Playwright from tests/e2e/ui, with the node cache keyed on both lockfiles. classify_changes.sh treats tests/e2e/ui as client so spec edits keep skipping backend jobs. The suite's mock LLM fixture is excluded from the e2e basedpyright zero-error gate in pyrightconfig.json since it belongs to the TS suite, not the typed Python harness.	2026-07-22 19:43:10 +00:00
yuneng-jiang	20a4666ec6	chore(ui): bump sharp to 0.35.x via npm override (#34193) ... sharp reaches the dashboard only as an optional dependency of next, which pins it to ^0.34.5. A caret range on a 0.x version cannot resolve past 0.34.x, and every stable next through 16.2.11 still declares that same range, so there is no transitive path to the 0.35 line. Add an overrides entry, matching how the other pinned transitives in this package are already handled. The dashboard builds with output: "export" and images.unoptimized, so sharp is never loaded; this keeps the lockfile current rather than changing runtime behaviour.	2026-07-21 17:03:31 -07:00
yuneng-jiang	72d458e416	feat(ui): add react-hook-form + zod form infrastructure (#34170) ... * feat(ui): add react-hook-form + zod form infrastructure Introduce the shared form layer the dashboard's antd forms will migrate onto, with no user-visible change yet. - pin react-hook-form, @hookform/resolvers, and zod (kept on 3.25.76 and imported via the zod/v4 entrypoint so openai's optional zod ^3 peer still resolves and npm ci stays clean) - vendor the base-vega Field family into components/shared/form as forwardRef components on the repo's cva.config, since base-vega ships no form primitive and its field source imports class-variance-authority and is React 19 style - add a FormField bridge that binds a react-hook-form Controller to the Field layer and wires label, description, and error ids into aria attributes - add pickDirty, which narrows a submitted body to the top-level keys the user actually touched so a partial update stops re-sending untouched fields pickDirty reads dirtiness at the top level because react-hook-form tracks it per leaf, so an edited array arrives as [true, false] and a cleared list as an empty array that still carries its default-length dirty markers; the falsy clear tokens (null, [], {}, 0, false) all survive. Tests cover the Field primitives, the FormField aria wiring against a live zod resolver, and pickDirty both as a unit and driven through a real react-hook-form instance. * test(ui): lock pickDirty behavior on a pure field-array reorder react-hook-form compares each array element to its default positionally by value, so useFieldArray move/swap and a reordered scalar array all mark the moved indices dirty and pickDirty sends the whole array; a swap of two equal elements is a value-level no-op and is correctly omitted. Covers the reorder case a review flagged as untested.	2026-07-21 15:17:58 -07:00
Yuneng Jiang	f2ca0a149b	build(deps-dev): bump js-yaml to 4.3.0 and brace-expansion to 5.0.7 ... Both are dev-only build and lint tooling in the dashboard, not part of the browser bundle. The bumps pull in upstream maintenance releases that address inefficient handling of certain inputs js-yaml is force-pinned through the overrides block because @redocly/openapi-core exact-pins an older copy; a plain lockfile change would not hold since the tree re-spawns a nested stale version on re-resolution, so the override moves from 4.2.0 to 4.3.0. brace-expansion is added to overrides at 5.0.7 so npm install does not leave the previously resolved 5.0.6 in place. Both resolve to a single deduped copy after the change	2026-07-20 17:46:20 -07:00
ryan-crabbe-berri	539bc30e04	refactor(ui): migrate callback debounce sites to react-pacer with regression tests (#33043) ... * refactor(ui): standardize debounce waits behind shared DEBOUNCE_WAIT_MS constant * build(ui): bump @tanstack/react-pacer from 0.2.0 to 0.22.1 * refactor(ui): migrate straightforward value debounces to react-pacer * refactor(ui): migrate callback debounce sites to react-pacer with regression tests * chore(ui): restore trailing newline in eslint-suppressions.json * test(ui): mock all pacer debounce hooks in VirtualKeysTable test * fix(ui): update merged debounce tests for OldTeams to Teams rename	2026-07-13 15:38:34 -07:00
ryan-crabbe-berri	3a42011350	build(ui): bump @tanstack/react-pacer from 0.2.0 to 0.22.1 (#33041) ... * refactor(ui): standardize debounce waits behind shared DEBOUNCE_WAIT_MS constant * build(ui): bump @tanstack/react-pacer from 0.2.0 to 0.22.1	2026-07-13 13:38:15 -07:00
ryan-crabbe-berri	8c5473f198	feat(ui): adopt openapi-react-query ($api) and convert useCustomers (#32949) ... * feat(ui): adopt openapi-react-query and convert useCustomers to $api Add openapi-react-query and expose $api = createQueryClient(fetchClient) alongside fetchClient. Rewrite useCustomers as $api.useQuery("get", "/customer/list", {}, { enabled, select }), which derives the query key from method + path (dropping the hand-written createQueryKeys entry and the manual key) and forwards the request signal for cancellation. The response type still flows from schema.d.ts as CustomerResponse[]. Tests assert the path, the admin/token enabled gate, and the empty-body select fallback. * test(ui): read the last render's options in useCustomers helper The lastCallOptions helper was named for the last call but read mock.calls[0]. Harmless while each test renders once, but it would silently assert against first-render options if a test ever re-renders. Read the final call instead.	2026-07-11 18:01:33 -07:00
ryan-crabbe-berri	0bf81e2496	feat(ui): typed openapi-fetch foundation (fetchClient) + first typed caller (useCustomers) (#29884) ... * feat(ui): add the typed openapi-fetch client (fetchClient) as the dashboard fetch foundation Introduces fetchClient (openapi-fetch) bound to schema.d.ts, used inside ordinary TanStack Query hooks so path/query/body types come from the proxy's OpenAPI spec. A small runtime registry feeds the client the base URL and auth header name (registered by networking) and the session token (published by AuthContext), so call sites carry no token plumbing; auth-header injection and ApiError mapping live in openapi-fetch middleware reusing deriveErrorMessage/ApiError from client.ts, and non-2xx maps to a thrown ApiError so query functions just read .data. The base URL default resolves from NEXT_PUBLIC_BASE_URL so a request still targets the right origin if it fires before networking registers its getter. AuthContext clears accessToken alongside the token on logout so no query fires unauthenticated after the session ends. Foundation only; callers migrate one at a time, each fully typed, in follow-up changes. * feat(ui): migrate useCustomers to the typed fetchClient Converts useCustomers from allEndUsersCall to fetchClient.GET("/customer/list"); the response is typed as LiteLLM_EndUserTable[] from the schema, so the hand-written Customer/CustomersResponse types are deleted. They were also inaccurate (allowed_model_region was string but is "eu"|"us", and a budget_id the table has no field for). No cast; the schema type flows to the one consumer. First caller on the new pattern. * fix(ui): route typed-client errors through the session-expiry handler The typed fetchClient middleware threw ApiError without invoking the handleError side effect that the legacy createApiClient wires via onError, so a migrated caller hitting an expired key no longer triggered the auto-logout. Add an error-handler seam to runtime.ts, register handleError from networking.tsx alongside the base-url/header getters, and call it in the middleware before throwing so both clients behave the same. Regression test asserts the handler fires with the derived message on non-2xx and stays silent on success * fix(ui): point the customers EndUser type at CustomerResponse The /customer/list response model was renamed to CustomerResponse on staging; the merged branch still aliased EndUser to LiteLLM_EndUserTable, so the exported type and its test mock had drifted from what the schema actually returns. CustomerResponse is also the accurate shape (it types allowed_model_region as 'eu' | 'us' and carries budget_id) * chore(ui): refresh eslint-metrics baseline after staging merge The recorded baseline predated the litellm_internal_staging merge, so its no-explicit-any and no-large-inline-object-arg counts were higher than the merged tree actually has. Regenerate via npm run lint:metrics so the gate reflects current reality * refactor(ui): source the typed client token from the session cookie, not AuthContext The typed client read its bearer from a runtime value that AuthContext pushed via setAuthToken, but migrated hooks gate enabled on useAuthorized, which decodes the cookie directly. Two independent derivations of the same cookie with different timing: on first load the query fires (useAuthorized sees the token) before AuthContext's async effect publishes it, so the first request goes out unauthenticated and only succeeds on a React Query retry. Make the token a registered getter like the base-url and header-name getters, reading the same cookie useAuthorized decodes, so the client's token and the gate can't diverge. Revert the AuthContext changes entirely; nothing is pushed from React state anymore.	2026-07-11 10:50:51 -07:00
ryan-crabbe-berri	e2eee36438	chore(ui): make knip trustworthy and enforce dead-code in CI (#32727) ... knip was producing garbage locally and was never wired into CI, so nobody trusted it. Two structural problems: it silently degrades when deps are missing (a partial worktree install flagged all 436 test files as unused), and its config had blind spots that surfaced as false positives. Fixes so a knip run means something: - Register every playwright config (serverRootPath + migration variants), not just the main one. serverRootPath.config.ts is invoked via --config in test_server_root_path.yml, which knip can't see; it was falsely flagged as an unused file - Treat src/components/ui/** as entry points. These are shadcn design-system primitives, intentionally part of the palette before every one is consumed; knip was flagging not-yet-used ones (e.g. select.tsx) as dead files and their sub-exports as unused. Marking the directory as the design-system surface is the correct fix, not deleting components someone is about to use - Declare @ant-design/icons as a direct dependency. It was imported in ~198 files but only resolved via antd hoisting, so every one showed up as an "unlisted dependency" - Add an explicit vitest plugin block so test-file classification no longer rides on auto-detection - Stage severities via rules: gate the now-clean categories (files, dependencies, unlisted, unresolved) as errors and keep exports/types/ duplicates as warnings, so CI enforces what's at zero today while the remaining findings ratchet down in follow-ups - Run npm run knip in the frontend-lint CI job, which installs with npm ci so it never sees a partial tree knip now exits 0 with the gated categories clean	2026-07-10 11:50:27 -07:00
ryan-crabbe-berri	592510ec18	feat(ui): shadcn charts foundation with tremor-compatible wrappers (#32668)	2026-07-09 18:18:52 -07:00
ryan-crabbe-berri	cfe9e39e55	refactor(ui): switch shadcn primitives from Radix to Base UI (#32124) ... * refactor(ui): switch shadcn primitives from Radix to Base UI shadcn made Base UI the default primitive library in July 2026 and our only shadcn component so far is the Button canary, so this is the last cheap moment to switch before the primitives phase adds the full set. components.json style moves from new-york (a legacy alias that resolves to the Radix variant) to base-vega. Button is regenerated from the base-vega registry with the same local adaptations as before: cva beta object form via lib/cva.config and a React 18 forwardRef wrapper. The polymorphic asChild prop becomes Base UI's render prop. radix-ui is replaced by @base-ui/react 1.6.0. Base UI optionally peers on date-fns 4 while tremor pins 3, so date-fns is bumped to 4.4.0 with an npm override; our only usage (add) is API-identical and the override can go away when tremor does. * refactor(ui): convert chat UI shadcn components from Radix to Base UI The chat UI migration landed 11 components/ui files generated against the old Radix registry config after this branch cut over to Base UI, which would have left them importing a deleted package. All 11 (dialog, alert-dialog, select, popover, tooltip, tabs, switch, scroll-area, collapsible, separator, label) are regenerated from the base-vega registry, with the repo conventions re-applied where relevant (cva beta object form from lib/cva.config in tabs; the Button canary keeps its React 18 forwardRef adaptation). Chat feature call sites move from the Radix asChild pattern to Base UI's render prop, and TooltipProvider delayDuration becomes delay. * fix(ui): restore security override pins clobbered by the date-fns override The date-fns 4 override was written by replacing the whole overrides object, dropping the ten security pins (prismjs, js-yaml, glob, minimatch, lodash, ws, braces, axios, postcss, esbuild) that keep patched versions in the lockfile; osv-scan caught the vulnerable versions resurfacing. Restores the pins alongside date-fns and regenerates the lockfile. * test(ui): pin tremor DateRangePicker behavior on date-fns 4 The date-fns 4 override forces react-day-picker 8 (authored against v3) onto v4 at runtime, which a build or lint pass cannot validate. This renders the shared UsageDatePicker wrapper, opens the calendar, checks the month grid, and selects a day, so a date-fns API break in the tremor date path fails tests instead of throwing in production. Delete alongside the override when tremor is removed. * fix(ui): close the alert dialog when AlertDialogAction is clicked The base-vega registry template renders AlertDialogAction as a plain Button with no Close binding, so confirm buttons fired their onClick but left the dialog open; both consumers (conversation delete, MCP credential revoke) were written against the Radix semantics where Action dismisses on click. Binds Action to AlertDialogPrimitive.Close via the render prop, mirroring AlertDialogCancel, and pins the behavior with a test so a future shadcn add --overwrite cannot silently reintroduce the template's non-closing Action.	2026-07-07 09:55:41 -07:00
Krrish Dholakia	e06adb5588	feat(ui): re-add chat UI, allow simple UI for MCP OBO auth (#31893)	2026-07-03 12:36:36 -07:00
ryan-crabbe-berri	27069bd74f	feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix (#31995) ... * feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix Upgrade the dashboard from Tailwind v3 to v4 with CSS-first config: the official upgrade codemod renamed utilities across 151 files, and tailwind.config.js (plus the dead tailwind.config.ts) is replaced by @theme tokens, @source globs, and @plugin directives in globals.css. The Tremor safelist becomes @source inline patterns and the legacy tremor theme tokens carry over verbatim. ui_colors.json was build-time only and fed the dying Tremor palette, so its brand values are inlined and the file removed; runtime theming replaces that path next. shadcn is initialized with a hand-authored components.json (rsc, cssVariables, baseColor gray) pointing utils at the existing lib/cva.config.ts, which now exports cn (cva beta cx + twMerge) instead of adding class-variance-authority as a second variant library. The two ad-hoc cn helpers fold into it. Button lands as the canary primitive, adapted to cva beta and React 18 forwardRef, with tests covering the variant, twMerge, asChild, and ref seams. --radius is 0.5rem so the shadcn radius scale reproduces Tailwind defaults and legacy rounded-* classes render unchanged. antd v5 emits unlayered CSS-in-JS that would beat every layered v4 utility, so AntdGlobalProvider now wraps the app in StyleProvider layer and ConfigProvider cssVar, and globals.css declares @layer theme, base, antd, components, utilities. antd wins over preflight but yields to utilities, which is what lets migrated shadcn pages coexist with legacy antd pages. Preflight stays global with the three v3 behaviors pinned (default border color, button cursor, placeholder color). * fix(ui): restore tremor opacity tints removed by tailwind v4 Tailwind v4 removed the *-opacity-* utilities, but the precompiled @tremor/react dist still composes them with shade-500 palette classes (bg-opacity-10 over bg-<color>-500 etc.), so Badge, BadgeDelta, Callout, light Icon and Button, BarList, and ProgressBar lost their tints and rendered solid 500-shade fills. Adversarial review caught it; the original smoke pages only exercised antd Tags. tremor-v3-compat.css restores exactly the pairs tremor emits: for each of the 22 safelisted colors, bg-opacity-{10,20,40}, hover/group-hover bg-opacity-{20,30}, and ring-opacity-{20,40} against the -500 shade, via color-mix into the utilities layer. Tremor's colorPalette maps both background and iconRing to 500, so the -500 pairing covers every composition in the dist; dark: variants are inert until dark mode ships. The shim dies with @tremor/react at the end of the migration. The upgrade codemod also missed two hand-rolled modal scrims using bg-black bg-opacity-{30,50} (solid black under v4); now bg-black/30 and bg-black/50. Removed the docker/build_admin_ui.sh copy of enterprise_colors.json into the deleted ui_colors.json; that build-time rebrand path is retired and its runtime replacement lands with the theming phase. * fix(ui): pair ring-opacity-40 with shade 300 in tremor compat shim Tremor's colorPalette maps ring to shade 300, and the only consumer of ring-opacity-40 (Icon variant outlined) composes it with that shade, so the shade-500 rows were dead and outlined icon rings would render at full opacity. Latent today (no dashboard usage of the outlined variant); caught by adversarial review. ring-opacity-20 stays at 500 (iconRing), matching Badge and BadgeDelta.	2026-07-02 19:02:27 -07:00
ryan-crabbe-berri	2a9dbc4c0d	chore(ui): remove unused dep, delete dead file, and unblock knip (#31933) ... Knip flagged remark-gfm as unused and date-fns as imported-but-undeclared, so drop remark-gfm (which prunes its transitive markdown subtree from the lockfile) and declare date-fns, which keyExpiryUtils.ts imports but only received transitively. Also delete the dead memory/components/index.tsx barrel, since nothing imports it once the page pulls MemoryView from its module directly Knip itself could not run: its Playwright plugin imports every config referenced by a --config flag in package.json scripts, and migration.serverRootPath.config.ts threw at import time when SERVER_ROOT_PATH was unset. Move that guard into a config-specific globalSetup so importing the config is side-effect-free; the check still fires loudly before any test runs when the prefix is missing	2026-07-01 20:13:58 -07:00
yucheng-berri	a8a1472428	fix(deps): bump osv-flagged dependencies to clear known CVEs (#31122) ... Bumps the 12 packages osv-scanner flags on litellm_internal_staging, taking the scan from 24 known vulnerabilities to zero. vcrpy goes to 8.2.1 first so aiohttp can move to 3.14.1 (vcrpy <= 8.1.1 cannot import aiohttp 3.14), then the two aiohttp ignore entries are dropped from osv-scanner.toml. The langchain stack moves together since langchain 1.3.9 requires langgraph 1.2.x. Runtime deps cryptography (48.0.1), starlette (1.3.1), python-multipart (0.0.32), pydantic-settings (2.14.2) and pypdf (6.13.3) are bumped via relock, and the dashboard's js-yaml, ws and form-data overrides are bumped too. Also removes the paths filter on the OSV workflow so it runs on every PR rather than only when a lockfile changes, which is why it never showed up on recent code-only PRs	2026-06-23 15:50:50 -07:00
yuneng-jiang	2fad75ffda	build(ui): pin esbuild to 0.28.1 via overrides (#30390) ... esbuild reaches the dashboard tree transitively through vite (the vitest stack), and vite 7.x still requests ^0.27.0, so it never advances past 0.27.x on its own. An overrides entry forces 0.28.1, which the osv-scan job reports as the fixed version for the dev dependency. Keeping it in package.json (not just the lockfile) means a lockfile regeneration cannot silently roll it back.	2026-06-13 11:52:40 -07:00
yuneng-jiang	d96ab467f1	chore(deps): bump vitest, brace-expansion, pypdf and tornado (#30220) ... * chore(deps): bump aiohttp to 3.14.1 and vitest to 3.2.6 Lockfile-only bump for aiohttp (3.13.5 -> 3.14.1, within the existing pyproject constraint) and dashboard devDependency bumps for vitest, @vitest/coverage-v8, @vitest/ui (3.2.4 -> 3.2.6) plus transitive brace-expansion (5.0.5 -> 5.0.6). Clears the currently published advisories flagged by osv.dev against uv.lock and the dashboard lockfile. Verified: 154 custom_httpx unit tests and all 3943 dashboard vitest tests pass; live proxy completion and streaming calls succeed on the bumped venv * chore(deps): raise aiohttp floor to 3.14.0 The lockfile bump alone only protects environments built from uv.lock. Raising the pyproject floor extends the same minimum to package consumers installing litellm from PyPI, and prevents a future lockfile regeneration from resolving below 3.14.0 * Revert "chore(deps): raise aiohttp floor to 3.14.0" This reverts commit d6c1c9dc0c. * revert(deps): roll back aiohttp to 3.13.5 vcrpy is incompatible with aiohttp >= 3.14 (the aiohttp_stubs module imports a symbol removed in 3.14) and the upstream fix is merged but unreleased, so every cassette-based test suite fails on 3.14. Hold aiohttp at 3.13.5 until a vcrpy release ships; the vitest and brace-expansion bumps stay * chore(deps): bump pypdf to 6.13.1 and tornado to 6.5.7 Lockfile-only bumps clearing the advisories published for both since this branch was opened * chore(deps): add regression guards for the bumped versions Raise the pypdf floor to 6.12.0 (direct dependency, applies to package consumers too) and add uv constraint-dependencies for the transitive pins: tornado >= 6.5.6, and aiohttp held in [3.13.5, 3.14) so a lockfile regeneration can neither fall back below the current version nor move onto 3.14 while vcrpy is incompatible. Constraints live in [tool.uv] and only affect this repo's resolution, not published metadata. Verified: uv lock -P with each out-of-range version fails to resolve; in-range resolutions unchanged (pypdf 6.13.1, tornado 6.5.7, aiohttp 3.13.5)	2026-06-12 17:48:00 -07:00
ryan-crabbe-berri	9e0d92c129	chore(ui): remove dead dashboard files and unused dependencies (#30047) ... * chore(ui): remove dead dashboard files and unused dependencies knip flagged seven orphaned source/config files with no importers and five declared dependencies that nothing in the tree uses. Removing them shrinks the dashboard bundle's source surface and keeps the manifest honest; vite stays installed transitively via vitest, so test tooling is unaffected. * fix(ci): restore serverRootPath.config.ts referenced by SERVER_ROOT_PATH workflow The dead-code sweep removed e2e_tests/serverRootPath.config.ts, but its spec (tests/login/serverRootPathRedirect.spec.ts) and the test_server_root_path.yml workflow step still depend on it, so the redirect e2e job failed to load a config that no longer existed.	2026-06-09 17:54:38 -07:00
yuneng-jiang	bac2590b39	build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 (#29982) ... Raise the PyJWT floor in pyproject (>=2.13.0,<3.0) and re-resolve uv.lock so the proxy installs 2.13.0 instead of 2.12.0. Bump the ws transitive-version override in the dashboard from 8.19.0 to 8.20.1 and regenerate package-lock; jsdom and openai both dedupe onto the single 8.20.1 copy. Both are routine dependency maintenance bumps to keep pinned versions current.	2026-06-08 16:39:21 -07:00
ryan-crabbe-berri	e53bd7cbd1	feat(ui): generate dashboard API types from the proxy OpenAPI spec (#29816) ... * feat(ui): generate dashboard API types from the proxy OpenAPI spec Introduces the shared type foundation for the dashboard without touching any runtime code. The proxy's FastAPI app is the source of truth; app.openapi() emits the spec and openapi-typescript turns it into src/lib/http/schema.d.ts. Adds an npm run gen:api script (a Python spec dump piped into openapi-typescript) and a Check UI API Types Sync CI job that regenerates the file from the live spec and fails if it drifts, so the committed types can never silently fall out of step with the backend. The generated file is pinned to openapi-typescript 7.13.0 and excluded from prettier, eslint, and knip, and marked linguist-generated so it collapses in diffs. No openapi-fetch and no call-site changes yet; this only makes the types exist. * chore(ui): tidy gen-api-types script per review Write the spec dump inside a with-block and clean up the temp dir in a finally, so repeated local runs don't leave stray ~MB JSON files behind.	2026-06-05 17:20:01 -07:00
ryan-crabbe-berri	c7f1bcfd0d	build(ui): migrate eslint to flat config and bump eslint-config-next to 16 (#29626) ... ESLint 9 defaults to flat config and eslint-config-next was pinned at 15 while Next is on 16, so eslint only ran with ESLINT_USE_FLAT_CONFIG=false and next lint is gone on Next 16. Replace .eslintrc.json with a native flat eslint.config.mjs (config-next 16 ships flat configs, so no FlatCompat shim is needed), bump eslint-config-next to 16.2.6, add @eslint/js and typescript-eslint as explicit devDeps for the recommended rule sets, and point the lint script at eslint directly. This only makes eslint runnable on modern tooling; it does not wire it into CI. The same rules carry over (next/core-web-vitals, eslint and typescript-eslint recommended, prettier, unused-imports)	2026-06-03 15:50:20 -07:00
yuneng-jiang	0715ed3359	build(deps): bump next from 16.2.4 to 16.2.6 in /ui/litellm-dashboard (#27665) (#28524) ... Bumps [next](https://github.com/vercel/next.js) from 16.2.4 to 16.2.6. - [Release notes](https://github.com/vercel/next.js/releases) - [Changelog](https://github.com/vercel/next.js/blob/canary/release.js) - [Commits](https://github.com/vercel/next.js/compare/v16.2.4...v16.2.6) --- updated-dependencies: - dependency-name: next dependency-version: 16.2.6 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-05-22 00:13:56 +00:00
Yuneng Jiang	faba1fda8a	Merge remote-tracking branch 'origin/main' into litellm_merge_main_staging ... Restore lineage between main and internal staging so the next staging->main promotion (#27436) can merge without conflicts. main was 2 commits ahead: - 6ff668c7aa squash-merge of the previous staging->main promotion (#27245) - 8c9830eef9 feat(xai): add grok-4.3 (#27396), already present on staging The squash-merge has no shared lineage with the individual commits that went into staging, which is why git surfaced 13 textual conflicts despite both sides having the same logical content. Every conflicting file's main-side change came from 6ff668c7aa only, and the matching staging-side changes are the post-promotion evolution. Resolved all 13 with --ours (staging's version is the latest evolution; main's snapshot is stale). The grok-4.3 entries auto-merged in model_prices_and_context_window.json and its backup, but were already on staging via an independent commit, so the net diff vs HEAD is empty for those files. Net new content from this merge: 12 lines added to ui/litellm-dashboard/package-lock.json -- npm 11 libc array tags on four existing entries, no functional impact.	2026-05-07 17:49:48 -07:00
user	63bda3f001	Merge remote-tracking branch 'upstream/litellm_internal_staging' into cve-sweep-2026-05 ... # Conflicts: # uv.lock	2026-05-07 23:03:28 +00:00
Krrish Dholakia	454ce5073f	fix(anthropic, mcp): sanitize tool names to match Anthropic's [a-zA-Z0-9_-]{1,128} pattern (#26788) ... * fix(anthropic, mcp): sanitize tool names to match Anthropic's `^[a-zA-Z0-9_-]{1,128}$` Tool names with characters like `/` or `.` (commonly produced by the OpenAPI -> MCP generator from `operationId`s such as `actions/download-job-logs-for-workflow-run`) caused Anthropic to reject requests with `tools.N.custom.name: String should match pattern '^[a-zA-Z0-9_-]{1,128}$'`. Two layers of fix: 1. Anthropic transformation: build a per-request forward map (original -> sanitized, disambiguated by suffix on collisions) and a reverse map (only for names actually rewritten). Forward map is applied to tool defs, `tool_choice`, and historical assistant tool_calls in messages. Reverse map is threaded through both the non-streaming and streaming response paths so callers continue to see their original tool names in `tool_use` blocks. 2. OpenAPI -> MCP generator: sanitize `operationId` (and the method+path fallback) at registration time so generated MCP tools are valid for any strict-name provider, not just Anthropic. The dashboard preview endpoint applies the same sanitization for parity. Includes unit tests covering: collision disambiguation between `foo_bar` and `foo/bar` in the same request, reverse-map only firing for actually-rewritten names, message rewrite for historical tool_calls, streaming chunk_parser reverse-mapping, and sanitization of OpenAPI operationIds plus the preview endpoint output. Made-with: Cursor * fix(anthropic): build tool-name maps in transform_request, not optional_params The previous patch stashed the per-request forward and reverse tool-name maps under ``optional_params["_anthropic_tool_name_forward_map"]`` and ``optional_params["_anthropic_tool_name_map"]``. ``optional_params`` is the dict that becomes the JSON body via ``data = {**optional_params}``, so those internal keys leaked over the wire and Anthropic 400'd with: _anthropic_tool_name_forward_map: Extra inputs are not permitted Worse, this meant *every* request whose tool list contained any name with an invalid character (the exact case the patch was meant to fix) regressed into a confusing meta-error pointing at LiteLLM's internal map instead of the offending tool. Fix: move all tool-name sanitization into ``transform_request``, which is the single chokepoint already shared by ``AnthropicConfig``, ``AmazonAnthropicConfig`` (Bedrock invoke), ``VertexAIAnthropicConfig``, and ``AzureAnthropicConfig`` (all call ``super().transform_request`` / ``AnthropicConfig.transform_request(self, ...)``). New static helper ``_sanitize_tool_names_in_request`` walks the already-Anthropic-shaped ``optional_params["tools"]`` (only ``type=="custom"`` entries -- hosted tool names are reserved by Anthropic and must not be touched), builds the per-request forward/reverse maps, and applies the forward map in place to ``tools[*].name`` and ``tool_choice.name``. The reverse map is stashed exclusively on ``litellm_params`` (which is never serialized to a provider) under ``_anthropic_tool_name_map`` for the response paths to consume. Side effect of this restructure: ``map_openai_params`` is now a pure OpenAI->Anthropic param translator with no side-channel state, which matches its contract everywhere else in the codebase. Tests: replaced the now-incorrect "stashes maps in optional_params" tests with regressions that assert no underscore-prefixed keys appear in either ``optional_params`` after ``map_openai_params`` or in the final ``transform_request`` body. Added end-to-end coverage for: sanitization in ``transform_request``, ``tool_choice`` rewriting, historical ``tool_calls`` rewriting in messages, and hosted-tool passthrough. Made-with: Cursor * fix(anthropic): always sanitize empty text content blocks Anthropic 400s on `{"role": "user", "content": ""}` with: "messages: text content blocks must be non-empty" LiteLLM already had `_sanitize_empty_text_content` to rewrite empty text to a placeholder, but it was gated behind `litellm.modify_params=True`. With that flag off (default), empty content from upstream agent frameworks (e.g. pydantic-ai) flowed straight through and tripped the Anthropic validator. Fix: - Always run `_sanitize_empty_text_content` at the top of `anthropic_messages_pt`, independent of `modify_params`. There is no way to "pass through" an empty text block, so this is non-optional. The richer tool-call sanitizations (Cases A/B/D, which actually mutate conversation structure) remain gated on `modify_params`. - Extend `_sanitize_empty_text_content` to also handle list-of-blocks content (`[{"type": "text", "text": ""}]`), not just string content. Adds 3 regression tests covering string content, list-of-blocks content, and the no-op case (non-empty messages with modify_params off). Made-with: Cursor * fix(anthropic): drop dead tool-name forward-map params, fix mypy + caller-mutation - remove unused `name_forward_map` param from `_map_tool_choice`, `_map_tool_helper`, `_map_tools` and the `_apply_anthropic_tool_name_forward` helper. Production sanitization runs in `_sanitize_tool_names_in_request` at `transform_request`; these params were never threaded through. - handler.py: use `ANTHROPIC_TOOL_NAME_REVERSE_MAP_KEY` constant instead of the hardcoded `"_anthropic_tool_name_map"` string. - fix mypy `"object" has no attribute "__iter__"` in `_rewrite_tool_names_in_messages` by guarding `tool_calls` with `isinstance(..., list)`. - `_sanitize_tool_names_in_request`: build a new tools list with copy-on- change entries (and copy `tool_choice` on rewrite) so a caller reusing the same tool list/dicts across requests doesn't see its inputs permanently rewritten. - doc-comment `_build_request_tool_name_maps` clarifying it operates on OpenAI-format tools (vs `_sanitize_tool_names_in_request` which runs on Anthropic-format tools post-`_map_tools`). - tests: drop 3 tests pinning the now-removed param paths; add coverage for tool_calls + None function_call rewrite and caller-dict immutability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(mcp): inherit stored credentials in test/tools/list for edit flow When editing an existing MCP server, the Tool Configuration preview calls POST /mcp-rest/test/tools/list with server_id but no credentials (management API redacts them). The endpoint now calls _inherit_credentials_from_existing_server() so stored bearer tokens and OAuth2 M2M credentials are loaded from global_mcp_server_manager automatically — tools load without re-entering credentials. New servers (no server_id) and requests with explicit credentials are unaffected (function is a no-op in both cases). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(mcp): show all tools in edit panel, not just allowed tools Edit flow was passing externalTools (from GET /tools/list, filtered by allowed_tools) to MCPToolConfiguration, disabling the internal hook. Remove the external props so the internal hook fires via POST /test/tools/list, which returns all tools unfiltered. Combined with the credential inheritance fix, tools load automatically without re-entering credentials and all tools are visible for re-configuration. existingAllowedTools still pre-checks previously allowed tools. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix order-dependent collision in _build_anthropic_tool_name_maps Use a two-pass approach: first pre-register all already-valid tool names in the 'used' set, then sanitize/disambiguate names that need rewriting. This ensures valid names always have priority regardless of input order, preventing duplicate tool names on the wire when e.g. 'foo/bar' appears before 'foo_bar' in the tool list. Add regression test for the reversed ordering case. * Fix OpenAPI tool name collision: disambiguate sanitized names with numeric suffixes sanitize_openapi_tool_name replaces all invalid chars with '_', but when two operationIds differ only by sanitized characters (e.g. 'foo/list' and 'foo.list' both become 'foo_list'), the second registration silently overwrites the first in the tool registry. Add collision disambiguation in register_tools_from_openapi that appends _2, _3, ... suffixes when a sanitized name is already taken, mirroring the existing logic in _build_anthropic_tool_name_maps. * Fix preview endpoint missing collision disambiguation for tool names Add used_names tracking and _2/_3 suffix disambiguation to _preview_openapi_tools, matching the logic in register_tools_from_openapi. Without this, two operationIds that sanitize to the same string (e.g. 'foo/list' and 'foo.list' both becoming 'foo_list') would show duplicate names in the preview while registration would disambiguate them. * Align preview HTTP method order with register_tools_from_openapi The preview endpoint and register_tools_from_openapi both use order-dependent collision disambiguation (_2, _3 suffixes). When the iteration order differs, two operations on the same path with sanitized names that collide get different suffixes in preview vs registration, so the dashboard shows names that don't match what actually got registered. Also adds a regression test that fails on the swapped order. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Skip duplicate originals in _build_anthropic_tool_name_maps If the same invalid tool name appeared twice in original_names (e.g. ['foo/bar', 'foo/bar']), the second occurrence overwrote the forward map entry with a freshly-suffixed name (foo_bar_2), leaving foo_bar orphaned in 'used' with no reverse mapping. _sanitize_tool_names_in_request then rewrote both tool entries to foo_bar_2, and Anthropic 400'd on duplicate tool names. Skip the rewrite if forward already has the original mapped. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>	2026-05-06 00:00:36 +00:00
yuneng-jiang	6ff668c7aa	[Infra] Promote internal staging to main (#27245) ... * default requested_model to empty string on litellm-side rejects * Update litellm/router.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: scope key access_group_ids override by team's assigned groups A team member could set any access_group_ids on their key (e.g. a group assigned only to a different team) and override the team's model restriction. Intersect the key's access_group_ids with team_object.access_group_ids in _key_access_group_grants_model so foreign groups are dropped before model expansion. Adds a regression test that asserts expansion is never called for foreign groups. * [Fix] Proxy: Skip Personal Budget Hook When Reservation Covers Counter The reservation path (PR #26845) atomically pre-fills `spend:user:{user_id}` and admits at the strict-`<` boundary. The legacy `_PROXY_MaxBudgetLimiter` pre-call hook re-reads the same counter with `>=`, so a reservation that fills the counter to exactly `max_budget` (e.g. a request without a `max_tokens` cap that falls back to reserving the smallest remaining headroom) is rejected by the hook even though the reservation already admitted it. Skip the hook when the request's active `budget_reservation` covers `spend:user:{user_id}`. The reservation is the source of truth for that counter cross-pod; the legacy `>=` path remains in place for requests without a reservation (e.g. paths that bypass the reservation entirely). Reproduces as `tests/otel_tests/test_prometheus.py::test_user_budget_metrics` on a fresh user with `max_budget=10` calling `fake-openai-endpoint` without `max_tokens`. Adds focused unit coverage in `tests/test_litellm/proxy/hooks/test_max_budget_limiter.py`. * harden bedrock file bucket validation * Fix syntax errors from botched merge in router.py * Fix Vertex batch output edge cases * [Fix] RBAC: Drop management_routes Write Fallback for Admin Viewer Greptile P1: the unsafe-method branch of `_check_proxy_admin_viewer_access` ended with a blanket `if route in management_routes: return`. That set is a mix of reads (info/list — handled via the safe-method GET branch above) and writes. The fallback let Admin Viewer POST to write endpoints not enumerated in `_ADMIN_VIEWER_BLOCKED_WRITE_ROUTES`, including: - /team/block, /team/unblock, /team/permissions_update - /jwt/key/mapping/{new,update,delete} - /key/bulk_update - /key/{key_id}/reset_spend Remove the fallback. The two remaining allow sets (admin_viewer_routes and global_spend_tracking_routes) are both read-only, so removal does not affect the legitimate POST-as-read cases (e.g. /spend/calculate, which is in spend_tracking_routes ⊂ admin_viewer_routes). Tests: - 8 new parametrized cases pinning each previously-leaking management write endpoint to 403 on POST for PROXY_ADMIN_VIEW_ONLY. * fix(tests): anchor VCR redis cassette key to repo root `os.path.relpath` with no `start` arg uses the current working directory, so running pytest from a subdirectory produced a different Redis key than running from the repo root. CI-recorded cassettes and locally-replayed runs would silently miss each other's cache. Anchor the path to the repo root (derived from `__file__`) so the key is stable regardless of CWD. https://claude.ai/code/session_018uCx7pcrkdUJZrCVMaTdPx * fix: gate key access_group override on group's own assignment Replaces the previous intersect-with-team.access_group_ids check, which made the override unreachable in practice (the team-gate fallback already covered every case the intersection allowed). The override now resolves each of the key's access_group_ids via get_access_object and accepts the group only if its assigned_team_ids includes the key's team_id, or its assigned_key_ids includes the key's token. This fulfills the original ask (a key can extend a team's allow-list via a group the admin granted to that team or that specific key) while still rejecting foreign groups referenced by team members of other teams. * [Fix] Proxy/Key Management: Honor team_member_permissions /key/list In /key/list Endpoint When a team grants /key/list via team_member_permissions, non-admin members should see all keys for that team — same as a team admin. Previously the classification in list_keys() only checked admin status, so permitted members fell into the service-account-only path and could not see other members' personal keys. Routes those members into the full-visibility set. * Fix access-group bypass via litellm-model fallback path When _get_all_deployments returns 0 candidates and the litellm-model fallback branch (_get_deployment_by_litellm_model) finds deployments that the access-group filter then empties, _access_group_filter_emptied_candidates remained False (it was captured before that branch ran). The router would then proceed to default fallbacks; the fallback model could have no access_groups and short-circuit the filter, silently serving a caller blocked by access-group restrictions. Update the flag inside the litellm-model branch when filtering empties a non-empty candidate set so the default-fallback guard still triggers. * fix(proxy): redact MCP server URL and headers for non-admin viewers (VERIA-8) Many MCP integrations (Zapier, etc.) embed an upstream API key directly in the server URL, e.g. ``https://actions.zapier.com/mcp/<api-key>/sse``. The list and single-server endpoints were returning the full URL to any authenticated user — `_redact_mcp_credentials` only stripped the explicit ``credentials`` field, and `_sanitize_mcp_server_for_virtual_key` only ran for restricted virtual keys. Non-admin internal users could read the dashboard, click the unmask toggle, and exfiltrate the raw token. Add `_sanitize_mcp_server_for_non_admin` that runs on top of the existing credential redaction and clears the credential-bearing fields: - ``url`` (the primary leak vector) - ``spec_path`` (OpenAPI spec URLs that may carry tokens) - ``static_headers`` / ``extra_headers`` (Authorization) - ``env`` (arbitrary secrets) - ``authorization_url`` / ``token_url`` / ``registration_url`` Identity fields (``server_id``, ``alias``, ``mcp_info``, etc.) are preserved so the UI can still list servers a non-admin's team has access to. Apply the new sanitizer in `fetch_all_mcp_servers` and the per-server fetch path right after the existing virtual-key branch. Update the existing `test_list_mcp_servers_non_admin_user_filtered` assertions that previously checked URL visibility. Frontend defense-in-depth: hide the URL unmask toggle on `mcp_server_view.tsx` unless the viewer is a proxy admin. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix runtime policy attachment initialization Mark runtime-created policies and attachments initialized so global policy attachments created from the policy builder apply immediately without requiring a restart. Co-authored-by: Cursor <cursoragent@cursor.com> * test(router): cover _try_early_resolve_deployments_for_model_not_in_names The router_code_coverage CI check requires every function in router.py to be referenced by at least one test under tests/{local_testing, router_unit_tests,test_litellm} in a file with "router" in its name. The recently-extracted helper had no direct test, so the check failed with "0.45% of functions in router.py are not tested". Add a focused test that exercises the four return paths: model already in self.model_names, no fallback applies, pattern-router match, and default_deployment substitution (also asserting the stored default isn't mutated). https://claude.ai/code/session_019AVp1XL7RT9RxRe4qRLkay * Fix policy registry teardown in tests Reset the policy ID index during policy engine test cleanup so stale policy versions cannot leak between tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(batches): count non-chat tokens, validate batch-file model access (VERIA-39) (#27015) * fix(batches): count non-chat tokens and validate every model in batch file Two security control bypasses on POST /v1/batches: 1. `_get_batch_job_input_file_usage` only summed tokens for `body.messages` (chat completions). Embedding (`input`) and text completion (`prompt`) batches reported zero, letting massive non-chat workloads slip past TPM rate limits. Extend the counter to handle string and list shapes for both fields. 2. The batch input file was forwarded to the upstream provider without inspecting the models named inside the JSONL — only the outer `model` query parameter was checked against the caller's allowlist. A caller restricted to gpt-3.5 could submit a batch targeting gpt-4o and the upstream would execute it under the proxy's shared API key. Add `_get_models_from_batch_input_file_content` (returns the distinct `body.model` values) and call it from `_enforce_batch_file_model_access` in the pre-call hook, which runs each model through `can_key_call_model` so the same allowlist semantics (wildcards, access groups, all-proxy-models, team aliases) the proxy enforces on `/chat/completions` apply here too. Any unauthorized model raises a 403 before the file is forwarded. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(batches): count pre-tokenized prompt/input shapes, classify 403 logs Two follow-ups from the Greptile review on the batch validation PR: 1. P1 TPM bypass via integer token arrays. The OpenAI batch schema accepts ``prompt`` and ``input`` as ``list[int]`` (a single pre-tokenized prompt) or ``list[list[int]]`` (multiple) in addition to the string and ``list[str]`` shapes. Pre-fix only the string shapes were counted, so a caller could submit a batch with hundreds of millions of pre-tokenized tokens and the rate limiter would record zero. Extract the per-field logic into ``_count_prompt_or_input_tokens`` and count each int as one token. 2. P2 access-denial logs were indistinguishable from I/O failures. ``count_input_file_usage`` caught every exception under a generic "Error counting input file usage" message, so an intentional 403 from ``_enforce_batch_file_model_access`` looked the same in the logs as a missing file or a Prisma timeout. Catch ``HTTPException`` separately and log 403s at WARNING level with a security-relevant message before re-raising. Tests cover the new shapes: single ``list[int]``, ``list[list[int]]`` (the worst-case bypass vector), and embeddings ``input`` with pre-tokenized arrays. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proxy): re-validate user_id after /user/info re-parses query (#27009) * fix(proxy): re-validate user_id ownership after /user/info re-parses query The route-level access check in `RouteChecks.non_proxy_admin_allowed_routes_check` reads `request.query_params.get("user_id")`, which decodes literal `+` to spaces. The endpoint then re-parses the raw query string with `urllib.unquote` in `get_user_id_from_request` to preserve `+` characters (so plus-addressed emails work as user_ids). Those two paths produce different ids: a caller who registered a user_id containing a literal space could pass the route check and then read another user's row by sending the encoded `+` form. Add `_enforce_user_info_access` and call it after `_normalize_user_info_user_id` returns the final id. Proxy admin / view-only admin still bypass; everyone else must match the resolved user_id (or have no user_id, which falls back to the caller's own id later in the handler). Tests cover the admin bypass, owner-match path, and the cross-user lookup that this change blocks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proxy): apply user_info ownership check to PROXY_ADMIN_VIEW_ONLY `_enforce_user_info_access` was bypassing both PROXY_ADMIN and PROXY_ADMIN_VIEW_ONLY, but the upstream route check in `RouteChecks.non_proxy_admin_allowed_routes_check` only treats PROXY_ADMIN as a true admin for the `/user/info` route — view-only admins go through the `user_id == valid_token.user_id` enforcement along with regular users. Mirroring that asymmetry left the same encoded-`+` bypass open for view-only admins whose user_id contains a literal space. Drop the PROXY_ADMIN_VIEW_ONLY exemption so the post-decode re-check matches the upstream rule. Update tests: a view-only admin must now be blocked from cross-user lookups but still allowed to read their own row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(spend-logs): opt-in suppression of stack traces in spend-tracking error logs Adds LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS env var. When set to true and the proxy log level is INFO or above, spend-tracking error paths emit a single ERROR line without the full traceback. Stack traces are preserved at DEBUG and the Sentry / proxy_logging_obj.failure_handler path is unchanged. The new spend_log_error helper is wired through the spend write hot path: - DBSpendUpdateWriter (update_database, _update_*_db, batch upsert, redis-commit fallbacks) - _ProxyDBLogger._PROXY_track_cost_callback - get_logging_payload exception path - update_spend / update_daily_tag_spend / spend logs queue monitor Resolves LIT-2704. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(spend-logs): preserve no-traceback behavior for update_daily_tag_spend This call site previously logged a single-line error via verbose_proxy_logger.error() with no traceback. Switching it to spend_log_error(..., exc=e) caused a full stack trace to render by default (when LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS is unset), which contradicts the PR goal of leaving default behavior unchanged. Revert this specific site to the original error log call. * fix(spend-logs): preserve no-traceback behavior for update_daily_tag_spend Bugbot caught a regression: the previous error log here was a single-line verbose_proxy_logger.error(...) with no traceback. spend_log_error attaches the active exception's traceback by default (when the suppression env var is unset), so swapping it in changed default behavior. Revert this one site to its original .error() call to keep the PR strictly opt-in. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * feat(spend-logs): suppress traceback in SpendLogs error_information row Extend LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS to the failure callback so the per-row Metadata pane in the UI no longer shows the stack trace when the opt-in env var is set, matching the existing console-side suppression. https://claude.ai/code/session_014dztoRbRnRvq54HL9EyHx6 * [Fix] Proxy: Repair Merge Fallout In Router-Override Fallback Auth Conflict resolution for #26968 dropped the `Iterator` typing import (NameError at module load), left a dead `fallback_models = cast(...)` block, and the new tests called `_enforce_key_and_fallback_model_access` without the now-required `request` kwarg. * isolate dual OTEL handlers * harden cloud file compatibility path * harden cloud file compatibility path * [Fix] Proxy/Key Management: Align Key-Org Membership Checks On Generate And Regenerate Mirrors the membership rule on /key/update so that /key/generate and /key/{key}/regenerate apply the same `_validate_caller_can_assign_key_org` gate when the caller specifies an `organization_id`. Proxy admins bypass. The check no-ops when `organization_id` is not being set. * thread trusted params through vertex file content * trust only server legacy file flag * chore(proxy): keep public AI hub unauthenticated * fix(proxy): preserve low-detail readiness status * [Test] Anthropic: Replace Legacy Claude-4-Sonnet Alias With Haiku 4.5 Three live-API tests pinned to claude-4-sonnet-20250514, which is a non-canonical alias of claude-sonnet-4-20250514. Anthropic's main API no longer resolves the legacy form under freshly issued keys, so the tests fail with not_found_error. The token counter test pinned to claude-sonnet-4-20250514 itself (deprecation_date 2026-05-14, two weeks out) was on borrowed time too. Bump all four to claude-haiku-4-5-20251001 — capability superset for what these tests exercise (streaming, parallel tool calling, extended thinking, token counting), no upcoming deprecation, cheaper per-token. * chore(proxy): move URL-valued model/file_id guard from SDK to proxy The previous per-provider guards in HuggingFace, Oobabooga, and Gemini files lived in the SDK layer, breaking SDK callers who legitimately pass URL-valued model identifiers. Move the check to the proxy boundary in add_litellm_data_to_request so SDK users keep working while proxy users default-deny URL-valued model and file_id, with admin opt-in via litellm.provider_url_destination_allowed_hosts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [Chore] Proxy/UI: Drop stray _experimental/out/chat/index.html This file is a regenerable UI build artifact that should not be tracked in source. Removing so the merge into litellm_internal_staging stays clean. * [Test] Anthropic Passthrough: Bump Streaming Cost-Injection Test To Haiku 4.5 test_anthropic_messages_streaming_cost_injection hits the proxy's /v1/messages route, which routes via the anthropic/* wildcard to api.anthropic.com. The 404 surfaced in the test was Anthropic's own not_found_error propagated back through the proxy (visible from the x-litellm-model-id hash on the response — the proxy did route). Same root cause as the prior commit: the legacy claude-4-sonnet-20250514 alias is no longer recognized by Anthropic's main API under the new key. Swap to claude-haiku-4-5-20251001 — same routing path, canonical model. * fix(proxy): handle ownership-recording failures after upstream create If record_container_owner raises after the upstream container is created, the user previously got a 500 with no usable container — they were billed for an unreachable resource. Move ownership recording into the create path's exception handling and split the two failure modes: - HTTPException from the recorder (auth conflicts) propagates verbatim so the client sees the real status code, not a generic LLM error. - Unexpected exceptions are logged and swallowed; the response is returned to the caller so they aren't billed for a container they can't address. The DB row stays untracked until an operator reconciles. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(guardrails): close post-call coverage gaps * fix(types): add /team/permissions_bulk_update to management_routes The blocklist check in _check_proxy_admin_viewer_access only fires for routes that match LiteLLMRoutes.management_routes — the bulk-update endpoint was missing from that list, so the test for view-only admins on /team/permissions_bulk_update fell through to "allow." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [Test] Anthropic Passthrough: Bump Thinking Tests Off Legacy Sonnet 4 Alias base_anthropic_messages_test.test_anthropic_messages_with_thinking and test_anthropic_streaming_with_thinking still pinned to claude-4-sonnet-20250514 — the same legacy alias Anthropic no longer recognizes under freshly issued keys. The other four tests in this base class already use claude-sonnet-4-5-20250929; these two were missed. Bump to claude-haiku-4-5-20251001 (supports_reasoning=true, no upcoming deprecation). Subclasses including TestAnthropicPassthroughBasic inherit these methods. * fix(guardrails): cover multi-choice output variants * fix(proxy): preserve public ai hub ui setting * fix(scim): cascade FK cleanup on user delete and surface block status in UI SCIM DELETE /Users/{id} previously called litellm_usertable.delete without clearing rows that FK back to the user, so Postgres rejected the delete with LiteLLM_InvitationLink_user_id_fkey and the SCIM caller saw a 500. Add a helper to drop invitation_link, organization_membership, and team_membership rows before the user delete (mirrors /user/delete in internal_user_endpoints). Also add a Status column to the Virtual Keys and Internal Users tables so admins can see at a glance which keys are blocked and which users SCIM has deactivated. SCIM-blocked keys carry a tooltip explaining the origin. Pin the dashboard's Node version to 20 via .nvmrc to match CI. * chore: update Next.js build artifacts (2026-05-02 03:21 UTC, node v20.20.2) * perf(proxy): cache container/skill ownership reads on the hot path Container ownership and skill rows are looked up on every retrieve / delete / list / file-content / chat-completion-with-skill call. The new stores wrapped raw Prisma queries with no cache, putting one DB round-trip on each request. Add an in-process TTL'd cache mirroring the _byok_cred_cache pattern in mcp_server/server.py: per-key (value, monotonic_timestamp), 60s TTL, 10000-entry cap with full-clear on overflow, invalidated by every write. Negative results (`None`) are cached too so untracked-resource checks also skip the DB. Tests cover: cache-after-first-hit, negative caching, write invalidation, no-caching-on-DB-error, TTL expiry, capacity eviction. 56 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update Next.js build artifacts (2026-05-02 03:39 UTC, node v20.20.2) * fix: remove traceback key instead of it being "" * fix: linting error * fix(scim): preserve scim_active on PUT when client omits the field A SCIM PUT may legally omit `active` (full-replace with the field absent). Pydantic fills the SCIMUser.active default of True, so the PUT handler was overwriting metadata.scim_active with True even when the client never sent it — silently reactivating a previously SCIM-blocked user and unblocking their keys. Use model_fields_set to detect whether the client actually sent `active`. If omitted, preserve the prior scim_active value and skip the cascade to virtual keys. Also drop comments added in this PR that just narrate what the code does; keep only the docstrings and the SQL-NULL pitfall note that explain non-obvious behaviour. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(proxy): use set lookup for permitted agent filters * fix(mcp): redact command fields for non-admin server views * fix(proxy): forward decoded container ids after ownership checks * fix(caching): handle stale isolated Redis semantic index * fix(cloudflare): support response_text in streaming chunk parser Newer Cloudflare Workers AI models (e.g. Nemotron) emit 'response_text' instead of 'response' on streamed chunks. The non-streaming path was already updated to fall back to 'response_text' (#26385), but the streaming chunk parser still only read 'response', which caused streaming requests against those models to silently produce empty content. Mirror the non-streaming fallback in CloudflareChatResponseIterator.chunk_parser and add a streaming test for the response_text shape. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Fix code qa * Address bugbot: drop dead encode/decode helpers; preserve empty custom_id - Remove unused _encode_gcp_label_value / _decode_gcp_label_value singular helpers; only the _chunks variants are actually called. - Use 'is not None' check for custom_id so empty-string custom_ids are still labeled and round-trip through batch outputs. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Forward Vertex file content logging context * test vertex file content logging forwarding Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> * Fix Vertex batch output logging mutation * fix: don't mutate caller's logging_obj in _try_transform_vertex_batch_output_to_openai The method was overwriting logging_obj.optional_params, logging_obj.model, and logging_obj.start_time on the caller's Logging instance. When invoked from llm_http_handler.py's generic framework path, the framework's own logging_obj (which already went through pre_call) had its properties clobbered, causing model and start_time to reflect the last batch line's values rather than the original call context. Fix: create a fresh local Logging instance for the per-line transformation instead of mutating the incoming logging_obj. The caller's object is now left entirely untouched regardless of whether a logging_obj was passed in or not. Regression tests added to verify model, start_time, and optional_params are not mutated on the caller's logging_obj. Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> * feat: add opt-out flag for Vertex batch output transformation Adds litellm.disable_vertex_batch_output_transformation (default False). When True, afile_content returns raw Vertex predictions.jsonl untouched so users that parse candidates/modelVersion directly are not broken. * fix(anthropic,bedrock): omit thinking/output_config when reasoning_effort="none" Setting reasoning_effort="none" on Anthropic chat models (direct, Bedrock Invoke, Bedrock Converse, Vertex AI Anthropic, Azure AI Anthropic) crashed LiteLLM with: litellm.APIConnectionError: 'NoneType' object has no attribute 'get' Both the Anthropic chat transformation and Bedrock Converse called ``AnthropicConfig._map_reasoning_effort`` and assigned the ``None`` it returns for ``"none"`` directly to ``optional_params["thinking"]``. Downstream ``is_thinking_enabled`` then did ``optional_params["thinking"].get("type")`` and crashed. Pop ``thinking`` (and on Claude 4.6/4.7, ``output_config``) instead of assigning ``None``, restoring the documented contract that ``reasoning_effort="none"`` means "do not enable thinking". This also prevents downstream Anthropic 400s ("thinking: Input should be an object", "output_config.effort: Input should be ...") if the bug were ever masked. Verified end-to-end against the live Anthropic API and Bedrock Converse on claude-opus-4-{5,6,7} and claude-sonnet-4-6, plus Bedrock Invoke for Claude 4.5/4.6. Vertex AI Anthropic and Azure AI Anthropic inherit the fixed ``map_openai_params`` from ``AnthropicConfig`` and need no further changes. * fix(vertex-ai): set response=null on batch error entries per OpenAI spec The Vertex batch output transformer was emitting both a populated 'response' and 'error' for failed batch entries. The OpenAI Batch output spec defines them as mutually exclusive: on error 'response' MUST be null. This broke any consumer using 'result["response"] is None' to detect failures. * test(vertex-ai): cover transformation_error path emits response=null * fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers DotpromptManager was hardened to render through ImmutableSandboxedEnvironment. The three sibling managers (gitlab, arize, bitbucket) were missed and still instantiate plain jinja2.Environment(), leaving the same attribute-traversal SSTI primitive open: a template fetched from a GitLab/BitBucket repo or Arize Phoenix workspace can reach __class__.__init__.__globals__ and execute arbitrary Python on the proxy host. Match the dotprompt pattern by switching all three to ImmutableSandboxedEnvironment. The sandbox blocks the dunder-traversal chain while leaving normal {{ var }} substitution intact, so the template surface is unchanged for legitimate use. Adds tests/test_litellm/integrations/test_prompt_manager_ssti.py (18 cases) verifying each manager's jinja_env is a sandbox, that classic SSTI payloads raise SecurityError, and that ordinary variable rendering still works. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(proxy): drop client-supplied pricing fields from request bodies The proxy currently forwards request-body pricing parameters (the fields on `CustomPricingLiteLLMParams`, plus `metadata.model_info`) into the core call path. Those fields belong to deployment configuration, not to per-request input — sending them from a client mutates the request's recorded cost and, via `litellm.completion` → `register_model`, the process-wide `litellm.model_cost` map for every later caller in the worker. Strip them at the boundary. The strip set is built from `CustomPricingLiteLLMParams.model_fields` so pricing fields added later are covered automatically. Operators who do want clients to supply per-request pricing can opt back in per key or team via `metadata.allow_client_pricing_override = true`, mirroring the existing `allow_client_mock_response` and `allow_client_message_redaction_opt_out` flags. Tests cover the strip set's coverage, root and metadata strips, the opt-in skip on both key and team metadata, and a regression check that the global `litellm.model_cost` map is unmutated after a stripped request. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(proxy): log stripped pricing fields at debug for operator visibility Operators upgrading would otherwise see client-supplied pricing overrides silently stop applying with no diagnostic. Emit a debug-level line listing the dropped fields and pointing at the opt-in flag when any are stripped; stay silent on the no-op path so the log isn't filled with noise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proxy): move pricing strip below the litellm_metadata JSON-string parse The strip ran before the proxy parses ``litellm_metadata`` from a JSON string into a dict (a path used by multipart/form-data and ``extra_body`` callers), so ``isinstance(metadata, dict)`` was False and ``model_info`` survived the strip. Move the call to the same post-parse position the ``user_api_key_*`` strip already uses for the same reason. Adds a regression test exercising the JSON-string ``litellm_metadata`` path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(responses): replace legacy claude-4-sonnet alias in multiturn tool-call test Anthropic's main API no longer resolves the non-canonical 'claude-4-sonnet-20250514' alias for freshly issued keys, returning 404 not_found_error. PR #27031 already swept three other live tests pinned to this alias to claude-haiku-4-5-20251001 but missed test_multiturn_tool_calls in the responses API suite, which is now failing reliably on PR CI runs (e.g. PR #27074, job 1603363). Bump the two model references in test_multiturn_tool_calls to the same claude-haiku-4-5-20251001 snapshot used by PR #27031 -- it covers everything this test exercises (tool calling, multi-turn) and isn't on a deprecation schedule. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * chore(proxy): close callback-config and observability-credential side channels Two related gaps in the proxy's request bouncer: 1. ``is_request_body_safe`` (auth_utils.py) walked the request-body root and the ``litellm_embedding_config`` nested dict, but not ``metadata`` or ``litellm_metadata``. The same fields it bans at root — Langfuse / Langsmith / Arize / PostHog / Braintrust / Phoenix / W&B Weave / GCS / Humanloop / Lunary credentials and routing — were silently accepted when the caller put them inside metadata, retargeting observability callbacks to a caller-controlled host with caller-supplied creds. Walk both metadata containers (and parse the JSON-string form sent via multipart / ``extra_body``) through the same banned-params helper, so the existing ``allow_client_side_credentials`` opt-in covers both paths consistently. 2. The banned-params list was hand-maintained and lagged the canonical ``_supported_callback_params`` allow-list in ``initialize_dynamic_callback_params``. Derive the observability bans from that allow-list (minus a small ``_SAFE_CLIENT_CALLBACK_PARAMS`` set for informational fields like ``langfuse_prompt_version`` and ``langsmith_sampling_rate``) so future integrations are covered automatically; ``_EXTRA_BANNED_OBSERVABILITY_PARAMS`` carries the handful of fields integrations read but the allow-list hasn't caught up to. A guard test fails CI if a new entry is added to ``_supported_callback_params`` without an explicit safe-list decision. Separately in ``litellm_pre_call_utils.py``: add ``callbacks``, ``service_callback``, ``logger_fn``, and ``litellm_disabled_callbacks`` to ``_UNTRUSTED_ROOT_CONTROL_FIELDS``. The first three are appended to worker-wide ``litellm.{input,success,failure,_async_*,service}_callback`` lists / ``litellm.user_logger_fn`` from inside ``function_setup`` — one request poisons every subsequent caller in that worker. The last is the inverse primitive: the legitimate path reads it from key/team metadata, the request-body version silently disables admin-configured audit / observability for the call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(auth): per-param allow must continue, not return early A pre-existing logic bug in ``_check_banned_params``: when the deployment-level ``configurable_clientside_auth_params`` permitted one banned field, the loop ``return``-ed on the first match instead of ``continue``-ing, so any other banned param later in the same body or metadata dict was never checked. This PR's metadata walk multiplies the surface where that bypass matters — a body pairing an allowed ``api_base`` with an observability credential like ``langfuse_host`` would silently pass. Proxy-wide ``allow_client_side_credentials`` keeps ``return`` (it's a global opt-in for every banned param). The per-param branch becomes ``continue`` so only the one explicitly-permitted field is skipped. Adds a regression test that exercises the api_base + langfuse_host pair. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(vector_store): resolve embedding config at request time, never persist creds The vector store create/update path previously called ``_resolve_embedding_config`` against the admin-configured router/DB model and persisted the resolved ``litellm_embedding_config`` dict (``api_key`` / ``api_base`` / ``api_version``) into the ``litellm_managedvectorstorestable.litellm_params`` column. Because the resolver expanded ``os.environ/...`` references via ``get_secret``, the DB row carried cleartext provider credentials, and the ``/vector_store/{new,info,update,list}`` responses returned them to any authenticated caller who could supply a known admin model name. Move the auto-resolve out of ``create_vector_store_in_db`` and out of the update path. Persist only the user-supplied ``litellm_embedding_model`` reference. Resolve at request-handling time inside ``_update_request_data_with_litellm_managed_vector_store_registry`` so the resolved config lives in the per-request ``data`` dict and is garbage-collected after the response. Legacy rows that were created by an earlier proxy version and already carry a resolved ``litellm_embedding_config`` skip the re-resolution and pass through unchanged so embedding calls keep working. The ``new_vector_store`` response now also runs the existing ``_redact_sensitive_litellm_params`` masker (already used by ``info``, ``update``, and ``list``), defending against caller-supplied cleartext on the create path and against legacy rows whose persisted credentials are still in the database. Existing tests that asserted the old write-time-resolve behaviour are updated to assert the new persistence shape (no embedding config stored, just the model reference). Two new tests cover the use-time path: one asserting fresh resolution happens when a row carries only the model reference, the other asserting legacy rows with persisted config skip re-resolution and continue to work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(vector_store): tighten registry-mutation comment and dedupe test helpers * fix(vector_store): cache use-time embedding-config resolution Hold the resolved config in a process-memory TTL cache so the request-handling path doesn't run litellm_proxymodeltable.find_first on every vector-store call. * fix(anthropic,bedrock,vertex): forward output_config.effort + 400 on garbage reasoning_effort Follow-up bugs surfaced by the QA sweep on PR #27039 (https://github.com/BerriAI/litellm/pull/27039#issuecomment-4363363610). 1. Stop stripping output_config.effort on Bedrock + Vertex adaptive routes. - Vertex AI Claude 4.6/4.7 accepts output_config.effort on rawPredict (verified end-to-end against us-east5 / global). The strip helper now no-ops for effort. - Bedrock Converse routes output_config into additionalModelRequestFields for anthropic base models so the requested adaptive tier (low/medium/ high/xhigh/max) actually reaches the wire instead of all collapsing to identical thinking. - Bedrock Invoke chat transformation (AmazonAnthropicClaudeConfig) stops popping output_config from the post-AnthropicConfig request body. - Bedrock Invoke /v1/messages allowlist (BedrockInvokeAnthropicMessagesRequest) now lists output_config so the runtime allowlist filter forwards it. 2. Validate effort across Bedrock Converse so 'disabled' / 'invalid' / '' / unsupported tiers (xhigh/max on Sonnet 4.6 or budget-mode 4.5 models) surface as a clean 400 BadRequestError instead of 500. 3. ValueError -> BadRequestError throughout (AnthropicConfig.map_openai_params, _apply_output_config, AmazonConverseConfig._handle_reasoning_effort_parameter). Empty-string effort is now rejected (was silently passing the 'if effort and ...' short-circuit). 4. Floor reasoning_effort='minimal' at the Anthropic provider minimum (1024 budget_tokens) via new ANTHROPIC_MIN_THINKING_BUDGET_TOKENS so it's a usable tier on direct Anthropic / Azure AI Anthropic / Vertex AI Anthropic / Bedrock Invoke (all of which 400 below 1024). 5. model_prices: dedupe duplicate supports_max_reasoning_effort key on claude-opus-4-7 / claude-opus-4-7-20260416. Adds regression tests across all five affected paths; existing tests asserting the silent-strip behavior were updated to reflect the new pass-through and clean 400 surfaces. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(constants): make ANTHROPIC_MIN_THINKING_BUDGET_TOKENS a plain constant The documentation CI test (tests/documentation_tests/test_env_keys.py) asserts every os.getenv() key in the source has a matching entry in the litellm-docs config_settings.md table. ANTHROPIC_MIN_THINKING_BUDGET_TOKENS tracks Anthropic's published wire-protocol minimum (1024) — it's not a user-tunable, so making it env-overridable was wrong anyway. Drop the os.getenv() wrapper; the value is now a plain literal. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(anthropic,bedrock): correct effort error message and dedupe effort_map - Remove 'none' from the Bedrock _validate_anthropic_adaptive_effort error message; it was listed as a valid value but rejected by the membership check, leaving users in a feedback loop if they tried 'none'. - Hoist the duplicated reasoning_effort -> output_config.effort mapping out of AnthropicConfig.map_openai_params and AmazonConverseConfig._handle_reasoning_effort_parameter into a single AnthropicConfig.REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT class constant so the two routes cannot drift. * fix(anthropic): translate reasoning_effort on /v1/messages route Closes the remaining QA-sweep gap on PR #27074: Bedrock Invoke /v1/messages was silently ignoring ``reasoning_effort`` because the shared param filter only kept native Anthropic keys, so every effort tier collapsed to the same behavior on the wire (27/231 cells failing across opus-4-5 / opus-4-6 / sonnet-4-6). Map ``reasoning_effort`` to native Anthropic ``thinking`` / ``output_config.effort`` at the ``AnthropicMessagesConfig`` layer so all four /v1/messages routes (direct Anthropic, Azure AI, Vertex AI, Bedrock Invoke) inherit the same translation: - Add ``reasoning_effort`` to ``AnthropicMessagesRequestOptionalParams`` so the param filter in ``AnthropicMessagesRequestUtils.get_requested_anthropic_messages_optional_param`` no longer drops it before the transformation runs. - Add ``_translate_reasoning_effort_to_anthropic`` and call it from ``transform_anthropic_messages_request``. Mirrors ``AnthropicConfig.map_openai_params`` on the chat completion path (re-uses ``_map_reasoning_effort`` and ``REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT``) so the two routes cannot drift. Pops ``reasoning_effort`` so it never reaches the wire. - Caller-supplied native ``thinking`` / ``output_config.effort`` always win — same precedence as ``_translate_legacy_thinking_for_adaptive_model``. - Garbage values (``""``, ``"disabled"``, ``"invalid"``) raise ``AnthropicError(status_code=400)`` instead of falling through and surfacing as 500s from the provider. - ``"none"`` clears thinking + output_config so callers can opt out per request. Also restores the non-adaptive-model test coverage on Bedrock Invoke /v1/messages that the previous commit lost when ``test_bedrock_messages_strips_output_config`` was renamed to the ``forwards`` variant on Opus 4.7. Adds a new test file ``test_reasoning_effort_translation.py`` covering the translation at the shared config level (adaptive + non-adaptive models, none, garbage, caller precedence) so all four /v1/messages routes are exercised by a single suite. Adds parametrized + behavioral tests on the Bedrock Invoke /v1/messages suite covering: minimal/low/medium/high/xhigh/max mapping for adaptive models, thinking-budget mapping for non-adaptive Opus 4.5, ``none`` clears both, garbage raises 400, explicit ``output_config`` wins. Refs: https://github.com/BerriAI/litellm/pull/27074 * fix(anthropic,bedrock): reject unmapped reasoning_effort at mapping site Both the chat completion path (AnthropicConfig.map_openai_params) and the Bedrock Converse path (_handle_reasoning_effort_parameter) used REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get(value, value) which falls back to the raw input on unmapped keys. Combined with _map_reasoning_effort returning type='adaptive' for any string on Claude 4.6/4.7, garbage values (e.g. 'disabled') could leak into optional_params['output_config']['effort'] unvalidated if map_openai_params ran without the downstream transform_request or _validate_anthropic_adaptive_effort check. Mirror the /v1/messages pattern: use .get(value) (no fallback) and raise BadRequestError immediately when the value is unmapped, co-locating validation with the mapping for defense in depth. * style: black formatting Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(anthropic): stop class-attr leak; gate xhigh/max on every route The reasoning-effort mapping dict was a public class attribute on AnthropicConfig, so BaseConfig.get_config returned it as a request parameter and every Anthropic-backed call (Anthropic / Azure / Vertex / Bedrock Invoke) hit a 400 'REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT: Extra inputs are not permitted' from the provider. Move the mapping to a module-level constant. _supports_effort_level only looked the model up under custom_llm_provider='anthropic', so bedrock-prefixed model ids (e.g. bedrock/invoke/us.anthropic.claude-opus-4-7) returned False for both 'max' and 'xhigh' even when the underlying model entry has the flag set. Strip known provider prefixes and retry the lookup against litellm.model_cost directly so per-model gating works on every route. Mirror the per-model xhigh/max gate from AnthropicConfig._apply_output_config in AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic so the /v1/messages route also raises a clean 400 instead of forwarding the unsupported tier. * feat(anthropic,bedrock): strip output_config under drop_params for non-effort models When a proxy fronts Claude Code (which always sends `output_config.effort`) at a pre-4.5 Anthropic model — haiku-3, sonnet-3.5, opus-3, sonnet-4 — the forwarded knob causes a forced 400 the client can't fix. Gating a strip behind the existing `drop_params` flag lets operators opt into silent fixup once and stop worrying about per-model param hygiene. Default (`drop_params=False`) still forwards and surfaces the provider's error, preserving the strict, debuggable contract from #27074. Per https://platform.claude.com/docs/en/build-with-claude/effort the supporting set is Opus 4.5+, Sonnet 4.6+, and Mythos Preview; everything else is dropped (with a verbose_logger warning so the strip is visible). Recognition uses model-name patterns plus a fallback to any `supports_*_reasoning_effort` flag in the model map for forward compatibility with new entries. https://claude.ai/code/session_01WjHq31rvXT6xYNdVmSJvRp (cherry picked from commit 1233943e78) * fix(base_llm): filter all _-prefixed class attrs from get_config The drop_params strip work added `AnthropicConfig._EFFORT_SUPPORTING_MODEL_PATTERNS` as a private class-level lookup tuple. `BaseConfig.get_config()` only filtered the `__`-prefixed names plus `_abc` / `_is_base_class`, so `_EFFORT_SUPPORTING_MODEL_PATTERNS` would have leaked into the request body the same way `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT` did before the previous commit. Generalize the existing `_abc` / `_is_base_class` carve-outs to skip every `_`-prefixed name. `AmazonConverseConfig.get_config()` overrides the base method, so apply the same change there. Also unblocks future internal helpers from accidentally serialising into the wire body. * fix(anthropic): drive output_config.effort support from model map flags Replace hardcoded _EFFORT_SUPPORTING_MODEL_PATTERNS with a JSON-backed check that uses supports_*_reasoning_effort flags from the model map. Add supports_minimal_reasoning_effort: true to opus-4-5 and mythos-preview entries (which previously only carried supports_reasoning) so the JSON remains the single source of truth for effort capability. * fix(anthropic,bedrock,databricks): four reasoning_effort follow-ups - claude-sonnet-4-6 + reasoning_effort=max no longer 400s. Renamed _is_opus_4_6_model to _is_claude_4_6_model at three sites and added supports_max_reasoning_effort: true to 12 model entries in the JSON cost map (10 sonnet 4.6 ids + OpenRouter opus 4.6/4.7). - _map_reasoning_effort now raises BadRequestError(400) directly with llm_provider, instead of letting Databricks (and similar callers) surface its raw ValueError as a 500. - output_config.effort on Opus 4.5 over Bedrock no longer 400s for missing effort-2025-11-24 beta. Flipped JSON to "effort-2025-11-24" for bedrock + bedrock_converse and added an auto-attach branch in _process_tools_and_beta for non-adaptive Anthropic + output_config on Converse. - reasoning_effort=xhigh / =max on legacy budget-mode models (Haiku 4.5, Sonnet 4.5, Opus 4.5) now map to thinking.budget_tokens 8192 / 16384 instead of returning 400. Added two constants in litellm/constants.py. Tests updated for all four flips. Validated end-to-end via 306-cell live proxy matrix (6 model families x 3 routes x 17 effort cases), all pass. * fix(databricks): validate reasoning_effort and set output_config on adaptive Claude The Databricks path called `AnthropicConfig._map_reasoning_effort` for Claude models but never validated the effort string nor set `output_config.effort` for adaptive models (Claude 4.6/4.7). Since `_map_reasoning_effort` returns `type=adaptive` for ANY non-None / non-"none" string on adaptive models (including "disabled", "invalid", ""), Databricks silently accepted garbage and emitted a request without an `output_config.effort`, collapsing every adaptive tier to identical behavior. Match the Anthropic native, Bedrock Converse, Bedrock Invoke, and /v1/messages paths: when the resolved `thinking` is non-None on a 4.6/4.7 model, look up the value in `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT` and either raise a clean `BadRequestError` or set `optional_params["output_config"]`. * fix(azure): omit model from image generation and image edit deployment requests Azure OpenAI routes image gen/edit by deployment in the URL; sending the deployment id in model breaks gpt-image-2 (invalid_value). Strip model from JSON for deployments/.../images/generations and from multipart data for .../images/edits. Non-deployment URLs (e.g. Azure AI FLUX) unchanged. Fixes #26316. Co-authored-by: Cursor <cursoragent@cursor.com> * test(azure): exercise image gen JSON filter via HTTP client; dedupe image edit URL - Image generation tests patch HTTPHandler.post / get_async_httpx_client so make_*_azure_httpx_request runs and wire json is asserted on call kwargs. - Azure image edit: strip model in finalize_image_edit_multipart_data using the same URL string the handler passes to POST (no second get_complete_url in transform). BaseImageEditConfig default finalize is a no-op. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(azure_ai/anthropic): promote output_config out of extra_body so validation runs `azure_ai` is registered in `litellm.openai_compatible_providers`, so `add_provider_specific_params_to_optional_params` (litellm/utils.py) auto-stuffs any non-OpenAI kwarg (e.g. `output_config={"effort": "..."}`) into `optional_params["extra_body"]`. `AzureAnthropicConfig.transform_request` then strips `extra_body` entirely on the way out, silently dropping the param — and `AnthropicConfig._apply_output_config` never sees it, so `effort="invalid"` / `effort="xhigh"` on a non-supporting model quietly reaches the model with default behavior instead of returning a clean 400 (as the native `anthropic` provider does). Promote the keys back to top-level `optional_params` (using `setdefault` so explicit top-level values win) before delegating to the parent `AnthropicConfig`. Apply in both `validate_environment` and `transform_request` so flag detection (`is_mcp_server_used`, etc.) and output-config validation both run. Surfaced by the QA matrix expansion on PR #27074: 20 cells where Azure returned 200 while `anthropic` returned 400 — all `output_config` mode across haiku_4_5, sonnet_4_5, opus_4_5, sonnet_4_6, opus_4_6, opus_4_7 families with `effort` in {invalid, xhigh, max, low, medium, high}. Tests: * `test_output_config_promoted_from_extra_body`: valid effort reaches data * `test_invalid_output_config_effort_raises_via_extra_body`: 400 on bad effort * `test_unsupported_effort_xhigh_raises_via_extra_body`: 400 on xhigh-on-Sonnet-4.6 * `test_extra_body_promotion_does_not_clobber_top_level`: setdefault semantics * test(image_gen): expect no model in Azure image edit multipart (#26316) Align test_azure_image_edit_litellm_sdk with deployment-scoped Azure edits. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(anthropic): extract _validate_effort_for_model to prevent drift The chat completion path (`_apply_output_config`) and the /v1/messages pass-through (`AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic`) both gate `max` / `xhigh` per model. The two sites had diverged from near-identical copies into separately maintained blocks, creating a real drift risk when a new model tier (e.g. Claude 4.8) lands -- a contributor could update one site and miss the other. Centralise the gating in `AnthropicConfig._validate_effort_for_model`, which returns an error message string or `None`. Each call site keeps its own provider-appropriate exception type (`BadRequestError` for the chat path, `AnthropicError` for the /v1/messages pass-through) but the gating decision now comes from one place. Net -11 LOC. Adds a parametrised unit test exercising the helper directly across 4.5 / 4.6 / 4.7 model families and `max` / `xhigh` / lower-effort inputs. Existing tests at both call sites continue to pass unchanged. Addresses Greptile finding on PR #27074. * fix(databricks): narrow reasoning_effort_value to str for mypy `non_default_params.get("reasoning_effort")` returns `Any | None`, but `REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get()` expects `str`. Mypy flagged this on the strict pass. Narrow with `isinstance` before the lookup; non-strings fall through to the existing `BadRequestError` below with a clean validation message, so behavior is unchanged. Fixes a regression introduced by 1a10746e95 in this PR. * feat(proxy): add health_check_reasoning_effort for model health checks Co-authored-by: Cursor <cursoragent@cursor.com> * test(image_gen): align Azure image gen fixture with body omitting model Expected JSON matches deployment-scoped Azure POST (#26316). Co-authored-by: Cursor <cursoragent@cursor.com> * test(anthropic/chat): force PR-local model_cost map via autouse fixture CI runs without LITELLM_LOCAL_MODEL_COST_MAP=True, so litellm.model_cost is loaded from main-branch JSON (default model_cost_map_url) instead of the PR's checked-out model_prices_and_context_window.json. Tests that assert per-model flags added in this PR (supports_max_reasoning_effort, supports_xhigh_reasoning_effort) therefore pass locally but fail in CI with 'AssertionError: assert False is True' on 5 cases: - test_anthropic_model_supports_effort_param_recognizes_supporting_models [anthropic.claude-mythos-preview, bedrock/.../mythos-preview, claude-opus-4-5-20251101] - test_supports_effort_level_handles_provider_prefixes [bedrock/invoke/us.anthropic.claude-sonnet-4-6-max-True, claude-sonnet-4-6-max-True] Add an autouse fixture at tests/test_litellm/llms/anthropic/chat/conftest.py that monkey-patches litellm.model_cost to the PR-local JSON for every test in this directory. The parent conftest already snapshots+restores litellm.model_cost per-function, so the mutation is contained. This is a scoped workaround. The proper fix is to set the env var globally in the test workflow once the ~10 inline self-set test files are audited; tracking that as a follow-up issue. * [Fix] Docker: Pin Wolfi And Uv To Multi-Arch Index Digests The previous pins resolved to single-platform amd64 manifests, so buildx pulled the same amd64 base for both linux/amd64 and linux/arm64 targets. The published OCI index then advertised an arm64 entry whose layers are byte-identical to amd64 -- arm64 users got an amd64 binary. Switch all three Dockerfiles to the multi-arch image-index digests: - cgr.dev/chainguard/wolfi-base (index has linux/amd64 + linux/arm64) - ghcr.io/astral-sh/uv:0.11.7 (index has linux/amd64 + linux/arm64) Resolved with `docker buildx imagetools inspect <ref>` -- that returns the index digest. `docker pull` + `docker inspect` returns the per-host platform digest, which is what slipped in last time. * [Fix] Docker: Pin Uv To Multi-Arch Index Digest In Remaining Dockerfiles Apply the same fix to the three Dockerfiles not in the release pipeline today (alpine, dev, health_check) so they stay correct if/when they're built for arm64 in the future. Wolfi pins are not present in these files; the python:3.11-alpine and python:3.13-slim digests they already use are multi-arch indexes that include arm64/v8, so only the uv pin needed swapping. * fix(xai): fold reasoning_tokens into completion_tokens to satisfy OpenAI invariant xAI's chat completions API accounts reasoning_tokens separately from completion_tokens, but rolls them into total_tokens. This breaks the OpenAI invariant total_tokens == prompt_tokens + completion_tokens that downstream consumers (including litellm's own _usage_format_tests in tests/llm_translation/base_llm_unit_tests.py:58) rely on. Live capture (grok-3-mini-beta, 2026-05-04): prompt=14, completion=10, total=336, reasoning=312 14 + 10 = 24, NOT 336. OpenAI's o1/o3 reasoning models include reasoning_tokens in completion_tokens, leaving the prompt+completion=total invariant intact. xAI deviates. This patch aligns xAI to OpenAI semantics by folding reasoning_tokens into completion_tokens after the parent OpenAI parser runs. The fold is idempotent and defensive: - Only fires when total_tokens == prompt_tokens + completion_tokens + reasoning_tokens (the documented xAI shape). Refuses to fold if the gap doesn't match, guarding against silent corruption when xAI changes accounting. - Skips if completion_tokens already covers the gap (already normalised — e.g. cost calc replays a previously-folded Usage). xai.cost_calculator.cost_per_token already added reasoning_tokens to the visible completion count for billing. Post-fold the Usage block now satisfies that invariant directly, so the cost calc would double-bill. Updated cost_per_token to detect the OpenAI-normalised shape (total == prompt + completion) and skip the reasoning add-on in that case, falling through to the legacy raw-shape behaviour for callers that bypass the transformation (e.g. proxy log replay). Tests: - Adds TestXAIReasoningTokenFolding covering: gap-explained-fold, idempotent-no-double-fold, no-reasoning-skip, gap-mismatch-skip. - Adds test_already_normalised_usage_does_not_double_count_reasoning to lock the cost-calc idempotency. - Updates 7 pre-existing cost-calc tests whose total_tokens was internally inconsistent (used the OpenAI-normalised total but kept reasoning_tokens external) to use the documented xAI raw shape total = prompt + visible completion + reasoning. Pre-existing values masked the missing-fold by accident. Verified end-to-end against the live xAI API: LITELLM_LOCAL_MODEL_COST_MAP=False (CI default) + XAI_API_KEY set + pytest tests/llm_translation/test_xai.py::TestXAIChat::test_prompt_caching -> PASSED in 18.81s (was: AssertionError on usage.total_tokens == usage.prompt_tokens + usage.completion_tokens) 20/20 tests in tests/test_litellm/llms/xai/test_xai_cost_calculator.py and 8/8 in tests/test_litellm/llms/xai/test_xai_chat_transformation.py pass. * refactor(bedrock/converse): delegate effort gating to AnthropicConfig._validate_effort_for_model Removes the duplicated max/xhigh gating logic in _validate_anthropic_adaptive_effort and the now-unused _supports_effort_level_on_bedrock helper. Per-model gating now flows through the centralized AnthropicConfig._validate_effort_for_model (whose _supports_effort_level already strips Bedrock prefixes), so the chat completion, /v1/messages, and Bedrock Converse paths can't drift when a new gated effort tier is added. * Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. * Fix qdrant semantic cache miss metadata * chore(deps): refresh dependency locks * chore(deps): authorize pytest license * fix: preserve tokenizer decode round trips * refactor(anthropic): drive adaptive-thinking gate via supports_adaptive_thinking flag Three of greptile's open comments on #27074 (P2 converse:512, P1 databricks:361, and the underlying capability-flag policy rule) flagged the same pattern: _is_claude_4_6_model(...) or _is_claude_4_7_model(...) used inline as a runtime 'is this an adaptive-thinking model?' check. That requires a code release each time a new adaptive Claude lands. Consolidate the inline gating to AnthropicModelInfo._is_adaptive_thinking_model, and switch the helper itself to read a new supports_adaptive_thinking flag from `model_prices_and_context_window.json` via `_supports_factory`, falling back to the family pattern only when the model-map entry doesn't carry the flag (preserves OpenRouter / Vercel / Bedrock-prefixed variants that route through the same code path with non-canonical ids). Adds `supports_adaptive_thinking: true` to the four 4.6/4.7 anthropic entries (opus-4-6 + dated, opus-4-7 + dated, sonnet-4-6). Bedrock-prefixed and Vertex-prefixed entries don't need the flag because both fall back through the family pattern (the helper short-circuits early on True from either path) and the bedrock/vertex Claude IDs all match the existing opus-4-{6,7} / sonnet-4-{6,7} pattern. Affected call sites: - `bedrock/chat/converse_transformation.py:_handle_reasoning_effort_parameter` - `anthropic/chat/transformation.py:_map_reasoning_effort` - `anthropic/chat/transformation.py:map_openai_params` (output_config branch) - `databricks/chat/transformation.py:map_openai_params` (output_config branch) The remaining `_is_claude_4_6_model` / `_is_claude_4_7_model` references in `AnthropicConfig._validate_effort_for_model` and `AnthropicConfig.get_supported_openai_params` are intentionally retained: they're per-model gating fallbacks for variants whose model-map entries don't yet carry the `supports_max_reasoning_effort` / `supports_reasoning` flag. Those are documented in-place. Tests: 537 anthropic/bedrock/databricks/vertex/messages tests pass. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * chore(deps): address dependency review notes * test(model_prices): add supports_adaptive_thinking to schema `test_aaamodel_prices_and_context_window_json_is_valid` validates the model-map JSON against an explicit schema with `additionalProperties`, so the new `supports_adaptive_thinking` flag added in 98ced0ae43 needs a matching schema entry. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * refactor: remove unnecessary comments from #27074 Strip out the explanatory and historical comments that don't carry business-logic justification. Comments that simply narrate what code does — or that explain prior behavior, what was changed, or which PR introduced a fix — are removed. Docstrings are reduced to a one-line summary where the long form repeated information already evident from the code or test data. No code-behavior changes. All 643 affected unit tests still pass. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test: keep decode token test local * chore(deps): align dashboard node engine * feat: selectively apply routing strategy according to model name * style: make _model_supports_effort_param more concise * refactor(anthropic,bedrock): hoist drop_params output_config warning to module constant Three call sites (anthropic chat, bedrock converse, bedrock invoke messages) emitted the same '...Effort is only supported on Opus 4.5+, Sonnet 4.6+, and Mythos Preview' warning verbatim. Extract DROP_UNSUPPORTED_OUTPUT_CONFIG_WARNING in litellm/llms/anthropic/chat/transformation.py and import it from the bedrock sites so future copy edits live in one place. Addresses Michael's review on PR #27074. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * refactor(anthropic,bedrock,databricks): factor BadRequestError for unknown reasoning_effort Three call sites raised the same BadRequestError("Invalid reasoning_effort: ... Must be one of 'minimal', 'low', ...") block when REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT returned None: anthropic chat map_openai_params, bedrock converse _handle_reasoning_effort_parameter, and databricks chat reasoning_effort path. Extract AnthropicConfig._raise_invalid_reasoning_effort(model, value, llm_provider) so future copy edits / valid-set changes happen in one place. Typed as NoReturn so type-checkers correctly narrow control flow at call sites. Addresses Michael's review on PR #27074. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Clean up Redis semantic cache isolation fallback * fix(guardrails): align banned_keywords + azure_content_safety call_type gates with runtime route_type The hooks gated on ``call_type == "completion"`` but the proxy ingress passes ``route_type`` straight through as ``call_type`` — ``"acompletion"`` for /v1/chat/completions and ``"aresponses"`` for /v1/responses. Tests passed because they used the literal sync ``"completion"`` value, masking the gap. Switch both hooks to ``is_text_content_call_type`` (matches the canonical runtime values: completion / acompletion / aresponses) and update existing tests to assert against runtime values, plus parametrize a regression test that pins the gate. * fix: remove unused import * Add semantic cache legacy migration flag * Treat 0 team_member_budget as no cap * chore(caching): annotate qdrant quantization_params dict type Mypy infers the dict's value type from the first branch (Dict[str, bool]) which clashes with the scalar branch's mixed-type inner dict. Explicit Dict[str, Any] annotation lifts the inference. * chore(caching): remove allow_legacy_unscoped_cache_hits opt-in The flag was an opt-in escape hatch for the cross-tenant leak the rest of the patch closes — flipping it on (env var or constructor param) re-enables exactly the VERIA-54 primitive on either backend. There is no operational need that the secure path doesn't already meet: - Qdrant: legacy points without ``litellm_cache_key`` payload are excluded by the must-clause filter and treated as misses; new sets populate the cache key, so cold-start lasts only as long as the natural cache rebuild. - Redis: existing unscoped index can't carry the new schema; the init path falls back to ``{name}_isolated`` (and recreates it on stale schema), leaving the legacy index untouched. Drop the constructor param, env-var fallback, ``_using_legacy_unscoped_index`` flag, the legacy-reuse branch in ``_init_semantic_cache``, and the matching guards in set/get paths. Update tests to drop the legacy-mode cases and assert the secure-only behaviour. * fix(container): keep ownership-filter exceptions out of the LLM-error path filter_container_list_response runs after the upstream call has already succeeded; treating an ownership-lookup failure as an LLM-API error fires post_call_failure_hook for a successful upstream call and returns a misleading provider-shaped error to the client. Run the filter outside the try/except so genuine LLM errors stay scoped to the upstream call. * chore(container,skills): LRU eviction for owner caches; widen file_purpose Literal Two cleanups from the /simplify pass: * ``_CONTAINER_OWNER_CACHE`` and ``_SKILL_CACHE`` now LRU-evict via ``OrderedDict.popitem(last=False)`` instead of full ``clear()`` at capacity. Full clears converted a steady-state cached workload into a periodic full-DB-load oscillation as the cache repopulated from zero and cleared again. Reads now ``move_to_end`` so the just-touched entry survives the next eviction. Mirrors the pre-existing LRU pattern in ``_remember_container_owner``. * ``LiteLLM_ManagedObjectTable.file_purpose`` Literal now includes ``"container"`` so Pydantic validation accepts rows written by the ownership store. * chore(container,skills): drop legacy-access opt-out env vars LITELLM_ALLOW_UNTRACKED_CONTAINER_ACCESS and LITELLM_ALLOW_UNOWNED_SKILL_ACCESS were operator-toggleable opt-outs for the cross-tenant access primitive this PR closes — flipping either on re-enabled exactly the VERIA-20 read path. Default-secure with no escape hatch matches sibling fixes (vector-store cred isolation, semantic cache key isolation, user_config strip): all rejected the opt-out-of-security pattern. Untracked containers and unowned skills (rows that pre-date this enforcement) are admin-only. Non-admin owners need to either re-create via the now-tracked flow or have an admin assign ``created_by`` on the existing row. Update tests to assert the strict-only behaviour. * fix(ownership): reject identity-less callers instead of sharing a sentinel scope UNSCOPED_RESOURCE_OWNER_SCOPE collapsed every caller without an identity field (no user_id / team_id / org_id / api_key / token) into a single shared owner — a cross-tenant access primitive: any two such callers could see and delete each other's containers and skills. Drop the sentinel. ``get_primary_resource_owner_scope`` returns ``None`` and ``get_resource_owner_scopes`` returns ``[]`` for identity-less callers. ``record_container_owner`` and ``LiteLLMSkillsHandler.create_skill`` now reject creates from identity-less callers with a 403 instead of stamping the placeholder. Read paths already deny ``owner is None`` correctly so legacy rows (if any) are admin-only. * fix(proxy): include request-blocked callback params in auth bans * fix: keep skills handler FastAPI-free; fold gcs deny list into the body bouncer Two cleanups: * ``LiteLLMSkillsHandler.create_skill`` raised ``HTTPException`` for identity-less callers, importing FastAPI from a ``litellm/llms/`` module — that violates the project rule that FastAPI lives only under ``proxy/``. Switch to ``ValueError`` (the same shape the rest of the handler uses for not-found/forbidden) and update the test. * The proxy-auth body bouncer derived its observability ban list from ``_supported_callback_params`` only, missing ``_request_blocked_callback_params`` (where ``gcs_bucket_name`` and ``gcs_path_service_account`` live). Two recently-merged sibling PRs (#27019 added the deny list, #27081 added the test asserting these are rejected at the request body root) crossed without folding them together. Union the GCS deny list into the bouncer's derivation so the single source of truth covers both code paths. * fix(proxy): normalize managed resource team owner field * chore: simplify ownership tracking — drop thin stores, in-memory fallback, hand-rolled cache Substantial reduction (~765 LOC) without changing the security boundary: * Drop ContainerOwnershipStore and LiteLLMSkillsStore — both were one-method-per-Prisma-call wrappers. Inline the calls instead, matching the established pattern in vector_store_endpoints, agent_endpoints, and mcp_server/db.py. * Drop the prisma_client is None in-memory fallback. Production deploys always have Prisma; running ownership-critical paths on a process-local dict is a security footgun in the dev-mode case it was meant to support, and complicates every code path with a branch. Fail-secure: skip recording if Prisma is unavailable, and treat reads as "not found" (admin-only). * Drop the hand-rolled module-level cache. Replace with the existing litellm.caching.in_memory_cache.InMemoryCache, which already has TTL + max-size + eviction tested in its own module. Sentinel string for negative caching since InMemoryCache can't disambiguate "miss" from "cached as None". * Tests: drop coverage for removed code paths (in-memory fallback, hand-rolled cache internals). Keep tests for actual behavior (cache hit-rate, negative caching, owner check, list filtering, identity-less reject, admin bypass). * fix(container): cache list-allow-set, track admin-created containers Address Greptile P2 follow-ups from the prior round: * Cache ``_get_allowed_container_ids`` (60s LRU/TTL keyed by sorted owner-scope tuple) so ``GET /v1/containers`` doesn't issue a fresh ``find_many`` against ``litellm_managedobjecttable`` on every list call. Invalidate the caller's own cache entry when they record a new owner so the just-created container shows up on their next list. * Tighten the admin early-return in ``record_container_owner`` to skip ONLY when there's literally no container ID to stamp. An admin with identity (the master-key path populates ``user_id`` + ``api_key``) flows through the normal record path so admin-created containers are tracked like any other caller's. The truly-identity-less admin case still falls through to the 403 below — correct fail-secure default. Skill-cache invalidation gap (also flagged by Greptile) is moot: there is no skill update endpoint exposed; ownership-affecting mutations are only delete (already invalidates) and create (new ID, no cache entry to update). * chore(container): use delete_cache, json-encode scope key, clean test /simplify follow-ups: * Replace the two-``pop`` reach into ``cache_dict``/``ttl_dict`` with the existing public ``InMemoryCache.delete_cache(key)`` — the same idiom used elsewhere in the proxy. Bonus: ``delete_cache`` calls ``_remove_key`` which also handles ``expiration_heap`` consistency the direct pops were silently leaking. * JSON-encode the sorted scope list for the cache key instead of ``"|".join``. ``user_id`` / ``team_id`` / ``org_id`` / ``api_key`` are free-form strings and could contain a literal ``|`` — JSON quoting escapes any in-string separator unambiguously. * Extract ``_allowed_container_ids_cache_key()`` so the read and invalidation sites compute the key the same way. * Fix a placeholder-then-overwrite test construction: the ``__module__.split(".")[0] and "proxy_admin"`` line evaluated to a literal string that was immediately overwritten with the real enum value. Hoist the import and construct directly. * [Fix] Tests: Replace deprecated openrouter/claude-3.7-sonnet with claude-sonnet-4.5 OpenRouter has dropped active endpoints for anthropic/claude-3.7-sonnet, causing test_reasoning_content_completion to fail with a 404 "No endpoints found" error. Switch to anthropic/claude-sonnet-4.5, which is current and supports reasoning streaming. * feat: routing groups ui * fix(security): prevent secret_fields from leaking into spend logs secret_fields (containing raw HTTP headers including Authorization Bearer tokens) was being included in proxy_server_request['body'] because the body snapshot was a copy.copy(data) of the full request dict. This body gets serialized and persisted in the LiteLLM_SpendLogs table, exposing user credentials in the database. Root cause: data['secret_fields'] was set before the body snapshot at data['proxy_server_request']['body'] = copy.copy(data), so the full raw headers (including auth tokens) ended up in the snapshot. Fix (defense in depth): 1. Exclude 'secret_fields' when creating the body snapshot in litellm_pre_call_utils.py (primary fix) 2. Strip 'secret_fields' in _sanitize_request_body_for_spend_logs_payload as a secondary safeguard secret_fields remains available on the live data dict for legitimate downstream consumers (MCP, Responses API). Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com> * chore: update Next.js build artifacts (2026-05-05 02:13 UTC, node v20.20.2) * [Fix] Proxy: Break managed-resources import cycle on Python 3.13 The Python 3.13 CCI smoke matrix surfaces a partially-initialized-module ImportError when loading the managed files hook chain: litellm.proxy.hooks/__init__ (mid-import) -> enterprise.enterprise_hooks -> litellm_enterprise.proxy.hooks.managed_files -> litellm.llms.base_llm.managed_resources.isolation -> litellm.proxy.management_endpoints.common_utils -> litellm.proxy.utils (re-enters litellm.proxy.hooks) The except ImportError block in hooks/__init__.py silently swallowed the failure, leaving managed_files unregistered and POST /files returning 500 "Managed files hook not found". Two-layer fix: - Inline the 3-line _user_has_admin_view check in isolation.py instead of importing it from litellm.proxy.management_endpoints.common_utils. litellm.llms.* should not depend on litellm.proxy.* — removing this layering violation breaks the cycle at its root. - Define PROXY_HOOKS and get_proxy_hook before the conditional enterprise import in litellm/proxy/hooks/__init__.py, so any future re-entry resolves the public names instead of hitting an ImportError on a partially-initialized module. Also fold in two unrelated CCI repairs surfaced in the same staging run: - tests/otel_tests/test_key_logging_callbacks.py: per-key gcs_bucket_name / gcs_path_service_account are now stripped by initialize_dynamic_callback_params, so the GCS client falls through to the env-only branch. Update the assertion to match the new "GCS_BUCKET_NAME is not set" message. - .circleci/config.yml: tests/pass_through_tests now resolves google-auth-library@10.x via the @google-cloud/vertexai 1.12.0 bump, which uses dynamic ESM imports Jest 29 cannot load without --experimental-vm-modules. Pass that flag in the Vertex JS test step. Adds tests/test_litellm/proxy/hooks/test_proxy_hooks_init.py as a regression guard: managed_files / managed_vector_stores must register, and isolation.py must not transitively import litellm.proxy.utils. * [Fix] Proxy: Address Greptile feedback on hook-cycle PR - Move _user_has_admin_view to litellm.proxy._types as user_api_key_has_admin_view (single source of truth). common_utils.py and isolation.py both import from there now, removing the duplicated role-check that could silently diverge if new admin roles are added. - Add pytest.importorskip("litellm_enterprise") to the two regression tests that assert managed_files / managed_vector_stores are registered; those keys come from ENTERPRISE_PROXY_HOOKS so the tests would fail unconditionally in a checkout without the enterprise extra installed. * [Fix] Lint: Mark _user_has_admin_view re-export in common_utils Ruff F401 flagged the aliased import as unused within common_utils.py because the name is consumed only by external modules (~15 callers across guardrails, spend tracking, MCP, agents, management endpoints). Add `# noqa: F401 re-exported` so the alias survives lint while keeping a single source of truth in litellm.proxy._types. * refactor(azure): move image gen JSON helper; rename image edit finalize hook - Add image_generation/http_utils.azure_deployment_image_generation_json_body; call from azure.py (keeps AzureChatCompletion focused on chat). - Rename finalize_image_edit_multipart_data to finalize_image_edit_request_data with docstring covering multipart and JSON POST payloads (review feedback). Co-authored-by: Cursor <cursoragent@cursor.com> * test(proxy): cover health_check_reasoning_effort for completion mode Co-authored-by: Cursor <cursoragent@cursor.com> * [Fix] Tests: Use master key for /otel-spans in test_chat_completion_check_otel_spans /otel-spans now requires proxy admin (returns 401 'Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/otel-spans' for non-admin callers). Switch the GET call to use the master key sk-1234 while keeping the generated key for the chat-completion request that produces the spans. * [Fix] Tests: Pick chat-completion OTEL trace by content, not recency The /otel-spans endpoint returns process-wide spans and tags most_recent_parent by max start_time. After tightening that route to proxy_admin (sk-1234), the GET /otel-spans request itself emits auth spans that beat the chat-completion spans on start_time, so most_recent_parent now points at the request's own auth trace (['postgres', 'postgres']) and the >=5-span assertion fails. Pick the chat-completion trace by content: it is the only trace whose span list is a superset of {postgres, redis, raw_gen_ai_request, batch_write_to_db}. Verified locally end-to-end against otel_test_config.yaml + OTEL_EXPORTER=in_memory: 3/3 runs green. * [Fix] CI: Enable VCR replay for test_azure_o_series The Azure o-series tests were excluded from the conftest's VCR auto-marker because of a respx/vcrpy transport-patching conflict, but the only respx reference in the file was an unused `MockRouter` import. Drop the dead import and remove the file from the conflict set so cassettes record on first run and replay thereafter, eliminating the 60-95s live Azure latency that was crashing xdist workers under --timeout=120 thread-mode timeouts. * [Fix] Tests: Restore /metrics access for prometheus test suite /metrics now requires auth by default; tests/otel_tests/test_prometheus.py makes 4+ unauthenticated GETs against http://0.0.0.0:4000/metrics, so every prometheus test in CI now fails the metric assertion. Set require_auth_for_metrics_endpoint: false in otel_test_config.yaml to opt out for this test job, which scrapes /metrics directly. Verified locally: 8/8 prometheus tests green (one flaky retry on test_proxy_success_metrics that pre-dates this PR). Also drop the -x stop-on-first-failure flag from the otel test command so all failures in the job surface in a single CI run rather than hiding behind whichever one trips first. * [Perf] CI: Skip Redundant Playwright Apt Install in E2E UI Job The cimg/python:3.12-browsers base image already ships every Chromium system dependency Playwright needs (libnss3, libatk-bridge2.0-0, libcups2, etc. — the install log shows them all as "already the newest version"). Passing --with-deps to `npx playwright install` therefore runs an apt-get update + install for nothing, but pays the full cost of hitting Ubuntu mirrors. On a recent run those mirrors stalled hard: apt-get update alone took 6m53s at 81.5 kB/s with several archives returning connection refused. Drop --with-deps and persist ~/.cache/ms-playwright alongside node_modules so the Chromium binary is also reused across runs. Bump the cache key to v2 so the existing v1 entry (which only contained node_modules) is not loaded and skipped over the new browser path. * [Fix] Docker: Remove Hardcoded Prisma Binary Target For Multi-Arch Builds PRISMA_CLI_BINARY_TARGETS="debian-openssl-3.0.x" was hardcoded in docker/Dockerfile.non_root by #17695. On a buildx linux/arm64 leg this forces prisma to download the amd64 schema-engine into an arm64 image, so 'prisma migrate deploy' fails at startup with 'Could not find schema-engine binary'. Removing the env lets prisma auto-detect per build platform: amd64 builds still resolve to debian-openssl-3.0.x (Wolfi falls back to debian, same binary as before), and arm64 builds now correctly fetch linux-arm64-openssl-3.0.x. The offline-cache pre-warm goal of #17695 is preserved — only which binaries fill the cache changes. Fixes #19458 * [Fix] UI: Clear Admin Session Cookies Before Establishing Invited User's Session (#27227) The invite-signup form was writing the new user's token via raw `document.cookie` at `path=/`, while the rest of the auth surface uses `storeLoginToken` (which writes at `path=/ui` and mirrors to sessionStorage). After signup the inviter's `path=/ui` cookie kept winning path-specificity matching, and sessionStorage still held the inviter's token, so the dashboard rendered as the inviter rather than the newly created user. Treat invite signup as a principal-change boundary — clear prior session cookies first, then store the new token via the canonical helper. * test: add 24hr Redis-backed VCR cache to additional test suites (#27159) * test: add 24hr Redis-backed VCR cache to additional test suites Extracts the existing llm_translation VCR plumbing into a reusable helper (tests/_vcr_conftest_common.py) and wires it into the conftest.py files of the test directories listed in LIT-2787: audio_tests, batches_tests, guardrails_tests, image_gen_tests, litellm_utils_tests, local_testing, logging_callback_tests, pass_through_unit_tests, router_unit_tests, unified_google_tests The same helper is also adopted by the pre-existing llm_translation and llm_responses_api_testing conftests to remove the copy-pasted VCR setup. Each consuming conftest: - registers the Redis persister via pytest_recording_configure - auto-marks collected tests with pytest.mark.vcr (skipping respx-using files where applicable, since respx and vcrpy both patch httpx) - gates cassette writes on test success via _vcr_outcome_gate The cache is opt-in via CASSETTE_REDIS_URL; when unset, VCR is disabled and tests hit live providers as before. LITELLM_VCR_DISABLE=1 still forces a bypass for ad-hoc local runs. Test directories that run LiteLLM proxy in Docker (build_and_test, proxy_logging_guardrails_model_info_tests, proxy_store_model_in_db_tests) are intentionally not included: VCR.py patches the in-process httpx transport and cannot intercept calls made from inside a Docker container. The installing_litellm_on_python* jobs make no LLM calls and don't benefit from caching. https://linear.app/litellm-ai/issue/LIT-2787/add-24hr-caching-to-additional-test-suites * test(vcr): add safe-body matcher to handle JSONL and binary request bodies vcrpy's stock body matcher inspects Content-Type and unconditionally runs json.loads on application/json bodies. JSON Lines payloads (used by the Bedrock batch S3 PUT and other upload paths) crash that with json.JSONDecodeError: Extra data, before the matcher can return 'not a match'. This was the root cause of the batches_testing CI job failing on test_async_create_file once VCR auto-marking was applied to the batches_tests directory. Add a conservative byte-equality body matcher and use it in place of 'body' in the shared match_on tuple. The matcher is strictly more conservative than vcrpy's default — the only thing it gives up is 'different JSON key order is treated as the same body', which doesn't apply to deterministic litellm-built request payloads. It can never produce a false positive that the default would have rejected, so there is no cross-contamination risk. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): exclude tests that VCR replay actively breaks A few tests are incompatible with cassette replay and were failing on the latest CI run after VCR auto-marking was extended to local_testing and logging_callback_tests: - test_amazing_s3_logs.py (logging_callback_tests): the test asserts on a per-run response_id that should round-trip through a real S3 PUT/LIST. vcrpy's boto3 stub intercepts the PUT and the LIST replays stale keys, so the freshly-generated id is never found. - test_async_embedding_azure (logging_callback_tests) and test_amazing_sync_embedding (local_testing): the failure branches deliberately pass api_key='my-bad-key' to assert that the failure callback fires. We scrub auth headers from cassettes (so the bad-key request matches the prior good-key request), and vcrpy replays the recorded 200 — the failure callback never fires. - test_assistants.py (local_testing): the OpenAI Assistants polling APIs mint fresh thread/run IDs every recording session and then poll until status=='completed'. Replays of those polled GETs can never match a freshly-generated run id, so every CI run effectively re-records and the suite blows past the 15m no_output_timeout. Skip these from VCR auto-marking so they continue to hit live providers as they did before this change. The remaining tests in each directory still get cached. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): expand skip lists for second batch of incompatible tests Followup to the previous commit. After re-running CI on the rebuilt branch, three more tests surfaced as VCR-replay-incompatible: - litellm_utils_testing :: test_get_valid_models_from_dynamic_api_key Calls GET /v1/models with api_key='123' to assert the result is empty. We scrub auth headers, so the bad-key request matches the prior good-key cassette and replays the recorded model list. - litellm_utils_testing :: test_litellm_overhead.py Measures litellm_overhead_time_ms as a percentage of total wall-clock time. With cached responses the upstream 'network' time collapses to microseconds, blowing past the 40%% threshold the test asserts on. Skip the whole file (every parametrization is at risk). - local_testing_part1 :: test_async_custom_handler_completion and test_async_custom_handler_embedding Same bad-key failure-callback pattern as the already-skipped test_amazing_sync_embedding. - litellm_router_testing :: test_router_caching.py Asserts on litellm's own router-level response cache by comparing response1.id to response2.id across repeat upstream calls (test bypasses litellm cache via ttl=0 and expects upstream to return a *new* id). With VCR replay both upstream calls return the same cassette body, so the ids are identical. Skip the whole file. - logging_callback_tests :: test_async_chat_azure (preemptive) Same shape as already-skipped test_async_embedding_azure; was masked by upstream OpenAI rate-limit failures on baseline. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): use item.path and tighten matcher docstring - Replace pytest's deprecated item.fspath with item.path in apply_vcr_auto_marker_to_items so we don't emit deprecation warnings under pytest 8. - Clarify _safe_body_matcher docstring to reflect actual behavior (direct == first, then UTF-8 bytes comparison, no repr fallback). Addresses Greptile review feedback on PR #27159. * test(vcr): swallow all RedisError on cassette save/load Cassette persistence is strictly best-effort: any Redis-side failure (connection blip, timeout, OutOfMemoryError when the maxmemory cap is hit, READONLY replicas, etc.) should degrade to 'test passed but cassette not cached' rather than fail the test on teardown. Previously the persister only caught ConnectionError and TimeoutError, so OutOfMemoryError — which Redis Cloud raises when the cassette cache hits its memory cap and there are no evictable keys — propagated out of vcrpy's autouse fixture and ERRORed otherwise-passing tests on teardown. This caused the litellm_utils_testing CircleCI job to fail on the latest commit's run, even though the underlying test was a unit test that used mock_response and produced no real upstream traffic (the cassette was dirtied by a background langfuse callback). The rerun only succeeded because Redis evictions happened to free enough room before the SET — i.e. it was timing-dependent flakiness. Catch redis.exceptions.RedisError (the common base of all server- and client-side Redis exceptions) on both save and load, and parametrize the regression tests across ConnectionError, TimeoutError, and OutOfMemoryError to pin the new behavior. * test(vcr): surface cassette-cache failures with warnings + session banner When the persister silently swallows a Redis OOM (or any RedisError) on save/load there is otherwise no visible signal that the cache is degraded — tests pass, the cassette just isn't persisted, and the next session still hits the same Redis at the same near-cap memory. Add three layers of observability so that failure mode is loud: 1. Per-process health counters ("save_failures", "load_failures", and the last error string for each), exposed via cassette_cache_health() and reset via reset_cassette_cache_health(). The persister increments these in addition to logging. 2. VCRCassetteCacheWarning (UserWarning subclass) emitted via warnings.warn() inside the persister's except block. Pytest's built-in warnings summary at session end automatically lists every such warning, so the failure is visible in CI logs without any conftest-level wiring. 3. Session-end banner via emit_cassette_cache_session_banner() and a stderr-fallback atexit handler registered from register_persister_if_enabled(). Two states: - red "VCR CASSETTE CACHE DEGRADED" when save_failures or load_failures > 0 - yellow "VCR CASSETTE CACHE NEAR CAPACITY" (no failures, but used_memory >= 85% of maxmemory) so the next session knows the Redis is approaching OOM before any SET actually fails Capacity comes from a best-effort INFO memory probe (cassette_cache_capacity_snapshot) that returns None on any failure or when maxmemory is uncapped. The atexit handler skips xdist workers so only the controller emits. Tests: parametrize the existing save/load swallow-error tests across ConnectionError/TimeoutError/OutOfMemoryError, add direct tests for the health counters and warning emission, and a new test_vcr_conftest_common_banner.py covering banner output for every state (silent/red/yellow/disabled/xdist-worker). * test(vcr): bucket cassettes by API key fingerprint, drop bad-key skips Tests that deliberately call an LLM API with a bad key (e.g. to assert that the failure callback fires, or that check_valid_key returns False) were being silently served the prior good-key cassette: we scrub the real Authorization / x-api-key header from the cassette before storing it, so a follow-up bad-key call is byte-identical to the good-key call under the existing match_on tuple. Add a 'key_fingerprint' custom matcher that distinguishes requests by the SHA-256 of their API-key headers. The fingerprint is stamped into a synthetic 'x-litellm-key-fp' header by a new before_record_request hook, which then strips the real auth headers (we have to do the scrubbing here instead of via vcrpy's filter_headers knob, because filter_headers runs *first* and would erase the value we want to hash). Bad-key requests now get a different cassette bucket than good-key requests, so vcrpy will not replay a recorded 200 in place of the expected 401. The fingerprint is a one-way hash of the secret, so cassettes never contain the key. This permanently removes the 'bad-key' category of skips: - tests/local_testing: dropped ::test_amazing_sync_embedding, ::test_async_custom_handler_completion, ::test_async_custom_handler_embedding - tests/logging_callback_tests: dropped ::test_async_chat_azure, ::test_async_embedding_azure - tests/litellm_utils_tests: dropped ::test_get_valid_models_from_dynamic_api_key Coverage: 7 new unit tests in tests/test_litellm/test_vcr_safe_body_matcher.py covering header stripping, fingerprint determinism, no-auth bucketing, good-vs-bad key discrimination, x-api-key (Anthropic/Azure) discrimination, and idempotence under replay. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): drop redundant comments and docstrings Trim narration of code that is already self-evident from function and variable names. Keep the two genuinely non-obvious bits: - ordering constraint between filter_headers and before_record_request, which would invite a maintainer to re-introduce the bug if removed - the per-directory _VCR_INCOMPATIBLE_FILES rationale, since 'why exactly is this skipped' is not knowable from the test name alone Also drop the 40-line commented-out drop-in conftest snippet at the bottom of _vcr_conftest_common.py — the consuming conftests are the canonical reference. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): make _before_record_request idempotent vcrpy invokes before_record_request more than once per request: can_play_response_for calls it, then __contains__ / _responses (reached via play_response) call it again on the result. The second invocation sees a request whose auth headers we already stripped, so a naive recompute yields "no-key" and overwrites the real fingerprint stored in the header. This makes can_play_response_for and play_response disagree on matchability — the former says "yes, we have a stored response for this" (matching no-key to no-key) and the latter throws UnhandledHTTPRequestError because it computes a fresh real fingerprint that doesn't match the stored no-key. In CI this manifested as ~30 failing tests across guardrails_testing, audio_testing, batches_testing, image_gen_testing, llm_responses_api, litellm_router_unit_testing, etc. Skip the recompute when the header is already set, so re-applying the hook is a no-op. Adds a regression test that fires the hook twice on the same dict and asserts the fingerprint stays put. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): drop more redundant docstrings and headers * test(vcr): enable 24hr cache for ocr_tests and search_tests These two directories were the only non-dockerized test suites in the build_and_test workflow that make live LLM/provider API calls but were not VCR-enabled by this PR. Together they account for 96 tests: - tests/ocr_tests/ (31): Mistral OCR, Azure AI OCR, Azure Document Intelligence, Vertex AI OCR. Pure-unit tests inside the same files (e.g. TestAzureDocumentIntelligencePagesParam) make no HTTP calls and become benign VCR NOOPs. - tests/search_tests/ (65): Brave, DataForSEO, DuckDuckGo, Exa, Firecrawl, Google PSE, Linkup, Parallel.ai, Perplexity, SearchAPI, Searxng, Serper, Tavily. Both directories use the canonical minimal conftest pattern from tests/audio_tests/conftest.py with no skip lists. None of the test files use respx, none assert on per-call upstream non-determinism (no response1.id != response2.id, no overhead-as-fraction-of-total, no live polling), so the default match_on tuple should cache cleanly. If a flake surfaces during the first cassette-recording CI run, we can add a targeted skip the same way we did for the other dirs. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * [Fix] Team UI: handle legacy dict shape for metadata.guardrails (#27224) * [Fix] Team UI: handle legacy dict shape for metadata.guardrails A team can have metadata.guardrails stored as {"modify_guardrails": bool} (the permission-flag shape introduced in PR #4810) rather than the expected string[]. The opt-out logic added in PR #25575 calls .filter() on this field, which throws TypeError on a dict and crashes the team detail page. Add a safeGuardrailsList helper that returns [] when the field is not an array, and route the three read sites through it. * [Fix] Team UI: inline Array.isArray guards for guardrails metadata Replace the safeGuardrailsList helper with inline Array.isArray checks at each call site, and apply the same guard to opted_out_global_guardrails for consistency. No known legacy dict rows for opted_out_global_guardrails, but the unguarded `|| []` pattern is the same shape risk. Six call sites now defended directly: three for metadata.guardrails and three for metadata.opted_out_global_guardrails. * chore: update Next.js build artifacts (2026-05-05 22:45 UTC, node v20.20.2) (#27240) * [Infra] Bump deps (#27157) * bump: version 0.4.70 → 0.4.71 * bump: version 0.1.39 → 0.1.40 * uv lock --------- Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: user <70670632+stuxf@users.noreply.github.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: shivam <shivam@berri.ai> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> Co-authored-by: Michael-RZ-Berri <michael@berri.ai> Co-authored-by: harish-berri <harish@berri.ai> Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu> Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local> Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>	2026-05-05 16:15:03 -07:00
user	4af58e1f97	[Security] Clear AWS Inspector CVE findings on Docker image ... - Narrow /root/.cache COPY in Dockerfile to /root/.cache/prisma{,-python} only — drops ~660MB of uv build cache including a setuptools wheel that surfaced as CVE-2024-6345 / CVE-2025-47273 even though it was never on the runtime sys.path. - DiskCache: switch to dc.JSONDisk to neutralize the pickle code path (CVE-2025-69872, no upstream fix). Values must be JSON-serializable; cleanup get_cache to skip the now-dead json.loads(dict) branch by guarding on isinstance(str). - pyproject.toml: drop diskcache pin from [caching] extra (no fixed version exists). Stub kept so `pip install litellm[caching]` doesn't warn; users who want disk caching install diskcache themselves. - Bump black 24.10.0 → 26.3.1 (CVE-2026-32274) + apply 296-file mechanical reformat. Black is dev-only (not in the runtime image), but bumping clears the manifest-scan finding. - Refresh ui/litellm-dashboard/package-lock.json to pick up next 16.2.4 (was 16.1.7, GHSA-q4gf-8mx6-v5v3), uuid 14.0.0, postcss 8.5.13. - Refresh litellm-js/spend-logs/package-lock.json to pick up hono 4.12.16 (GHSA-458j-xx4x-4375). - uv lock: gitpython 3.1.46 → 3.1.49 (clears two High GHSAs), langchain-text-splitters 1.1.1 → 1.1.2. - Add tests/test_litellm/caching/test_disk_cache.py covering JSONDisk enforcement, dict/string round-trip, TTL, increment, delete/flush. Net delta on combined trivy + grype scans: 17 findings → 4 (all remaining 4 are Wolfi system python-3.13 CVEs marked WONTFIX upstream in CPython 3.14; CVE-2026-3298 is Windows-unreachable on Linux). Existing on-disk caches written by the previous pickle-format Disk will silently miss after upgrade — diskcache is intended to be ephemeral so impact is recreate-on-next-write. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-05 20:21:15 +00:00
user	1dcfc36393	chore(deps): align dashboard node engine	2026-05-04 13:21:03 -07:00
user	bfdd786962	chore(deps): refresh dependency locks	2026-05-04 11:36:18 -07:00
user	8d1493ed08	fix(security): bump vulnerable dependencies ... pip: - cryptography 43.0.3 → 46.0.7 (5 CVEs including CVSS 8.2 ECDH key leak) npm: - hono 4.1.4/4.12.7 → 4.12.12 (prototype pollution, cookie injection, path traversal, middleware bypass, IP matching bypass) - @hono/node-server 1.19.6 → 1.19.13 (serveStatic middleware bypass) - vite 7.3.1 → 7.3.2 (file read via WebSocket, path traversal, fs.deny bypass) - lodash override 4.17.23 → 4.18.1 (code injection via _.template, prototype pollution via _.unset/_.omit) mlflow left at 3.9.0 — 2 of 3 alerts have no upstream fix, and 3.11.1 is blocked by exclude-newer (transitive dep chain). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-09 19:35:19 +00:00
Yuneng Jiang	006d481025	[Fix] Remove neon CLI dependency and pin all JS dependencies ... Remove @neondatabase/api-client and neonctl to address CVE-2026-25639 (axios supply chain vulnerability). Pin all JS dependencies to exact versions across all package.json files to prevent future supply chain attacks via semver range resolution. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-01 16:15:32 -07:00
yuneng-jiang	62835ff03d	adding package-lock	2026-03-17 17:44:01 -07:00