* fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425)
* fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata
Batch create was failing with `Invalid type for 'metadata.applied_policies':
expected a string, but got an array instead` whenever a policy attachment
matched the request. The policy engine helpers wrote `applied_policies`,
`applied_guardrails`, and `policy_sources` into `data["metadata"]`
unconditionally, and `/v1/batches` forwarded that dict straight to OpenAI,
which only accepts string values.
- Route proxy-internal tracking into `litellm_metadata` for batch/file
routes via a shared `_get_or_create_proxy_metadata_bucket` helper.
- Sanitize `data["metadata"]` in `create_batch` to drop known internal
keys and non-string values before building the OpenAI request.
- Cover both behaviors with unit + endpoint tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(proxy): merge metadata buckets for batch policy response headers
Ensure get_logging_caching_headers reads both metadata and litellm_metadata so policy/guardrail headers are emitted on batch routes with user metadata, and log dropped non-string OpenAI metadata at debug level.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(model-edit): allow clearing custom pricing on wildcard models (#28719)
* fix(model-edit): allow clearing custom input/output cost on wildcard deployments
A user-set pricing override on a `/model/*` wildcard deployment could not
be removed: clearing the Input/Output Cost fields in the UI succeeded
visually, but the next read still showed the old values because both
`litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`)
retained the original rates.
UI: when the pricing field is touched but left empty, send `null` instead
of dropping it from the payload so the backend sees the clear intent. The
cache-read-cost fallback now guards against `null` as well as `undefined`
so a cleared input cost cannot silently wipe the cache-read override.
Backend: `update_db_model` honors explicit-null clears, but ONLY for
`SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the
null-clear path prevents a team-scoped caller from using this codepath to
null out privileged fields like `team_id` or access groups.
Tests cover both clear paths (`litellm_params` and `model_info`), the
SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and
the security guard that non-pricing nulls don't reach the merged dict.
Resolves LIT-3250
* fix(model-edit): run null-clears after both merges, not interleaved
The previous version cleared `model_info` from inside the litellm_params
merge block, but the subsequent `model_info.update(...)` re-injected the
old pricing because the UI's PATCH carries the full model_info blob with
the stale values still in it. Move the explicit-null clear pass to after
both merges so a model_info passthrough cannot resurrect cleared fields.
Adds a regression test for the realistic UI submit shape (both blobs in
the patch, model_info still holding the old pricing).
* test(e2e): clear-custom-pricing flow with create/delete cleanup
Covers the dashboard model edit form's pricing-clear flow end-to-end:
seeds a deployment with custom input/output pricing, drives the UI to
clear both fields, asserts the outgoing PATCH sends explicit nulls,
and confirms via /v2/model/info that the override is gone from both
litellm_params and model_info.
The dashboard DB persists across this suite, so beforeEach creates a
uniquely-named deployment and afterEach POSTs /model/delete to leave
the DB clean regardless of test outcome.
* fix(model-edit): extend pricing clear to cache_read and cache_write costs
Pre-existing parallel of the wildcard input/output cost bug: cleared
cache_read_input_token_cost and cache_creation_input_token_cost overrides
silently persisted because the UI omitted the key (delete or fallback) and
the backend null-clear allowlist did not cover them.
- types/router.py: add cache_read_input_token_cost and
cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are
mirrored between litellm_params and model_info by Deployment.__init__ and
honoured by the null-clear loop in update_db_model.
- model_info_view.tsx: emit explicit null for touched-but-empty cache_read
and cache_write fields. Preserve the input_cost->cache_read mirror only
when cache_read itself was not touched.
- model_management_endpoints.py: update the allowlist comment.
- Tests: three new unit tests for cache clear paths and a preserve check;
the e2e spec now seeds, clears, and asserts null PATCH + key-absence for
all four pricing fields.
---------
Co-authored-by: Shivam Rawat <shivam@berri.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(e2e): forward LITELLM_LICENSE to UI e2e proxy
The UI e2e job ran without LITELLM_LICENSE, so premium_user was always
false in the issued login JWT and premium-gated UI surfaces (Team-BYOK
Model switch, etc.) couldn't be driven through the UI. Forward the env
var from run_e2e.sh and the CircleCI e2e_ui_testing job, and add a
sanity test that decodes the admin storage state token and asserts
premium_user=true so the wiring fails loudly if it ever regresses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Update ui/litellm-dashboard/e2e_tests/tests/proxy-admin/license.spec.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* test(ui-e2e): add admin key creation with a specific proxy model
Adds Playwright coverage for creating a key (no team) scoped to a single
proxy model, complementing the existing All-Proxy-Models test. Uses a
DOM-dispatched click on the antd dropdown option since the popup
animation can render the option outside the viewport.
* test(ui-e2e): verify scoped key works against mock /chat/completions
Extend the "Create a key with a specific proxy model" test to extract
the new key from the success modal and POST to /chat/completions for
the scoped model, asserting 200 and the mock response body. Without
this the test could pass even if the model selection failed to register.
* chore(e2e): migrate runner to uv, add All Proxy Models key test
Switches the local e2e runner (run_e2e.sh) from poetry to uv to match
the rest of the repo and CI. Adds a Playwright test for creating an
admin key with no team selected (all-proxy-models flow), a SLOWMO env
hook for headed debugging, and a MIGRATION_TRACKING.md doc that maps
the manual UI QA checklist to e2e tests so future migration work has
a single source of truth.
* chore(e2e): address greptile feedback
- Remove MIGRATION_TRACKING.md (docs belong in litellm-docs repo)
- playwright.config.ts: fall back to 0 when SLOWMO is non-numeric
(parseInt returns NaN, which Playwright accepts silently)
- run_e2e.sh: add --frozen to uv sync for CI determinism
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
Reviewer flagged that cleanup failures were silently swallowed and
suggested asserting `delete.ok()`. While thinking through the fix, the
actual question turned out to be "does the cleanup matter at all?" —
and the answer is no.
The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per
invocation and tears it down at the end, so every local and CI run
starts with an empty DB. Playwright retries share the same DB but each
attempt creates a new model with a unique `Date.now()` name and only
queries its own model, so orphans from failed attempts never collide
with later attempts or other tests. Nothing else in the suite reads
the all-models table.
Keeping the cleanup would also turn every write test into an implicit
delete test, coupling responsibilities and inflating runtime — which
is probably why `teams.spec.ts` (create a team), `keys.spec.ts`
(update key limits), etc. all leave their entities in place. Matching
that convention, drop the try/finally block and the `createdModelId`
tracking. 12 lines removed, no behavior change.
Covers the full write-path flow for team-scoped models on the Models +
Endpoints page: create via /model/new, click the row to open the detail
view, click Edit Settings, change TPM/RPM, click Save Changes, assert
the new values render back. Cleans up via /model/delete in finally so
reruns stay deterministic.
Requires store_model_in_db: true in the fixture general_settings so the
proxy accepts /model/new and /model/delete. No existing test in the
dashboard e2e suite reads the all-models table or hits the model CRUD
endpoints, so enabling the flag has no cross-test impact.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
On case-sensitive Linux CI, the old regenerate_key_modal.tsx from main
can coexist with the new RegenerateKeyModal.tsx after merge. The old
modal renders "Copy Virtual Key" while the new one renders "Copy Key".
Use /Copy.*Key/ to match both.
- Label the key block with a small "Virtual Key" caption so the gray
box is clearly the key container.
- Move the Copy Key action to the modal footer as a primary button
with icon; inline copy icon next to the key is removed.
- Swap the button to "Copied" with a check icon on success instead of
firing a notification — less noisy and keeps feedback in place.
- Disable clicking outside the modal to close (maskClosable=false) so
users must explicitly dismiss via Close or X.
- Enlarge the key text and let its container span the full modal
width.
- Tests updated accordingly, including a new test for the copied-state
swap and the "Virtual Key" label.
Use Flex, Typography.Paragraph (with copyable), and Typography.Text
instead of raw divs + code block + CopyToClipboard wrapper. Drops the
direct react-copy-to-clipboard dependency in this component in favor
of antd's native copyable support.
Also fixes two test issues surfaced when running the e2e locally:
- RegenerateKeyModal.test.tsx no longer mocks react-copy-to-clipboard
(the component no longer imports it), removing the CJS require()
inside an ESM mock factory flagged by Greptile.
- keys.spec.ts scopes the Regenerate and Copy lookups to the modal.
The Regenerate button has an icon whose aria-label ("sync") is
concatenated into the button's accessible name, so an exact-match
lookup on "Regenerate" failed; and the new Paragraph copyable
renders a generic "Copy" button that collided with the other
copyable fields on the key info view.
Replace Tremor components in the regenerate key modal with Ant Design
equivalents and move the component to a new PascalCase file. The form
layout now uses Row/Col to place Max Budget, TPM Limit, and RPM Limit
on one row and Expire Key with Grace Period on another, reducing the
vertical footprint. The success view shows an Alert banner, the key
alias as secondary context, and the regenerated key in a monospace
block with an inline primary Copy button.
Also adds unit tests for the new component and updates the existing
Playwright spec to match the new banner and button text.
- Add constants.ts with all required exports (key aliases, team IDs)
- Add fixtures/users.ts with all role definitions and storage paths
- Add fixtures/seed.sql for deterministic test database seeding
- Remove Firefox project from playwright config (only Chromium installed)
- Remove unused variable in teams.spec.ts
- Rename CircleCI job to e2e_ui_testing
Add Playwright E2E tests covering proxy admin team and key management
workflows, with a self-contained test runner and CircleCI integration.
Tests cover: create team, invite user, edit/delete team members, create
key in team, regenerate key, update TPM/RPM limits, delete key, and
verify internal user keys are visible.
Infrastructure: run_e2e.sh builds the UI from source before starting
the proxy, ensuring tests always run against the latest UI changes.
Added data-testid attributes to key UI components for reliable selectors.
- Change no_output_timeout from 120m/60m to 15m across all test jobs
- Fix multi-line YAML formatting for no_output_timeout entries
- Reduce Playwright per-test timeout from 4min to 3min
- Add 15s actionTimeout and 30s navigationTimeout to Playwright config
Hanging tests now fail in 15min instead of 2hrs, cutting wasted CI time.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The createKey test broke because the Models field changed from required
to optional (PR #22826), removing the asterisk from the accessible name.
The unauthenticated redirect test broke because the login URL now
includes a redirect_to query param. Both selectors now use regex to be
resilient to these kinds of changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
