The UI vitest suite is CPU-bound; move it to a 16-core larger runner and raise vitest fork concurrency from 4 to 14 (leaving headroom for the coordinator, jsdom, and the OS) so the full suite and PR-scoped runs finish faster.
The SSO and Email Server settings pages read only stored config, so a gateway
configured entirely through environment variables rendered every field blank
even though both features were live. Rather than add per-endpoint env fallback,
resolve each setting through one typed config object.
A FieldDescriptor names, for one setting, where it lives in the stored row
(db_key), which process env var carries it (env_var), whether it is a secret,
and its effective default. A pure resolve_fields reconciles a descriptor table
against the stored row and the process environment with a fixed precedence and
reports per-field provenance (db, env, default, or unset). The SSO descriptor
table single-sources the field-to-env mapping that the read and write paths
previously duplicated, so they can no longer drift.
get_sso_settings and the /get/config/callbacks alerting block read through the
resolver instead of their own inline fallbacks. get_sso_settings no longer
decrypts stored values into os.environ; decryption happens once inside the
resolver via the pure helper, so a GET stops mutating the process environment.
The SSO response carries provenance so the UI can distinguish an env-sourced
value from a stored one, and secrets are masked at the endpoint (the resolver
returns them unmasked so the login path could consume them). os.environ remains
the runtime carrier; the SSO login and mail-send paths are unchanged.
The settings pages also submit only fields an admin actually edited, so a
rendered mask or env-sourced value is never written back over a working
secret, and generic_scope is a real SSO form field. Omitting a field from
/update/sso_settings clears it, which provider switching relies on; the deeper
write-path concern that behaviour points at is tracked in LIT-4498.
Relocates ui/litellm-dashboard/e2e_tests to tests/e2e/ui so all end to end
suites live under tests/e2e. The suite stays in TypeScript and becomes a
self-contained npm package with its own package.json, lockfile and tsconfig
instead of leaning on the dashboard's toolchain; the dashboard drops its
@playwright/test dependency, e2e scripts and knip/vitest/tsconfig carve-outs.
CI paths follow the move: both CircleCI jobs (main e2e and the
SERVER_ROOT_PATH migration smoke) and the test_server_root_path workflow now
install and run Playwright from tests/e2e/ui, with the node cache keyed on
both lockfiles. classify_changes.sh treats tests/e2e/ui as client so spec
edits keep skipping backend jobs. The suite's mock LLM fixture is excluded
from the e2e basedpyright zero-error gate in pyrightconfig.json since it
belongs to the TS suite, not the typed Python harness.
Adds get_external to e2e_http.py for absolute third-party GETs (no proxy base url or auth, same Result classification) and rewires fetch_agent_card through it, dropping the urllib.request escape hatch. Creates tests/code_coverage_tests/check_e2e_no_raw_requests.py, the checker tests/e2e/CLAUDE.md already referenced, and wires it into the code-quality workflow so raw HTTP client imports outside the transport fail CI; pre-existing uses (root conftest liveness probe, claude_code version resolver) are grandfathered and exception-type-only imports stay allowed.
* test(ui): run vitest unit tests in GitHub Actions and fix stale key-info tests
The dashboard's vitest suite only ran on CircleCI; GitHub Actions covered the
UI build, lint and api-types sync but never the unit tests. Add a UI Unit Tests
workflow that runs the suite, sharded across a matrix so the wall-clock is not
bound by a single 4-core runner.
Porting it surfaced 17 pre-existing failures. Adding the block/unblock key
action moved Delete Key and Reset Spend into a "More key actions" dropdown and
introduced a React Query hook; KeyInfoHeader's own test was updated but the two
KeyInfoView test files were not. Reach those actions through the dropdown and
stub the new hook the way the neighbouring hook is already stubbed.
The same refactor had quietly hollowed out assertions that still passed:
"should not show Reset Spend button for regular key owner" queried for a button
role that no longer exists, so it held green regardless of the permission
check. Those now open the menu and assert on the menu item, which fails when
canResetSpend is forced true.
Also add the missing cost-optimization page description; page_utils guards that
every navigable page carries one.
* ci(ui): scope PR runs to changed tests, run the full suite on staging
Running the whole vitest suite on every pull request costs about five minutes,
and none of it is recoverable through parallelism: vitest schedules by file and
create_mcp_server.test.tsx alone accounts for 252s of the 255s total, so shards
and extra cores cannot get under that floor. Measured on this branch, css:false,
pool=threads and isolate=false all landed within noise of the baseline.
Scope pull requests to tests reachable from the diff instead, which takes 11s
here, and keep a full run on pushes to litellm_internal_staging so nothing rots
behind a gap in the module graph. Backend-only pull requests match no test files
and exit zero; --passWithNoTests states that rather than leaning on it being the
current default. The checkout needs full history for --changed to resolve the
base commit.
grype defaults match.python.using-cpes to false, so PyPI packages are
matched only against the GitHub Advisory Database. When a CVE is
published to NVD but its GHSA has not propagated to the global advisory
database, the scan reports clean even though grype's own database
already carries the NVD record with the correct version ranges.
The pypdf CVEs (CVE-2026-59935 / 59936 / 59937 / 59938, analyzed in NVD
since 2026-07-08) are the case that exposed this; their GHSA IDs are
still repo-level and return 404 from the global advisory API, so the
ecosystem matcher has nothing to match on.
Enabling CPE matching for Python closes that gap. Measured against a
v1.91.1 build the finding count goes from 28 to 38; the additions are
mostly actionable, and the few cross-product CPE collisions cannot fail
the build because --only-fixed drops the ones carrying no fix version
and the remainder land below the --fail-on high threshold.
Removes the scheduled workflow that cut litellm_oss_daily_YYYY_MM_DD
branches and the guardrails workflow that only ran on them. The secret
scan and ruff checks that workflow duplicated already run on PRs to
litellm_internal_staging via test-linting.yml, so no coverage is lost.
Retargets contributor-facing messaging in CONTRIBUTING.md, CLAUDE.md,
and the guard-main-branch error output at litellm_internal_staging.
The eslint-metrics.json snapshot duplicated the violation counts already
enforced by eslint-budgets.json. Keeping it current added a CI drift check,
a pre-commit regenerate-and-flag step, and a standalone npm run lint:metrics
script, none of which caught anything the budget gate did not, yet all of
which failed noisily whenever the snapshot went stale. This drops the file
and that machinery while leaving eslint-budgets.json as the actual ratchet
gate
The guard-main-branch error messages and the contributor docs still
pointed people at litellm_oss_staging. Redirect them to the current
daily OSS branch (litellm_oss_daily_YYYY_MM_DD), a fresh one of which
is cut each weekday, so contributors should target the most recent
The lint job lived in test-litellm-ui-build.yml (workflow name "UI Build
Check") next to the build job, so its check surfaced as the misleading
"UI Build Check / frontend-lint" even though it does prettier, eslint, lint
budgets, and knip, not building. Split it into test-litellm-ui-lint.yml
(name "UI Lint") so the check reads "UI Lint / frontend-lint". The build
workflow keeps only build-ui; the lint job (including the knip step) moves
over unchanged.
Note for whoever manages branch protection: this renames the lint required-
check context from "UI Build Check / frontend-lint" to
"UI Lint / frontend-lint"; update the required-check entry so PRs don't strand.
knip was producing garbage locally and was never wired into CI, so nobody
trusted it. Two structural problems: it silently degrades when deps are
missing (a partial worktree install flagged all 436 test files as unused),
and its config had blind spots that surfaced as false positives.
Fixes so a knip run means something:
- Register every playwright config (serverRootPath + migration variants), not
just the main one. serverRootPath.config.ts is invoked via --config in
test_server_root_path.yml, which knip can't see; it was falsely flagged as
an unused file
- Treat src/components/ui/** as entry points. These are shadcn design-system
primitives, intentionally part of the palette before every one is consumed;
knip was flagging not-yet-used ones (e.g. select.tsx) as dead files and
their sub-exports as unused. Marking the directory as the design-system
surface is the correct fix, not deleting components someone is about to use
- Declare @ant-design/icons as a direct dependency. It was imported in ~198
files but only resolved via antd hoisting, so every one showed up as an
"unlisted dependency"
- Add an explicit vitest plugin block so test-file classification no longer
rides on auto-detection
- Stage severities via rules: gate the now-clean categories (files,
dependencies, unlisted, unresolved) as errors and keep exports/types/
duplicates as warnings, so CI enforces what's at zero today while the
remaining findings ratchet down in follow-ups
- Run npm run knip in the frontend-lint CI job, which installs with npm ci so
it never sees a partial tree
knip now exits 0 with the gated categories clean
The in-workflow regex list was hard to maintain and, because it runs on pull_request, could be modified by the same PR it inspects. Path gating for the OSS daily branches now lives in repository branch protection settings, so this workflow keeps only the OSS-safe checks: the hardcoded-secret test and ruff
* ci: skip unit test workflows when only docs or ui files change
Mirror the CircleCI backend path filter (.circleci/scripts/classify_changes.sh)
in the GitHub Actions unit test workflows by adding paths-ignore for ui/**,
docs/**, *.md and *.mdx to every test-unit-*.yml pull_request trigger
* ci: drop docs/** from unit test paths-ignore since the folder no longer exists
* feat(terraform): vendor terraform-provider-litellm as source of truth with endpoint drift CI
* fix(terraform): address review feedback on vendored provider
Replace deprecated io/ioutil with io. Remove the unused org/team CRUD
client methods so the endpoint audit only tracks live call sites
(54 -> 46). Redact request/response logs by parsing the JSON and
recursively masking sensitive fields, which fixes the nested-object
leak in the old credential_values regex, with a regex fallback for
non-JSON payloads; covered by new unit tests. Docs: stop showing
api_key inside vector store litellm_params and document that Sensitive
attributes still persist in plaintext state, recommending
litellm_credential_name and an encrypted state backend.
* fix(terraform): stop persisting server-returned litellm_params into vector store state
The vector store Read wrote litellm_params straight back from the API
response into state. The proxy redacts secrets in those responses, so
the readback overwrote user config with redaction sentinels and caused
perpetual diffs, and against a server that returns raw values it would
persist secrets into a non-Sensitive attribute. Read now preserves the
config value like the credential and model resources do, litellm_params
is marked Sensitive, and a regression test pins that a server-returned
api_key never lands in state
* fix(terraform): send role on team member update and stop persisting server env into MCP state
The team member update payload omitted role, and the proxy leaves role
unchanged when the field is absent, so a role downgrade reported as
applied by Terraform never took effect on the proxy. The update now
always sends the configured role (the attribute is Required).
The MCP server resource wrote env straight back from API responses
into a non-Sensitive attribute, pulling admin-visible secrets into
state and, for sanitized responses, blanking user config. Read now
preserves the config value, env is marked Sensitive, and the docs warn
against passing secrets via args. Regression tests cover both fixes
and fail against the previous behavior.
The componentized chart at helm/litellm had no way to mount extra
volumes into its deployments, so custom callback or SSO handler code
could not be mounted the way the docs describe for the monolithic
chart. Adds per-component volumes and volumeMounts values for gateway,
backend, and ui, merged with the existing gateway-config volume, plus
a helm-unittest suite for the chart wired into the helm unit test
workflow
Resolves LIT-4209
* chore: clarify the linear ticket instruction in pr template
* fix: make it more concise
* Update CLAUDE.md
lol
* chore: tell claude not to search for it if it doesn't have it
---------
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* fix(release): create tag before release and set make_latest post-publish
The Create Release workflow failed for every stable maintenance release
while pre-releases succeeded. Two independent bugs were behind that.
createRelease was minting the tag from target_commitish, and that path
returns "Resource not accessible by integration" (403) to the Actions
token, or 404 to a user token, for certain commits (cli/cli#9773). The
stable-line tips tripped it; the dev/rc commits happened not to. Create
the tag up front with git.createRef and drop target_commitish so the
release attaches to the existing tag instead of minting one. A 422 from
createRef (tag already exists) is tolerated so re-runs are idempotent.
make_latest is silently ignored during the draft-to-published
transition (cli/cli#8201), so a backport that published would seize the
repo "latest" badge from a newer line. Publish first, then set
make_latest in a separate call, and only for non-prereleases.
Each operation here is already runtime-proven: git.createRef under the
workflow token by prior release-branch jobs, the no-target createRelease
and non-prerelease publish and separate make_latest PATCH by a manual
1.89.5 cut.
* fix(release): pin tag_name on publish so the draft binding can't reset
Pre-creating the tag means the draft is edited while a tag ref already exists, and a draft PATCH that omits tag_name can reset it to the untagged placeholder. Send tag_name explicitly on both updateRelease calls so publish always attaches to the intended tag.
* fix(release): fail loudly when the tag exists at a different commit
The createRef 422 swallow kept re-runs idempotent but also masked a tag that already exists at the wrong SHA, which would publish the release against the wrong commit silently. On 422, compare the existing tag ref to the intended commit and error on a mismatch, keeping idempotency only for a genuine same-SHA re-run.
* ci(codspeed): pin benchmark runner to ubuntu-24.04
ubuntu-latest resolves to different runner images between the BASE
(main/staging) and HEAD (PR) runs, so CodSpeed reports 'Different
runtime environments detected' and emits false-positive regressions
(e.g. a -25.2% swing on test_completion_multi_turn in #31684, an MCP
auth fix with no LLM code changes). Pinning the runner to a fixed
image keeps BASE and HEAD on the same hardware so 1 ms swings on a
~3 ms benchmark stop blocking unrelated PRs.
Fixes#31738
* ci(codspeed): stop running benchmarks on litellm_internal_staging
The CodSpeed check flip-flops on internal staging and on PRs targeting
it (e.g. "+11.75% improvement" on one run, "-25.36% regression" on the
next) because the comparison flags "different runtime environments" and
the benchmarks are only 3-4 ms, so sub-millisecond runner noise swings
the result by 25-30%. Pinning the runner to ubuntu-24.04 in this PR
helps the head side, but the internal_staging base is still recorded on
the old unpinned runner, so comparisons keep flapping until the pin
merges and the base is re-baselined.
Until that settles, the red X's on internal staging make the OSS
project look unhealthy and confuse contributors, so drop the
litellm_internal_staging push and pull_request triggers and keep
CodSpeed running on main only.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Guard the per-request CPU cost of the chat completion, MCP tool and A2A
message transforms against regressions on every commit. All benchmarks are
pure in-process work with no network I/O so they stay deterministic under
CodSpeed's simulation mode, and they import under the base dependency set the
benchmark job installs.
Inference covers the full SDK overhead via mock_response (simple, multi-turn,
tools, streaming) plus convert_to_model_response_object as a deterministic
anchor. MCP covers the client-side tool translation and the proxy server-side
tool-name prefix round-trip. A2A covers the client request/response transforms
and the proxy server-ingress message conversion.
Adds the mcp and a2a-sdk packages to the benchmark run since those transform
modules need them, and broadens the workflow triggers to litellm_internal_staging
so the internal branch flow is benchmarked too.
* test(batches): add 1:1 test file scaffold for batches component paths
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add harness test for create batch endpoint
* Add retrieve endpoint harness tests
* Add list endpoint harness tests
* Add cancel endpoint harness tests
* Add cancel endpoint harness tests
* Add test for litellm/batches/main.py
* Add test for litellm/tests/test_litellm/batches/test_batch_utils.py
* Add handler and transformation tests for all providers
* Fix: run batches tests in cicd
* fix(tests): remove azure/__init__.py that shadowed azure namespace package
Adding __init__.py to tests/test_litellm/llms/azure/ caused pytest to
insert tests/test_litellm/llms/ into sys.path[0], making our empty
azure/ dir shadow the real azure-identity namespace package. Any test
that patched azure.identity.* would then fail with AttributeError.
* style(tests): apply ruff format to test_batch_utils.py
Base migrated the formatter from black to ruff format (#31317); reformat the
batches scaffold test file to match.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
