Mutation testing surfaced three factory functions whose tests ran against them
but asserted nothing that a mutation could break, so every planted bug survived.
- litellm/llms/litellm_proxy/skills/code_execution.py: the OpenAI and Anthropic
tool schemas were unpinned (the Anthropic one was not reached by any test at
all) and the handler's default fallbacks were unchecked
- litellm/containers/endpoint_factory.py: the endpoints.json contract, the
generated sync/async function set and the response-type mapping were unpinned
- litellm/llms/openai_like/dynamic_config.py: the generated Responses API config
class had no coverage of auth header, URL resolution or the store override
The openai_like tests clear _responses_config_cache around each test. Without
that, the module-level cache hands back a class built before the mutation and
the tests pass against mutated code.
Verified by re-running mutmut per scope:
llms/litellm_proxy 45.2% -> 62.8% (70 mutants newly killed)
containers 36.8% -> 84.3% (45 mutants newly killed)
llms/openai_like 55.7% -> 66.9% (34 mutants newly killed)
Three groups, all verified by running the suite rather than by inspection.
18 files whose every test function carries an unconditional @pytest.mark.skip,
39 test functions in total. They are collected on every CI run and always skip,
so they advertise coverage the suite does not have. Reasons on the marks include
"AWS Suspended Account", "lakera deprecated their v1 endpoint" and "moved to
using 'otel' for logging"; 26 of the marks predate 2025.
30 test functions with a byte-identical body and identical decorators to a
sibling in the same file and class, differing only in name. Deleting one of each
pair removes no coverage. Four further candidates were excluded because they
override an inherited test, where deleting the override un-shadows the base
class implementation instead of removing a duplicate.
9 test functions that a later definition of the same name shadows, so Python
never binds them and pytest cannot collect them.
One file that is a demo script rather than a test; its own docstring says to run
it with python.
Verification: collecting the 26 edited files gives 2,492 node IDs before and
2,462 after. The 30 duplicate deletions account for exactly 30 removals, the 9
shadowed deletions account for 0 (confirming at runtime that they were never
collectable), nothing unexplained disappeared, and nothing new appeared. No
other test or module imports any deleted symbol.
Substantial reduction (~765 LOC) without changing the security
boundary:
* Drop ContainerOwnershipStore and LiteLLMSkillsStore — both were
one-method-per-Prisma-call wrappers. Inline the calls instead,
matching the established pattern in vector_store_endpoints,
agent_endpoints, and mcp_server/db.py.
* Drop the prisma_client is None in-memory fallback. Production
deploys always have Prisma; running ownership-critical paths on a
process-local dict is a security footgun in the dev-mode case it
was meant to support, and complicates every code path with a
branch. Fail-secure: skip recording if Prisma is unavailable, and
treat reads as "not found" (admin-only).
* Drop the hand-rolled module-level cache. Replace with the existing
litellm.caching.in_memory_cache.InMemoryCache, which already has
TTL + max-size + eviction tested in its own module. Sentinel string
for negative caching since InMemoryCache can't disambiguate "miss"
from "cached as None".
* Tests: drop coverage for removed code paths (in-memory fallback,
hand-rolled cache internals). Keep tests for actual behavior (cache
hit-rate, negative caching, owner check, list filtering,
identity-less reject, admin bypass).
Two cleanups:
* ``LiteLLMSkillsHandler.create_skill`` raised ``HTTPException`` for
identity-less callers, importing FastAPI from a ``litellm/llms/``
module — that violates the project rule that FastAPI lives only
under ``proxy/``. Switch to ``ValueError`` (the same shape the rest
of the handler uses for not-found/forbidden) and update the test.
* The proxy-auth body bouncer derived its observability ban list from
``_supported_callback_params`` only, missing
``_request_blocked_callback_params`` (where ``gcs_bucket_name`` and
``gcs_path_service_account`` live). Two recently-merged sibling PRs
(#27019 added the deny list, #27081 added the test asserting these
are rejected at the request body root) crossed without folding them
together. Union the GCS deny list into the bouncer's derivation so
the single source of truth covers both code paths.
UNSCOPED_RESOURCE_OWNER_SCOPE collapsed every caller without an
identity field (no user_id / team_id / org_id / api_key / token) into
a single shared owner — a cross-tenant access primitive: any two such
callers could see and delete each other's containers and skills.
Drop the sentinel. ``get_primary_resource_owner_scope`` returns
``None`` and ``get_resource_owner_scopes`` returns ``[]`` for
identity-less callers. ``record_container_owner`` and
``LiteLLMSkillsHandler.create_skill`` now reject creates from
identity-less callers with a 403 instead of stamping the placeholder.
Read paths already deny ``owner is None`` correctly so legacy rows
(if any) are admin-only.
LITELLM_ALLOW_UNTRACKED_CONTAINER_ACCESS and
LITELLM_ALLOW_UNOWNED_SKILL_ACCESS were operator-toggleable opt-outs
for the cross-tenant access primitive this PR closes — flipping either
on re-enabled exactly the VERIA-20 read path. Default-secure with no
escape hatch matches sibling fixes (vector-store cred isolation, semantic
cache key isolation, user_config strip): all rejected the
opt-out-of-security pattern.
Untracked containers and unowned skills (rows that pre-date this
enforcement) are admin-only. Non-admin owners need to either re-create
via the now-tracked flow or have an admin assign ``created_by`` on the
existing row. Update tests to assert the strict-only behaviour.
Two cleanups from the /simplify pass:
* ``_CONTAINER_OWNER_CACHE`` and ``_SKILL_CACHE`` now LRU-evict via
``OrderedDict.popitem(last=False)`` instead of full ``clear()`` at
capacity. Full clears converted a steady-state cached workload into a
periodic full-DB-load oscillation as the cache repopulated from zero
and cleared again. Reads now ``move_to_end`` so the just-touched
entry survives the next eviction. Mirrors the pre-existing LRU
pattern in ``_remember_container_owner``.
* ``LiteLLM_ManagedObjectTable.file_purpose`` Literal now includes
``"container"`` so Pydantic validation accepts rows written by the
ownership store.
Container ownership and skill rows are looked up on every retrieve /
delete / list / file-content / chat-completion-with-skill call. The new
stores wrapped raw Prisma queries with no cache, putting one DB
round-trip on each request. Add an in-process TTL'd cache mirroring the
_byok_cred_cache pattern in mcp_server/server.py: per-key (value,
monotonic_timestamp), 60s TTL, 10000-entry cap with full-clear on
overflow, invalidated by every write. Negative results (`None`) are
cached too so untracked-resource checks also skip the DB.
Tests cover: cache-after-first-hit, negative caching, write
invalidation, no-caching-on-DB-error, TTL expiry, capacity eviction.
56 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* build: migrate packaging metadata to uv
* ci: move automation and local tooling to uv
* docker: migrate image builds and runtime setup to uv
* docs: update install and deployment guidance for uv
* chore: align auxiliary scripts and tests with uv
* test: harden test_litellm isolation
* fix: keep release and health check images self-contained
* build: pin uv tooling and health check deps
* test: isolate bedrock image request formatting from suite state
* test: cover sandbox executor requirements flow
* ci: fix circleci no-op command steps
* ci: fix circleci publish workflow parsing
* fix: stabilize remaining uv migration CI checks
* ci: increase matrix test timeout headroom
* fix: restore published docker and license coverage
* fix: restore proxy runtime build parity
* fix: restore proxy extras parity and venv migrations
* ci: persist uv path across circleci steps
* fix: keep psycopg binary in default test env
* docker: preserve prisma cache across stages
* test: run local proxy checks through uv python
* build: restore runtime deps moved into ci
* build: refresh uv lock after upstream merge
* fix: restore module import in test_check_migration after merge
The conflict resolution imported only the function but the test body
references check_migration as a module throughout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching
- Move google-generativeai, Pillow, tenacity back to ci group (they are
lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate uv.lock after removing nodejs-wheel-binaries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): use cache/restore instead of cache to prevent cache poisoning
The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert
The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): disable setup-uv cache in publish workflow
Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(test): remove duplicate verbose_logger mock in test_check_migration
The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): free disk space before Docker build in test-server-root-path
The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params
Fixes issue where 'user' was not being sent in request to litellm proxy via sdk
* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default
* docs: cleanup docs
* fix(proxy_cli.py): remove --use_prisma_migrate flag
* refactor: remove references to use_prisma_migrate env var
This is now the default flow for db migrations
* fix(factory.py): handle qs:.. in mime type
Fixes https://github.com/BerriAI/litellm/issues/11839
* feat(litellm_proxy/): don't transform messages client-side
leave litellm proxy messages untouched - allow proxy to handle transformation
prevents double transformation
* feat(tag_management_endpoints.py): support adding models to tag by adding model_name
Closes https://github.com/BerriAI/litellm/issues/11884
* test(test_tag_management_endpoints.py): add unit tests for adding new model by public model name
* test: update test
