/simplify follow-ups:
* Replace the two-``pop`` reach into ``cache_dict``/``ttl_dict`` with
the existing public ``InMemoryCache.delete_cache(key)`` — the same
idiom used elsewhere in the proxy. Bonus: ``delete_cache`` calls
``_remove_key`` which also handles ``expiration_heap`` consistency
the direct pops were silently leaking.
* JSON-encode the sorted scope list for the cache key instead of
``"|".join``. ``user_id`` / ``team_id`` / ``org_id`` / ``api_key``
are free-form strings and could contain a literal ``|`` — JSON
quoting escapes any in-string separator unambiguously.
* Extract ``_allowed_container_ids_cache_key()`` so the read and
invalidation sites compute the key the same way.
* Fix a placeholder-then-overwrite test construction: the
``__module__.split(".")[0] and "proxy_admin"`` line evaluated to a
literal string that was immediately overwritten with the real enum
value. Hoist the import and construct directly.
Address Greptile P2 follow-ups from the prior round:
* Cache ``_get_allowed_container_ids`` (60s LRU/TTL keyed by sorted
owner-scope tuple) so ``GET /v1/containers`` doesn't issue a fresh
``find_many`` against ``litellm_managedobjecttable`` on every list
call. Invalidate the caller's own cache entry when they record a
new owner so the just-created container shows up on their next list.
* Tighten the admin early-return in ``record_container_owner`` to skip
ONLY when there's literally no container ID to stamp. An admin with
identity (the master-key path populates ``user_id`` + ``api_key``)
flows through the normal record path so admin-created containers are
tracked like any other caller's. The truly-identity-less admin case
still falls through to the 403 below — correct fail-secure default.
Skill-cache invalidation gap (also flagged by Greptile) is moot: there
is no skill update endpoint exposed; ownership-affecting mutations are
only delete (already invalidates) and create (new ID, no cache entry
to update).
Substantial reduction (~765 LOC) without changing the security
boundary:
* Drop ContainerOwnershipStore and LiteLLMSkillsStore — both were
one-method-per-Prisma-call wrappers. Inline the calls instead,
matching the established pattern in vector_store_endpoints,
agent_endpoints, and mcp_server/db.py.
* Drop the prisma_client is None in-memory fallback. Production
deploys always have Prisma; running ownership-critical paths on a
process-local dict is a security footgun in the dev-mode case it
was meant to support, and complicates every code path with a
branch. Fail-secure: skip recording if Prisma is unavailable, and
treat reads as "not found" (admin-only).
* Drop the hand-rolled module-level cache. Replace with the existing
litellm.caching.in_memory_cache.InMemoryCache, which already has
TTL + max-size + eviction tested in its own module. Sentinel string
for negative caching since InMemoryCache can't disambiguate "miss"
from "cached as None".
* Tests: drop coverage for removed code paths (in-memory fallback,
hand-rolled cache internals). Keep tests for actual behavior (cache
hit-rate, negative caching, owner check, list filtering,
identity-less reject, admin bypass).
UNSCOPED_RESOURCE_OWNER_SCOPE collapsed every caller without an
identity field (no user_id / team_id / org_id / api_key / token) into
a single shared owner — a cross-tenant access primitive: any two such
callers could see and delete each other's containers and skills.
Drop the sentinel. ``get_primary_resource_owner_scope`` returns
``None`` and ``get_resource_owner_scopes`` returns ``[]`` for
identity-less callers. ``record_container_owner`` and
``LiteLLMSkillsHandler.create_skill`` now reject creates from
identity-less callers with a 403 instead of stamping the placeholder.
Read paths already deny ``owner is None`` correctly so legacy rows
(if any) are admin-only.
LITELLM_ALLOW_UNTRACKED_CONTAINER_ACCESS and
LITELLM_ALLOW_UNOWNED_SKILL_ACCESS were operator-toggleable opt-outs
for the cross-tenant access primitive this PR closes — flipping either
on re-enabled exactly the VERIA-20 read path. Default-secure with no
escape hatch matches sibling fixes (vector-store cred isolation, semantic
cache key isolation, user_config strip): all rejected the
opt-out-of-security pattern.
Untracked containers and unowned skills (rows that pre-date this
enforcement) are admin-only. Non-admin owners need to either re-create
via the now-tracked flow or have an admin assign ``created_by`` on the
existing row. Update tests to assert the strict-only behaviour.
Two cleanups from the /simplify pass:
* ``_CONTAINER_OWNER_CACHE`` and ``_SKILL_CACHE`` now LRU-evict via
``OrderedDict.popitem(last=False)`` instead of full ``clear()`` at
capacity. Full clears converted a steady-state cached workload into a
periodic full-DB-load oscillation as the cache repopulated from zero
and cleared again. Reads now ``move_to_end`` so the just-touched
entry survives the next eviction. Mirrors the pre-existing LRU
pattern in ``_remember_container_owner``.
* ``LiteLLM_ManagedObjectTable.file_purpose`` Literal now includes
``"container"`` so Pydantic validation accepts rows written by the
ownership store.
Container ownership and skill rows are looked up on every retrieve /
delete / list / file-content / chat-completion-with-skill call. The new
stores wrapped raw Prisma queries with no cache, putting one DB
round-trip on each request. Add an in-process TTL'd cache mirroring the
_byok_cred_cache pattern in mcp_server/server.py: per-key (value,
monotonic_timestamp), 60s TTL, 10000-entry cap with full-clear on
overflow, invalidated by every write. Negative results (`None`) are
cached too so untracked-resource checks also skip the DB.
Tests cover: cache-after-first-hit, negative caching, write
invalidation, no-caching-on-DB-error, TTL expiry, capacity eviction.
56 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If record_container_owner raises after the upstream container is created,
the user previously got a 500 with no usable container — they were billed
for an unreachable resource. Move ownership recording into the create
path's exception handling and split the two failure modes:
- HTTPException from the recorder (auth conflicts) propagates verbatim
so the client sees the real status code, not a generic LLM error.
- Unexpected exceptions are logged and swallowed; the response is
returned to the caller so they aren't billed for a container they
can't address. The DB row stays untracked until an operator reconciles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Use decoded managed container model_id to resolve deployment credentials for container file calls and add regressions to verify provider/model metadata decoding and api_base selection.
Made-with: Cursor
* feat(containers): Azure container routing, managed IDs, and delete response wire format
- Add AzureContainerConfig and safe URL joining for paths with api-version query
- Encode/decode managed container IDs in responses, streaming, and proxy handlers
- Accept OpenAI delete response object literal container.file.deleted
- Tests for Azure URL regression and DeleteContainerFileResponse parsing
Made-with: Cursor
* fix(responses): gate response id update on parsed_chunk having response
Delta stream events do not include a response body; Mock-based tests
(and any truthy synthetic .response on transforms) must not trigger
_update_responses_api_response_id_with_model_id. Fixes
test_stop_async_iteration_not_logged_as_failure (TypeError: Mock not iterable).
Made-with: Cursor
* feat(containers): encode container IDs in SDK responses for routing
- Add ContainerRequestUtils.encode_container_id_in_response utility
- Encode container_id in create/retrieve/delete responses (SDK path)
- Fix streaming iterator: gate response ID update on parsed_chunk key
- Follows responses API pattern (encode after handler, not in handler)
Made-with: Cursor
* fix(containers): module-level imports and managed cntr_ ID encoding
- Move ResponsesAPIRequestUtils imports to module scope (utils, main, handler_factory).
- Serialize absent model_id as empty segment instead of literal None; decode empty
and legacy "None" segments as missing for router affinity.
- Add unit tests for build/decode round-trip and legacy IDs.
Made-with: Cursor
* fix(containers): decode managed IDs in endpoint_factory SDK path
- Add decode_managed_container_id_for_request in containers/utils and reuse from main.
- Strip LiteLLM cntr_ wrappers before generic_container_handler (64-char API limit).
- Resolve provider for logging/errors; add unit test for decode helper.
- Use resolved_custom_llm_provider after decode for mypy-safe provider typing.
Made-with: Cursor
* Fix p1 concern
* Fix p1 concern
The test was making real API calls instead of using mocks because the
conftest.py reloads litellm at module scope, causing stale module
references. The mock was patching the old reference while the actual
code used the new one.
Fix: Reload litellm.containers.main inside the test to get a fresh
reference to base_llm_http_handler, then re-import create_container
after the reload.
* fix(tests): Mock async_container_create_handler for async router test
The test was mocking container_create_handler (sync), but
router.acreate_container uses _is_async=True which calls
async_container_create_handler. This caused the test to hit
the real OpenAI API.
Fixed by using AsyncMock on async_container_create_handler.
* fix(tests): Use uuid for unique model name in scientific notation test
The test was using a static "unique" model name which could cause
conflicts when running tests in parallel (-n 16 in CI). Using uuid
ensures truly unique names to prevent test pollution.
---------
Co-authored-by: Shin <shin@openclaw.ai>
* litellm_fix_mapped_tests_core: fix test isolation and mock injection issues
## Problem
Four tests in litellm_mapped_tests_core were failing:
1. test_register_model_with_scientific_notation - KeyError due to test isolation issues
2. test_search_uses_registry_credentials - Mock not being called due to incorrect patch path
3. test_send_email_missing_api_key - Real API calls despite mocking
4. test_stream_transformation_error_sync - Mock not effective, real API called
## Solution
### test_register_model_with_scientific_notation
- Use unique model name to avoid conflicts with other tests
- Clear LRU caches before test to prevent stale data
- Clean up model_cost entry after test
### test_search_uses_registry_credentials
- Use patch.object() on the actual base_llm_http_handler instance
- String-based patching for instance methods can fail; direct object patching is more reliable
### test_send_email_missing_api_key
- Directly inject mock HTTP client into logger instance
- This bypasses any caching issues that could cause the fixture mock to be ineffective
### test_stream_transformation_error_sync
- Patch litellm.completion directly instead of the handler module's litellm reference
- This ensures the mock is effective regardless of import order
## Regression
These tests were affected by LRU caching added in #19606 and HTTP client caching.
* fix(test): use patch.object for container API tests to fix mock injection
## Problem
test_retrieve_container_basic tests were failing because mocks weren't
being applied correctly. The tests used string-based patching:
patch('litellm.containers.main.base_llm_http_handler')
But base_llm_http_handler is imported at module level, so the mock wasn't
intercepting the actual handler calls, resulting in real HTTP requests
to OpenAI API.
## Solution
Use patch.object() to directly mock methods on the imported handler
instance. Import base_llm_http_handler in the test file and patch like:
patch.object(base_llm_http_handler, 'container_retrieve_handler', ...)
This ensures the mock is applied to the actual object being used,
regardless of import order or caching.
* fix(test): add missing Prometheus metric labels to test_proxy_failure_metrics
Add client_ip, user_agent, model_id labels to expected metric patterns.
These labels were added in PRs #19717 and #19678 but test wasn't updated.
* fix(test_resend_email): use direct mock injection for all email tests
Extend the mock injection pattern used in test_send_email_missing_api_key
to all other tests in the file:
- test_send_email_success
- test_send_email_multiple_recipients
Instead of relying on fixture-based patching and respx mocks which can
fail due to import order and caching issues, directly inject the mock
HTTP client into the logger instance. This ensures mocks are always used
regardless of test execution order.
* fix(test): use patch.object for image_edit and vector_store tests
- test_image_edit_merges_headers_and_extra_headers: import base_llm_http_handler
and use patch.object instead of string path patching
- test_search_uses_registry_credentials: import module and patch via
module.base_llm_http_handler to ensure we patch the right instance
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
## Problem
Tests using mocked HTTP clients were hitting real APIs because:
1. HTTP client cache was returning previously cached real clients
2. isinstance checks failed due to module identity issues from sys.path
### Tests affected:
- test_send_email_missing_api_key
- test_send_email_multiple_recipients (resend & sendgrid)
- test_search_uses_registry_credentials
- test_vector_store_create_with_simple_provider_name
- test_vector_store_create_with_provider_api_type
- test_vector_store_create_with_ragflow_provider
- test_image_edit_merges_headers_and_extra_headers
- test_retrieve_container_basic (container API tests)
## Solution
1. Add clear_client_cache fixture (autouse=True) to clear
litellm.in_memory_llm_clients_cache before each test
2. Fix isinstance checks to use type name comparison
(avoids module identity issues from sys.path.insert)
## Why not disable_aiohttp_transport
The default transport is aiohttp, so tests should work with it.
Clearing the cache ensures mocks are used instead of cached real clients.
## Regression
PR #19829 (commit f95572e3ed) added @respx.mock but cached clients
from earlier tests were being reused, bypassing the mocks.
Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
* fix: make HTTPHandler mockable in OIDC secret manager tests
- Add _get_oidc_http_handler() factory function to make HTTPHandler
easily mockable in tests
- Update test_oidc_github_success to patch factory function instead
of HTTPHandler directly
- Update Google OIDC tests for consistency
- Fixes test_oidc_github_success failure where mock was bypassed
This change allows tests to properly mock HTTPHandler instances used
for OIDC token requests, fixing the test failure where the mock was
not being used.
* fix: patch base_llm_http_handler method directly in container tests
- Use patch.object to patch container_create_handler method directly
on the base_llm_http_handler instance instead of patching the module
- Fixes test_provider_support[openai] failure where mock wasn't applied
- Also fixes test_error_handling_integration with same approach
The issue was that patching 'litellm.containers.main.base_llm_http_handler'
didn't work because the module imports it with 'from litellm.main import',
creating a local reference. Using patch.object patches the method on the
actual object instance, which works regardless of import style.
* fix: resolve flaky test_openai_env_base by clearing cache
- Add cache clearing at start of test_openai_env_base to prevent cache pollution
- Ensures no cached clients from previous tests interfere with respx mocks
- Fixes intermittent failures where aiohttp transport was used instead of httpx
- Test-only change with low risk, no production code modifications
Resolves flaky test marked with @pytest.mark.flaky(retries=3, delay=1)
Both parametrized versions (OPENAI_API_BASE and OPENAI_BASE_URL) now pass consistently
* test: add explicit mock verification in test_provider_support
- Capture mock handler with 'as mock_handler' for explicit validation
- Add assert_called_once() to verify mock was actually used
- Ensures test verifies no real API calls are made
- Follows same pattern as test_openai_env_base validation
* Add v1 cut of container api
* fix lint errors
* Add proxy support to container apis & logging support (#16049)
* Add proxy support to container apis
* Add logging support
* Add cost tracking support for containers and documentation
* Add new constant documentation
* Add container cost in model map
* fix failing azure tests
* Update tests based on model map changes
* fix model map tests
* fix model map tests
* Container modeshould be container
* Container tests fix
* Merge branch 'main' into litellm_sameer_oct_staging_2
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
