- test_fire_and_forget_silent_on_network_error: mock threading.Thread
entirely so no real thread or network call can escape the test boundary
- test_semaphore_released_on_thread_start_failure: replace _inflight._value
(private CPython attr) with a non-blocking acquire/release assertion
- Semaphore leak: wrap t.start() in try/except and release _inflight on
RuntimeError so the slot is not permanently lost if the OS thread limit
is hit
- Remove 500 from _ERROR_TYPE_MAP: 500 is a generic internal server error
(crash/bug/misconfiguration), not a capacity/overload condition — sending
'overloaded' for 500s would corrupt crowd-sourced signal
- Rewrite semaphore cap test to actually invoke real _fire_and_forget with
exhausted semaphore and assert threading.Thread is never called
- Add test for semaphore release on thread start failure
- Update error-type test to explicitly assert 500 is excluded
Covers: provider normalization, status code extraction, latency
calculation with both datetime and float types, error type mapping,
payload construction, semaphore cap, silent network failure, and
async hook wiring.
- _latency_ms(): handle both datetime objects and floats — LiteLLM
passes datetime, not float, causing TypeError on every invocation
- _ERROR_TYPE_MAP fallback: removed default "overloaded" for unmapped
codes (400, 404, 502, etc.) — now returns None for unrecognised codes
- _fire_and_forget(): added semaphore cap (_MAX_INFLIGHT=5) to prevent
unbounded thread spawning during burst failures
- Removed dead is_resolution parameter from _report() — only called
from failure hooks so it was always False
Adds Tickerr as a built-in LiteLLM callback. Tickerr is a crowd-sourced
outage radar for AI agents — when an agent hits a 5xx error, it reports
anonymously to Tickerr and receives live signal from other agents hitting
the same issue (including a fallback model recommendation).
Changes:
- litellm/integrations/tickerr.py — TickerrLogger (stdlib-only, no new deps)
- litellm/__init__.py — adds "tickerr" to _custom_logger_compatible_callbacks_literal
- litellm/litellm_core_utils/custom_logger_registry.py — registers TickerrLogger
- litellm/integrations/callback_configs.json — UI config entry
- docs/my-website/docs/observability/tickerr.md — integration docs
Usage:
litellm.callbacks = ["tickerr"]
No API key required. Anonymous. Non-blocking (daemon thread, 5s timeout).
https://tickerr.ai
PRISMA_CLI_BINARY_TARGETS="debian-openssl-3.0.x" was hardcoded in
docker/Dockerfile.non_root by #17695. On a buildx linux/arm64 leg this
forces prisma to download the amd64 schema-engine into an arm64 image,
so 'prisma migrate deploy' fails at startup with 'Could not find
schema-engine binary'.
Removing the env lets prisma auto-detect per build platform: amd64
builds still resolve to debian-openssl-3.0.x (Wolfi falls back to
debian, same binary as before), and arm64 builds now correctly fetch
linux-arm64-openssl-3.0.x. The offline-cache pre-warm goal of #17695 is
preserved — only which binaries fill the cache changes.
Fixes#19458
The cimg/python:3.12-browsers base image already ships every Chromium
system dependency Playwright needs (libnss3, libatk-bridge2.0-0,
libcups2, etc. — the install log shows them all as "already the newest
version"). Passing --with-deps to `npx playwright install` therefore
runs an apt-get update + install for nothing, but pays the full cost of
hitting Ubuntu mirrors. On a recent run those mirrors stalled hard:
apt-get update alone took 6m53s at 81.5 kB/s with several archives
returning connection refused.
Drop --with-deps and persist ~/.cache/ms-playwright alongside
node_modules so the Chromium binary is also reused across runs. Bump
the cache key to v2 so the existing v1 entry (which only contained
node_modules) is not loaded and skipped over the new browser path.
/metrics now requires auth by default; tests/otel_tests/test_prometheus.py
makes 4+ unauthenticated GETs against http://0.0.0.0:4000/metrics, so
every prometheus test in CI now fails the metric assertion.
Set require_auth_for_metrics_endpoint: false in otel_test_config.yaml
to opt out for this test job, which scrapes /metrics directly. Verified
locally: 8/8 prometheus tests green (one flaky retry on
test_proxy_success_metrics that pre-dates this PR).
Also drop the -x stop-on-first-failure flag from the otel test command
so all failures in the job surface in a single CI run rather than
hiding behind whichever one trips first.
The Azure o-series tests were excluded from the conftest's VCR auto-marker
because of a respx/vcrpy transport-patching conflict, but the only respx
reference in the file was an unused `MockRouter` import. Drop the dead
import and remove the file from the conflict set so cassettes record on
first run and replay thereafter, eliminating the 60-95s live Azure latency
that was crashing xdist workers under --timeout=120 thread-mode timeouts.
The /otel-spans endpoint returns process-wide spans and tags
most_recent_parent by max start_time. After tightening that route to
proxy_admin (sk-1234), the GET /otel-spans request itself emits auth
spans that beat the chat-completion spans on start_time, so
most_recent_parent now points at the request's own auth trace
(['postgres', 'postgres']) and the >=5-span assertion fails.
Pick the chat-completion trace by content: it is the only trace whose
span list is a superset of {postgres, redis, raw_gen_ai_request,
batch_write_to_db}. Verified locally end-to-end against
otel_test_config.yaml + OTEL_EXPORTER=in_memory: 3/3 runs green.
/otel-spans now requires proxy admin (returns 401 'Only proxy admin
can be used to generate, delete, update info for new keys/users/teams.
Route=/otel-spans' for non-admin callers). Switch the GET call to use
the master key sk-1234 while keeping the generated key for the
chat-completion request that produces the spans.
- Add image_generation/http_utils.azure_deployment_image_generation_json_body; call
from azure.py (keeps AzureChatCompletion focused on chat).
- Rename finalize_image_edit_multipart_data to finalize_image_edit_request_data with
docstring covering multipart and JSON POST payloads (review feedback).
Co-authored-by: Cursor <cursoragent@cursor.com>
Ruff F401 flagged the aliased import as unused within common_utils.py
because the name is consumed only by external modules (~15 callers
across guardrails, spend tracking, MCP, agents, management endpoints).
Add `# noqa: F401 re-exported` so the alias survives lint while
keeping a single source of truth in litellm.proxy._types.
- Move _user_has_admin_view to litellm.proxy._types as
user_api_key_has_admin_view (single source of truth). common_utils.py
and isolation.py both import from there now, removing the duplicated
role-check that could silently diverge if new admin roles are added.
- Add pytest.importorskip("litellm_enterprise") to the two regression
tests that assert managed_files / managed_vector_stores are registered;
those keys come from ENTERPRISE_PROXY_HOOKS so the tests would fail
unconditionally in a checkout without the enterprise extra installed.
The Python 3.13 CCI smoke matrix surfaces a partially-initialized-module
ImportError when loading the managed files hook chain:
litellm.proxy.hooks/__init__ (mid-import)
-> enterprise.enterprise_hooks
-> litellm_enterprise.proxy.hooks.managed_files
-> litellm.llms.base_llm.managed_resources.isolation
-> litellm.proxy.management_endpoints.common_utils
-> litellm.proxy.utils (re-enters litellm.proxy.hooks)
The except ImportError block in hooks/__init__.py silently swallowed the
failure, leaving managed_files unregistered and POST /files returning
500 "Managed files hook not found".
Two-layer fix:
- Inline the 3-line _user_has_admin_view check in isolation.py instead
of importing it from litellm.proxy.management_endpoints.common_utils.
litellm.llms.* should not depend on litellm.proxy.* — removing this
layering violation breaks the cycle at its root.
- Define PROXY_HOOKS and get_proxy_hook before the conditional
enterprise import in litellm/proxy/hooks/__init__.py, so any future
re-entry resolves the public names instead of hitting an
ImportError on a partially-initialized module.
Also fold in two unrelated CCI repairs surfaced in the same staging run:
- tests/otel_tests/test_key_logging_callbacks.py: per-key
gcs_bucket_name / gcs_path_service_account are now stripped by
initialize_dynamic_callback_params, so the GCS client falls through
to the env-only branch. Update the assertion to match the new
"GCS_BUCKET_NAME is not set" message.
- .circleci/config.yml: tests/pass_through_tests now resolves
google-auth-library@10.x via the @google-cloud/vertexai 1.12.0 bump,
which uses dynamic ESM imports Jest 29 cannot load without
--experimental-vm-modules. Pass that flag in the Vertex JS test step.
Adds tests/test_litellm/proxy/hooks/test_proxy_hooks_init.py as a
regression guard: managed_files / managed_vector_stores must register,
and isolation.py must not transitively import litellm.proxy.utils.
secret_fields (containing raw HTTP headers including Authorization
Bearer tokens) was being included in proxy_server_request['body']
because the body snapshot was a copy.copy(data) of the full request
dict. This body gets serialized and persisted in the LiteLLM_SpendLogs
table, exposing user credentials in the database.
Root cause: data['secret_fields'] was set before the body snapshot at
data['proxy_server_request']['body'] = copy.copy(data), so the full
raw headers (including auth tokens) ended up in the snapshot.
Fix (defense in depth):
1. Exclude 'secret_fields' when creating the body snapshot in
litellm_pre_call_utils.py (primary fix)
2. Strip 'secret_fields' in _sanitize_request_body_for_spend_logs_payload
as a secondary safeguard
secret_fields remains available on the live data dict for legitimate
downstream consumers (MCP, Responses API).
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
OpenRouter has dropped active endpoints for anthropic/claude-3.7-sonnet,
causing test_reasoning_content_completion to fail with a 404 "No endpoints
found" error. Switch to anthropic/claude-sonnet-4.5, which is current and
supports reasoning streaming.
/simplify follow-ups:
* Replace the two-``pop`` reach into ``cache_dict``/``ttl_dict`` with
the existing public ``InMemoryCache.delete_cache(key)`` — the same
idiom used elsewhere in the proxy. Bonus: ``delete_cache`` calls
``_remove_key`` which also handles ``expiration_heap`` consistency
the direct pops were silently leaking.
* JSON-encode the sorted scope list for the cache key instead of
``"|".join``. ``user_id`` / ``team_id`` / ``org_id`` / ``api_key``
are free-form strings and could contain a literal ``|`` — JSON
quoting escapes any in-string separator unambiguously.
* Extract ``_allowed_container_ids_cache_key()`` so the read and
invalidation sites compute the key the same way.
* Fix a placeholder-then-overwrite test construction: the
``__module__.split(".")[0] and "proxy_admin"`` line evaluated to a
literal string that was immediately overwritten with the real enum
value. Hoist the import and construct directly.
Address Greptile P2 follow-ups from the prior round:
* Cache ``_get_allowed_container_ids`` (60s LRU/TTL keyed by sorted
owner-scope tuple) so ``GET /v1/containers`` doesn't issue a fresh
``find_many`` against ``litellm_managedobjecttable`` on every list
call. Invalidate the caller's own cache entry when they record a
new owner so the just-created container shows up on their next list.
* Tighten the admin early-return in ``record_container_owner`` to skip
ONLY when there's literally no container ID to stamp. An admin with
identity (the master-key path populates ``user_id`` + ``api_key``)
flows through the normal record path so admin-created containers are
tracked like any other caller's. The truly-identity-less admin case
still falls through to the 403 below — correct fail-secure default.
Skill-cache invalidation gap (also flagged by Greptile) is moot: there
is no skill update endpoint exposed; ownership-affecting mutations are
only delete (already invalidates) and create (new ID, no cache entry
to update).
Substantial reduction (~765 LOC) without changing the security
boundary:
* Drop ContainerOwnershipStore and LiteLLMSkillsStore — both were
one-method-per-Prisma-call wrappers. Inline the calls instead,
matching the established pattern in vector_store_endpoints,
agent_endpoints, and mcp_server/db.py.
* Drop the prisma_client is None in-memory fallback. Production
deploys always have Prisma; running ownership-critical paths on a
process-local dict is a security footgun in the dev-mode case it
was meant to support, and complicates every code path with a
branch. Fail-secure: skip recording if Prisma is unavailable, and
treat reads as "not found" (admin-only).
* Drop the hand-rolled module-level cache. Replace with the existing
litellm.caching.in_memory_cache.InMemoryCache, which already has
TTL + max-size + eviction tested in its own module. Sentinel string
for negative caching since InMemoryCache can't disambiguate "miss"
from "cached as None".
* Tests: drop coverage for removed code paths (in-memory fallback,
hand-rolled cache internals). Keep tests for actual behavior (cache
hit-rate, negative caching, owner check, list filtering,
identity-less reject, admin bypass).
