* build(deps): bump pyasn1 from 0.6.4 to 0.6.5
Bumps [pyasn1](https://github.com/pyasn1/pyasn1) from 0.6.4 to 0.6.5.
- [Release notes](https://github.com/pyasn1/pyasn1/releases)
- [Changelog](https://github.com/pyasn1/pyasn1/blob/main/CHANGES.rst)
- [Commits](https://github.com/pyasn1/pyasn1/compare/v0.6.4...v0.6.5)
---
updated-dependencies:
- dependency-name: pyasn1
dependency-version: 0.6.5
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump pyasn1, pypdf and minimatch to clear osv-scan
pyasn1 0.6.4 -> 0.6.5 and pypdf 6.21.2 -> 6.21.4 in uv.lock, with a dated
exclude-newer-package cutoff that admits pypdf 6.21.4 before it clears the
3-day window. minimatch has no fixed 10.x release, so the dashboard's override
and a new vscode-extension override move it to 11.0.2.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(token_counter): price PDF pages for OpenAI and Azure models the way they bill
* fix(token_counter): price PDF pages for OpenAI and Azure models the way they bill
* fix(token_counter): count azure, data-zone and dated GPT names with their family's encoding
* fix(token_counter): price only drawn PDF images and keep other gateways' GPT pricing
Detect raster images from the page content stream, including inline
images on pages with empty resources, and walk each form once. Strip
only openai/, azure/ and Azure data-zone prefixes, bound the model
family scan, read Azure-only chat models from the cost map, and keep
cl100k_base for listed non-chat OpenAI models.
* fix(token_counter): price GPT names behind any prefix like the bare name and walk shared forms per resources
Prices a PDF by the model name's last path segment, the same name the proxy counts with,
keeps listed non-chat models on Claude pricing, walks each form once per inherited resources
dictionary so a shared child form drawing an image is not missed, and resolves PDF pricing
only when a message carries a PDF
* fix(token_counter): keep the rendered-page count for image pages, parse each form once, and match chat models only
A PDF page that draws a raster image keeps the rendered-page estimate, since newer GPT models bill such a page near
it. The image walk parses each form once and stops at a fixed number of drawn XObjects, so a form shared under many
resources dictionaries no longer slows the count. The family prefix match skips non-chat entries, and ft: fine-tunes
count like their base model
* fix(token_counter): count Azure-only chat models with o200k like other OpenAI chat models
* fix(token_counter): count reservation and project ITPM with the serving deployment's model
Budget reservation and project ITPM counted with the requested group name, so a
GPT-named group served by Claude reserved a PDF at OpenAI's price. They now count
with each deployment's model minus its provider prefix, the name count_tokens uses,
and take the largest. A custom OpenAI tokenizer with no model counts with cl100k
again instead of raising, and listed Azure gpt- chat entries price as OpenAI.
* perf(token_counter): find drawn PDF images with a byte scan instead of parsing every operator
pypdf parsed every operator of a page's content stream to find the images it draws, so a
compressed page of a million operators took about 30 seconds to price for an OpenAI model.
A linear scan of the decoded stream for name Do and inline images finds the same draws.
* fix(token-counter): count each team deployment by its PDF pricing and honor base_model
Budget reservation and the TPM limiter now look up the deployments that serve the
caller's team, by base_model when set. A deployment that reads PDF pages and prices
them unlike the group name counts with its own model, any other with the group name.
A comment between an XObject name and Do now reads as whitespace, and the timing
tests moved to CodSpeed benchmarks.
* fix(benchmarks): install pypdf in the benchmark environment
The PDF token-count benchmarks parse pages with pypdf, which only the proxy-runtime extra installed, so CodSpeed took the no-parser path.
* fix(proxy): count PDF pages for deployments served under multi-segment model names
* fix(proxy): count a team's wildcard route the way routing serves it and reserve OpenAI chat's per-PDF charge
* fix(proxy): count routing-group and alias members with the caller's team, as routing serves them
* build: allow filelock 4.x
filelock 3.x is the only line the current cap permits, and three CVEs
(CVE-2026-112940, CVE-2026-112913, CVE-2026-112907) are fixed only in
4.2.1+. The cap means every downstream that depends on litellm is held
on a vulnerable filelock with no way out, since pip has no override.
litellm uses FileLock(path, timeout=..., mode=...) and Timeout, in
litellm/proxy/client/cli/commands/configure_profiles.py and
claude_settings.py. Both keyword arguments still exist in 4.2.4 and
Timeout is still exported, so the two call sites are unaffected. The
CVEs are in the SQLite-backed ReadWriteLock / AsyncReadWriteLock, which
litellm does not use.
<5 rather than a floor bump because litellm supports Python 3.10 and
filelock 4.x requires >=3.11: widening the cap lets 3.11+ resolve 4.x
while 3.10 keeps a 3.x filelock automatically, through filelock's own
requires_python.
* build: sync core filelock bound and dependency lock
---------
Co-authored-by: Speculator55005 <50082482+fas89@users.noreply.github.com>
* chore(caching)!: remove the disk cache backend
Drop the disk cache type and its diskcache dependency, along with the
caching install extra that existed only to pull it in. diskcache has not
had a release since August 2023
Cache now raises a ValueError listing the supported types when it gets a
type it does not know. Before, an unknown type built a Cache with no
backend, which only failed on first use
BREAKING CHANGE: cache_params type "disk" and the litellm[caching] extra
are gone. A proxy configured with type disk now refuses to start. Use
local or redis instead
* test(proxy): expect the unsupported cache type error from _init_cache
The test pinned the old fallthrough, where an unknown type left the cache
without a backend and the proxy hit an AttributeError reading it
* chore(deps): drop the diskcache scan ignore main extended
diskcache is no longer in the lock, so the ignore matches nothing
* fix(caching): ignore a leftover disk_cache_dir instead of crashing redis and s3
Dropping the disk_cache_dir parameter sent the key into **kwargs, which
the redis and s3 branches hand to their clients, so a config that moved
from the disk cache to redis or s3 and kept the key failed with a
TypeError. Accept the key again, ignore it, and log a warning
The unsupported type test now checks the whole supported list, and two
proxy docstrings no longer name disk as a cache backend
* fix(caching): warn about a leftover disk_cache_dir only once the cache is built
The warning ran before the type check, so a config with type disk logged
that the key was ignored and then failed on the type anyway. It now logs
only after a supported backend is built. The test also checks that a
cache without the key logs nothing
* fix(proxy): default the uvicorn worker healthcheck timeout to 60 s so multi-worker boots survive a slow import
* fix(proxy): warn on uvicorn below 0.37 when multi-worker boots cannot get the longer healthcheck
uvicorn 0.33 to 0.36 has no timeout_worker_healthcheck, so the 60 s default never reaches it and a slow worker start still crash-loops with no hint. Print an upgrade warning when the proxy runs more than one uvicorn worker on such a version, keep granian out of the uvicorn-only path, and type the new tests.
* fix(proxy): name --run_granian in the healthcheck help text and add per-worker grace guidance to the helm chart
* fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping
Each worker uvicorn spawns re-runs the parent's main script before it starts
answering health pings. The litellm console script imported the whole SDK at
the top, so on a busy or CPU-limited host a new worker missed uvicorn's 5 s
ping and was killed in a loop. The script now runs litellm_cli.main:proxy_server,
which imports the proxy only when called, and gateway.launch imports LiteLLM
inside main the same way.
This replaces the earlier 60 s healthcheck default, so uvicorn's 5 s
healthcheck and SIGTERM timing stay as they are.
* fix(proxy): load the worker app before its log config and without GC pauses
* fix(gateway): ship litellm_cli in the gateway image
gateway.launch imports without_gc_pauses from litellm_cli, which the
gateway image never copied, so python -m gateway.launch failed at import.
Also build the gateway image when the launcher or litellm_cli changes.
* fix(proxy): apply the worker log config before the proxy app import, as earlier releases did
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* build(deps): suppress filelock Windows advisories and drop stale certifi entry to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): suppress dev-only msal-node advisory in vscode-extension until azure identity allows 7.1
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): bump ddtrace and msgpack to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): bump pylint to 4.0.9 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): lock dnspython 2.9.0 on Python 3.11+ and defer 3.10 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): bump qs in vscode-extension to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): extend the pillow 12.3.0 override for Python 3.10 to 2026-11-27
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): extend the diskcache ignore, drop the withdrawn braces entry, and name the filelock cap in its reasons
* build(deps): bump langgraph and add dated ignores for the 2026-10-28 filelock advisories
* build(deps): move tornado to 6.6 on Python 3.11+, name huggingface_hub's filelock use, and scope a grype rule to the filelock High
* build(deps): say which packages import tornado in its ignore reasons
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(rust-bridge): require the native extension
_native ships in every wheel and the sdist will not install without it, yet the bridge loaded it optionally and carried None paths that could fail open for fork safety, exception mapping and OTLP errors. Delete loader.py and import _native directly. Tests inject a fake module through NativeBinding(source=...) or with_source() instead of patching the loader
The install smoke checks now verify the compiled module, and tests fail instead of skipping when _native is missing. The interactions spec canary installs the project so it has the extension
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ci): build Rust bridge in Windows smoke tests
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http
The workspace moves to reqwest 0.13, so litellm-http no longer carries a
second reqwest just for rmcp. The mcp feature, its pool and its proxy
conversion go away, and the gateway hands rmcp a pooled NoRedirect client
that already carries the TLS, proxy and timeout settings
The Client test helpers collapse into for_test, a no-redirect client
without TLS or proxy settings, and wrap_for_test for the few tests that
need a hand-built reqwest client
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* build(deps): refresh compatible dependencies flagged by OSV
* refactor(rust): remove redundant client construction comment
* test(rust): cover configured MCP proxy and CA bundle
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(deps): upgrade vulnerable Python dependencies
* fix(deps): lock patched Python dependency versions
* fix(otel): preserve JSON export with the updated SDK
* test(otel): cover JSON export with the updated SDK
* test(otel): update SDK log records and worker discovery
* fix(ui): align Lens peer dependency with patched Next.js
* fix(otel): hand the AgentOps exporter a session it owns for the JWT header
OpenTelemetry 1.45 moved the OTLP HTTP exporter's requests.Session behind a
transport, so writing the minted JWT to the exporter's private _session raised,
was swallowed, and every AgentOps batch went out unauthenticated. The exporter
now creates the session, passes it through the public session= argument and
sets the Authorization header on it
The tests now drive a real export, serving the token endpoint with respx and
capturing the OTLP POST with requests_mock, instead of reading SDK internals
* fix(otel): skip FastAPI instrumentation once the app has started
opentelemetry-instrumentation-fastapi 0.66b1 wraps build_middleware_stack instead
of calling add_middleware, so instrumenting an app whose middleware stack is
already built no longer raises. The call silently did nothing while we still
registered the global response propagator. Check the built stack up front and
skip both, which is what the existing test expects
* test(otel): assert exported requests and span flags instead of SDK internals
OpenTelemetry 1.45 dropped the exporter's private _headers and sets the W3C
random-trace-id flag on new traces, so traceresponse ends in -03. The Langtrace
and Langfuse tests now export through requests_mock and check the URL and headers
the collector receives, and the traceresponse tests expect the server span's own
trace flags
* fix(deps): bump dashboard zod to 4.6.6 and make the vendored Lens use it
zod 4.6.5 is flagged by the dependency scan and blocks osv-scan on every PR. A
root bump alone leaves the vendored Lens tarball, which pins zod 4.6.5 exactly,
on its own nested copy, so a top-level zod override points every consumer at the
root version and the lock carries a single zod 4.6.6
* test(ui): stub the tag summary fetch the client panel retry refreshes
Retrying the model breakdown refetches /tag/summary, which this integration test
left on the real network. When that request failed before the assertion, the
panel switched to its summary error and unmounted the model row, so the test
passed or failed by timing
* fix(deps): drop the duplicated Lens override from the dashboard package.json
* style(ui): format the client panel fetch stub
---------
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* fix(security): bump grpcio and boto3 so their bundled CA files drop the compromised eMudhra roots
grpc ignores SSL_CERT_FILE and verifies every channel against its own bundled roots.pem. In 1.78.0 that file still trusts emSign ECC Root CA - G3 and AAA Certificate Services, so Vertex AI, Google KMS, Chirp STT and OTLP gRPC exporters (Arize) accept all 14 advisory chains. 1.84.1 is the first release whose roots.pem drops both roots.
botocore uses certifi when it is installed and falls back to its bundled cacert.pem otherwise. Through 1.43.116 that file still trusts AAA Certificate Services. Requiring boto3 1.43.117 pulls in botocore 1.43.117, the first release whose cacert.pem rejects all 14 advisory chains, so the fallback stays safe if certifi is ever absent
* fix(docker): provide OpenSSL 3 when generating Prisma engines
Update certifi, Rust trust roots, and the Docker CA bundle to remove emSign ECC Root CA - G3
Require the fixed certifi release in litellm-core and retain libssl3 in Docker runtimes for Prisma compatibility
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Pillow 13.0.0 requires Python >= 3.11, so the ci group pin is split by
Python version and the 3.10 resolution stays on 12.3.0 with a 7-day
osv-scanner override. Dashboard vite moves from 7.3.5 to 7.3.7. The
certifi fix (2026.10.19) is still inside the 3-day uv exclude-newer
window, so its advisory is ignored until 2026-10-23
* fix(mcp): upgrade to MCP SDK 2.3.0 and raise the per-event SSE size cap
MCP SDK 2.x reads Streamable HTTP responses through httpx2's SSE parser,
which rejects any single event over 1 MiB, so a large tools/call result
failed with 'SSE stream ended without a response'. mcp 2.3.0 exposes
max_sse_event_size, and MCPClient now passes a 64 MiB default,
configurable with LITELLM_MCP_MAX_SSE_EVENT_SIZE_BYTES
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): fall back for invalid SSE event limits
* test(mcp): verify configured SSE limits and recovery
* chore(mcp): remove redundant dependency comment
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* test(proxy): cover gzip and zstd request bodies on /v1/systemone
* fix(proxy): decode zstd and gzip request bodies before auth reads them
* fix(proxy): bound request body decompression and keep it inside CORS
Limit the decoded size by an expansion ratio and the operator's max_request_size_mb, pass bodies without the coding's magic number through, answer 415 when zstd is unavailable, and register the middleware innermost so errors carry CORS and security headers
* fix(proxy): budget decoded request bodies per worker and forward plugin bodies as sent
Each worker holds at most MAX_DECOMPRESSED_REQUEST_BODIES_IN_FLIGHT_MB of
decoded bodies and answers 503 past it. /plugin-proxy/ bodies reach the
plugin as sent, a zstd skippable frame counts as zstd, the decoded-size 413
names the decoded limit, and an integer max_request_size_mb stays an integer
in the 413 text.
* fix(proxy): read the magic number before size limits and reserve declared decoded sizes
* test(proxy): pin gzip mtime so parametrized test ids match across xdist workers
* fix(proxy): parse max_request_size_mb the same way in the auth size check
* fix(proxy): pass Claude Code OTLP export bodies through undecoded and skip the size parse without premium
* test(proxy): cover client disconnects, chunked bodies past the cap, and body-less request messages in the decompression middleware
* test(proxy): name the decompression test cases so CI logs stay small
anyio below 4.14.0 can leave a cancelled waiter queued on an asyncio Lock or
Semaphore, so the next task to acquire waits forever on a lock nobody holds
(agronholm/anyio#1145, fixed in 4.14.0). httpcore takes these locks for every
httpx request, including the prisma query-engine client
litellm imports anyio directly but never declared it, so pip and uv kept an
already installed anyio 4.13.0 when installing litellm and accepted an
explicit anyio==4.13.0 next to it. uv.lock already resolves 4.14.2, so Docker
and CI installs do not change. Both the litellm and litellm-core manifests get
the floor, and the lock only records the new direct dependency
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* perf(cli): start lite without initializing the litellm SDK
lite and litellm-proxy now enter through a small litellm_cli package that imports the CLI against an uninitialized litellm package and runs the SDK __init__ in place only when something first uses it, so commands start in a fraction of a second instead of loading about 3,000 modules first.
* perf(cli): import only the lite subcommand being run
Each lite subcommand now loads its module when Click looks it up, so
lite logout no longer imports the autoroute wizard, agents, or the other
command groups. importlib.import_module and attribute reads on the SDK
also initialize it on first use, and an already imported litellm is
kept as is.
* fix(cli): initialize the SDK on a star import and test command bodies in thin mode
A from litellm import * after lite loaded the CLI used to see only the names the thin package had, so it now runs the full SDK init first. The thin-vs-full parity test now also runs logout and whoami, not just --help
* fix(cli): keep the import hooks until the lazy SDK init finishes
A thread that imported SDK code after the init restored the interpreter
imports could hold that module's import lock while waiting for the init,
and the init then waited on the same lock, so both threads hung. Other
threads now wait for the init before they load any new SDK code
* fix(cli): retry a failed lazy SDK init on the next use instead of leaving it half-run
* test(cli): bind the patched credentials client and autoroute dir in an order-independent way
OSV started flagging claude-agent-sdk 0.1.44 today, which turns OSV Scan red on every PR. 0.2.165 is the newest release outside the 3-day uv cooldown. The package sits only in the ci dependency group and is only imported by the cookbook
* feat(ui): load the Home page What's new launches from /public/whats_new at runtime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the three newest What's new launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build: ship whats_new.json in the wheel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: support OpenAI SDK 3.x while retaining 2.x compatibility
* fix: preserve HTTPX compatibility across OpenAI SDK versions
* refactor: trim OpenAI SDK compatibility patch
* refactor: simplify SDK client defaults
* test: cover SDK compatibility across provider routes
* fix: use shared names for SDK transport helpers
* fix: address SDK compatibility review feedback
* test(openai): send requests through the SDK API factory clients
* fix(exceptions): keep the provider response on PaymentRequiredError under httpx 2
* ci(base-sdk): count httpx2 as base-only when a base dependency brings it in
OpenAI SDK 3 depends on httpx2, so the litellm-core install at highest resolution pulls it in through openai. The base-only check now resolves the declared base dependency closure of litellm and litellm-core and accepts extras-only modules owned by a distribution inside that closure
* test: cover OpenAI SDK 3 client paths end to end and read SDK HTTP headers case-insensitively
* test: read C02 spend rows once, wait out the worker healthcheck on respawn, and resolve module owners on Python 3.10
---------
Co-authored-by: Marcus Wood <marcuswood@openai.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci(lint): derive gate ceilings from merge-base counts and drop the budget files
The four lint gates (ruff strict, type discipline, basedpyright, test quality)
now fail a branch only when a rule's codebase count grows past its count at the
merge-base with litellm_internal_staging plus a fixed per-rule headroom, which
is zero everywhere except the LIT010/LIT011 and reportAny/reportExplicitAny
seeds. Base counts come from a disk cache, then the CI artifact the renamed
publish-lint-base-counts workflow uploads for every staging push (all four
checkers, one artifact per checker and sha), then a scan of the base worktree.
The four *-budget.json files, make lint-budget-update, budget_ratchet_check.py,
and the unratcheted check are gone, so no PR carries a budget edit again.
* test(lint-gates): assert checker identities by behavior and type the new gate tests
* test(lint-gates): resolve base points over an injected git so unit tests stay in-process
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(lens): use async-timeout on python 3.10 for budget reservation timeouts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): keep uv.lock diff to the async-timeout entry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(lens): cover real request deadline expiry in reserved_budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lens): finish Python 3.10 timeout coverage and dependency checks
* test(lens): control event-loop time for deadline regressions
* test(tracing): include priced call count in trace fixture
* fix(lens): limit timeout compatibility changes to PR scope
---------
Co-authored-by: Moe Khalil <moe@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
uv only rebuilds the editable litellm package when its cache keys change, and
the default keys are pyproject.toml, setup.py and setup.cfg. Editing or
switching to a branch with different Rust code left the old
litellm/rust_bridge/_native.abi3.so installed. Key the build on the Rust
toolchain pin, Cargo config, lockfile, workspace manifest and every file
under litellm-rust/crates, since crates embed .sql, .json and .jinja files at
compile time.
Co-authored-by: Nate Armstrong <narmstrong@Nates-MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(proxy): serve a Codex-native model catalog with per-model service_tiers from /v1/models
GET /v1/models and /models answer Codex CLI's catalog fetch (the request
carrying its client_version query parameter) with Codex's own
{"models": [...]} shape: a model Codex knows keeps the metadata of its
bundled 0.159.3 catalog (vendored), any other model gets Codex's fallback
entry, and model_info.service_tiers becomes each entry's service tiers so
Codex offers them as slash commands that send service_tier upstream.
Without the parameter the OpenAI list shape is unchanged. The CLI's
litellm agents codex catalog shares the same builder.
* fix(proxy): offer a Codex service tier only when every deployment of the model lists it
* fix(codex-catalog): an invalid service_tiers value offers no tier for the model
* fix(codex-catalog): read service tiers off the deployments the key's team can route to
A tier is offered to Codex only when every deployment of the model name a
request from the key's team can route to lists it, so another team's
deployment of the name and a deployment an admin paused via model_info.blocked
no longer withhold or add tiers for requests that never reach them
The catalog's always-null fields are annotated NoneType so the module imports
under pydantic 2.12.0 on Python 3.14, the lowest pin the MCP resolve job
installs, which rejects a None annotation with a None default
* test(codex-catalog): drop the redundant module docstring and sort the imports
* test(integration): add the Codex catalog audit cells and the multi-worker convergence note
* test(integration): clean up every catalog test model and answer the refresh GET
* fix(proxy): keep tiered models under Codex's catalog cut and resolve alias tiers
Under Codex's 1 MiB catalog limit the entries offering a service tier are kept
ahead of those offering none, each group in model_list order, with every kept
entry at its listing position, so the model an operator configured tiers for
survives a wide key's long listing. A model_group_alias row reads its target's
deployments, so it carries the target's tiers and stock metadata under the
alias name.
* fix(proxy): pick Codex catalog metadata per team and skip entries too large for the cut
The upstream model that selects Codex's stock entry was read off the first deployment of a name
without checking the key's team, so a team whose requests route to a different deployment could be
handed another team's prompt, reasoning levels, and tiers. The upstream model and the tiers now come
from the same team-aware selection routing uses, and a caller with no team reads the deployments no
team owns
The byte cut kept a prefix of the tier-first order, so one entry larger than the whole limit emptied
the catalog. An entry too large for the bytes left is now passed over and the smaller ones after it
are still kept
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(lens): move analysis prompts into markdown files
* feat(lens): ask the investigator for a scoped agent fix brief with two options
* test(lens): cover the agent fix brief through investigation and merges
* chore(ui): regenerate api types for the lens fix brief
* feat(ui): build copyable lens fix prompts
* feat(ui): show the lens fix brief with copy buttons for claude code and codex
* test(ui): cover copying a lens fix option
* refactor(lens): replace the fix options with a plain issue brief
* feat(lens): ask for problem, user goal, outcome and test cases without prescribing code changes
* test(lens): cover the issue brief through investigation and merges
* chore(ui): regenerate api types for the lens issue brief
* refactor(ui): drop the lens fix prompt builders
* feat(ui): add a lens issue brief panel
* feat(ui): show the lens issue brief in the finding drawer
* test(ui): cover the lens issue brief and the legacy fallback
* feat(ui): render a lens issue brief as a markdown document
* test(ui): pin the lens issue brief markdown layout
* feat(ui): show the issue brief as a copyable file with claude code and codex buttons
* feat(ui): pass the finding title into the issue brief
* test(ui): cover copying the issue brief for claude code and codex
* feat(ui): bold the input and expected labels in lens test cases
* test(ui): pin the bold test case labels in the issue brief
* feat(ui): render the issue brief as formatted markdown
* test(ui): cover the rendered issue brief sections and raw markdown copy
The pytest-postgresql based proxy tests moved to tests/integration on the real
Postgres harness in #43996, so nothing loads the plugin anymore. uv.lock is
edited by hand to drop the package and its now orphaned mirakuru and port-for
deps; uv lock --check passes and a full relock resolves the same package set
* test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stub HIBP through respx by disabling the aiohttp transport
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): share the httpx transport fixture across proxy unit tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): restore proxy globals without a missing-value sentinel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): package moved dirs and stub the login breach check at the HTTP boundary
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): isolate the mcp server manager per test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): move management_endpoints, management_helpers and guardrails tests into tests/unit/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): reuse the shared httpx transport fixture in moved proxy tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stub outbound HTTP and package moved test dirs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): restore the config server hostname in the mcp resolution test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin the completion tokenizer model in the straiker screening test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python
pip install litellm fails on Microsoft Store Python because its user
site-packages is already 134 chars plus the profile name, and the content
filter guardrail ships YAML five directories deep under
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/. The
existing wheel guard assumed a 100-char install prefix, so it never saw it.
Move categories/ and policy_templates/ to
litellm/proxy/guardrails/content_filter_data/ and drop the benchmark
fixtures from the wheel. Old category_file paths keep resolving because
the resolver only keys on the trailing categories/<file> or
policy_templates/<file> suffix.
Derive the guard's worst-case prefix from the Store Python site-packages
path with a 15-char profile name (149), fail files at 260 and directories
at 248 (CreateDirectoryW), and fix the off-by-one that let a 260-char
path through.
Fixes#43851
* ci: run the Windows wheel install guard on pull requests
The two Windows jobs live in CircleCI, which never runs on pull requests,
so nothing installs the wheel on Windows before merge. Add a GitHub Actions
job on windows-latest that builds the wheel and runs the guard.
Two things make the run deterministic instead of image dependent. The job
turns the LongPathsEnabled registry key off first, because runner images
ship with it on and python.exe is long-path aware, so a 300-char path would
install fine. The guard installs with pip instead of uv, because uv writes
files from Rust, which switches to extended-length paths on its own and can
never hit MAX_PATH.
* fix(guardrails): keep the old content filter package dir as a category search root
Deployments that copied their own category YAML into
guardrail_hooks/litellm_content_filter/ before the data move would have had
that file rejected by the new directory jail and missing from by-name loads,
inherit_from lookups, the UI category listing and the category YAML endpoint.
Every lookup now searches the bundled data dir first and the old package dir
second, with the bundled copy winning on a name clash.
* fix(guardrails): resolve category files through safe_join
By-name category lookups and the suffix search in the category_file resolver now go through safe_join, so a name or suffix that would escape its data root never reaches the filesystem. The LITELLM_CONTENT_FILTER_ALLOW_EXTERNAL_PATHS opt-out keeps its unjailed search. Clears the two CodeQL path-injection findings on the new lookup code.
* fix(guardrails): keep symlinked category files loadable by name
By-name category lookups resolved symlinks through safe_join, so a category file symlinked into the categories folder from elsewhere stopped loading. Those lookups now only reject names that leave the folder lexically and return the link untouched, matching how by-name loads behaved before the data move. The category_file resolver keeps its realpath jail as before.
* fix(guardrails): keep the category viewer inside the category folders
GET /guardrails/ui/category_yaml/{name} hands raw file contents to any valid key, and on main it refused a symlink whose target left the categories folder. The previous commit let by-name lookups follow symlinks again, which also let the viewer read whatever a symlink in a legacy categories folder pointed at. The viewer now checks the found file's real path against every categories folder it searches and answers 400 as before, while the guardrail's own by-name loads keep following symlinks
The roots come in through a FastAPI dependency so the check is testable against a temp folder, and the content filter's realpath containment moves to path_utils.is_within so both surfaces share it. The test that patched os.path.commonpath covered a branch that no longer exists and goes with it
* ci: drop the Windows wheel install job from pull requests
The job took about 13 minutes on every PR to guard an edge case. The
guard still runs its path-length check on Linux in base_sdk_install and
on Windows in the CircleCI windows_release_wheel job.
* feat(proxy): add model leaderboard analytics
* feat: add model insights task and range constants
* feat: record task type from task tags in model usage rollup
* feat: serve 365 days of model insights by UTC date
* test: cover task tag resolution in model usage rollup
* test: update model insights range limit test to 365 days
* chore: regenerate dashboard api types for model insights
* feat: add model insights aggregation helpers
* test: cover model insights aggregation helpers
* feat: redesign model leaderboard with stacked bars, treemap and ranking
* test: update model leaderboard view test
* feat: mark model leaderboard as beta in sidebar
* chore: sync schema.prisma copies from root
* fix: only treat task: prefixed tags as model insight tasks
* feat: add metric type for model insights ranking
* fix: rank model insights by selected metric and scope detail queries to ranked deployments
* test: plain tags are not model insight tasks
* test: cover metric ranking, deployment scoping and rollup round trip
* fix: build model insights weeks and halves from the requested date range
* test: cover empty weeks and range-based change comparison
* fix: refetch by metric, show load errors and ignore stale responses
* test: cover metric refetch and error state
* feat: define model insight tasks in a JSON file
* feat: return task labels and categories from model insights
* feat: load model insight tasks from JSON
* refactor: validate rollup task tags against the JSON task list
* feat: serve the task list with model insights
* refactor: drop hardcoded task list from constants
* build: ship model insight tasks JSON in the wheel
* test: cover model insight task JSON
* refactor: take task labels and categories from the API
* test: pass task info to task tile builder
* refactor: color treemap by API-provided category
* test: include tasks in model leaderboard fixture
* fix: make daily model usage migration idempotent
* feat: bound the model insights task query size
* fix: compute task breakdown independent of the chart metric
* test: task breakdown is stable across chart metrics
* chore: regenerate lazy openapi snapshot for model insights
* chore: regenerate dashboard api types for model insights
* fix: keep previous ranking dimmed while a new metric loads
* test: cover stale metric state in model leaderboard
* refactor: drop task row cap constant
* fix: return the full task breakdown instead of a truncated one
* test: task query is not truncated
* feat: add task summary types for model insights
* feat: summarise tasks server-side on a separate model insights endpoint
* test: cover the model insights tasks endpoint
* chore: regenerate lazy openapi snapshot for model insights tasks
* chore: regenerate dashboard api types for model insights tasks
* refactor: drop client-side task aggregation
* test: remove client-side task aggregation tests
* feat: load task breakdown separately from the chart metric
* test: task breakdown is not refetched on chart metric change
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
