mutmut's gather_coverage() looks each source file's covered lines up by
absolute path, but [tool.coverage.run] sets relative_files = true, so every
lookup misses. With mutate_only_covered_lines = true that leaves no line
eligible for mutation, and the run ends on "Stopping early, because we could
not find any test case for any mutant" after spending 26 minutes collecting
coverage. The last four dispatches all died that way.
Point COVERAGE_RCFILE at a small rc file for mutation runs only, so the
coverage instance mutmut builds stores absolute paths. Scoped to one module
locally this takes the run from 0 mutants to 8 generated and 8 killed.
Also give the mutmut step a deadline inside the job's own. mutmut records
each mutant's verdict to mutants/mutmut-stats.json as it finishes, so a run
that outlasts its budget still scores what it got through, but a cancelled
job skips the report and upload steps and publishes nothing. That is how the
two runs before these four ended.
Ignore mutants/ and .venv-mutmut, which a local run leaves behind untracked.
E2E_FIXTURE_MODE selects the transport every e2e client is built on: live
(default, unchanged behavior), record (pass through to the live proxy while
writing every interaction to a fixture bundle), or replay (serve every
interaction from the bundle with no proxy and no provider spend). Both new
transports fulfil the existing Transport protocol, so no test changes shape.
A bundle is a directory with a manifest (record timestamp, harness version,
format version) and one JSON file per interaction, grouped per test in call
order. Replay against a manifest older than seven days hard-fails at
collection time naming the bundle age. Record always wipes and never reads
the previous bundle, refusing to wipe a directory that is not a bundle.
Auth header values are redacted on write; uploads store a sha256 digest.
unique_marker() becomes deterministic per test in record/replay modes so a
replay run regenerates exactly the requests the record run sent.
Content-based match keys, streaming chunk fidelity, and provider-scoping are
follow-ups (LIT-5741, LIT-5742, LIT-5745).
The gate previously measured whatever environment the caller happened to
have. Locally that is the fat bootstrap venv (--extra proxy pulls in
fastapi-sso, whose type info flips a reportUnnecessaryIsInstance
diagnostic in ui_sso.py), while CI's publisher venv only has the
proxy-dev and e2e-dev groups, so identical trees measured 866 locally vs
865 in CI and every local gate run breached by a phantom +1
scripts/type_check_gate.py now provisions .venv-typecheck itself: a
frozen uv sync of the canonical proxy-dev and e2e-dev groups, the
interpreter pinned to pyrightconfig.json's pythonVersion, plus the
generated Prisma client. Every measurement pass is pinned to that env
with --pythonpath, because basedpyright auto-detects a .venv in the
project root and that auto-detection beats both PATH order and
VIRTUAL_ENV, so the CLI flag is the only pin that actually works. The
dependency-group set is folded into the environment fingerprint, so
artifacts or caches recorded under a different group set never match
and the gate falls back to computing base counts locally instead of
comparing mismatched environments
The publisher workflow drops its own install and prisma steps and lets
the script build the measurement env, and the node heap for the
full-tree pass drops from 12GB to 8GB (peak RSS measured at 5.4GB)
* feat(ui): chart failed requests as their own series on the cache dashboard
Spend logs for failed requests are stored with an empty call_type, so the
Cache Hits vs API Requests chart lumped them into an Unknown bar that read
as normal LLM API traffic. The activity query now also returns a per-group
failed_rows count (status = 'failure') and the dashboard charts it as a
third stacked series, so failures are visibly separate from successful
requests and cache hits. The chart data transform moves into a pure
summarizeCacheActivity helper with unit tests; header stats keep their
existing semantics (cache hit ratio still counts failures in the
denominator).
* refactor(ui): move cache dashboard aggregation server-side with a typed response
The /global/activity/cache_hits endpoint previously returned raw per
(key, call_type, model) spend-log aggregates typed as LiteLLM_SpendLogs
(wrong), and the dashboard reduced them in the browser: grouping by
call_type, relabeling empty call_type as Unknown, and computing the stat
card totals. All of that now happens server-side. The SQL groups per
call_type and splits cache hits vs successful vs failed requests, a new
cache_activity module validates rows into Pydantic models and computes
totals plus the key-alias/model filter options, and the endpoint declares
a real response_model so schema.d.ts types it correctly. The dashboard
consumes it through a typed $api react-query hook (filters ride the
query key and are applied in SQL instead of the browser), the hand-rolled
summarizeCacheActivity transform and the adminGlobalCacheActivity fetch
helper are deleted, and the refresh button now actually refetches.
The endpoint is UI-internal (hidden from the public swagger), so the
response reshape is not a public API break.
dist/litellm-1.79.1.tar.gz (a 64-byte build artifact) was committed by
mistake. Release CI wipes dist/ before building, so it never affected
published artifacts, but it doesn't belong in version control. Add dist/
to .gitignore to prevent a repeat.
* test(claude_code): move the Claude Code compatibility matrix under tests/e2e
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(claude_code): drop the CircleCI compat PR gate; the matrix runs in the scheduled e2e suite instead
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: restore the upload-coverage job dropped by mistake with the compat gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(e2e/claude_code): print rate-limit summary on failed compat runs and fix stale run_daily.sh header comments
* test(claude_code): assert fine-grained tool streaming via input_json_delta instead of an event-count floor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix
Upgrade the dashboard from Tailwind v3 to v4 with CSS-first config: the
official upgrade codemod renamed utilities across 151 files, and
tailwind.config.js (plus the dead tailwind.config.ts) is replaced by
@theme tokens, @source globs, and @plugin directives in globals.css. The
Tremor safelist becomes @source inline patterns and the legacy tremor
theme tokens carry over verbatim. ui_colors.json was build-time only and
fed the dying Tremor palette, so its brand values are inlined and the
file removed; runtime theming replaces that path next.
shadcn is initialized with a hand-authored components.json (rsc,
cssVariables, baseColor gray) pointing utils at the existing
lib/cva.config.ts, which now exports cn (cva beta cx + twMerge) instead
of adding class-variance-authority as a second variant library. The two
ad-hoc cn helpers fold into it. Button lands as the canary primitive,
adapted to cva beta and React 18 forwardRef, with tests covering the
variant, twMerge, asChild, and ref seams. --radius is 0.5rem so the
shadcn radius scale reproduces Tailwind defaults and legacy rounded-*
classes render unchanged.
antd v5 emits unlayered CSS-in-JS that would beat every layered v4
utility, so AntdGlobalProvider now wraps the app in StyleProvider layer
and ConfigProvider cssVar, and globals.css declares
@layer theme, base, antd, components, utilities. antd wins over
preflight but yields to utilities, which is what lets migrated shadcn
pages coexist with legacy antd pages. Preflight stays global with the
three v3 behaviors pinned (default border color, button cursor,
placeholder color).
* fix(ui): restore tremor opacity tints removed by tailwind v4
Tailwind v4 removed the *-opacity-* utilities, but the precompiled
@tremor/react dist still composes them with shade-500 palette classes
(bg-opacity-10 over bg-<color>-500 etc.), so Badge, BadgeDelta, Callout,
light Icon and Button, BarList, and ProgressBar lost their tints and
rendered solid 500-shade fills. Adversarial review caught it; the
original smoke pages only exercised antd Tags.
tremor-v3-compat.css restores exactly the pairs tremor emits: for each
of the 22 safelisted colors, bg-opacity-{10,20,40}, hover/group-hover
bg-opacity-{20,30}, and ring-opacity-{20,40} against the -500 shade,
via color-mix into the utilities layer. Tremor's colorPalette maps both
background and iconRing to 500, so the -500 pairing covers every
composition in the dist; dark: variants are inert until dark mode ships.
The shim dies with @tremor/react at the end of the migration.
The upgrade codemod also missed two hand-rolled modal scrims using
bg-black bg-opacity-{30,50} (solid black under v4); now bg-black/30 and
bg-black/50. Removed the docker/build_admin_ui.sh copy of
enterprise_colors.json into the deleted ui_colors.json; that build-time
rebrand path is retired and its runtime replacement lands with the
theming phase.
* fix(ui): pair ring-opacity-40 with shade 300 in tremor compat shim
Tremor's colorPalette maps ring to shade 300, and the only consumer of
ring-opacity-40 (Icon variant outlined) composes it with that shade,
so the shade-500 rows were dead and outlined icon rings would render
at full opacity. Latent today (no dashboard usage of the outlined
variant); caught by adversarial review. ring-opacity-20 stays at 500
(iconRing), matching Badge and BadgeDelta.
* chore: remove _experimental/out
* fix(ci): recreate _experimental/out before copying UI build output
The build scripts cp the Next.js output into litellm/proxy/_experimental/out,
which was removed from git. cp failed because the target directory no longer
existed; mkdir -p recreates it before the copy.
* fix(proxy): make UI serving resilient to a missing _experimental/out
Removing the committed UI export means the source/test tree no longer
ships litellm/proxy/_experimental/out. Three things assumed it was always
present and broke once it was gone:
- get_favicon hard-coded the built favicon path and 404'd without it; it
now falls back to the bundled swagger/favicon.ico
- the /_next and /ui static mounts raised at construction when the export
was absent, so the whole UI-setup block was swallowed and no mounts
registered; they now use check_dir=False
- _restructure_ui_html_files was a nested function only exposed as a
module attribute when that block happened to succeed; it is now a real
module-level function
test_admin_ui_export_serves_nested_extensionless_routes validated the
committed artifact, whose premise this PR removes; it now drives the same
MCP OAuth callback restructure guarantee through a synthetic export.
* chore(greptile): ignore generated _experimental/out so review fits the file limit
* Revert "chore(greptile): ignore generated _experimental/out so review fits the file limit"
ignorePatterns is applied after Greptile counts the files changed, so it
does not bring the diff under the file limit; the config had no effect.
Ignore the compiled, platform-specific Rust extension output (litellm/rust_bridge/_native*.so/.pyd) and the litellm-rust/target/ build dir so local maturin/cargo builds don't show up as untracked files.
Also drop the two stale self-referential .gitignore entries; .gitignore is tracked, so ignoring it did nothing except add confusion.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* feat(proxy): add logging_endpoints package init
* feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through the success/failure callback fan-out
* feat(proxy): register callback_logs_router
* test(proxy): add logging_endpoints test package init
* test(proxy): cover /v1/callbacks/logs replay, admin guard, and partial-failure handling
* refactor(proxy): move callback-logs request/response models to litellm/types/proxy
* refactor(proxy): wrap callback-logs replay in CallbackLogsReplayer class with payload logging
* test(proxy): update callback-logs tests for class-based replayer and separated types
* fix(proxy): cover /v1/callbacks/ in backend component allowlist
The new /v1/callbacks/logs route was dropped by both component
allowlists, failing test_gateway_plus_backend_covers_full_app. It's an
admin-only spend-logging route, so it belongs on the backend (control
plane) alongside the existing /callbacks family.
* refactor(proxy): use builtin dict/list generics in callback-logs endpoint
Switch Dict/List from typing to builtin dict/list to satisfy the ruff
strict-rule budget (UP006).
* refactor(proxy): use builtin dict/list generics in callback-logs types
UP006: builtin generics over typing.Dict/List.
* chore(ui): regenerate schema.d.ts for /v1/callbacks/logs
Run npm run gen:api to add the CallbackLogRecord/CallbackLogsRequest/
CallbackLogsResponse types and the /v1/callbacks/logs path, keeping the
dashboard types in sync with the proxy OpenAPI spec.
* fix(proxy): force stream=False when replaying callback logs
A replayed StandardLoggingPayload is a terminal, fully-aggregated event —
the producer (e.g. the rust realtime gateway) already collected the whole
session before POSTing. Marking the rebuilt Logging object as streaming made
async_success_handler wait for a complete_streaming_response that never
arrives, so the spend log was never written. Realtime sessions now land in
LiteLLM_SpendLogs.
* feat(litellm-rust): CustomLogger callback layer posting to /v1/callbacks/logs
integrations/ mirrors litellm/integrations/: a sync, typed CustomLogger trait
(base contract), a typed StandardLoggingPayload, and LiteLLMPythonProxyAPILogger
— the first concrete logger, owning a bounded channel + background worker that
batches and POSTs to the Python proxy's /v1/callbacks/logs.
* feat(litellm-rust): RealTimeStreaming per-session log collector
1:1 with Python's RealTimeStreaming: observe() accumulates O(1) usage/model/id
per event (never buffers frames); log_messages() builds one StandardLoggingPayload
on session close and fans out to the CustomLogger callbacks. request_id == the
OpenAI realtime session id (sess_…), with the gateway id as fallback.
* feat(litellm-rust): wire realtime logging into the splice (lock-free observe)
The collector is owned on the splice task and observed via a synchronous &mut
callback threaded through providers::realtime::realtime() — no Arc/Mutex/atomic
on the per-frame hot path. On session close the bridge flushes one payload.
AppState carries the registered loggers; main spawns the proxy logger.
* docs(litellm-rust): ai-gateway realtime logging architecture
* docs(litellm-rust): document request-log egress to the LiteLLM control plane
Add a 'Request logging' guide to the ai-gateway README: how to point the gateway
at a LiteLLM proxy via LITELLM_PROXY_BASE_URL (+ LITELLM_MASTER_KEY for the
admin-only /v1/callbacks/logs POST), and the non-blocking / one-payload-per-session
behavior.
* feat(litellm-rust): make log-egress tunables env-overridable
Channel capacity, batch size, and flush interval now read from
LITELLM_LOG_CHANNEL_CAPACITY / LITELLM_LOG_BATCH_SIZE / LITELLM_LOG_FLUSH_INTERVAL_MS,
falling back to the DEFAULT_* consts on missing/invalid/non-positive values.
Grouped behind an EgressTunables::from_env() read once at logger construction.
* docs(litellm-rust): document log-egress tuning env vars
* docs(litellm-rust): require constants in a crate-level constants.rs
Mirror of Python's litellm/constants.py rule — magic numbers and fixed strings
go in src/constants.rs, not inline in feature modules; env-overridable tunables
keep their DEFAULT_* value there.
* refactor(litellm-rust): move ai-gateway constants into constants.rs
Per the new rule: the log-egress defaults (proxy base, ingest path, channel
capacity, batch size, flush interval) and the realtime provider default move to
crates/ai-gateway/src/constants.rs; modules import from it.
* ci: run logging_endpoints tests in the proxy-infra coverage shard
tests/test_litellm/proxy/logging_endpoints wasn't in any coverage-uploading
job, so callback_logs_endpoints.py showed only import-level coverage (~35%) on
codecov/patch despite being ~98% covered locally. Add it to proxy-infra's
test-path so the test is exercised under --cov.
* fix(litellm-rust): hash the master key before logging — never send the raw credential
Greptile/Veria P1: user_api_key_hash was the plaintext LITELLM_MASTER_KEY, which
fans out to spend logs and every callback (Langfuse/Datadog) and could be
recovered from logs. SHA-256 it (auth::hash_token, matching the proxy's
hash_token); the field is named *_hash and the proxy stores it verbatim when it
isn't sk-prefixed, so the DB value is identical with zero plaintext exposure.
* fix(litellm-rust): observe realtime logging on upstream events only
Greptile P1: observe ran on the client->upstream arm too, so an authenticated
client could send a fabricated response.done and inflate its own spend log.
session.created/response.done are server->client events; observe the upstream
arm only.
* feat(proxy): bound callback-logs batch + return per-record failures
Greptile P2: cap /v1/callbacks/logs at MAX_CALLBACK_LOG_RECORDS (default 1000,
env-overridable) so one POST can't trigger an unbounded callback/DB fan-out; and
return per-record {index, error} failures so a caller (the rust gateway) can
distinguish a transient callback error from a structurally bad payload.
* chore(ui): regenerate schema.d.ts for CallbackLogFailure / failures field
* fix(constants): make MAX_CALLBACK_LOG_RECORDS a plain constant
It doesn't need to be env-configurable (only the rust egress tunables are). As an
os.getenv var it tripped tests/documentation_tests/test_env_keys.py, which requires
every env key to be documented in the (separate-repo) config_settings.md. Plain
constant → not scanned → code-quality + documentation checks pass.
* docs(litellm-rust): trim ai-gateway ARCHITECTURE.md to one diagram + notes
* docs(litellm-rust): tighten the README request-logging section
* docs(litellm-rust): ARCHITECTURE.md is just the diagram (gateway = inference, spend = callback)
* docs(litellm-rust): drop em-dashes from the request-logging section
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* ci: drop redundant mypy type-check gate, standardize on basedpyright
Type checking ran both mypy (via the pydantic.mypy plugin) and basedpyright.
pydantic v2 emits dataclass_transform, so basedpyright understands models
natively with no plugin, and its gated rules already cover what the mypy pass
caught (no-untyped-def, no-any-return, valid-type, import-not-found all map to
basedpyright equivalents). Running both meant two checkers, two budgets, and a
plugin only mypy could load.
This removes the mypy type-check gate: the lint-mypy/lint-mypy-budget-update
Makefile targets, the CI MyPy step, mypy-code-budget.json, the budget-ratchet
entry, and the vestigial [tool.mypy] pydantic plugin block (the gating pass used
litellm/mypy.ini, which never loaded the plugin). type_check_gate.py is
specialized to basedpyright since the mypy parsing path is now unused.
mypy stays a dev dependency because the Any-discipline gate
(scripts/check_any_discipline.py) imports it as a library to detect Any-typed
values; it is no longer run as a type checker.
* ci: remove the Any-discipline gate, rely on basedpyright's reportAny
The Any-discipline gate (scripts/check_any_discipline.py) was the last consumer
of mypy: it imported mypy as a library to detect values whose inferred type
contains Any, gated per-file against any-discipline-budget.json. basedpyright
already reports the same class of finding through reportAny/reportExplicitAny,
which are gated tree-wide in basedpyright-code-budget.json, so the separate gate
(and the mypy dependency behind it) is redundant.
Removes the gate end to end: check_any_discipline.py and its test, the
any-discipline CI job, the lint-any/lint-any-budget-update Makefile targets,
any-discipline-budget.json, litellm/mypy.ini, the .mypy_cache_any references,
and mypy from the dev dependencies. budget_ratchet_check.py drops the
any-discipline entry and the now-unused zero-floor mechanism (rewritten as a
comprehension). check_type_discipline.py drops the any-ok suppression token,
since # any-ok suppressed only the deleted gate; the 134 now-orphaned
# any-ok comments across 14 files are stripped (they never affected
basedpyright, which uses # pyright: ignore).
uv.lock is intentionally left untouched: uv still considers it consistent with
the mypy-removed pyproject (uv lock --check and uv sync --frozen both pass), and
a relock bumps 30+ unrelated packages because of the moving exclude-newer window.
A future intentional relock will prune the now-unreferenced mypy entry.
* build: relock to drop mypy from uv.lock
CI's uv 0.10.9 honors the repo's exclude-newer window and correctly flags the
lockfile as out of sync once mypy leaves pyproject; my earlier local uv 0.8.17
could not parse exclude-newer and silently passed --check. Relocking with the
pinned CI version removes only mypy and its transitive librt, with no other
version changes.
* ci: enable ruff preview rules under the budgeted strict gate
Turn on ruff preview in the strict-budget lane (ruff-strict.toml) only,
leaving the clean gate (ruff.toml) untouched so make lint-ruff stays at
zero. Enumerate the 118 firing codes explicitly with
explicit-preview-rules so the gate is deterministic and stable across
ruff upgrades rather than depending on preview auto-selecting the broad
catalog.
Grandfather the existing 58438 violations into ruff-strict-budget.json
as per-rule baselines with headroom, so only net-new violations fail CI.
The existing ten rules keep their hand-tuned slack; the new rules get
slack 10 when the baseline is 50 or more and 3 otherwise.
* ci: add ANN return-type rules to the budgeted strict gate
Add ANN201/202/204/205/206 (missing return annotations) to the strict
lane and grandfather the existing counts into ruff-strict-budget.json so
the codebase ratchets toward explicit return types without breaking CI.
* ci: add mypy (disallow_untyped_defs) and basedpyright strict gates with baselines
Add two type-check gates, each grandfathering the current tree so only
net-new violations fail CI, matching the ruff strict-budget ratchet.
mypy gains disallow_untyped_defs in litellm/mypy.ini (the config the CI
invocation actually reads; the root [tool.mypy] is not picked up from the
litellm/ working dir). The 4885 existing missing-annotation errors are
captured in litellm/.mypy-baseline.txt and the run is piped through
mypy-baseline filter so new untyped defs are rejected.
basedpyright runs in strict mode over litellm/, with
enableTypeIgnoreComments disabled so it only honors '# pyright: ignore'
and never polices mypy's '# type: ignore'. The existing strict diagnostics
are grandfathered into .basedpyright/baseline.json.
Both tools are pinned in the dev group and uv.lock; the lint workflow and
Makefile run them filtered through their baselines, with
lint-mypy-baseline-update and lint-basedpyright-baseline-update to ratchet.
* ci: raise lint job timeout to 15m for the basedpyright strict pass
* ci: pin pythonVersion 3.12 and regenerate baselines against merged base
Merge litellm_internal_staging so the baselines cover code the CI merge
includes (e.g. the cisco_ai_defense guardrail), which otherwise tripped
the mypy gate with 3 ungrandfathered no-untyped-def errors. Pin
pythonVersion 3.12 in pyrightconfig so basedpyright's strict analysis is
reproducible across interpreter versions (CI runs 3.12).
* ci: regenerate basedpyright baseline against the frozen lint env
The previous baseline was generated with optional provider deps (azure,
google, anthropic, mcp, numpydoc, google-genai) installed locally, so CI's
dev-only env surfaced ~3500 reportUnknown*/reportMissingTypeStubs errors
not in the baseline. Regenerate after uv sync --frozen so the baseline
reflects the same dependency set the lint job sees.
* ci: regenerate basedpyright baseline on python 3.12 frozen env
The prior baseline still carried proxy-dev packages (e.g. prisma) that the
lint job's dev-only, python 3.12 env lacks, leaving 2 unresolved-import
errors ungrandfathered. Regenerate in a python 3.12 venv synced to the
frozen lock with default groups only, so the baseline matches exactly what
CI sees.
* ci: replace type-check baselines with per-file count budgets
The mypy and basedpyright baselines were position-sensitive (and the
basedpyright one was a 27MB file), so ordinary line shifts churned them.
Replace both with a per-file count gate: scripts/type_check_gate.py reduces
each tool's output to errors-per-file and checks it against a committed
{file: max} budget, ignoring line and column numbers. A file fails only
when it gains more errors than its ceiling; debt can't be shuffled between
files because each file has its own cap and new files default to zero.
Budgets (mypy-file-budget.json 48K, basedpyright-file-budget.json 96K) are
generated in the python 3.12 frozen lint env so they match CI. Drops the
mypy-baseline dependency; basedpyright runs without its native baseline.
ratchet via make lint-mypy-budget-update / lint-basedpyright-budget-update.
* ci: add a small per-file slack to the type-check gate
Allow each file to drift PER_FILE_SLACK (5) errors past its recorded count
before failing, so a basedpyright inference ripple in an unrelated file
doesn't break the build over a couple of errors. Budgets still record exact
counts; the tolerance is applied at check time.
* ci: move type-check slack into the budget json and trim lint timeout
Make slack declarative: the budget is now {"slack": N, "files": {path: count}}
so the tolerance is tuned in JSON without editing the script, mirroring how
ruff-strict-budget.json carries its slack. --update preserves the existing
slack. Also drop the lint job timeout from 15m to 10m; the mypy and
basedpyright passes add ~2m, leaving the job around 4-5m, so 10m is a
comfortable margin.
* ci: collapse fully-adopted ruff categories and drop inert preview flag
ANN (all nine non-removed rules) and BLE (its only rule) were spelled out
code-by-code; replace each with its category selector, which is exactly
equivalent in 0.15.3 (the removed ANN101/ANN102 are skipped by a category
selector and error when named explicitly). explicit-preview-rules was inert:
every selected rule is stable and nothing is selected by category, so the flag
had nothing to gate. Verified the strict-rule counts are identical before and
after (62379 each, zero per-rule drift), so no budget change.
* ci: drop redundant pyright dev dependency
Nothing invokes bare pyright in the Makefile, the linting workflow, or
scripts; the basedpyright gate added on this branch is the only type
checker that runs. basedpyright is a superset fork that reads the same
pyrightconfig.json and honors the same "# pyright: ignore" comments, so
pyright==1.1.408 in the ci group was dead weight. Regenerated uv.lock
under the same exclude-newer cutoff so the only change is removing
pyright and its package stanza
* ci: un-weaken mypy and error on Any in basedpyright
mypy: enable warn_return_any, drop the valid-type silencer, and stop globally ignoring missing first-party imports via [mypy-litellm.*] ignore_missing_imports = False, which surfaced eight real broken litellm.* imports the blanket ignore was hiding; third-party imports stay ignored. The per-file budget moves 4888 -> 5799 (902 no-any-return, 1 valid-type, 8 import-not-found), all grandfathered so only net-new errors fail and the ceilings ratchet down
basedpyright: error on reportExplicitAny and reportAny. The per-file budget moves 117033 -> 148946 (6931 explicit-Any, 24954 Any-typed expressions), grandfathered the same way
* ci: add Any-discipline gate on changed lines under litellm/
Add scripts/check_any_discipline.py, a type-aware gate that fails when a
changed line holds a value typed Any -- including the X | Any unions that
mypy --strict / basedpyright accept (e.g. re.Match.group() -> str | Any,
json.loads() -> Any, bare dict -> dict[Any, Any]).
It reuses the repo's mypyc-compiled mypy 1.19 via a custom generic AST
walker (mypyc precludes subclassing TraverserVisitor), loads litellm/mypy.ini
for parity with lint-mypy, and uses a dedicated incremental cache
(.mypy_cache_any) with mtime+hash invalidation to force re-checks. Scope is
changed-lines-only so editing a legacy file never forces cleaning its
existing Any debt; suppress a genuine typed/untyped boundary with
# any-ok: <reason> (ANY002 requires the reason).
Wire it into the Makefile (lint-any, lint, lint-dev), a parallel
any-discipline CI job with its own actions/cache, .gitignore, and the
CLAUDE.md / CONTRIBUTING.md docs.
* ci: move Any-gate codes into the shared LIT namespace
Renumber the Any-discipline checker into the LIT*** scheme owned by
scripts/check_type_discipline.py (PR #30500) so the two checkers share one
rule namespace and suppression convention:
ANY001 -> LIT002 (Any-typed value; LIT002 was the retired/free slot)
ANY002 -> LIT005 (any-ok without a reason; the shared suppression-reason code)
ANY000 -> LIT000 (setup/build/read error; the shared error code)
Messages and behavior are unchanged; LIT005's text already matches the
"<token> requires a reason" shape used for cast-ok/guard-ok.
* ci: gate mypy and basedpyright per error rule, not per file
Switch the mypy/basedpyright budget gate from per-file error counts to
per-rule-code totals, mirroring the {rule: {baseline, slack}} shape of
ruff-strict-budget.json. A rule fails when its codebase-wide error count
exceeds baseline + slack, so violations are tracked by category rather
than by file location.
scripts/type_check_gate.py now parses mypy from its text output (trailing
[code]) and basedpyright from --outputjson (the JSON `rule` field), since
basedpyright's wrapped text diagnostics mis-attribute the rule on
continuation lines. Replace the *-file-budget.json files with freshly
captured *-code-budget.json baselines and update the Makefile, CI, and
CLAUDE.md accordingly.
* docs: prefer Pydantic validation over any-ok suppression
Point the Any-discipline guidance at validating Any with Pydantic (a model
or TypeAdapter that returns a typed value or raises) and frame
# any-ok as a last resort that should ideally never be used.
* chore: remove extraneous comment
* chore: make the CLAUDE.md more concise
* chore: clean up bloated CONTRIBUTING.md additions
* chore: make Makefile more concise
* ci: add the lint-budget-update target CLAUDE.md references
CLAUDE.md tells contributors to run make lint-budget-update, but the
target was never defined. Add it as an aggregate that re-captures the
ruff, mypy, and basedpyright budgets in one shot.
* ci: recapture mypy and basedpyright budgets in the lint env
The per-rule baselines were captured in a richer dependency env than the
CI lint job's uv sync --frozen, so CI resolved fewer types and reported
more errors than the budgets allowed (no-any-return 902 over cap 900, plus
several basedpyright reportUnknown* rules). Regenerate both in the frozen
env so they grandfather the true CI debt: mypy 5786 -> 5799 (no-any-return
890 -> 902, valid-type 1 restored), basedpyright 146213 -> 148942.
* ci: check out PR head sha in lint and any-discipline jobs
The default pull_request checkout uses refs/pull/N/merge, which folds the
latest base commits into HEAD. The diff-based gates (ruff delta, Any
discipline) then diff against the event's older base.sha and blame base's
own new commits on this branch; staging's otel-v2 and streaming changes
(#30326, #30485) tripped the Any gate on files this branch never touched.
Checking out the PR head sha makes the gates diff the real branch tip
against base, and pins the tree the mypy/basedpyright budgets were captured
against so their counts stay deterministic as the base advances.
* ci(lint): renumber Any-typed-value rule LIT002 -> LIT009
Free up LIT002 for the sibling type-discipline gate (check_type_discipline.py,
#30500), which groups its mutable-collection family at LIT001 (annotation) and
LIT002 (construction). This gate's Any-typed-value rule moves to LIT009 so the
shared LIT namespace stays contiguous with no holes; LIT000 and LIT005 are
unchanged.
* style: rename lint-strict-budget -> lint-ruff-budget
* ci: harden type-check gates against silent passes (greptile review)
type_check_gate.py: refuse to certify a vacuous run. The CI pipe swallows
the tool's exit code ('tool || true'), so a crashed mypy/basedpyright that
emits nothing would parse to zero errors, breach no ceiling, and pass.
is_vacuous_run() now fails when nothing was parsed but the budget expects
errors. Also wrap basedpyright's json.loads in a JSONDecodeError handler
that prints the offending output instead of dumping a raw traceback.
check_any_discipline.py: ALL_LINES was None, which dict.get() also returns
for a path absent from the line map, so a path-normalisation mismatch could
let a violation on an unchanged file pass the scope filter. Make ALL_LINES a
distinct sentinel object so 'whole file' and 'path missing' are unambiguous.
Adds tests for all three.
* test(proxy/proxy_server): pin forwarding routes (PR2) (#28887)
* test(proxy): pin proxy_server.py forwarding-route behavior
PR2 of the proxy_server.py behavior-pinning project: fills the 12
forwarding-route test files added by the harness PR with happy + error
pins for all 52 LLM-facing routes (models, chat/completions, completions,
embeddings, moderations, audio, assistants, threads, utils, model-info,
model-metrics, queue). Every happy-path test asserts the full response
dict via normalize() so the gate enforces real shape pinning rather
than status codes.
* test(proxy): drop task-plumbing comments from PR2 test files
* test(proxy): tighten PR2 error-path status-code pins
Apply the same review feedback Greptile gave on PR1 (#28856) and PR3
(#28850) to PR2's forwarding-route tests:
- Replace permissive `>= 400` / `in (X, Y)` status assertions with the
exact 500/405 the handler actually returns, so a regression that
silently shifts the code now fails the pin.
- Add a body-presence check alongside each tightened status assertion
to satisfy _pin_check.py's no-status-only rule.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* test(proxy): pin proxy_server.py non-route surface behavior (PR1) (#28856)
* test(proxy): pin proxy_server.py non-route surface behavior (PR1)
Fills the 7 PR1 placeholder files under tests/test_litellm/proxy/proxy_server/
with behavior pins for the non-route surface of proxy_server.py:
lifecycle/init/shutdown, ProxyConfig class methods, DB-overlay config scrubbers,
spend counters, background-health helpers, OpenAPI customization, exception
handlers, and streaming-generator helpers.
233 tests cover 101 pin-list symbols (1+ happy + 1+ error each). New-tests-only
coverage on litellm/proxy/proxy_server.py: 32.80% line / 20.91% branch (PR1
gate: 25% line / 18% branch). Full directory runs in ~22s with -n 4.
Plan: https://www.notion.so/Plan-Pin-proxy_server-py-behavior-2026-05-25-36c43b8acdab81ee845fd5365128a2fc
* test(proxy): address Greptile review comments on test_lifecycle.py
- test_initialize_signature_is_async_with_expected_params: hard-code
expected_param_count so a signature change actually trips the gate
(previously both sides of the comparison were len(sig.parameters)).
- test_check_request_disconnection_invalid_when_connected_times_out:
patch asyncio.sleep so the test no longer spins for ~1.2 s of real
wall-clock; timeout lowered to 0.05 s.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* test(proxy/proxy_server): pin control-plane routes (PR3) (#28850)
* test(proxy/proxy_server): pin misc routes (PR3, partial)
Adds happy + error tests for the misc control-plane routes:
GET /, /routes, /adaptive_router/state, /get_logo_url,
/get_image, /get_favicon.
Also gitignores .pin_list.txt (used by the pin gate).
* test(proxy/proxy_server): pin login/SSO routes (PR3, partial)
Adds happy + error tests for the 5 login/SSO control-plane routes:
GET /fallback/login, POST /login, POST /v2/login, POST /v3/login,
POST /v3/login/exchange. Mocks authenticate_user and
create_ui_token_object at their imported location.
* test(proxy/proxy_server): pin onboarding routes (PR3, partial)
Adds happy + error tests for the 2 onboarding control-plane routes:
GET /onboarding/get_token, POST /onboarding/claim_token. Wires a
MagicMock async context manager for prisma_client.db.tx() and
signs the onboarding JWT with the patched master_key.
* test(proxy/proxy_server): pin model_cost_map reload routes (PR3, partial)
Adds happy + error tests for the 5 model-cost-map control-plane routes:
POST /reload/model_cost_map, POST|DELETE|GET
/schedule/model_cost_map_reload(/status), GET /model/cost_map/source.
Attaches litellm_config to mock_prisma per-test (the table is not in
the default _PRISMA_TABLES fixture).
* test(proxy/proxy_server): pin anthropic_beta_headers reload routes (PR3, partial)
Adds happy + error tests for the 4 anthropic-beta-headers control-plane
routes: POST /reload/anthropic_beta_headers, POST|DELETE|GET
/schedule/anthropic_beta_headers_reload(/status). Stubs
db.litellm_config (not in default _PRISMA_TABLES) and monkeypatches
reload_beta_headers_config so no network calls fire.
* test(proxy/proxy_server): pin invitation routes (PR3, partial)
Adds happy + error tests for the 4 invitation control-plane routes:
POST /invitation/new, GET /invitation/info, POST /invitation/update,
POST /invitation/delete. Patches _user_has_admin_privileges /
_user_has_admin_view to avoid extensive get_user_object mocking.
* test(proxy/proxy_server): pin config CRUD routes (PR3, partial)
Adds happy + error tests for the 8 config-CRUD control-plane routes:
POST /config/update, POST|GET /config/field/update|info, GET /config/list,
POST /config/field/delete, POST /config/callback/delete,
GET /get/config/callbacks, GET /config/yaml. Attaches litellm_config
to mock_prisma per-test.
* test(proxy/proxy_server): tighten pin assertions per review
- test_routes_misc.py: `b"" in response.content` is trivially true;
replace with `len(response.content) > 0` so an empty 405 body trips
the gate.
- test_routes_login_sso.py: `len(response.content) >= 0` is trivially
true; tighten to `> 0`.
- test_routes_anthropic_beta.py: replace brittle string-literal checks
on the serialized JSON (`'"interval_hours": 12' in payload`) with
`json.loads` + dict access so the assertion survives any serializer
spacing.
- test_routes_config.py: `assert status_code in (404, 500)` was too
permissive; the handler re-raises HTTPException(404) verbatim, so
pin 404 strictly.
---------
Co-authored-by: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Add AWS ECS Fargate stack with Aurora Postgres (IAM auth), ElastiCache Redis, S3, ALB with path-based routing to gateway/backend/ui components, Application Auto Scaling, and automated DB bootstrap + prisma migration via local-exec provisioners
- Add GCP Cloud Run stack with Cloud SQL Postgres (password auth), Memorystore Redis, GCS, external HTTPS load balancer with serverless NEGs and URL map routing, and automated prisma migration via Cloud Run Job
- Both stacks support typed proxy_config input mirroring the helm chart's gateway.config.proxy_config, per-component extra env vars, and Secret Manager references for provider API keys
- Gateway/backend services depend on terraform_data.migration so they never start before the schema is in place, eliminating crash-loop windows on first apply
- AWS stack uses IAM database authentication with a one-shot Fargate bootstrap task that creates and grants the rds_iam role to the application user; GCP stack uses password auth assembled at container startup to avoid Cloud SQL Auth Proxy sidecar complexity
- Add .gitignore rules for Terraform state files, plan files, tfvars inputs, provider binaries, and crash logs while explicitly keeping .terraform.lock.hcl for provider version pinning
- Include terraform.tfvars.example files, provider lock files, and comprehensive README documentation covering architecture, TLS setup, image pull strategies, and quick-start instructions for both stacks
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
* Refactor Bedrock response stream shape handling
- Introduced a module-level constant `BEDROCK_RESPONSE_STREAM_SHAPE` to cache the response stream shape, eliminating the need for per-instance caching in `BedrockEventStreamDecoderBase`.
- Updated relevant methods to utilize the new constant, improving performance by avoiding redundant loading of the shape.
- Added tests to ensure the shape is loaded correctly at import time and is consistent across different modules.
- Added a new mock server script for testing Bedrock pass-through functionality.
* Refactor response parsing for Bedrock and SageMaker
- Improved code readability by formatting the parsing method calls in `AWSEventStreamDecoder` for both Bedrock and SageMaker response stream shapes.
- Added blank lines for better separation of code blocks in `invoke_handler.py` and `common_utils.py` to enhance maintainability.
* Enhance error handling for Bedrock and SageMaker response stream shape loading
- Wrapped the loading logic in `_load_bedrock_response_stream_shape` and `_load_sagemaker_response_stream_shape` with try-except blocks to gracefully handle exceptions.
- Added logging to warn when the response stream shape cannot be pre-loaded, ensuring the module imports cleanly.
- Updated tests to verify that loading failures return `None` instead of propagating exceptions.
* Implement error handling for missing response stream shapes in Bedrock and SageMaker
- Added checks in `_parse_message_from_event` methods to raise appropriate errors when `BEDROCK_RESPONSE_STREAM_SHAPE` or `SAGEMAKER_RESPONSE_STREAM_SHAPE` is None, ensuring clearer error reporting.
- Updated logging messages to reflect the unavailability of event-stream decoding for both Bedrock and SageMaker.
- Enhanced unit tests to verify that the correct exceptions are raised when the response stream shapes are not loaded.
- New reusable workflow that spins up a LiteLLM container from the
release image, exposes it via cloudflared tunnel, and triggers
test runs on the Railway-hosted observatory
- Integrates into ghcr_deploy.yml for RC and stable releases
- Can also be triggered manually via workflow_dispatch
- Add placeholder litellm_config.yaml for observatory test models
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add /v1/vector_store/list route for OpenAI API compatibility (fixes test_routes_on_litellm_proxy)
- Fix Bedrock Converse API model format (bedrock_converse/ → bedrock/converse/)
- Fix Nova Premier inference profile prefix (amazon. → us.amazon.)
- Add STABILIZATION_TODO.md to .gitignore
Tested locally - all affected tests now pass
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm
new vector store provider
* feat(vector_store/): use vector store id for vertex ai search api
* fix: transformation.py
cleanup
* fix: implement abstract function
* fix: fix linting error
* fix: main.py
fix check
* Get the basics of the integration working.
* Cleanup bytez integration.
* Update user agent for Bytez integration.
* Use the config class directly. Create the start of the docs.
* Finish up bytez documentation. Include a provider integration guide.
* Fix typing bug in custom_logger_utils. Add tests for bytez integration.
* Add token tracking for model usage for Bytez integration.
* Create a units test for the Bytez config.
* Make changes to Bytez transformation code per PR feedback.
* Cleanup coment in Bytez transformation test.
* Remove LRU usage for bytez integration.
* Consolidate Bytez tests into a single file. Conform to project structure for tests.
* Fix linting error with Bytez impl.
* Schedule budget resets at expectable times (#10331)
* Enhance budget reset functionality with timezone support and standardized reset times
- Added `get_next_standardized_reset_time` function to calculate budget reset times based on specified durations and timezones.
- Introduced `timezone_utils.py` to manage timezone retrieval and budget reset time calculations.
- Updated budget reset logic in `reset_budget_job.py`, `internal_user_endpoints.py`, `key_management_endpoints.py`, and `team_endpoints.py` to utilize the new timezone-aware reset time calculations.
- Added unit tests for the new reset time functionality in `test_duration_parser.py`.
- Updated `.gitignore` to include `test.py` and made minor formatting adjustments in `docker-compose.yml` for consistency.
* Fixed linting
* Fix for mypy
* Fixed testcase for reset
* fix(duration_parser.py): move off zoneinfo - doesn't work with python 3.8
* test: update test
* refactor: improve budget reset time calculation and update related tests for accuracy
* clean up imports in team_endpoints.py
* test: update budget remaining hours assertions to reflect new reset time logic
* build(model_prices_and_context_window.json): update model
---------
Co-authored-by: Prathamesh Saraf <pratamesh1867@gmail.com>
* build(litellm-proxy-extras/utils.py): correctly generate baseline migration for non-empty db
* fix(litellm-proxy-extras/utils.py): Fix issue in migration, where if a migration fails during baselining, all are still marked as applied
* fix(prisma_client.py): don't pass separate schema.prisma to litellm-proxy-extras
use the one in litellm-proxy-extras
* fix(litellm-proxy-extras/utils.py): support passing custom dir for baselining db in read-only fs
Fixes https://github.com/BerriAI/litellm/issues/9885
* fix(utils.py): give helpful warning message when permission denied error raised in fs
* build(model_prices_and_context_window.json): add vertex ai gemini-2.5-flash pricing
* build(model_prices_and_context_window.json): add gemini reasoning token pricing
* fix(vertex_and_google_ai_studio_gemini.py): support counting thinking tokens for gemini
allows accurate cost calc
* fix(utils.py): add reasoning token cost calc to generic cost calc
ensures gemini-2.5-flash cost calculation is accurate
* build(model_prices_and_context_window.json): mark gemini-2.5-flash as 'supports_reasoning'
* feat(gemini/): support 'thinking' + 'reasoning_effort' params + new unit tests
allow controlling thinking effort for gemini-2.5-flash models
* test: update unit testing
* feat(vertex_and_google_ai_studio_gemini.py): return reasoning content if given in gemini response
* test: update model name
* fix: fix ruff check
* test(test_spend_management_endpoints.py): update tests to be less sensitive to new keys / updates to usage object
* fix(vertex_and_google_ai_studio_gemini.py): fix translation
