some users do not use make pre-commit as it is a multi-minute process. I personally use it but I want users themselves to decide whether to pre-commit before each commit or not, based on what works best for them
LIT001/LIT002 and the typing.Dict ban all steered dict-shaped values to
frozen dataclasses or suppression even though the checker already accepts
MappingProxyType as a freezing wrapper; the messages now name it so the
dict-shaped freeze path is actually discoverable at fix time.
The 15-25 word cap previously applied only to replies/rebuttals to AI PR
review bots; it now covers every GitHub comment (issue comments and PR
discussion comments included). The public-writing punctuation bullet also
gains a warning that a word cap is not a one-sentence cap, so tight budgets
should be met with period splits or conjunctions rather than ";" splices,
at most one ";" per message
* feat(pre-commit): save full lint output to a per-worktree log file
* docs(claude): point agents at the pre-commit log instead of rerunning
* fix(pre-commit): warn when the log cannot be created or fully written
For a backend-only commit, staging the regenerated schema.d.ts newly
satisfies the ui file triggers, so the folder-wide dashboard lint
budgets run locally for the first time and CI's frontend-lint job
(budgets plus knip) activates on the PR. Those can only fail from
pre-existing dashboard-tree state, never from the regenerated file,
but the guidance should say so instead of implying a re-run is
always redundant.
The stale-types failure already writes the regenerated schema.d.ts to the
working tree, and staging it cannot introduce a new failure: the file is
listed in .prettierignore and the eslint config ignores, so no lint pass
sees it, and gen:api derives it purely from the Python proxy code, so a
second regeneration is a no-op. The only reason left to re-run is when
other checks also failed, so say exactly that in the script message and
CLAUDE.md instead of prescribing an unconditional re-run.
Removes the scheduled workflow that cut litellm_oss_daily_YYYY_MM_DD
branches and the guardrails workflow that only ran on them. The secret
scan and ruff checks that workflow duplicated already run on PRs to
litellm_internal_staging via test-linting.yml, so no coverage is lost.
Retargets contributor-facing messaging in CONTRIBUTING.md, CLAUDE.md,
and the guard-main-branch error output at litellm_internal_staging.
The guard-main-branch error messages and the contributor docs still
pointed people at litellm_oss_staging. Redirect them to the current
daily OSS branch (litellm_oss_daily_YYYY_MM_DD), a fresh one of which
is cut each weekday, so contributors should target the most recent
* chore: add latest model rule to CLAUDE.md
* chore: correct grammar mistake
* chore: make the rule more concise
* chore: replace rule instead
* chore: revise wording to override memories, etc.
* chore: slightly adjust wording to be more precise
* fix: zero out crash-class basedpyright rules across litellm/
* feat(lint): add LIT009 banning inert type: ignore comments
* docs: require bracketed rule and reason on every suppression
* chore(lint): ratchet budgets down and zero crash-class pyright limits
* fix: narrow auto router routelayer through a local before calling
* test: add regression tests for crash-class fixes
* fix: drop dead AZURE_AD_TOKEN lookups and word-bound the type-ignore regex
* chore: clarify the linear ticket instruction in pr template
* fix: make it more concise
* Update CLAUDE.md
lol
* chore: tell claude not to search for it if it doesn't have it
---------
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* chore(e2e): untrack gateway config and document e2e test location
Stop tracking tests/e2e/gateway/litellm-config.yml so the local proxy config stays on the machine
Add a note to CLAUDE.md that new e2e tests belong in tests/e2e/ and must follow that directory's conventions
* chore(e2e): add self-contained docker compose stack for local runs
Ship a docker-compose.yml that starts the proxy with a throwaway Postgres and Redis and inlines the proxy config with example models, so contributors can bring up a local gateway with nothing but a .env. Update CONTRIBUTING.md to match the inline-config flow
* chore(e2e): drop the second gemini deployment; one key is enough locally
* docs(e2e): make pre-commit steps ordered and require flagging internally found issues
* chore(lint): remove dead E501 config, fix stale blame-ignore SHAs, note 120 width in CLAUDE.md
E501 sat in both lint.ignore and lint.extend-select in ruff.toml; ignore wins,
so no line length was linted at all (verified: a 130-char line passes ruff
check while T201 fires). Remove it from both lists so the config tells the
truth: the formatter's wrap width is the only line-length control, matching
how the repo has actually behaved since E501 was ignored in Oct 2024
.git-blame-ignore-revs listed the pre-squash PR-head SHAs for the two ruff
reformat commits (#31317, #31518), which never landed on the branch, so git
blame ignored nothing. Replace them with the squash-merge SHAs that are
actually in history
Also document in CLAUDE.md that the line length is 120 (ruff.toml), not 88,
so agents stop wrapping to the old Black width
* fix: make CLAUDE.md more concise
* fix: make the guideline more clear
* chore(lint): raise basedpyright per-rule slack to 50% of baseline
The per-rule ceilings in basedpyright-code-budget.json sat at roughly 10% slack over baseline, which several in-flight PRs are already bumping into. Raise the slack on every rule to at least 50% of its baseline so there is ample headroom for a long while, while never lowering any rule that already had more generous slack (e.g. reportReturnType stays at 100).
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* refactor(lint): collapse type/lint budgets to a single per-rule limit
The three non-frontend budget files (ruff-strict, type-discipline, basedpyright-code) tracked a per-rule baseline and slack whose sum was the ceiling. Nothing consumed the split beyond that sum, so this replaces both keys with a single limit equal to the old baseline + slack; the original baselines live in git history if anyone needs them.
The gate scripts and the ratchet guard now read limit directly. lint-budget-update no longer re-captures raw counts; it ratchets each rule's limit down by the number of violations this branch cleared since its branch point (the merge-base), so the granted headroom shrinks by exactly what was fixed and a limit never rises. The ratchet guard reads either schema so it still compares correctly across the migration boundary.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* chore(lint): surface staged-vs-working parity for pre-commit and budget-update
make pre-commit selects which checks to run from the staged index but runs the linters over the working tree, so unstaged edits to tracked files and untracked files skew a green/red away from what a commit of only the staged changes would produce. There is no safe in-place way to lint the index, so the script now warns when unstaged or untracked changes are present, and CLAUDE.md documents that you must stage everything first for both make pre-commit and make lint-budget-update to predict CI correctly.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* docs(lint): list type-discipline budget in lint-budget-update instruction
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* feat(guardrails/headroom): add CCR (compress-cache-retrieve) support via agentic loop
When Headroom's /v1/compress returns messages containing hash markers
(hash=[a-f0-9]{24}), inject a headroom_retrieve tool into the request.
When the LLM calls that tool, intercept via async_should_run_agentic_loop
and async_build_agentic_loop_plan, call GET /v1/retrieve/{hash} on the
Headroom sidecar, and replay the LLM with the original content as a tool
result -- all transparent to the caller.
* style: run ruff format on headroom guardrail and tests
* fix(guardrails/headroom): detect headroom_retrieve calls in both OpenAI and Anthropic response formats
* test(guardrails/headroom): add test for Anthropic content block format detection in CCR loop
* ci: trigger CI checks
* fix(guardrails/headroom): replace List/Dict with list/dict to fix UP006 ruff violations
* fix(guardrails/headroom): replace except Exception with except ValueError to fix BLE001
* fix(guardrails/headroom): add Responses API output format detection for CCR tool calls
* refactor(guardrails/headroom): extract format-specific helpers to fix C901 complexity
* fix(guardrails/headroom): scope CCR retrieval to hashes produced by current request
Previously any LLM-supplied hash in a headroom_retrieve tool call was
forwarded to the Headroom retrieve API, letting a crafted tool call
fetch arbitrary cached content. Validate the hash against the set
produced by compressing the current request's messages before calling
retrieve.
* fix(guardrails/headroom): track issued hashes server-side, fix Responses API replay shape
Hash validation now also checks an in-memory cache of hashes actually
returned by /v1/compress, not just whether the hash text appears
somewhere in the request's messages. The message-text check alone is
forgeable: an attacker can plant a hash-shaped string in their own
prompt and have it treated as valid.
Responses API follow-up now emits function_call/function_call_output
items keyed by call_id instead of chat-style assistant/tool messages,
since the Responses API does not accept the latter as input. Also
fixes call_id/id field priority when extracting tool calls from
Responses API output, since call_id (not id) is what must match
between the function_call and its output.
* fix(guardrails/headroom): drop redundant quoted type annotations
UP037 flags quotes on annotations that are already lazily evaluated
via `from __future__ import annotations`.
* test(guardrails/headroom): add missing pytest.mark.asyncio decorators
Functional under asyncio_mode=auto, but every other async test in the
file has the decorator for consistency.
* fix(guardrails/headroom): scope CCR hashes per call_id, fix Anthropic replay shape
Two real gaps found in review:
1. The instance-wide issued-hash cache combined with a message-text
check did not actually scope retrieval to the request that produced
the hash. A hash issued for request A stays in the shared cache
until TTL expiry, and the message-text check is satisfied by any
request whose own messages happen to echo that hash string. Request
B could plant A's hash in its own prompt and retrieve A's content.
Fixed by keying the issued-hash cache by litellm_call_id, matching
the pattern already used in compression_interception: a hash is
only honored when it was issued under the exact call_id resolving
for the current request.
2. The Anthropic Messages replay path fell through to the chat-style
assistant/tool-message builder, which Anthropic does not accept.
Anthropic requires the tool_use block echoed in an assistant message
paired with a tool_result block in a user message, keyed by
tool_use_id. Added a dedicated branch for this shape.
* docs: note proactive API-fragmentation helper convention
Add a bullet to the coding-conventions list: look for or add a shared
helper when logic branches on API surface (chat completions vs
Anthropic Messages vs Responses API), instead of duplicating
format-detection per module.
* fix(guardrails/headroom): fix Anthropic tool-shape detection, extract shared cross-API tool util
Live e2e testing against the real Anthropic API surfaced two bugs the
mocked unit tests couldn't catch because they used MagicMock responses
instead of realistic response shapes:
1. has_headroom_retrieve_tool only recognized OpenAI-shaped function
tools. By the time an Anthropic Messages response reaches the
agentic-loop gate, the tool this guardrail injected has already been
transformed into Anthropic's native shape (type: "custom", top-level
"name"), so the gate never fired for real Anthropic requests.
2. AnthropicMessagesResponse is a TypedDict, so real responses are
plain dicts at runtime, not objects with attribute access. The
extractors and format detectors used bare getattr(), which silently
returns nothing for dict responses instead of reading the actual
key.
Extracted the cross-API-surface tool-call extraction and tool-presence
check into litellm/litellm_core_utils/prompt_templates/factory.py
(get_tool_calls_from_response, has_tool_with_name) so this format
fragmentation is handled in one place instead of being duplicated
per-guardrail, and reused the existing repair-aware
parse_tool_call_arguments from common_utils instead of a naive
json.loads. headroom.py now delegates to these shared helpers.
Confirmed live against the real Anthropic API: the retrieve loop now
fires and successfully retrieves the correct hash's content through
the full compress -> tool-call -> retrieve -> replay round-trip.
* fix(guardrails/headroom): fix ruff-strict UP006/I001 budget violations
Use lowercase list/dict generics in the new factory.py tool-call
helpers instead of typing.List/Dict, drop the now-unused Tuple import
in headroom.py, and reorder the new factory import ahead of the
llms.custom_httpx import to satisfy import sorting.
* fix(guardrails/headroom): match Anthropic tools without a type field
Anthropic's documented client tool format is just name + input_schema;
type: "custom" is only one possible value, not a requirement. Match
any non-OpenAI-shaped tool on its top-level name instead of requiring
type == "custom".
* chore: shift CI lint left with a pre-commit hook and CLAUDE.md rule
Add an opt-in pre-commit hook (.githooks/pre-commit, active after
make install-hooks) that runs the CI-equivalent checks against staged
files: make lint for Python, prettier plus eslint for the dashboard,
and a gen:api drift check for the proxy OpenAPI types. Document the
same expectation in CLAUDE.md so reds surface locally instead of in CI.
* fix: make `make lint` isomorphic to the CI lint job
`make lint` diverged from test-linting.yml in ways that produced both
false reds and false greens: its format-check ran over the whole repo
(CI scopes it to changed files vs the base), its ruff-strict budget ran
in absolute mode (CI runs it as a delta vs base), and it omitted the
type-discipline gate entirely. Recompose `lint` to replay CI's exact
sequence: diff-scoped ruff format check, whole-tree ruff check, the
strict / type-discipline / basedpyright budgets as a delta resolved the
same way CI resolves it (merge-base with origin/litellm_internal_staging),
then circular-import and import-safety. Factor the repeated base fetch
into one shared prerequisite so the chain hits the network once.
Align the pre-commit hook's eslint invocation with the CI frontend-lint
job (`--pass-on-unpruned-suppressions`) and fix the CLAUDE.md guidance to
point at the diff-scoped frontend commands instead of the whole-folder
npm scripts, which are broader than CI.
* fix(githooks): make pre-commit 1:1 with CI frontend-lint, lint, and type-gen
The shift-left pre-commit hook diverged from the CI jobs it claims to mirror, so a clean commit did not actually mean a green CI lint.
The dashboard block only ran prettier and eslint over js/jsx/ts/tsx/mjs/cjs, but CI's frontend-lint runs prettier over a wider set (also json, css, scss, md, mdx, yml, yaml, html) and additionally gates the whole-folder eslint lint budgets via scripts/check-lint-budgets.mjs. The hook now mirrors that split and runs the budget check, so a dashboard commit that passes locally passes the job.
The API-types block ran npm run gen:api without LITELLM_PYTHON, so it shelled out to the system python3 which has no litellm installed and always failed with a false 'could not regenerate API types' red. It now passes LITELLM_PYTHON="uv run --no-sync python" the way check-ui-api-types.yml does.
make lint format-checks the files in origin/base...HEAD, which at pre-commit time predates the staged change, so a brand-new commit's formatting went unchecked. The Python block now also runs ruff format --check over the staged litellm files directly to cover that case, and its trigger is scoped to staged litellm/ files (the only tree CI's lint job inspects) so a tests-only or scripts-only commit skips the slow make lint instead of wasting time on a run that could not catch anything.
CLAUDE.md's shift-left rule was cut off mid-sentence and understated the frontend checks; it now describes all three gates accurately and points agents at make install-hooks to run them automatically before each commit.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(githooks): scope the API-types check to all of check-ui-api-types.yml's triggers
spec_files was filtered from the staged Python files, so the gen:api drift
check only fired for .py changes under litellm/proxy or litellm/types. CI's
check-ui-api-types.yml triggers on any file under those directories (Prisma
schema, configs) plus the generator script and the dashboard package files,
so a non-Python proxy/types change could pass the hook and still fail CI.
Match the workflow's full trigger set instead.
* fix(pre-commit): run prisma generate before gen:api to mirror CI
* refactor(githooks): run shift-left lint via on-demand make pre-commit, not an auto-firing hook
The pre-commit hook ran make lint plus the dashboard eslint budgets, which are minutes of work (basedpyright over litellm/, a whole-folder eslint . pass at ~40s). Wiring that into core.hooksPath via make install-hooks meant every human commit, not just an agent's, paid that cost, which is real friction for interactive committers.
Move the staged-file checks out of .githooks/ into scripts/pre_commit_lint.sh and expose them as make pre-commit, and keep .githooks/ to only the fast Conventional Commits / Branches hooks so make install-hooks no longer makes commits slow. Agents run make pre-commit right before each commit (CLAUDE.md instructs this), so the slow gates fire only for the commits an agent is making and never auto-fire for a human typing git commit. The script stays hook-compatible for anyone who still wants it to fire automatically via a symlink.
Preferred this over sniffing an agent env var to auto-fire only for agents: that is fragile (misses agents when the var is unset, fires on humans when it leaks into their shell, and silently no-ops a hook a human deliberately installed), whereas an on-demand command achieves the same humans-never, agents-per-commit outcome deterministically.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(pre-commit): run make lint last so it can't prune the proxy deps gen:api needs
make lint's install-dev prerequisite runs uv sync --frozen, which prunes the proxy extras (prisma, websockets, ...) from the venv. With the Python block running first, the subsequent API-types block then failed: gen:api imports litellm.proxy.proxy_server, which needs those deps, so every litellm/proxy change (the main trigger for the API-types check) hit a false 'could not regenerate API types' red. Run the dashboard and API-types blocks before the Python block so gen:api sees an intact env; CI is unaffected because there the lint and check-ui-api-types jobs run in separate environments.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix: make CLAUDE.md more concise
* fix(makefile): give make lint the CI lint env and stop it pruning the venv
make lint diverged from test-linting.yml's lint job in two ways: it never generated the Prisma client (so basedpyright resolved the DB wrappers as Unknown, drifting from CI's counts), and its bare uv sync --frozen pruned the proxy extras (prisma, websockets, ...) out of the venv on every run, which broke the gen:api step that imports litellm.proxy.proxy_server and left a dev unable to run the proxy until re-syncing.
Add a lint-install target that mirrors the job's environment (the proxy-dev group plus prisma generate) and runs before the checks, and make both it and install-dev use uv sync --inexact so they top up the venv instead of tearing packages out. CI is unaffected since it installs its own env per job.
Because make lint no longer prunes, the pre-commit reorder that ran it last (to dodge the prune) is no longer needed, so restore the original block order.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(makefile): drop lint-install so make lint matches CI's slimmer env
test-linting.yml's lint job installs deps with a bare uv sync --frozen
(default dev group only, no proxy-dev, no prisma generate), but the
lint-install target chained into make lint pulled in --group proxy-dev
and ran prisma generate. Because the basedpyright budget step compares
head and base counts against fixed thresholds, the extra symbols and
Prisma client locally resolved can shift error counts away from CI's,
producing false greens or false reds on the type-check gate.
Remove the lint-install target and its slot in lint. The remaining
sub-targets already chain install-dev, which now uses
uv sync --inexact --frozen, so the venv still isn't pruned but the
installed set stays aligned with what CI sees.
* ci(linting): install proxy-dev and generate prisma in lint job, matching make lint
make lint now installs the proxy-dev group and generates the Prisma client so basedpyright resolves the DB wrappers; the lint job here still installed only the base env, so a local pre-commit could pass while the required CI lint failed (or vice versa). Bring this job in line, which is the same environment litellm_internal_staging's lint job already uses.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(makefile): keep make lint on the proxy-dev + prisma env to match CI
A concurrent change dropped lint-install to match what looked like CI's slim env, but test-linting.yml's lint job (and the merge ref this PR's CI actually runs) installs --group proxy-dev and generates the Prisma client. With make lint slim and CI fat, basedpyright resolves fewer symbols locally than CI, so a prisma-typed error can stay Unknown locally (green) while CI catches it (red). Restore lint-install so make lint installs the same env CI does; the previous commit also brought this PR's test-linting.yml in line with that env, so the two now match.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* ci: drop redundant mypy type-check gate, standardize on basedpyright
Type checking ran both mypy (via the pydantic.mypy plugin) and basedpyright.
pydantic v2 emits dataclass_transform, so basedpyright understands models
natively with no plugin, and its gated rules already cover what the mypy pass
caught (no-untyped-def, no-any-return, valid-type, import-not-found all map to
basedpyright equivalents). Running both meant two checkers, two budgets, and a
plugin only mypy could load.
This removes the mypy type-check gate: the lint-mypy/lint-mypy-budget-update
Makefile targets, the CI MyPy step, mypy-code-budget.json, the budget-ratchet
entry, and the vestigial [tool.mypy] pydantic plugin block (the gating pass used
litellm/mypy.ini, which never loaded the plugin). type_check_gate.py is
specialized to basedpyright since the mypy parsing path is now unused.
mypy stays a dev dependency because the Any-discipline gate
(scripts/check_any_discipline.py) imports it as a library to detect Any-typed
values; it is no longer run as a type checker.
* ci: remove the Any-discipline gate, rely on basedpyright's reportAny
The Any-discipline gate (scripts/check_any_discipline.py) was the last consumer
of mypy: it imported mypy as a library to detect values whose inferred type
contains Any, gated per-file against any-discipline-budget.json. basedpyright
already reports the same class of finding through reportAny/reportExplicitAny,
which are gated tree-wide in basedpyright-code-budget.json, so the separate gate
(and the mypy dependency behind it) is redundant.
Removes the gate end to end: check_any_discipline.py and its test, the
any-discipline CI job, the lint-any/lint-any-budget-update Makefile targets,
any-discipline-budget.json, litellm/mypy.ini, the .mypy_cache_any references,
and mypy from the dev dependencies. budget_ratchet_check.py drops the
any-discipline entry and the now-unused zero-floor mechanism (rewritten as a
comprehension). check_type_discipline.py drops the any-ok suppression token,
since # any-ok suppressed only the deleted gate; the 134 now-orphaned
# any-ok comments across 14 files are stripped (they never affected
basedpyright, which uses # pyright: ignore).
uv.lock is intentionally left untouched: uv still considers it consistent with
the mypy-removed pyproject (uv lock --check and uv sync --frozen both pass), and
a relock bumps 30+ unrelated packages because of the moving exclude-newer window.
A future intentional relock will prune the now-unreferenced mypy entry.
* build: relock to drop mypy from uv.lock
CI's uv 0.10.9 honors the repo's exclude-newer window and correctly flags the
lockfile as out of sync once mypy leaves pyproject; my earlier local uv 0.8.17
could not parse exclude-newer and silently passed --check. Relocking with the
pinned CI version removes only mypy and its transitive librt, with no other
version changes.
* ci(lint): grandfather any-discipline with a per-file ratchet budget (50% headroom)
The any-discipline gate previously failed on any Any-typed value touched on a
changed line, which tripped on merely editing a legacy `X | Any` line. Switch it
to a per-file budget: `any-discipline-budget.json` records each file's current
Any count and a changed file fails only when its count exceeds `baseline + slack`
(50% headroom, rounded up). New/unbudgeted files have baseline 0, so they stay
airtight, while editing legacy files no longer forces cleaning pre-existing debt.
Only changed files are re-type-checked (per-PR cost unchanged); the whole-tree
scan to recapture the budget runs under `--update` (`make lint-any-budget-update`).
The budget is a one-way ratchet guarded by `budget_ratchet_check.py`, matching the
ruff/mypy/basedpyright budgets, and folds into `make lint-budget-update`.
Also fixes a RecursionError in `contains_any` (recursive type aliases yield fresh
objects per unfold, defeating the id() cycle guard) by walking iteratively with a
depth cap, exposed by the whole-tree scan.
* chore: make CLAUDE.md more concise
* chore: rearrange Makefile
* ci(lint): make any-budget --update git-failure-safe; clarify over-budget message
all_litellm_py_files now returns None when git is unavailable (mirroring
changed_line_map) instead of letting CalledProcessError/FileNotFoundError escape
as a raw traceback, and update_budget reports a clean setup error (exit 2) for
that case. The list-files dependency is injected so the path is unit-testable
without monkeypatching. The over-budget diagnostic now reads "N value(s) total,
over budget" so the count isn't misread as the excess over the ceiling.
* ci(lint): exempt the file-keyed any-discipline budget from the ratchet's dropped-entry rule
budget_ratchet_check treats a vanished budget entry as a loosening (an untracked
rule whose ceiling is now unbounded). That holds for the rule-keyed budgets, but
the any-discipline budget is keyed by file and its gate treats an absent file as
ceiling 0 (the file must be Any-free). Cleaning a file to zero drops its entry on
the next --update, so the generic rule flagged that as a regression: a false-
positive red on exactly the cleanup the ratchet exists to encourage. Exempt the
file-keyed budget from the dropped-entry rule while still catching a raised
ceiling.
* ci(lint): enforce type-discipline budget for casts and type guards
Add a ratcheted gate that blocks net-new typing.cast() usage and bans
TypeGuard/TypeIs outright, layered on the existing ruff-strict budget setup.
- ruff-strict.toml: ban cast/TypeGuard/TypeIs (typing + typing_extensions)
via flake8-tidy-imports banned-api (TID251) for a coarse import-level freeze.
- ruff-strict-budget.json: bump TID251 baseline 2404 -> 2662 to absorb the
~258 pre-existing usages now matched by the new banned-api entries.
- scripts/check_type_discipline.py: AST checker adding LIT006 (cast call sites,
suppress with `# cast-ok: <reason>`) and LIT007 (TypeGuard/TypeIs annotations,
suppress with `# guard-ok: <reason>`) for per-call-site granularity.
- scripts/type_discipline_gate.py: baseline+slack gate with delta-vs-base,
mirroring ruff_strict_gate.py.
- type-discipline-budget.json: LIT006 baseline 1013 (slack 10), LIT007 0/0.
- test-linting.yml: run the gate in CI against the PR base SHA.
* ci(lint): enforce suppression-reason budgets and guard budgets against loosening
- wire the **kwargs ban (LIT008) into the vendored type-discipline checker so it
matches the budget that already referenced it
- freeze LIT003/LIT004 (noqa / type-ignore without codes or reason) and LIT005
(*-ok suppression without a reason) at slack 0 so any net-new unexplained
suppression trips the type-discipline gate
- add scripts/budget_ratchet_check.py and a separate, non-gating budget-ratchet CI
job that turns red when any *-budget.json ceiling is raised, a rule is dropped,
or a budget file is deleted
* ci(lint): ban mutable collections in annotations and all mutable construction
Expand LIT001 from coarse builtins at interfaces to any mutable collection
in any annotation (builtins, typing aliases, collections concretes, mutable
ABCs) across signatures, class attributes, locals, and globals. Add LIT009 to
flag mutable-collection construction (literals, comprehensions, constructors)
so the unannotated seed-then-mutate pattern is caught too. Enumerate any-ok in
LIT005 so its reason requirement holds even when only the stdlib checker runs.
Budget LIT001 (21452) and LIT009 (25222) with slack 10 to ratchet down.
* ci(lint): recommend pydantic at boundaries and add functional-refactor guidance
Drop the msgspec mention from the cast banned-api messages so the recommended
validation path matches the codebase's primary pattern (pydantic). Add a note to
CLAUDE.md that lint / type-discipline failures should be resolved by refactoring
to functional, immutable patterns rather than reaching for mutable structures or
`# mutable-ok`.
* style: make CLAUDE.md more concise
* chore: update CLAUDE.md guidelines
* ci(lint): renumber mutable construction LIT009 -> LIT002 next to LIT001
Group the mutable-collection family together: LIT001 (mutable collection in any
annotation) and the construction rule now sit adjacent at LIT001/LIT002. The
freed LIT009 slot is taken by the sibling Any gate (check_any_discipline.py,
#30379), which moves its Any-typed-value rule LIT002 -> LIT009 in lockstep so
the shared LIT namespace stays contiguous with no holes. Budget, gate docstring,
and the checker's own docstring/messages are updated to match.
* fix: numbering in CLAUDE.md
* test(lint): test type-discipline checker, scope LIT007 to return types
Add regression tests for check_type_discipline.py (every LIT rule, its
suppression, and the comment scanner) and for budget_ratchet_check.py.
Confine LIT007 to function return annotations, the only place TypeGuard/TypeIs
are valid, so a runtime name that merely reads those identifiers is no longer
flagged. Switch scan_comments to io.StringIO(source).readline, the standard
readline that returns '' at EOF, dropping the iter(...).__next__ idiom.
* fix(lint): best-effort worktree teardown so cleanup can't mask the real error
base_counts ran `git worktree remove` through the raising `_run` in its finally,
so a failed `git worktree add` (or a failure in the body) was masked by a second
SystemExit from the cleanup. Tear the worktree down best-effort, like the sibling
rmtree, so the original error propagates.
* fix(lint): ratchet fails loudly on an unresolvable base; drop dead checker state
Verify the merge-base ref resolves to a commit before trusting a missing-file
result from git show, so an invalid or empty BASE_SHA now turns the budget-ratchet
guard red instead of skipping every budget and passing vacuously
Also drop the unused Comments.by_line field and the phantom --changed-only usage
line from check_type_discipline's docstring, and cover the ref handling with tests
* fix(lint): degrade malformed source to LIT000 instead of crashing the checker
tokenize.generate_tokens raises IndentationError (a SyntaxError subclass) on a dedent
mismatch, which escaped scan_comments' tokenize.TokenError handler and crashed the whole
checker run, zeroing the gate for that invocation. Catch SyntaxError too so the file
falls through to ast.parse and is reported as LIT000, matching the checker's
graceful-degradation contract. Also add the trailing newline ruff-strict.toml lacked
* perf(lint): skip the base worktree scan when no rule is over its ceiling
cmd_check created a git worktree and re-scanned the base tree on every run, but a
rule can only breach when its head count is already over baseline + slack; when none
are, the base comparison cannot change the verdict. Short-circuit to OK in that case,
which is every green PR, roughly halving the gate's work. Extract over_ceiling and
cover it (and evaluate's drift-safety) with tests
* fix(lint): exempt .dict()/.list()/.set() method calls from LIT002
_construction_kind matched dict/list/set as constructors via func.attr too, flagging
common method calls like pydantic's model.dict() as mutable construction; 200 such
false positives existed in litellm. Recognize dict/list/set construction only when
unqualified while keeping the collections concretes (deque/defaultdict/...) matchable
as attributes, since those are rarely method names. Ratchet the LIT002 baseline down
25222 -> 25022 to reflect the removed false positives
* chore(lint): bump basedpyright ceilings to absorb staging base drift
The basedpyright gate added in #30379 is a total-count check against
basedpyright-code-budget.json and the linting workflow runs only on
pull_request, so pushes to litellm_internal_staging never re-baseline it.
Merging staging into this branch surfaced that drift: seven
reportAny/reportUnknown* rules sit 10-149 errors above their committed ceiling
even though this PR changes no files under litellm/, the only path basedpyright
scans (pyrightconfig include is litellm). The new baselines match the counts CI
measured on the merge commit, with the existing per-rule slack preserved
* fix(lint): ratchet guard watches every budget file, not just two
DEFAULT_BUDGETS only listed ruff-strict-budget.json and
type-discipline-budget.json, so mypy-code-budget.json and
basedpyright-code-budget.json were unguarded and their ceilings could rise with
no signal, which is exactly the failure mode this guard exists to prevent. The
gap became concrete when this PR bumped basedpyright-code-budget.json to absorb
staging drift. All four budgets are now watched, so the budget-ratchet job
surfaces that basedpyright bump for human review the same way it surfaces the
TID251 raise. A regression test pins that every *-budget.json on disk is in
DEFAULT_BUDGETS, failing loudly if a future budget escapes the ratchet
* fix: add a lot more slack
* fix(lint): restore LIT003 frozen slack to 0
The blanket slack bump set LIT003 (bare # noqa without codes or a reason) to a
slack of 50, which contradicts the documented zero-tolerance invariant: the gate
docstring and the PR description table both freeze LIT003/LIT004/LIT005 at slack
0 so any net-new unexplained suppression trips the gate. Slack 50 would let 50
new bare noqas through silently. The actual LIT003 count is 397, well under the
516 baseline, so restoring slack to 0 keeps the gate green while putting the
freeze back. LIT004/LIT005/LIT007 were already correct at 0
* fix(lint): restore documented slack 10 for the buffered LIT rules
The slack bump left LIT001/LIT002/LIT006/LIT008 at 2000/2500/100/100, 10-250x the
"/ 10" the PR description table and the gate docstring document. That buffer was
never needed: the gate already blames a rule only when its count exceeds the
ceiling and grew vs the merge-base, so the violations the staging merge added in
litellm/ sit in both head and base and are never charged to this PR. With slack
back at the documented 10 the gate stays green, and the ceiling is tight again
(LIT006 no longer waves through 99 net-new cast() calls). Baselines are
unchanged; only the slack returns to its documented value
* fix(lint): ratchet LIT003 baseline down to its actual count
The LIT003 baseline was 516 while the current bare-noqa count is 397, leaving
~119 units of headroom that undercut the documented zero-tolerance freeze: the
gate docstring claims any net-new bare noqa trips the gate, but with cap 516 a PR
could add over a hundred first. Drop the baseline to the measured 397 so the
freeze is exact (cap = 397 + slack 0), the same hard-zero-at-the-boundary shape
LIT005 and LIT007 already use and pass in CI. PR table row updated to 397 / 0
* fix: increase slack
* fix: increase slack
* docs(lint): align gate docstring with buffered LIT003/LIT004 slack
The budget now gives LIT003/LIT004 nonzero slack, so the gate's prose no
longer claims they are frozen at slack 0; LIT005 remains the reasonless-
suppression freeze and LIT007 the hard zero.
