Commit graph

115 commits

Author SHA1 Message Date
Mateo Wang
a0d08b8143 chore: remove pre-commit and bootstrap advisories
They were taking too long
2026-08-04 19:38:34 -07:00
mateo-berri
38cd75342d Note the one case where staging schema.d.ts changes what runs
For a backend-only commit, staging the regenerated schema.d.ts newly
satisfies the ui file triggers, so the folder-wide dashboard lint
budgets run locally for the first time and CI's frontend-lint job
(budgets plus knip) activates on the PR. Those can only fail from
pre-existing dashboard-tree state, never from the regenerated file,
but the guidance should say so instead of implying a re-run is
always redundant.
2026-08-04 19:38:34 -07:00
mateo-berri
4a2ceed595 Stop advising a pre-commit re-run for stale dashboard API types
The stale-types failure already writes the regenerated schema.d.ts to the
working tree, and staging it cannot introduce a new failure: the file is
listed in .prettierignore and the eslint config ignores, so no lint pass
sees it, and gen:api derives it purely from the Python proxy code, so a
second regeneration is a no-op. The only reason left to re-run is when
other checks also failed, so say exactly that in the script message and
CLAUDE.md instead of prescribing an unconditional re-run.
2026-08-04 19:38:34 -07:00
Mateo Wang
4395e974db
Merge pull request #35825 from BerriAI/litellm_claude_md_em_dash_order
docs(CLAUDE.md): prefer commas over semicolons when replacing em dashes
2026-08-04 14:47:32 -07:00
mateo-berri
ae54f0c95d docs(CLAUDE.md): add colon to em dash replacement list 2026-08-04 14:34:01 -07:00
mateo-berri
bd7d270e17 docs(CLAUDE.md): weight punctuation variety instead of defaulting to comma 2026-08-04 14:24:29 -07:00
Mateo Wang
98fed43ae7
chore: make it more concise 2026-08-04 14:19:56 -07:00
mateo-berri
09388532d2 docs(CLAUDE.md): prefer commas over semicolons when replacing em dashes 2026-08-04 14:10:03 -07:00
Mateo Wang
c1450e9fa9
chore: fix formatting 2026-08-04 13:24:15 -07:00
mateo-berri
741aa901cd docs(claude): state the Final-binding and frozen-parameter conventions 2026-08-04 13:20:59 -07:00
Mateo Wang
d33e6fe16c
chore: make it concise 2026-07-30 12:37:58 -07:00
mateo
23bf657112 docs(claude): require 15-25 word human-readable replies to AI PR review bots
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-30 19:36:36 +00:00
Yuneng Jiang
8a0bb4cc56
chore(ci): retire daily OSS branches in favor of litellm_internal_staging
Removes the scheduled workflow that cut litellm_oss_daily_YYYY_MM_DD
branches and the guardrails workflow that only ran on them. The secret
scan and ruff checks that workflow duplicated already run on PRs to
litellm_internal_staging via test-linting.yml, so no coverage is lost.

Retargets contributor-facing messaging in CONTRIBUTING.md, CLAUDE.md,
and the guard-main-branch error output at litellm_internal_staging.
2026-07-20 11:38:52 -07:00
Mateo Wang
1bff68c0ce
chore: keep it brief 2026-07-11 20:23:41 -07:00
mateo-berri
c9beaf85ff build(dev-env): add make bootstrap and unprovisioned-checkout preflight to pre-commit 2026-07-11 19:25:39 -07:00
Yuneng Jiang
f72c7cc2db
docs: point OSS contributors at the daily OSS branch
The guard-main-branch error messages and the contributor docs still
pointed people at litellm_oss_staging. Redirect them to the current
daily OSS branch (litellm_oss_daily_YYYY_MM_DD), a fresh one of which
is cut each weekday, so contributors should target the most recent
2026-07-10 14:54:37 -07:00
Mateo Wang
ee69a62304
docs(CLAUDE.md): warn that harness-injected PR template copies strip HTML comments (#32373) 2026-07-07 14:24:09 -07:00
ryan-crabbe-berri
65be4c16cd
docs(claude): note UI dev server command in run guidance (#32344) 2026-07-07 16:23:24 +00:00
Mateo Wang
03271de527
chore: add latest model rule to CLAUDE.md (#32164)
* chore: add latest model rule to CLAUDE.md

* chore: correct grammar mistake

* chore: make the rule more concise

* chore: replace rule instead

* chore: revise wording to override memories, etc.

* chore: slightly adjust wording to be more precise
2026-07-05 02:07:47 +00:00
Mateo Wang
5f864c83ce
chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)
* fix: zero out crash-class basedpyright rules across litellm/

* feat(lint): add LIT009 banning inert type: ignore comments

* docs: require bracketed rule and reason on every suppression

* chore(lint): ratchet budgets down and zero crash-class pyright limits

* fix: narrow auto router routelayer through a local before calling

* test: add regression tests for crash-class fixes

* fix: drop dead AZURE_AD_TOKEN lookups and word-bound the type-ignore regex
2026-07-04 16:56:12 -07:00
Mateo Wang
5f4b9ad51c
chore: clarify the linear ticket instruction in pr template (#32076)
* chore: clarify the linear ticket instruction in pr template

* fix: make it more concise

* Update CLAUDE.md

lol

* chore: tell claude not to search for it if it doesn't have it

---------

Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-07-03 14:14:40 -07:00
mubashir1osmani
a86dc4c15e
chore(e2e): untrack gateway config and document e2e test location (#31914)
* chore(e2e): untrack gateway config and document e2e test location

Stop tracking tests/e2e/gateway/litellm-config.yml so the local proxy config stays on the machine

Add a note to CLAUDE.md that new e2e tests belong in tests/e2e/ and must follow that directory's conventions

* chore(e2e): add self-contained docker compose stack for local runs

Ship a docker-compose.yml that starts the proxy with a throwaway Postgres and Redis and inlines the proxy config with example models, so contributors can bring up a local gateway with nothing but a .env. Update CONTRIBUTING.md to match the inline-config flow

* chore(e2e): drop the second gemini deployment; one key is enough locally

* docs(e2e): make pre-commit steps ordered and require flagging internally found issues
2026-07-02 19:22:02 -07:00
Mateo Wang
bd9db3691e
chore(lint): remove dead E501 config, fix stale blame-ignore SHAs, note 120 line width (#31927)
* chore(lint): remove dead E501 config, fix stale blame-ignore SHAs, note 120 width in CLAUDE.md

E501 sat in both lint.ignore and lint.extend-select in ruff.toml; ignore wins,
so no line length was linted at all (verified: a 130-char line passes ruff
check while T201 fires). Remove it from both lists so the config tells the
truth: the formatter's wrap width is the only line-length control, matching
how the repo has actually behaved since E501 was ignored in Oct 2024

.git-blame-ignore-revs listed the pre-squash PR-head SHAs for the two ruff
reformat commits (#31317, #31518), which never landed on the branch, so git
blame ignored nothing. Replace them with the squash-merge SHAs that are
actually in history

Also document in CLAUDE.md that the line length is 120 (ruff.toml), not 88,
so agents stop wrapping to the old Black width

* fix: make CLAUDE.md more concise

* fix: make the guideline more clear
2026-07-01 18:44:57 -07:00
Mateo Wang
700afbb6b2
chore: make CLAUDE.md rules more concise (#31892) 2026-07-01 16:26:38 -07:00
Mateo Wang
e141596204
refactor(lint): collapse type/lint budgets to a single per-rule limit (#31883)
* chore(lint): raise basedpyright per-rule slack to 50% of baseline

The per-rule ceilings in basedpyright-code-budget.json sat at roughly 10% slack over baseline, which several in-flight PRs are already bumping into. Raise the slack on every rule to at least 50% of its baseline so there is ample headroom for a long while, while never lowering any rule that already had more generous slack (e.g. reportReturnType stays at 100).

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* refactor(lint): collapse type/lint budgets to a single per-rule limit

The three non-frontend budget files (ruff-strict, type-discipline, basedpyright-code) tracked a per-rule baseline and slack whose sum was the ceiling. Nothing consumed the split beyond that sum, so this replaces both keys with a single limit equal to the old baseline + slack; the original baselines live in git history if anyone needs them.

The gate scripts and the ratchet guard now read limit directly. lint-budget-update no longer re-captures raw counts; it ratchets each rule's limit down by the number of violations this branch cleared since its branch point (the merge-base), so the granted headroom shrinks by exactly what was fixed and a limit never rises. The ratchet guard reads either schema so it still compares correctly across the migration boundary.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* chore(lint): surface staged-vs-working parity for pre-commit and budget-update

make pre-commit selects which checks to run from the staged index but runs the linters over the working tree, so unstaged edits to tracked files and untracked files skew a green/red away from what a commit of only the staged changes would produce. There is no safe in-place way to lint the index, so the script now warns when unstaged or untracked changes are present, and CLAUDE.md documents that you must stage everything first for both make pre-commit and make lint-budget-update to predict CI correctly.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* docs(lint): list type-discipline budget in lint-budget-update instruction

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-07-01 18:12:35 +03:00
Krrish Dholakia
50b936c75e
feat(guardrails/headroom): add CCR (compress-cache-retrieve) via agentic loop (#31681)
* feat(guardrails/headroom): add CCR (compress-cache-retrieve) support via agentic loop

When Headroom's /v1/compress returns messages containing hash markers
(hash=[a-f0-9]{24}), inject a headroom_retrieve tool into the request.
When the LLM calls that tool, intercept via async_should_run_agentic_loop
and async_build_agentic_loop_plan, call GET /v1/retrieve/{hash} on the
Headroom sidecar, and replay the LLM with the original content as a tool
result -- all transparent to the caller.

* style: run ruff format on headroom guardrail and tests

* fix(guardrails/headroom): detect headroom_retrieve calls in both OpenAI and Anthropic response formats

* test(guardrails/headroom): add test for Anthropic content block format detection in CCR loop

* ci: trigger CI checks

* fix(guardrails/headroom): replace List/Dict with list/dict to fix UP006 ruff violations

* fix(guardrails/headroom): replace except Exception with except ValueError to fix BLE001

* fix(guardrails/headroom): add Responses API output format detection for CCR tool calls

* refactor(guardrails/headroom): extract format-specific helpers to fix C901 complexity

* fix(guardrails/headroom): scope CCR retrieval to hashes produced by current request

Previously any LLM-supplied hash in a headroom_retrieve tool call was
forwarded to the Headroom retrieve API, letting a crafted tool call
fetch arbitrary cached content. Validate the hash against the set
produced by compressing the current request's messages before calling
retrieve.

* fix(guardrails/headroom): track issued hashes server-side, fix Responses API replay shape

Hash validation now also checks an in-memory cache of hashes actually
returned by /v1/compress, not just whether the hash text appears
somewhere in the request's messages. The message-text check alone is
forgeable: an attacker can plant a hash-shaped string in their own
prompt and have it treated as valid.

Responses API follow-up now emits function_call/function_call_output
items keyed by call_id instead of chat-style assistant/tool messages,
since the Responses API does not accept the latter as input. Also
fixes call_id/id field priority when extracting tool calls from
Responses API output, since call_id (not id) is what must match
between the function_call and its output.

* fix(guardrails/headroom): drop redundant quoted type annotations

UP037 flags quotes on annotations that are already lazily evaluated
via `from __future__ import annotations`.

* test(guardrails/headroom): add missing pytest.mark.asyncio decorators

Functional under asyncio_mode=auto, but every other async test in the
file has the decorator for consistency.

* fix(guardrails/headroom): scope CCR hashes per call_id, fix Anthropic replay shape

Two real gaps found in review:

1. The instance-wide issued-hash cache combined with a message-text
   check did not actually scope retrieval to the request that produced
   the hash. A hash issued for request A stays in the shared cache
   until TTL expiry, and the message-text check is satisfied by any
   request whose own messages happen to echo that hash string. Request
   B could plant A's hash in its own prompt and retrieve A's content.

   Fixed by keying the issued-hash cache by litellm_call_id, matching
   the pattern already used in compression_interception: a hash is
   only honored when it was issued under the exact call_id resolving
   for the current request.

2. The Anthropic Messages replay path fell through to the chat-style
   assistant/tool-message builder, which Anthropic does not accept.
   Anthropic requires the tool_use block echoed in an assistant message
   paired with a tool_result block in a user message, keyed by
   tool_use_id. Added a dedicated branch for this shape.

* docs: note proactive API-fragmentation helper convention

Add a bullet to the coding-conventions list: look for or add a shared
helper when logic branches on API surface (chat completions vs
Anthropic Messages vs Responses API), instead of duplicating
format-detection per module.

* fix(guardrails/headroom): fix Anthropic tool-shape detection, extract shared cross-API tool util

Live e2e testing against the real Anthropic API surfaced two bugs the
mocked unit tests couldn't catch because they used MagicMock responses
instead of realistic response shapes:

1. has_headroom_retrieve_tool only recognized OpenAI-shaped function
   tools. By the time an Anthropic Messages response reaches the
   agentic-loop gate, the tool this guardrail injected has already been
   transformed into Anthropic's native shape (type: "custom", top-level
   "name"), so the gate never fired for real Anthropic requests.

2. AnthropicMessagesResponse is a TypedDict, so real responses are
   plain dicts at runtime, not objects with attribute access. The
   extractors and format detectors used bare getattr(), which silently
   returns nothing for dict responses instead of reading the actual
   key.

Extracted the cross-API-surface tool-call extraction and tool-presence
check into litellm/litellm_core_utils/prompt_templates/factory.py
(get_tool_calls_from_response, has_tool_with_name) so this format
fragmentation is handled in one place instead of being duplicated
per-guardrail, and reused the existing repair-aware
parse_tool_call_arguments from common_utils instead of a naive
json.loads. headroom.py now delegates to these shared helpers.

Confirmed live against the real Anthropic API: the retrieve loop now
fires and successfully retrieves the correct hash's content through
the full compress -> tool-call -> retrieve -> replay round-trip.

* fix(guardrails/headroom): fix ruff-strict UP006/I001 budget violations

Use lowercase list/dict generics in the new factory.py tool-call
helpers instead of typing.List/Dict, drop the now-unused Tuple import
in headroom.py, and reorder the new factory import ahead of the
llms.custom_httpx import to satisfy import sorting.

* fix(guardrails/headroom): match Anthropic tools without a type field

Anthropic's documented client tool format is just name + input_schema;
type: "custom" is only one possible value, not a requirement. Match
any non-OpenAI-shaped tool on its top-level name instead of requiring
type == "custom".
2026-06-30 19:19:34 -07:00
Mateo Wang
59f51b2d72
chore: prevent CLAUDE.md comment bloat (#31729)
The existing comment rule is not strict enough
2026-06-30 10:46:47 -07:00
Mateo Wang
0965a4d1f4
chore: shift CI lint left with an opt-in make pre-commit and CLAUDE.md rule (#31544)
* chore: shift CI lint left with a pre-commit hook and CLAUDE.md rule

Add an opt-in pre-commit hook (.githooks/pre-commit, active after
make install-hooks) that runs the CI-equivalent checks against staged
files: make lint for Python, prettier plus eslint for the dashboard,
and a gen:api drift check for the proxy OpenAPI types. Document the
same expectation in CLAUDE.md so reds surface locally instead of in CI.

* fix: make `make lint` isomorphic to the CI lint job

`make lint` diverged from test-linting.yml in ways that produced both
false reds and false greens: its format-check ran over the whole repo
(CI scopes it to changed files vs the base), its ruff-strict budget ran
in absolute mode (CI runs it as a delta vs base), and it omitted the
type-discipline gate entirely. Recompose `lint` to replay CI's exact
sequence: diff-scoped ruff format check, whole-tree ruff check, the
strict / type-discipline / basedpyright budgets as a delta resolved the
same way CI resolves it (merge-base with origin/litellm_internal_staging),
then circular-import and import-safety. Factor the repeated base fetch
into one shared prerequisite so the chain hits the network once.

Align the pre-commit hook's eslint invocation with the CI frontend-lint
job (`--pass-on-unpruned-suppressions`) and fix the CLAUDE.md guidance to
point at the diff-scoped frontend commands instead of the whole-folder
npm scripts, which are broader than CI.

* fix(githooks): make pre-commit 1:1 with CI frontend-lint, lint, and type-gen

The shift-left pre-commit hook diverged from the CI jobs it claims to mirror, so a clean commit did not actually mean a green CI lint.

The dashboard block only ran prettier and eslint over js/jsx/ts/tsx/mjs/cjs, but CI's frontend-lint runs prettier over a wider set (also json, css, scss, md, mdx, yml, yaml, html) and additionally gates the whole-folder eslint lint budgets via scripts/check-lint-budgets.mjs. The hook now mirrors that split and runs the budget check, so a dashboard commit that passes locally passes the job.

The API-types block ran npm run gen:api without LITELLM_PYTHON, so it shelled out to the system python3 which has no litellm installed and always failed with a false 'could not regenerate API types' red. It now passes LITELLM_PYTHON="uv run --no-sync python" the way check-ui-api-types.yml does.

make lint format-checks the files in origin/base...HEAD, which at pre-commit time predates the staged change, so a brand-new commit's formatting went unchecked. The Python block now also runs ruff format --check over the staged litellm files directly to cover that case, and its trigger is scoped to staged litellm/ files (the only tree CI's lint job inspects) so a tests-only or scripts-only commit skips the slow make lint instead of wasting time on a run that could not catch anything.

CLAUDE.md's shift-left rule was cut off mid-sentence and understated the frontend checks; it now describes all three gates accurately and points agents at make install-hooks to run them automatically before each commit.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(githooks): scope the API-types check to all of check-ui-api-types.yml's triggers

spec_files was filtered from the staged Python files, so the gen:api drift
check only fired for .py changes under litellm/proxy or litellm/types. CI's
check-ui-api-types.yml triggers on any file under those directories (Prisma
schema, configs) plus the generator script and the dashboard package files,
so a non-Python proxy/types change could pass the hook and still fail CI.
Match the workflow's full trigger set instead.

* fix(pre-commit): run prisma generate before gen:api to mirror CI

* refactor(githooks): run shift-left lint via on-demand make pre-commit, not an auto-firing hook

The pre-commit hook ran make lint plus the dashboard eslint budgets, which are minutes of work (basedpyright over litellm/, a whole-folder eslint . pass at ~40s). Wiring that into core.hooksPath via make install-hooks meant every human commit, not just an agent's, paid that cost, which is real friction for interactive committers.

Move the staged-file checks out of .githooks/ into scripts/pre_commit_lint.sh and expose them as make pre-commit, and keep .githooks/ to only the fast Conventional Commits / Branches hooks so make install-hooks no longer makes commits slow. Agents run make pre-commit right before each commit (CLAUDE.md instructs this), so the slow gates fire only for the commits an agent is making and never auto-fire for a human typing git commit. The script stays hook-compatible for anyone who still wants it to fire automatically via a symlink.

Preferred this over sniffing an agent env var to auto-fire only for agents: that is fragile (misses agents when the var is unset, fires on humans when it leaks into their shell, and silently no-ops a hook a human deliberately installed), whereas an on-demand command achieves the same humans-never, agents-per-commit outcome deterministically.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(pre-commit): run make lint last so it can't prune the proxy deps gen:api needs

make lint's install-dev prerequisite runs uv sync --frozen, which prunes the proxy extras (prisma, websockets, ...) from the venv. With the Python block running first, the subsequent API-types block then failed: gen:api imports litellm.proxy.proxy_server, which needs those deps, so every litellm/proxy change (the main trigger for the API-types check) hit a false 'could not regenerate API types' red. Run the dashboard and API-types blocks before the Python block so gen:api sees an intact env; CI is unaffected because there the lint and check-ui-api-types jobs run in separate environments.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix: make CLAUDE.md more concise

* fix(makefile): give make lint the CI lint env and stop it pruning the venv

make lint diverged from test-linting.yml's lint job in two ways: it never generated the Prisma client (so basedpyright resolved the DB wrappers as Unknown, drifting from CI's counts), and its bare uv sync --frozen pruned the proxy extras (prisma, websockets, ...) out of the venv on every run, which broke the gen:api step that imports litellm.proxy.proxy_server and left a dev unable to run the proxy until re-syncing.

Add a lint-install target that mirrors the job's environment (the proxy-dev group plus prisma generate) and runs before the checks, and make both it and install-dev use uv sync --inexact so they top up the venv instead of tearing packages out. CI is unaffected since it installs its own env per job.

Because make lint no longer prunes, the pre-commit reorder that ran it last (to dodge the prune) is no longer needed, so restore the original block order.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(makefile): drop lint-install so make lint matches CI's slimmer env

test-linting.yml's lint job installs deps with a bare uv sync --frozen
(default dev group only, no proxy-dev, no prisma generate), but the
lint-install target chained into make lint pulled in --group proxy-dev
and ran prisma generate. Because the basedpyright budget step compares
head and base counts against fixed thresholds, the extra symbols and
Prisma client locally resolved can shift error counts away from CI's,
producing false greens or false reds on the type-check gate.

Remove the lint-install target and its slot in lint. The remaining
sub-targets already chain install-dev, which now uses
uv sync --inexact --frozen, so the venv still isn't pruned but the
installed set stays aligned with what CI sees.

* ci(linting): install proxy-dev and generate prisma in lint job, matching make lint

make lint now installs the proxy-dev group and generates the Prisma client so basedpyright resolves the DB wrappers; the lint job here still installed only the base env, so a local pre-commit could pass while the required CI lint failed (or vice versa). Bring this job in line, which is the same environment litellm_internal_staging's lint job already uses.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(makefile): keep make lint on the proxy-dev + prisma env to match CI

A concurrent change dropped lint-install to match what looked like CI's slim env, but test-linting.yml's lint job (and the merge ref this PR's CI actually runs) installs --group proxy-dev and generates the Prisma client. With make lint slim and CI fat, basedpyright resolves fewer symbols locally than CI, so a prisma-typed error can stay Unknown locally (green) while CI catches it (red). Restore lint-install so make lint installs the same env CI does; the previous commit also brought this PR's test-linting.yml in line with that env, so the two now match.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-06-30 09:37:12 -07:00
Mateo Wang
c5833a9d70
fix: inverted rule in CLAUDE.md (#31370) 2026-06-25 17:00:12 -07:00
Mateo Wang
0a92734691
fix: clarify further that customer names shouldn't be made public (#31365)
* fix: make it clearer that customer names should not be put in PR descriptions

* fix: revise the policy to be stricter
2026-06-25 16:27:27 -07:00
Mateo Wang
c0a146929c
chore: clarify rule about trailing periods (#31175)
I notice Claude applies the rule irregularly. This aims to fix that
2026-06-23 22:05:59 -07:00
Mateo Wang
b8d79d1e0c
ci: drop mypy entirely, standardize type checking on basedpyright (#30648)
* ci: drop redundant mypy type-check gate, standardize on basedpyright

Type checking ran both mypy (via the pydantic.mypy plugin) and basedpyright.
pydantic v2 emits dataclass_transform, so basedpyright understands models
natively with no plugin, and its gated rules already cover what the mypy pass
caught (no-untyped-def, no-any-return, valid-type, import-not-found all map to
basedpyright equivalents). Running both meant two checkers, two budgets, and a
plugin only mypy could load.

This removes the mypy type-check gate: the lint-mypy/lint-mypy-budget-update
Makefile targets, the CI MyPy step, mypy-code-budget.json, the budget-ratchet
entry, and the vestigial [tool.mypy] pydantic plugin block (the gating pass used
litellm/mypy.ini, which never loaded the plugin). type_check_gate.py is
specialized to basedpyright since the mypy parsing path is now unused.

mypy stays a dev dependency because the Any-discipline gate
(scripts/check_any_discipline.py) imports it as a library to detect Any-typed
values; it is no longer run as a type checker.

* ci: remove the Any-discipline gate, rely on basedpyright's reportAny

The Any-discipline gate (scripts/check_any_discipline.py) was the last consumer
of mypy: it imported mypy as a library to detect values whose inferred type
contains Any, gated per-file against any-discipline-budget.json. basedpyright
already reports the same class of finding through reportAny/reportExplicitAny,
which are gated tree-wide in basedpyright-code-budget.json, so the separate gate
(and the mypy dependency behind it) is redundant.

Removes the gate end to end: check_any_discipline.py and its test, the
any-discipline CI job, the lint-any/lint-any-budget-update Makefile targets,
any-discipline-budget.json, litellm/mypy.ini, the .mypy_cache_any references,
and mypy from the dev dependencies. budget_ratchet_check.py drops the
any-discipline entry and the now-unused zero-floor mechanism (rewritten as a
comprehension). check_type_discipline.py drops the any-ok suppression token,
since # any-ok suppressed only the deleted gate; the 134 now-orphaned
# any-ok comments across 14 files are stripped (they never affected
basedpyright, which uses # pyright: ignore).

uv.lock is intentionally left untouched: uv still considers it consistent with
the mypy-removed pyproject (uv lock --check and uv sync --frozen both pass), and
a relock bumps 30+ unrelated packages because of the moving exclude-newer window.
A future intentional relock will prune the now-unreferenced mypy entry.

* build: relock to drop mypy from uv.lock

CI's uv 0.10.9 honors the repo's exclude-newer window and correctly flags the
lockfile as out of sync once mypy leaves pyproject; my earlier local uv 0.8.17
could not parse exclude-newer and silently passed --check. Relocking with the
pinned CI version removes only mypy and its transitive librt, with no other
version changes.
2026-06-17 09:42:00 -07:00
Mateo Wang
17b88719a2
ci(lint): grandfather any-discipline with a per-file ratchet budget (50% headroom) (#30582)
* ci(lint): grandfather any-discipline with a per-file ratchet budget (50% headroom)

The any-discipline gate previously failed on any Any-typed value touched on a
changed line, which tripped on merely editing a legacy `X | Any` line. Switch it
to a per-file budget: `any-discipline-budget.json` records each file's current
Any count and a changed file fails only when its count exceeds `baseline + slack`
(50% headroom, rounded up). New/unbudgeted files have baseline 0, so they stay
airtight, while editing legacy files no longer forces cleaning pre-existing debt.

Only changed files are re-type-checked (per-PR cost unchanged); the whole-tree
scan to recapture the budget runs under `--update` (`make lint-any-budget-update`).
The budget is a one-way ratchet guarded by `budget_ratchet_check.py`, matching the
ruff/mypy/basedpyright budgets, and folds into `make lint-budget-update`.

Also fixes a RecursionError in `contains_any` (recursive type aliases yield fresh
objects per unfold, defeating the id() cycle guard) by walking iteratively with a
depth cap, exposed by the whole-tree scan.

* chore: make CLAUDE.md more concise

* chore: rearrange Makefile

* ci(lint): make any-budget --update git-failure-safe; clarify over-budget message

all_litellm_py_files now returns None when git is unavailable (mirroring
changed_line_map) instead of letting CalledProcessError/FileNotFoundError escape
as a raw traceback, and update_budget reports a clean setup error (exit 2) for
that case. The list-files dependency is injected so the path is unit-testable
without monkeypatching. The over-budget diagnostic now reads "N value(s) total,
over budget" so the count isn't misread as the excess over the ceiling.

* ci(lint): exempt the file-keyed any-discipline budget from the ratchet's dropped-entry rule

budget_ratchet_check treats a vanished budget entry as a loosening (an untracked
rule whose ceiling is now unbounded). That holds for the rule-keyed budgets, but
the any-discipline budget is keyed by file and its gate treats an absent file as
ceiling 0 (the file must be Any-free). Cleaning a file to zero drops its entry on
the next --update, so the generic rule flagged that as a regression: a false-
positive red on exactly the cleanup the ratchet exists to encourage. Exempt the
file-keyed budget from the dropped-entry rule while still catching a raised
ceiling.
2026-06-16 19:23:20 -07:00
Mateo Wang
be4fa702e7
ci(lint): ratcheted type-discipline gate (mutable collections, casts, guards, kwargs, suppressions) (#30500)
* ci(lint): enforce type-discipline budget for casts and type guards

Add a ratcheted gate that blocks net-new typing.cast() usage and bans
TypeGuard/TypeIs outright, layered on the existing ruff-strict budget setup.

- ruff-strict.toml: ban cast/TypeGuard/TypeIs (typing + typing_extensions)
  via flake8-tidy-imports banned-api (TID251) for a coarse import-level freeze.
- ruff-strict-budget.json: bump TID251 baseline 2404 -> 2662 to absorb the
  ~258 pre-existing usages now matched by the new banned-api entries.
- scripts/check_type_discipline.py: AST checker adding LIT006 (cast call sites,
  suppress with `# cast-ok: <reason>`) and LIT007 (TypeGuard/TypeIs annotations,
  suppress with `# guard-ok: <reason>`) for per-call-site granularity.
- scripts/type_discipline_gate.py: baseline+slack gate with delta-vs-base,
  mirroring ruff_strict_gate.py.
- type-discipline-budget.json: LIT006 baseline 1013 (slack 10), LIT007 0/0.
- test-linting.yml: run the gate in CI against the PR base SHA.

* ci(lint): enforce suppression-reason budgets and guard budgets against loosening

- wire the **kwargs ban (LIT008) into the vendored type-discipline checker so it
  matches the budget that already referenced it
- freeze LIT003/LIT004 (noqa / type-ignore without codes or reason) and LIT005
  (*-ok suppression without a reason) at slack 0 so any net-new unexplained
  suppression trips the type-discipline gate
- add scripts/budget_ratchet_check.py and a separate, non-gating budget-ratchet CI
  job that turns red when any *-budget.json ceiling is raised, a rule is dropped,
  or a budget file is deleted

* ci(lint): ban mutable collections in annotations and all mutable construction

Expand LIT001 from coarse builtins at interfaces to any mutable collection
in any annotation (builtins, typing aliases, collections concretes, mutable
ABCs) across signatures, class attributes, locals, and globals. Add LIT009 to
flag mutable-collection construction (literals, comprehensions, constructors)
so the unannotated seed-then-mutate pattern is caught too. Enumerate any-ok in
LIT005 so its reason requirement holds even when only the stdlib checker runs.
Budget LIT001 (21452) and LIT009 (25222) with slack 10 to ratchet down.

* ci(lint): recommend pydantic at boundaries and add functional-refactor guidance

Drop the msgspec mention from the cast banned-api messages so the recommended
validation path matches the codebase's primary pattern (pydantic). Add a note to
CLAUDE.md that lint / type-discipline failures should be resolved by refactoring
to functional, immutable patterns rather than reaching for mutable structures or
`# mutable-ok`.

* style: make CLAUDE.md more concise

* chore: update CLAUDE.md guidelines

* ci(lint): renumber mutable construction LIT009 -> LIT002 next to LIT001

Group the mutable-collection family together: LIT001 (mutable collection in any
annotation) and the construction rule now sit adjacent at LIT001/LIT002. The
freed LIT009 slot is taken by the sibling Any gate (check_any_discipline.py,
#30379), which moves its Any-typed-value rule LIT002 -> LIT009 in lockstep so
the shared LIT namespace stays contiguous with no holes. Budget, gate docstring,
and the checker's own docstring/messages are updated to match.

* fix: numbering in CLAUDE.md

* test(lint): test type-discipline checker, scope LIT007 to return types

Add regression tests for check_type_discipline.py (every LIT rule, its
suppression, and the comment scanner) and for budget_ratchet_check.py.

Confine LIT007 to function return annotations, the only place TypeGuard/TypeIs
are valid, so a runtime name that merely reads those identifiers is no longer
flagged. Switch scan_comments to io.StringIO(source).readline, the standard
readline that returns '' at EOF, dropping the iter(...).__next__ idiom.

* fix(lint): best-effort worktree teardown so cleanup can't mask the real error

base_counts ran `git worktree remove` through the raising `_run` in its finally,
so a failed `git worktree add` (or a failure in the body) was masked by a second
SystemExit from the cleanup. Tear the worktree down best-effort, like the sibling
rmtree, so the original error propagates.

* fix(lint): ratchet fails loudly on an unresolvable base; drop dead checker state

Verify the merge-base ref resolves to a commit before trusting a missing-file
result from git show, so an invalid or empty BASE_SHA now turns the budget-ratchet
guard red instead of skipping every budget and passing vacuously

Also drop the unused Comments.by_line field and the phantom --changed-only usage
line from check_type_discipline's docstring, and cover the ref handling with tests

* fix(lint): degrade malformed source to LIT000 instead of crashing the checker

tokenize.generate_tokens raises IndentationError (a SyntaxError subclass) on a dedent
mismatch, which escaped scan_comments' tokenize.TokenError handler and crashed the whole
checker run, zeroing the gate for that invocation. Catch SyntaxError too so the file
falls through to ast.parse and is reported as LIT000, matching the checker's
graceful-degradation contract. Also add the trailing newline ruff-strict.toml lacked

* perf(lint): skip the base worktree scan when no rule is over its ceiling

cmd_check created a git worktree and re-scanned the base tree on every run, but a
rule can only breach when its head count is already over baseline + slack; when none
are, the base comparison cannot change the verdict. Short-circuit to OK in that case,
which is every green PR, roughly halving the gate's work. Extract over_ceiling and
cover it (and evaluate's drift-safety) with tests

* fix(lint): exempt .dict()/.list()/.set() method calls from LIT002

_construction_kind matched dict/list/set as constructors via func.attr too, flagging
common method calls like pydantic's model.dict() as mutable construction; 200 such
false positives existed in litellm. Recognize dict/list/set construction only when
unqualified while keeping the collections concretes (deque/defaultdict/...) matchable
as attributes, since those are rarely method names. Ratchet the LIT002 baseline down
25222 -> 25022 to reflect the removed false positives

* chore(lint): bump basedpyright ceilings to absorb staging base drift

The basedpyright gate added in #30379 is a total-count check against
basedpyright-code-budget.json and the linting workflow runs only on
pull_request, so pushes to litellm_internal_staging never re-baseline it.
Merging staging into this branch surfaced that drift: seven
reportAny/reportUnknown* rules sit 10-149 errors above their committed ceiling
even though this PR changes no files under litellm/, the only path basedpyright
scans (pyrightconfig include is litellm). The new baselines match the counts CI
measured on the merge commit, with the existing per-rule slack preserved

* fix(lint): ratchet guard watches every budget file, not just two

DEFAULT_BUDGETS only listed ruff-strict-budget.json and
type-discipline-budget.json, so mypy-code-budget.json and
basedpyright-code-budget.json were unguarded and their ceilings could rise with
no signal, which is exactly the failure mode this guard exists to prevent. The
gap became concrete when this PR bumped basedpyright-code-budget.json to absorb
staging drift. All four budgets are now watched, so the budget-ratchet job
surfaces that basedpyright bump for human review the same way it surfaces the
TID251 raise. A regression test pins that every *-budget.json on disk is in
DEFAULT_BUDGETS, failing loudly if a future budget escapes the ratchet

* fix: add a lot more slack

* fix(lint): restore LIT003 frozen slack to 0

The blanket slack bump set LIT003 (bare # noqa without codes or a reason) to a
slack of 50, which contradicts the documented zero-tolerance invariant: the gate
docstring and the PR description table both freeze LIT003/LIT004/LIT005 at slack
0 so any net-new unexplained suppression trips the gate. Slack 50 would let 50
new bare noqas through silently. The actual LIT003 count is 397, well under the
516 baseline, so restoring slack to 0 keeps the gate green while putting the
freeze back. LIT004/LIT005/LIT007 were already correct at 0

* fix(lint): restore documented slack 10 for the buffered LIT rules

The slack bump left LIT001/LIT002/LIT006/LIT008 at 2000/2500/100/100, 10-250x the
"/ 10" the PR description table and the gate docstring document. That buffer was
never needed: the gate already blames a rule only when its count exceeds the
ceiling and grew vs the merge-base, so the violations the staging merge added in
litellm/ sit in both head and base and are never charged to this PR. With slack
back at the documented 10 the gate stays green, and the ceiling is tight again
(LIT006 no longer waves through 99 net-new cast() calls). Baselines are
unchanged; only the slack returns to its documented value

* fix(lint): ratchet LIT003 baseline down to its actual count

The LIT003 baseline was 516 while the current bare-noqa count is 397, leaving
~119 units of headroom that undercut the documented zero-tolerance freeze: the
gate docstring claims any net-new bare noqa trips the gate, but with cap 516 a PR
could add over a hundred first. Drop the baseline to the measured 397 so the
freeze is exact (cap = 397 + slack 0), the same hard-zero-at-the-boundary shape
LIT005 and LIT007 already use and pass in CI. PR table row updated to 397 / 0

* fix: increase slack

* fix: increase slack

* docs(lint): align gate docstring with buffered LIT003/LIT004 slack

The budget now gives LIT003/LIT004 nonzero slack, so the gate's prose no
longer claims they are frozen at slack 0; LIT005 remains the reasonless-
suppression freeze and LIT007 the hard zero.
2026-06-16 16:59:21 -07:00
Mateo Wang
d0c2e87810
ci: ratchet lint and type-check gates (ruff preview, ANN, mypy, basedpyright) (#30379)
* ci: enable ruff preview rules under the budgeted strict gate

Turn on ruff preview in the strict-budget lane (ruff-strict.toml) only,
leaving the clean gate (ruff.toml) untouched so make lint-ruff stays at
zero. Enumerate the 118 firing codes explicitly with
explicit-preview-rules so the gate is deterministic and stable across
ruff upgrades rather than depending on preview auto-selecting the broad
catalog.

Grandfather the existing 58438 violations into ruff-strict-budget.json
as per-rule baselines with headroom, so only net-new violations fail CI.
The existing ten rules keep their hand-tuned slack; the new rules get
slack 10 when the baseline is 50 or more and 3 otherwise.

* ci: add ANN return-type rules to the budgeted strict gate

Add ANN201/202/204/205/206 (missing return annotations) to the strict
lane and grandfather the existing counts into ruff-strict-budget.json so
the codebase ratchets toward explicit return types without breaking CI.

* ci: add mypy (disallow_untyped_defs) and basedpyright strict gates with baselines

Add two type-check gates, each grandfathering the current tree so only
net-new violations fail CI, matching the ruff strict-budget ratchet.

mypy gains disallow_untyped_defs in litellm/mypy.ini (the config the CI
invocation actually reads; the root [tool.mypy] is not picked up from the
litellm/ working dir). The 4885 existing missing-annotation errors are
captured in litellm/.mypy-baseline.txt and the run is piped through
mypy-baseline filter so new untyped defs are rejected.

basedpyright runs in strict mode over litellm/, with
enableTypeIgnoreComments disabled so it only honors '# pyright: ignore'
and never polices mypy's '# type: ignore'. The existing strict diagnostics
are grandfathered into .basedpyright/baseline.json.

Both tools are pinned in the dev group and uv.lock; the lint workflow and
Makefile run them filtered through their baselines, with
lint-mypy-baseline-update and lint-basedpyright-baseline-update to ratchet.

* ci: raise lint job timeout to 15m for the basedpyright strict pass

* ci: pin pythonVersion 3.12 and regenerate baselines against merged base

Merge litellm_internal_staging so the baselines cover code the CI merge
includes (e.g. the cisco_ai_defense guardrail), which otherwise tripped
the mypy gate with 3 ungrandfathered no-untyped-def errors. Pin
pythonVersion 3.12 in pyrightconfig so basedpyright's strict analysis is
reproducible across interpreter versions (CI runs 3.12).

* ci: regenerate basedpyright baseline against the frozen lint env

The previous baseline was generated with optional provider deps (azure,
google, anthropic, mcp, numpydoc, google-genai) installed locally, so CI's
dev-only env surfaced ~3500 reportUnknown*/reportMissingTypeStubs errors
not in the baseline. Regenerate after uv sync --frozen so the baseline
reflects the same dependency set the lint job sees.

* ci: regenerate basedpyright baseline on python 3.12 frozen env

The prior baseline still carried proxy-dev packages (e.g. prisma) that the
lint job's dev-only, python 3.12 env lacks, leaving 2 unresolved-import
errors ungrandfathered. Regenerate in a python 3.12 venv synced to the
frozen lock with default groups only, so the baseline matches exactly what
CI sees.

* ci: replace type-check baselines with per-file count budgets

The mypy and basedpyright baselines were position-sensitive (and the
basedpyright one was a 27MB file), so ordinary line shifts churned them.
Replace both with a per-file count gate: scripts/type_check_gate.py reduces
each tool's output to errors-per-file and checks it against a committed
{file: max} budget, ignoring line and column numbers. A file fails only
when it gains more errors than its ceiling; debt can't be shuffled between
files because each file has its own cap and new files default to zero.

Budgets (mypy-file-budget.json 48K, basedpyright-file-budget.json 96K) are
generated in the python 3.12 frozen lint env so they match CI. Drops the
mypy-baseline dependency; basedpyright runs without its native baseline.
ratchet via make lint-mypy-budget-update / lint-basedpyright-budget-update.

* ci: add a small per-file slack to the type-check gate

Allow each file to drift PER_FILE_SLACK (5) errors past its recorded count
before failing, so a basedpyright inference ripple in an unrelated file
doesn't break the build over a couple of errors. Budgets still record exact
counts; the tolerance is applied at check time.

* ci: move type-check slack into the budget json and trim lint timeout

Make slack declarative: the budget is now {"slack": N, "files": {path: count}}
so the tolerance is tuned in JSON without editing the script, mirroring how
ruff-strict-budget.json carries its slack. --update preserves the existing
slack. Also drop the lint job timeout from 15m to 10m; the mypy and
basedpyright passes add ~2m, leaving the job around 4-5m, so 10m is a
comfortable margin.

* ci: collapse fully-adopted ruff categories and drop inert preview flag

ANN (all nine non-removed rules) and BLE (its only rule) were spelled out
code-by-code; replace each with its category selector, which is exactly
equivalent in 0.15.3 (the removed ANN101/ANN102 are skipped by a category
selector and error when named explicitly). explicit-preview-rules was inert:
every selected rule is stable and nothing is selected by category, so the flag
had nothing to gate. Verified the strict-rule counts are identical before and
after (62379 each, zero per-rule drift), so no budget change.

* ci: drop redundant pyright dev dependency

Nothing invokes bare pyright in the Makefile, the linting workflow, or
scripts; the basedpyright gate added on this branch is the only type
checker that runs. basedpyright is a superset fork that reads the same
pyrightconfig.json and honors the same "# pyright: ignore" comments, so
pyright==1.1.408 in the ci group was dead weight. Regenerated uv.lock
under the same exclude-newer cutoff so the only change is removing
pyright and its package stanza

* ci: un-weaken mypy and error on Any in basedpyright

mypy: enable warn_return_any, drop the valid-type silencer, and stop globally ignoring missing first-party imports via [mypy-litellm.*] ignore_missing_imports = False, which surfaced eight real broken litellm.* imports the blanket ignore was hiding; third-party imports stay ignored. The per-file budget moves 4888 -> 5799 (902 no-any-return, 1 valid-type, 8 import-not-found), all grandfathered so only net-new errors fail and the ceilings ratchet down

basedpyright: error on reportExplicitAny and reportAny. The per-file budget moves 117033 -> 148946 (6931 explicit-Any, 24954 Any-typed expressions), grandfathered the same way

* ci: add Any-discipline gate on changed lines under litellm/

Add scripts/check_any_discipline.py, a type-aware gate that fails when a
changed line holds a value typed Any -- including the X | Any unions that
mypy --strict / basedpyright accept (e.g. re.Match.group() -> str | Any,
json.loads() -> Any, bare dict -> dict[Any, Any]).

It reuses the repo's mypyc-compiled mypy 1.19 via a custom generic AST
walker (mypyc precludes subclassing TraverserVisitor), loads litellm/mypy.ini
for parity with lint-mypy, and uses a dedicated incremental cache
(.mypy_cache_any) with mtime+hash invalidation to force re-checks. Scope is
changed-lines-only so editing a legacy file never forces cleaning its
existing Any debt; suppress a genuine typed/untyped boundary with
# any-ok: <reason> (ANY002 requires the reason).

Wire it into the Makefile (lint-any, lint, lint-dev), a parallel
any-discipline CI job with its own actions/cache, .gitignore, and the
CLAUDE.md / CONTRIBUTING.md docs.

* ci: move Any-gate codes into the shared LIT namespace

Renumber the Any-discipline checker into the LIT*** scheme owned by
scripts/check_type_discipline.py (PR #30500) so the two checkers share one
rule namespace and suppression convention:

  ANY001 -> LIT002  (Any-typed value; LIT002 was the retired/free slot)
  ANY002 -> LIT005  (any-ok without a reason; the shared suppression-reason code)
  ANY000 -> LIT000  (setup/build/read error; the shared error code)

Messages and behavior are unchanged; LIT005's text already matches the
"<token> requires a reason" shape used for cast-ok/guard-ok.

* ci: gate mypy and basedpyright per error rule, not per file

Switch the mypy/basedpyright budget gate from per-file error counts to
per-rule-code totals, mirroring the {rule: {baseline, slack}} shape of
ruff-strict-budget.json. A rule fails when its codebase-wide error count
exceeds baseline + slack, so violations are tracked by category rather
than by file location.

scripts/type_check_gate.py now parses mypy from its text output (trailing
[code]) and basedpyright from --outputjson (the JSON `rule` field), since
basedpyright's wrapped text diagnostics mis-attribute the rule on
continuation lines. Replace the *-file-budget.json files with freshly
captured *-code-budget.json baselines and update the Makefile, CI, and
CLAUDE.md accordingly.

* docs: prefer Pydantic validation over any-ok suppression

Point the Any-discipline guidance at validating Any with Pydantic (a model
or TypeAdapter that returns a typed value or raises) and frame
# any-ok as a last resort that should ideally never be used.

* chore: remove extraneous comment

* chore: make the CLAUDE.md more concise

* chore: clean up bloated CONTRIBUTING.md additions

* chore: make Makefile more concise

* ci: add the lint-budget-update target CLAUDE.md references

CLAUDE.md tells contributors to run make lint-budget-update, but the
target was never defined. Add it as an aggregate that re-captures the
ruff, mypy, and basedpyright budgets in one shot.

* ci: recapture mypy and basedpyright budgets in the lint env

The per-rule baselines were captured in a richer dependency env than the
CI lint job's uv sync --frozen, so CI resolved fewer types and reported
more errors than the budgets allowed (no-any-return 902 over cap 900, plus
several basedpyright reportUnknown* rules). Regenerate both in the frozen
env so they grandfather the true CI debt: mypy 5786 -> 5799 (no-any-return
890 -> 902, valid-type 1 restored), basedpyright 146213 -> 148942.

* ci: check out PR head sha in lint and any-discipline jobs

The default pull_request checkout uses refs/pull/N/merge, which folds the
latest base commits into HEAD. The diff-based gates (ruff delta, Any
discipline) then diff against the event's older base.sha and blame base's
own new commits on this branch; staging's otel-v2 and streaming changes
(#30326, #30485) tripped the Any gate on files this branch never touched.
Checking out the PR head sha makes the gates diff the real branch tip
against base, and pins the tree the mypy/basedpyright budgets were captured
against so their counts stay deterministic as the base advances.

* ci(lint): renumber Any-typed-value rule LIT002 -> LIT009

Free up LIT002 for the sibling type-discipline gate (check_type_discipline.py,
#30500), which groups its mutable-collection family at LIT001 (annotation) and
LIT002 (construction). This gate's Any-typed-value rule moves to LIT009 so the
shared LIT namespace stays contiguous with no holes; LIT000 and LIT005 are
unchanged.

* style: rename lint-strict-budget -> lint-ruff-budget

* ci: harden type-check gates against silent passes (greptile review)

type_check_gate.py: refuse to certify a vacuous run. The CI pipe swallows
the tool's exit code ('tool || true'), so a crashed mypy/basedpyright that
emits nothing would parse to zero errors, breach no ceiling, and pass.
is_vacuous_run() now fails when nothing was parsed but the budget expects
errors. Also wrap basedpyright's json.loads in a JSONDecodeError handler
that prints the offending output instead of dumping a raw traceback.

check_any_discipline.py: ALL_LINES was None, which dict.get() also returns
for a path absent from the line map, so a path-normalisation mismatch could
let a violation on an unchanged file pass the scope filter. Make ALL_LINES a
distinct sentinel object so 'whole file' and 'path missing' are unambiguous.

Adds tests for all three.
2026-06-16 12:07:46 -07:00
ryan-crabbe-berri
c90eb7e96f
feat: ruff strict-rule suppressions baseline gate (#30303)
* feat: add ruff strict-rule suppressions baseline gate

Introduce a stricter ruff rule set (typed params, no Any, complexity and
arg-count caps, mutable-default and global-rebinding checks) grandfathered
against the current tree and enforced as a budget rather than zero-tolerance

ruff-strict.toml defines the 9 rules separately from ruff.toml so the existing
ruff check stays green. scripts/ruff_suppressions.py builds the per-file,
per-rule baseline in ruff-suppressions.json and gates CI by failing when the
total grows past the baseline plus a 0.5% slack margin. The baseline ratchets
down via `make lint-suppressions-update` after fixes

* fix: surface per-file drift as a warning on a passing suppressions check

Greptile flagged that cmd_check computed per-file regressions but only printed
them on failure, so violations shifted between files (or a brand-new file under
the slack) passed with a silent OK. Print them as a non-fatal warning on the
pass path too; pass/fail behavior is unchanged

* refactor: gate strict ruff rules on the delta vs base, not a frozen baseline

The committed total-count baseline went stale against a moving base. CI lints the
PR merged with the current staging tip, so violations merged by other PRs counted
against this PR and tripped the budget even though nothing here touched them

Replace it with a drift-proof gate. scripts/ruff_strict_gate.py runs ruff on the
head, keeps only violations on lines this change adds relative to the merge-base,
and fails when a rule exceeds its per-rule allowance in ruff-strict-budget.json
(all 0 today). Because the base is measured live, base drift cancels out and only
what the change introduces is gated. Drops ruff-suppressions.json and the old
suppressions script

* chore: allow 5 new ANN001/ANN003/ANN401 per change

Give the three annotation-completeness rules a small per-change allowance so a
large new module is not blocked over a few untyped params or kwargs, while the
correctness and structural rules (B006, C901, PLR0913, PLW0603, RUF012, ANN002)
stay at 0

* feat: add TID251 typing.Any/Dict import ban and widen annotation budgets

Add TID251 (flake8-tidy-imports banned-api) to ruff-strict.toml, banning new
imports of typing.Any and typing.Dict and steering new code toward structured
types. It counts the import site, about one per file, so it is set non-blocking
at 50 as a forward-looking signal

Widen the annotation-completeness budgets so they nudge rather than block:
ANN001 50, ANN401 50, ANN003 25. Correctness and structural rules stay at 0

* refactor: make the strict gate a drift-safe per-rule total ceiling

Switch the gate from a per-change allowance to a hard ceiling on each rule's
total count across the codebase. The ceiling is baseline + slack in
ruff-strict-budget.json, with baseline captured from today's tree

To stay drift-safe, the gate counts each rule on the head and on the merge-base
(via a throwaway git worktree) and fails a rule only when its head total is over
the ceiling and higher than the base, so base drift never blames a change that
did not add to that rule. Annotation rules keep generous slack (ANN001 and
ANN401 50, ANN003 25, TID251 50); structural and correctness rules are frozen at
today's count. Add make lint-strict-budget-update to re-capture baselines

* chore: give the structural strict rules a cushion of 3

To be liberal to start, B006, C901, PLR0913, PLW0603, RUF012, and ANN002 each get
a slack of 3 instead of 0, so an occasional legitimate case is not hard-blocked.
The annotation budgets are unchanged, and these ratchet down later

* feat: ban more typing collection aliases and tighten annotation slack to 10

Add typing.List, typing.Set, typing.MutableSequence, and typing.MutableMapping to
the TID251 banned-api list, steering new code toward tuple, Sequence, Mapping,
frozenset, and frozen dataclasses. This raises TID251's baseline to 2404

Bring the three rules that were at slack 50 (ANN001, ANN401, TID251) down to 10

* docs: document the strict-gate ratchet and Any-avoidance in CLAUDE.md

Add a line on running make lint-strict-budget-update to knock baselines down
after fixes, and a line on validating untyped inputs in the caller rather than
spending the Any budget

* feat: make it a bit more strict

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-06-12 20:14:45 -07:00
Mateo Wang
6f8e6adf23
feat: strengthen coding conventions in CLAUDE.md (#30333)
* feat: strengthen coding conventions in CLAUDE.md

* fix: make hand-roll rule clearer
2026-06-12 18:11:06 -07:00
Mateo Wang
ba72ccf52c
feat: add conventional commits and coding guidelines (#30159)
* feat: add guideline for conventional commits

* feat: add functional programming coding conventions
2026-06-10 16:34:08 -07:00
Mateo Wang
8259d6cd85
fix: small CLAUDE.md nit (#29749) 2026-06-05 06:30:05 +00:00
Mateo Wang
84c4c12f90
fix: small CLAUDE.md nits (#29504)
* fix: small CLAUDE.md nits

* fix: make test rule more concise

* fix: add arrow rule
2026-06-02 09:02:47 -07:00
Sameer Kankute
68952a55d7
docs(agents): clarify when to create new test files (#29472)
* docs(agents): clarify when to create new test files in CLAUDE.md

Document that bug fixes should extend existing mapped test files while new
features may add files under the mirrored tests/test_litellm/ layout.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(agents): clarify test file naming conventions in CLAUDE.md

Address Greptile feedback: document test_<filename>.py vs descriptive
test_*_transformation.py patterns and when to match existing names.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-01 21:10:42 -07:00
Mateo Wang
f27df8d516
docs: hand-written CLAUDE.md; point GEMINI.md and AGENTS.md at it (#29252)
Some checks are pending
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
* docs: replace generated CLAUDE.md with hand-written guidance, remove AGENTS.md

Swap the auto-generated CLAUDE.md for a concise hand-written version that captures how we actually want agents to work in this repo: minimal comments, simplicity first, meaningful tests with a high mutation kill rate, PRs based off litellm_internal_staging rather than main, and curl against a live proxy as proof of fix instead of pasted pytest output. Remove AGENTS.md so there is one source of truth for agent guidance. The customer and company name confidentiality policy, along with the MCP available_on_public_internet note, are carried over from the previous CLAUDE.md.

* fix: further clarify communication guidelines

* docs: point GEMINI.md at CLAUDE.md instead of duplicating guidance

Replace the standalone GEMINI.md copy, which had already drifted from the new CLAUDE.md, with a one-line pointer so Gemini reads the same single source of truth.

* docs: simplify PR template test checklist item

Replace the rigid "at least 1 test is a hard requirement" checklist line with "I have added meaningful tests", which matches the testing guidance in CLAUDE.md, and tidy a comma into a semicolon in the scope-isolation item.

* docs: point AGENTS.md at CLAUDE.md instead of deleting it

Keep AGENTS.md so tools that read it still resolve guidance, but collapse it to the same one-line pointer to CLAUDE.md used by GEMINI.md, keeping a single source of truth.

* fix: make AI-generated rules more concise

* fix: spelling

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: make the .env usage more careful

* docs: restore MCP available_on_public_internet note to CLAUDE.md

The PR description states this note was carried over verbatim from the
previous CLAUDE.md, but it was dropped in the rewrite. Restore it so the
file matches the description and the team guidance is not lost.

* docs: restore browser storage and CI supply-chain safety notes to CLAUDE.md

These security-relevant rules were dropped in the rewrite. Restore the
sessionStorage-over-localStorage (XSS) guidance and the CI supply-chain
rules (no curl|bash, pin versions, verify checksums) so agents editing UI
or CI code are still steered away from those pitfalls.

* docs: move area-specific guidance into nested CLAUDE.md files

The MCP, browser-storage, and CI supply-chain notes are scoped to
particular parts of the tree, so move each into a nested CLAUDE.md that
Claude Code loads on demand when those files are touched: the MCP note
under the mcp_server gateway, the browser-storage rule under the UI
dashboard, and the CI supply-chain rules under .circleci. Keeps the root
CLAUDE.md focused on general guidance while the area notes surface where
they are relevant.

* docs: keep CI supply-chain note in root CLAUDE.md

CI guidance applies beyond .circleci (it also covers downloads in GitHub
workflows and any CI script), and CI work does not reliably touch a single
subtree, so a nested file under .circleci would not surface it dependably.
Keep it in the always-loaded root instead. The MCP and browser-storage
notes stay nested where they map cleanly to one area of the tree.

* fix: make it clear we prefer httpOnly

* chore: make ci rule more concise

* chore: make concise

Fix formatting and punctuation in MCP note.

* fix: don't include Claude attribution

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-05-29 00:05:05 -07:00
yuneng-jiang
b6fd7f7746
docs(agents): require consent before writing new third-party names (#28908)
Adds a rule to CLAUDE.md and AGENTS.md instructing AI coding agents to
pause and ask the user before introducing any third-party organization
name that does not already appear in the repository. Names already
established in the codebase (existing LLM providers, etc.) remain fine
to use without prompting.
2026-05-26 16:40:07 -07:00
Sameer Kankute
d855e56333
chore(mcp): warn on internal + upstream PKCE delegate
Log verbose_logger.warning when loading oauth2 interactive servers with
available_on_public_internet=false and delegate_auth_to_upstream=true
(config + DB). Dashboard Alert for the same combo. CLAUDE note for
operators. Tests for log and M2M skip.
2026-05-15 10:05:35 +05:30
Yassin Kortam
fa5eae8bc9
chore: remove legacy deployment artifacts and litellm-js packages (#27541)
- Remove litellm-js/proxy and litellm-js/spend-logs TypeScript packages that provided Cloudflare Worker proxy and Node.js spend logging services, as these are no longer maintained
- Remove deprecated Docker variants (Dockerfile.alpine, Dockerfile.dev, Dockerfile.custom_ui, Dockerfile.health_check, Dockerfile.ghcr_base) that have been superseded by the primary Dockerfile
- Remove legacy Kubernetes manifests (kub.yaml, service.yaml) from deploy/kubernetes in favor of the Helm chart
- Remove stale index.yaml Helm chart index pinned to an old version (v1.43.18)
- Remove dev_config.yaml development configuration file that contained hardcoded credentials and example endpoints
- Clean up ~3,500 lines of unused code and configuration to reduce repository maintenance burden

Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
2026-05-09 20:51:34 +00:00
Yuneng Jiang
c35f3a50ae docs: remove docs/my-website, point contributors to litellm-docs
The documentation source has moved to a separate repository,
BerriAI/litellm-docs, served at docs.litellm.ai. This PR removes
docs/my-website/ from this repo and updates README.md, AGENTS.md,
and CLAUDE.md to direct doc contributions to the new repo.

Also fixes a broken relative link in
litellm/integrations/levo/README.md.

The existing CI symlink in .github/workflows/test-code-quality.yml
(which clones litellm-docs and symlinks docs/my-website to it for
tests/documentation_tests/*) continues to work without change.
2026-04-24 14:17:46 -07:00
Ryan Crabbe
17568e81f2
chore(ui): use antd Typography in GuardrailTestPlayground
- Replace plain text/heading tags with Typography.Text, Title, Paragraph
- Document Typography preference in CLAUDE.md UI guidelines
2026-04-15 08:45:11 -07:00
yuneng-jiang
a306092d47
Merge pull request #25463 from BerriAI/litellm_oss_staging_04_09_2026
Litellm oss staging 04 09 2026
2026-04-13 17:25:53 -07:00
Ryan Crabbe
842523a918
chore(ui): use antd in GuardrailSettingsView and document the rule
Converts GuardrailSettingsView from @tremor/react (Badge, Text) to
antd (Tag, plain spans) as part of the Tremor migration. Also
captures the "no new Tremor imports" rule in CLAUDE.md and expands
the existing note in AGENTS.md with the specific antd equivalents
and the yellow→gold gotcha.
2026-04-13 11:49:39 -07:00
Yuneng Jiang
78c282f400
[Infra] CI: harden supply chain — remove dockerize, pin tools, verify checksums
- Remove dockerize entirely. Replace all 26 `dockerize -wait` calls with
  a new `wait_for_service` CircleCI command using built-in bash + curl.
- Replace `curl | bash` Docker install with a `docker version` check
  (Docker is pre-installed on the ubuntu-2204 machine image).
- Pin Helm v3.17.3, Kind v0.20.0, kubectl v1.31.4 with SHA-256
  checksum verification. Replace `curl | bash` helm install.
- Add reusable commands: wait_for_service, install_helm, install_kind.
- Add CI supply-chain safety guidelines to CLAUDE.md.
2026-04-10 21:42:33 -07:00