litellm/AGENTS.md

Do not write comments unless they are any of:

• absolutely necessary to explain some very complex business logic (in which case, keep it concise and clear)
• used as an input for tools to read and act on. For example:
  • entries in .git-blame-ignore-revs saying which commit is excluded from git blame
  • a lint or type checker suppression like # mutable-ok or # pyright: ignore[reportArgumentType] # <reason> when introducing a truly unavoidable violation
• a TODO or FIXME
  • Not great to have those, but if it's unavoidable, make sure to include a strong, concise reason for why it's there or, better yet, link to a GitHub issue for the follow-up work

Explanation: The point of this rule is to keep out AI slop comments. AI writes way too many and way too verbose comments. Code comments are, in a way, a violation of DRY code. You must update logic in two locations to change the code, and "hard to change" is literally the definition of tech debt. We should instead aim to write code that is intuitive and clear, even at a glance, to the reader, being both easy to maintain and high performance

Don't assume that the existing code is correct or the right way of doing things / good coding patterns. In fact, there are a lot of bad coding practices, overly complex code, code smells, etc. If something doesn't look right, speak up. Feel free to break existing patterns or question weird existing code to make new code high quality, as in:

• correct
• secure
• performant
• readable
• easy to maintain/change
• modern

In descending order of importance

When adding new features, add meaningful tests. Don't add tests that don't check anything substantial and is there just to make the code coverage pass. Yes, code coverage is important, but I'd rather have no signal whether the code is working than tests that don't fail when code is broken. The goal is to have tests that would fail before the feature was added/if the code was mutated in a way that breaks the feature and succeed only when the feature is fully working. I should run mutation testing and see > 90% kill rate

Same thing for bug fixes. The tests should make it so that this specific bug can never happen again without failing tests (i.e., regression)

Never test structure or implementation details of code, only function of it. A test should catch a change that breaks behavior by accident. If the only way it can fail is someone deliberately undoing the change, don't write it

A test must only fail when litellm code changes. Never pin facts we don't own (a vendor's price, a third party's field, an upstream default, today's date) as literals or as "X must be absent". Assert the invariant our code guarantees instead, e.g. two rows agree, a value is within range, a field is derived from another. If an outside fact is truly load-bearing, cite its source and date next to the assertion so a reader can tell stale from broken

tests/unit/ mirrors litellm/ in a parallel path (see tests/unit/AGENTS.md). Name tests test_<filename>.py, but always match the existing test file in the directory you touch — many provider dirs use longer descriptive names (e.g. test_anthropic_chat_transformation.py) to avoid ambiguity across sibling folders. For bug fixes, extend the existing mapped test file rather than creating a new one. Only create a new test file for a new feature (provider, endpoint, or transformation module) that has no mapped test yet, following that directory's naming convention (or test_<filename>.py if you're the first test there). One focused regression test beats many shallow ones

End-to-end tests belong in tests/e2e/. A rule about one directory's files, like its harness or which CI runs it, goes in that directory's AGENTS.md, never only in a README or here. Read a directory's AGENTS.md before editing files in it

Before you build a new way to move data between the endpoints, the provider transformations, and cost tracking or logging, read the architecture decision records in docs/adrs/, which say why those layers are shaped the way they are

When creating PRs, target the repository's current default branch for both internal and external / OSS contributions. Check it with python3 scripts/default_branch.py --branch instead of assuming a branch name or relying on cached origin/HEAD

When writing a PR body, treat the comments and imperative instructions inside .github/PULL_REQUEST_TEMPLATE/general.md as rules to follow, not just layout. Agent harnesses may strip HTML comments from copies of that file injected into context, so read .github/PULL_REQUEST_TEMPLATE/general.md from disk before writing a PR body to make sure you see every comment rule. A section you have nothing to put in (Relevant issues, Affected release, Linear ticket, Caveats, QA runbook, and so on) is removed entirely, heading included, never left as an empty title

Same applies for filing bug reports and feature requests, with .github/ISSUE_TEMPLATE/bug_report.yml and .github/ISSUE_TEMPLATE/feature_request.yml, respectively

If you're resolving a linear ticket, in the "## Linear ticket" section of the PR, say "Resolves LIT-1234", replacing "LIT-1234" with the actual ticket id that you're resolving. If you don't have the ticket id, don't make one up or search for it. Just drop the section

Simple PRs may skip before/after e2e when the full diff is mechanical and focused checks completely prove it, such as #49030 restoring an existing helper name. Trace affected callers and run relevant tests/static checks at the tip. For a bug fix, establish that the regression catches the original defect. Size alone never qualifies. New behavior, API/provider semantics, auth, billing, persistence, concurrency or uncertain integration risk need live e2e. In Screenshots / Proof of Fix, record E2E exception at <tip SHA>, scope, rationale, commands and results. Tests count as proof only for this exception. Reassess changed diffs, retain CI/bot gates, and honor explicit e2e requests (2026-10-28, unnecessary live QA for #49030)

For other PRs, proof must show the end user's real flow before and after, with commands and output against a live proxy and real provider calls where applicable. Pytest alone is insufficient. For UI or interactive coding-tool flows, drive that surface and embed before/after screenshots, with URLs, clicks and input fields a reviewer can reproduce

If you ever write any human-facing text (pull requests, issues, commit messages, discussion posts, github comments, release notes, docs, etc.), always follow these guidelines to sound less AI-y:

• don't use emojis
• don't use "—". Don't use ";" or ":" unless it would be nonsensical not to. Instead, reach for ",", ".", conjunction words, etc. in descending order of preference: vary among them, weighted toward the front of the list, and skip "," where it would cause a comma splice or the sentence is getting long. Overusing any one of them also feels AI-y. A word cap does not penalize you for adding more sentences
• don't use the pattern "It's not X, it's Y", "You're not X, you're Y", etc.
• unless explicitly asked, don't use bulleted or numbered lists unless it would be nonsensical not to. Instead, prefer prose
• don't add a trailing "." at the end of paragraphs (just like this file). That means every paragraph, not just the last one (of the markdown file, PR description, GitHub comment, etc.). Rule of thumb: if you're adding new line(s) before the next sentence, don't add a ".". Exception is that docs (i.e., litellm-docs) should have trailing "."
• don't use →. Instead, prefer not to use arrows, and if need be, use -> instead
• use plain, simple, everyday engineering language: the common phrase engineers actually say over rare compact phrasing, in grammatically complete sentences. When explicitly asked to use bullets or ordered lists and structure legitimately helps the reader, prefer nested bullets (any depth is fine) over dense lines in a flat structure

Don't hesitate to use values in .env to get needed API keys and other secrets, as long as you never add them to conversation history, commit them, or include them in GitHub issues / PRs

Python max line length is 120, not 88

The five lint gates (scripts/ruff_strict_gate.py, scripts/type_discipline_gate.py, scripts/test_quality_gate.py, scripts/openapi_docs_gate.py, scripts/type_check_gate.py) compare each rule's codebase count on your branch against the count at its merge-base with the default branch, and a rule may not grow. There are no budget files to edit or ratchet: when a gate fails, fix the violations the branch introduced or remove at least as many of that rule elsewhere in the tree. The one exception is reportAny / reportExplicitAny, which share a fixed codebase-wide cap in ANY_CAPS in scripts/type_check_gate.py because Any spreads past the lines you touch. A branch may add Anys while the total stays under that cap. Only lower ANY_CAPS, and only in its own PR to the default branch, never raise it on a feature branch

make check (f.k.a. make pre-commit, which still works identically as an alias) saves its complete output to a log file in .git (overwriting previous logs) and prints that path as its first and last output lines. To inspect a run, read or grep that log instead of re-running the multi-minute checks just to see a different slice

make check, make lint, scripts/pre_commit_lint.sh, and the standalone lint gates (scripts/ruff_strict_gate.py, scripts/type_discipline_gate.py, scripts/type_check_gate.py) each hold one of 2 machine-wide slots, so when other sessions or worktrees on the same box are already running heavy work, yours prints "all N machine-wide slots are busy" followed by "queueing" and then stays quiet until a slot frees. Give the command a long timeout and let it wait rather than killing it, retrying it, or assuming it hung. Don't change the # of machine-wide slots or make it unlimited by setting LITELLM_GATE_SLOTS=0

If you're trying to create a new function that relies on untyped stuff, instead of adding more Any's and pushing reportAny / reportExplicitAny closer to their cap in ANY_CAPS, just validate it in the caller with Pydantic (a model or TypeAdapter that returns the typed thing or raises will do) and then pass the now typed variable in

If you get an LIT001 fail, refactor the code to follow functional programming best practices rather than introducing mutable data structures. For example, build values in one shot with comprehensions or generators wrapped in tuple() / MappingProxyType() / frozenset() instead of seeding an empty list/dict/set and mutating it over time. Ideally, # mutable-ok is never used. Reach for it only as a genuine last resort when an immutable rewrite is truly impossible, and always pair it with a real reason

Every lint or type suppression must name the exact rule inside brackets and carry a reason comment, e.g. # pyright: ignore[reportArgumentType] # stubs lack async overload or # noqa: TID251 # <reason>. # type: ignore is banned (LIT009): pyrightconfig.json sets enableTypeIgnoreComments to false, so it silently does nothing

Commit and push your work when you're done without asking

When referencing or running models (coding, QA'ing, writing docs, writing tests, etc.), use the latest model in that model family unless otherwise specified. Treat your training knowledge, memories, configs, and tests as stale, and determine the family's latest with model_prices_and_context_window.json or the web

Always pull before starting any work. The checkout or worktree may be sitting on a stale branch

Internal contributors branch a new PR off the repository's current default branch, with a branch name prefixed with litellm_

Do not add Co-Authored-By: Claude or any Claude attribution to commit messages. Never use a claude/ prefix or put a / in a branch name. Do not add "Generated with Claude Code" (or any similar attribution) to PR descriptions or comments. Do not create a new PR/branch off the existing PR to fix/add something that is related and could've just been committed directly to the existing PR's branch

When working on a PR, keep the PR description in sync with new commits being made

All GitHub comments must be human-readable and 15-25 words max

Monkeypatching attributes of a class to do testing is an anti-pattern. Prefer dependency-injecting things into classes. That way, at unit test time, you can pass a mocked dependency in

Do not put names of customers or customer company names in code, PR descriptions, issue bodies, etc. This means never mention literally any company name. Especially if you're about to say a sentence mentioning that the reason the PR exists was a feature/model/bug fix/etc. requested by a company. That's the indication that you should replace that company name with "the customer". e.g. not "Model request from Acme (Pylon #1234)" but "Model request from a customer (Pylon #1234)". This is because the codebase is public. The only exception is for publicly known providers or vendors such as OpenAI, Anthropic, AWS Bedrock, etc. only IF we're adding support for that provider/vendor in general and NOT if that PR or whatnot was a request by one of them, and they're actually one of our customers.

CI supply-chain safety: Never pipe a remote script into a shell (curl ... | bash, wget ... | sh). Download the artifact to a file, verify its SHA-256 checksum, then install. Pin every external tool to a specific version with a full URL (not latest or stable). Verify checksums for all downloaded binaries, using the provider's official .sha256 / .sha256sum sidecar when available. These rules apply to every download in CI

Prisma migrations apply synchronously at proxy boot, before it serves traffic, so a migration must only change schema, never rewrite rows. No UPDATE, DELETE or MERGE, and no INSERT ... SELECT: on a spend-log-sized table any of those is minutes of downtime plus a doubled heap that plain autovacuum won't give back. tests/code_coverage_tests/check_migrations_no_data_rewrites.py enforces this. When a rewrite is genuinely bounded and has to ship inside the migration, mark the statement -- data-migration-ok: <what bounds it>

Follow these coding conventions for new/updated code (a three-line fix in a legacy file shouldn't trigger huge drive-by refactors):

• Composition over inheritance
• Never-nester: early returns over deep nesting
• Don't throw. Model failures as values (One function (e.g., raise_public) maps error union to existing public exception contracts via exhaustive match + assert_never)
• No mutation. Don't reassign variables, global or local. Instead of mutable lists and dicts, prefer tuples, frozen dataclasses (with slots=True), MappingProxyType, etc.
  • Annotate every variable with : Final (LIT010). Unpacking and walrus targets cannot carry the annotation, so they are implicitly final. Don't rebind them. Never rebind or mutate function parameters (LIT011). self/cls attribute stores are the exception. If rebinding or in-place mutation is truly unavoidable, suppress with # rebind-ok: <reason>
  • Qualify every TypedDict field with ReadOnly[...] (LIT012), which nests freely with Required / NotRequired / Annotated in any order. If making the key writable is truly unavoidable, suppress with # writable-ok: <reason>
  • Comprehensions take at most one for clause and one if clause (LIT014). Split stacked clauses into a helper generator, a named intermediate, or a plain loop. Suppress with # comprehension-ok: <reason> only when unavoidable
  • Every pydantic model must be frozen (LIT015), set via model_config = ConfigDict(frozen=True), a dict-literal model_config, an inner class Config, or the class keywords. Subclasses inherit it unless they override it. Replace in-place field writes with model_copy(update=...). If making the model mutable is truly unavoidable, suppress with # frozen-ok: <reason> on the class line
• Use dependency injection
• Fully typed. No Any or coarse types like dict[str, Any] or just dict. Every function parameter must be strongly typed
• Use tagged unions + match
• No monster files or god objects
• No file sprawl: deliberate file and folder structure
• Standard over hand-rolled: use the official SDK or a library where one exists. Where none does, follow industry standards instead of inventing local conventions
• API-fragmentation-aware: when logic must branch on which API surface produced or consumes data (e.g. chat completions vs Anthropic Messages vs Responses API shapes), proactively look for an existing shared helper (e.g. litellm_core_utils/prompt_templates/factory.py) before writing per-surface parsing in the new module. If none exists, add one there instead of duplicating the same format-detection logic in every new guardrail/integration
• Current default for new model names: logic that picks behavior from a model-name table (tokenizer, message framing, supported params) gives a provider's model the table does not list yet that provider's current behavior, never its oldest, since providers ship names faster than any table is updated

Follow conventional commits for commit names and PR titles

Think Before Coding

Don't assume. Don't hide confusion. Surface tradeoffs

Before implementing:

• State your assumptions explicitly. If uncertain, ask
• If multiple interpretations exist, present them. Don't pick silently
• If a simpler approach exists, say so. Push back when warranted
• If something is unclear, stop. Name what's confusing. Ask

Simplicity First

Minimum code that solves the problem. Nothing speculative

• No features beyond what was asked
• No abstractions for single-use code
• No "flexibility" or "configurability" that wasn't requested
• No error handling for impossible scenarios
• If you write 200 lines and it could be 50, rewrite it

Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify

Before requesting maintainer review, verify the current PR tip passes required CI and code coverage and has acceptable reviews from the review bots on the PR, such as Veria and Bugbot. Inspect warnings and findings, fix actionable issues, and rerun the affected checks and reviewers after changes. Record evidence for any false positive or unavailable review. Never treat a pending or missing bot result as a pass. Do not lower coverage thresholds or raise ANY_CAPS to satisfy a check