* ci: enable ruff preview rules under the budgeted strict gate
Turn on ruff preview in the strict-budget lane (ruff-strict.toml) only,
leaving the clean gate (ruff.toml) untouched so make lint-ruff stays at
zero. Enumerate the 118 firing codes explicitly with
explicit-preview-rules so the gate is deterministic and stable across
ruff upgrades rather than depending on preview auto-selecting the broad
catalog.
Grandfather the existing 58438 violations into ruff-strict-budget.json
as per-rule baselines with headroom, so only net-new violations fail CI.
The existing ten rules keep their hand-tuned slack; the new rules get
slack 10 when the baseline is 50 or more and 3 otherwise.
* ci: add ANN return-type rules to the budgeted strict gate
Add ANN201/202/204/205/206 (missing return annotations) to the strict
lane and grandfather the existing counts into ruff-strict-budget.json so
the codebase ratchets toward explicit return types without breaking CI.
* ci: add mypy (disallow_untyped_defs) and basedpyright strict gates with baselines
Add two type-check gates, each grandfathering the current tree so only
net-new violations fail CI, matching the ruff strict-budget ratchet.
mypy gains disallow_untyped_defs in litellm/mypy.ini (the config the CI
invocation actually reads; the root [tool.mypy] is not picked up from the
litellm/ working dir). The 4885 existing missing-annotation errors are
captured in litellm/.mypy-baseline.txt and the run is piped through
mypy-baseline filter so new untyped defs are rejected.
basedpyright runs in strict mode over litellm/, with
enableTypeIgnoreComments disabled so it only honors '# pyright: ignore'
and never polices mypy's '# type: ignore'. The existing strict diagnostics
are grandfathered into .basedpyright/baseline.json.
Both tools are pinned in the dev group and uv.lock; the lint workflow and
Makefile run them filtered through their baselines, with
lint-mypy-baseline-update and lint-basedpyright-baseline-update to ratchet.
* ci: raise lint job timeout to 15m for the basedpyright strict pass
* ci: pin pythonVersion 3.12 and regenerate baselines against merged base
Merge litellm_internal_staging so the baselines cover code the CI merge
includes (e.g. the cisco_ai_defense guardrail), which otherwise tripped
the mypy gate with 3 ungrandfathered no-untyped-def errors. Pin
pythonVersion 3.12 in pyrightconfig so basedpyright's strict analysis is
reproducible across interpreter versions (CI runs 3.12).
* ci: regenerate basedpyright baseline against the frozen lint env
The previous baseline was generated with optional provider deps (azure,
google, anthropic, mcp, numpydoc, google-genai) installed locally, so CI's
dev-only env surfaced ~3500 reportUnknown*/reportMissingTypeStubs errors
not in the baseline. Regenerate after uv sync --frozen so the baseline
reflects the same dependency set the lint job sees.
* ci: regenerate basedpyright baseline on python 3.12 frozen env
The prior baseline still carried proxy-dev packages (e.g. prisma) that the
lint job's dev-only, python 3.12 env lacks, leaving 2 unresolved-import
errors ungrandfathered. Regenerate in a python 3.12 venv synced to the
frozen lock with default groups only, so the baseline matches exactly what
CI sees.
* ci: replace type-check baselines with per-file count budgets
The mypy and basedpyright baselines were position-sensitive (and the
basedpyright one was a 27MB file), so ordinary line shifts churned them.
Replace both with a per-file count gate: scripts/type_check_gate.py reduces
each tool's output to errors-per-file and checks it against a committed
{file: max} budget, ignoring line and column numbers. A file fails only
when it gains more errors than its ceiling; debt can't be shuffled between
files because each file has its own cap and new files default to zero.
Budgets (mypy-file-budget.json 48K, basedpyright-file-budget.json 96K) are
generated in the python 3.12 frozen lint env so they match CI. Drops the
mypy-baseline dependency; basedpyright runs without its native baseline.
ratchet via make lint-mypy-budget-update / lint-basedpyright-budget-update.
* ci: add a small per-file slack to the type-check gate
Allow each file to drift PER_FILE_SLACK (5) errors past its recorded count
before failing, so a basedpyright inference ripple in an unrelated file
doesn't break the build over a couple of errors. Budgets still record exact
counts; the tolerance is applied at check time.
* ci: move type-check slack into the budget json and trim lint timeout
Make slack declarative: the budget is now {"slack": N, "files": {path: count}}
so the tolerance is tuned in JSON without editing the script, mirroring how
ruff-strict-budget.json carries its slack. --update preserves the existing
slack. Also drop the lint job timeout from 15m to 10m; the mypy and
basedpyright passes add ~2m, leaving the job around 4-5m, so 10m is a
comfortable margin.
* ci: collapse fully-adopted ruff categories and drop inert preview flag
ANN (all nine non-removed rules) and BLE (its only rule) were spelled out
code-by-code; replace each with its category selector, which is exactly
equivalent in 0.15.3 (the removed ANN101/ANN102 are skipped by a category
selector and error when named explicitly). explicit-preview-rules was inert:
every selected rule is stable and nothing is selected by category, so the flag
had nothing to gate. Verified the strict-rule counts are identical before and
after (62379 each, zero per-rule drift), so no budget change.
* ci: drop redundant pyright dev dependency
Nothing invokes bare pyright in the Makefile, the linting workflow, or
scripts; the basedpyright gate added on this branch is the only type
checker that runs. basedpyright is a superset fork that reads the same
pyrightconfig.json and honors the same "# pyright: ignore" comments, so
pyright==1.1.408 in the ci group was dead weight. Regenerated uv.lock
under the same exclude-newer cutoff so the only change is removing
pyright and its package stanza
* ci: un-weaken mypy and error on Any in basedpyright
mypy: enable warn_return_any, drop the valid-type silencer, and stop globally ignoring missing first-party imports via [mypy-litellm.*] ignore_missing_imports = False, which surfaced eight real broken litellm.* imports the blanket ignore was hiding; third-party imports stay ignored. The per-file budget moves 4888 -> 5799 (902 no-any-return, 1 valid-type, 8 import-not-found), all grandfathered so only net-new errors fail and the ceilings ratchet down
basedpyright: error on reportExplicitAny and reportAny. The per-file budget moves 117033 -> 148946 (6931 explicit-Any, 24954 Any-typed expressions), grandfathered the same way
* ci: add Any-discipline gate on changed lines under litellm/
Add scripts/check_any_discipline.py, a type-aware gate that fails when a
changed line holds a value typed Any -- including the X | Any unions that
mypy --strict / basedpyright accept (e.g. re.Match.group() -> str | Any,
json.loads() -> Any, bare dict -> dict[Any, Any]).
It reuses the repo's mypyc-compiled mypy 1.19 via a custom generic AST
walker (mypyc precludes subclassing TraverserVisitor), loads litellm/mypy.ini
for parity with lint-mypy, and uses a dedicated incremental cache
(.mypy_cache_any) with mtime+hash invalidation to force re-checks. Scope is
changed-lines-only so editing a legacy file never forces cleaning its
existing Any debt; suppress a genuine typed/untyped boundary with
# any-ok: <reason> (ANY002 requires the reason).
Wire it into the Makefile (lint-any, lint, lint-dev), a parallel
any-discipline CI job with its own actions/cache, .gitignore, and the
CLAUDE.md / CONTRIBUTING.md docs.
* ci: move Any-gate codes into the shared LIT namespace
Renumber the Any-discipline checker into the LIT*** scheme owned by
scripts/check_type_discipline.py (PR #30500) so the two checkers share one
rule namespace and suppression convention:
ANY001 -> LIT002 (Any-typed value; LIT002 was the retired/free slot)
ANY002 -> LIT005 (any-ok without a reason; the shared suppression-reason code)
ANY000 -> LIT000 (setup/build/read error; the shared error code)
Messages and behavior are unchanged; LIT005's text already matches the
"<token> requires a reason" shape used for cast-ok/guard-ok.
* ci: gate mypy and basedpyright per error rule, not per file
Switch the mypy/basedpyright budget gate from per-file error counts to
per-rule-code totals, mirroring the {rule: {baseline, slack}} shape of
ruff-strict-budget.json. A rule fails when its codebase-wide error count
exceeds baseline + slack, so violations are tracked by category rather
than by file location.
scripts/type_check_gate.py now parses mypy from its text output (trailing
[code]) and basedpyright from --outputjson (the JSON `rule` field), since
basedpyright's wrapped text diagnostics mis-attribute the rule on
continuation lines. Replace the *-file-budget.json files with freshly
captured *-code-budget.json baselines and update the Makefile, CI, and
CLAUDE.md accordingly.
* docs: prefer Pydantic validation over any-ok suppression
Point the Any-discipline guidance at validating Any with Pydantic (a model
or TypeAdapter that returns a typed value or raises) and frame
# any-ok as a last resort that should ideally never be used.
* chore: remove extraneous comment
* chore: make the CLAUDE.md more concise
* chore: clean up bloated CONTRIBUTING.md additions
* chore: make Makefile more concise
* ci: add the lint-budget-update target CLAUDE.md references
CLAUDE.md tells contributors to run make lint-budget-update, but the
target was never defined. Add it as an aggregate that re-captures the
ruff, mypy, and basedpyright budgets in one shot.
* ci: recapture mypy and basedpyright budgets in the lint env
The per-rule baselines were captured in a richer dependency env than the
CI lint job's uv sync --frozen, so CI resolved fewer types and reported
more errors than the budgets allowed (no-any-return 902 over cap 900, plus
several basedpyright reportUnknown* rules). Regenerate both in the frozen
env so they grandfather the true CI debt: mypy 5786 -> 5799 (no-any-return
890 -> 902, valid-type 1 restored), basedpyright 146213 -> 148942.
* ci: check out PR head sha in lint and any-discipline jobs
The default pull_request checkout uses refs/pull/N/merge, which folds the
latest base commits into HEAD. The diff-based gates (ruff delta, Any
discipline) then diff against the event's older base.sha and blame base's
own new commits on this branch; staging's otel-v2 and streaming changes
(#30326, #30485) tripped the Any gate on files this branch never touched.
Checking out the PR head sha makes the gates diff the real branch tip
against base, and pins the tree the mypy/basedpyright budgets were captured
against so their counts stay deterministic as the base advances.
* ci(lint): renumber Any-typed-value rule LIT002 -> LIT009
Free up LIT002 for the sibling type-discipline gate (check_type_discipline.py,
#30500), which groups its mutable-collection family at LIT001 (annotation) and
LIT002 (construction). This gate's Any-typed-value rule moves to LIT009 so the
shared LIT namespace stays contiguous with no holes; LIT000 and LIT005 are
unchanged.
* style: rename lint-strict-budget -> lint-ruff-budget
* ci: harden type-check gates against silent passes (greptile review)
type_check_gate.py: refuse to certify a vacuous run. The CI pipe swallows
the tool's exit code ('tool || true'), so a crashed mypy/basedpyright that
emits nothing would parse to zero errors, breach no ceiling, and pass.
is_vacuous_run() now fails when nothing was parsed but the budget expects
errors. Also wrap basedpyright's json.loads in a JSONDecodeError handler
that prints the offending output instead of dumping a raw traceback.
check_any_discipline.py: ALL_LINES was None, which dict.get() also returns
for a path absent from the line map, so a path-normalisation mismatch could
let a violation on an unchanged file pass the scope filter. Make ALL_LINES a
distinct sentinel object so 'whole file' and 'path missing' are unambiguous.
Adds tests for all three.
10 KiB
Contributing to LiteLLM
Thank you for your interest in contributing to LiteLLM! We welcome contributions of all kinds - from bug fixes and documentation improvements to new features and integrations.
Checklist before submitting a PR
Here are the core requirements for any PR submitted to LiteLLM:
- Sign the Contributor License Agreement (CLA) - see details
- Keep scope isolated - Your changes should address 1 specific problem at a time
Proxy (Backend) PRs
- Add testing - Adding at least 1 test is a hard requirement - see details
- Ensure your PR passes all checks:
- Unit Tests -
make test-unit - Linting / Formatting -
make lint
- Unit Tests -
UI PRs
- Ensure the UI builds successfully -
npm run build - Ensure all UI unit tests pass -
npm run test - Add tests for new components or logic - If you are adding a new component or new logic, add corresponding tests
Contributor License Agreement (CLA)
Before contributing code to LiteLLM, you must sign our Contributor License Agreement (CLA). This is a legal requirement for all contributions to be merged into the main repository.
Important: We strongly recommend reviewing and signing the CLA before starting work on your contribution to avoid any delays in the PR process.
Quick Start
1. Setup Your Local Development Environment
# Fork the repository on GitHub (click the Fork button at https://github.com/BerriAI/litellm)
# Then clone your fork locally
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm
# Create a new branch for your feature (see "Commit and Branch Conventions" below)
git checkout -b feature/your-feature
# Install development dependencies
make install-dev
# Install git hooks that enforce commit + branch conventions (one-time, opt-in)
make install-hooks
# Verify your setup works
make help
That's it! Your local development environment is ready.
Commit and Branch Conventions
Commits follow Conventional Commits and branches follow Conventional Branches. Run make install-hooks once per clone to enable the local git hooks that enforce these — see the contributor docs for the full type list, examples, the protected-branch bypass list, and how to opt out.
2. Development Workflow
Here's the recommended workflow for making changes:
# Make your changes to the code
# ...
# Format your code (auto-fixes formatting issues)
make format
# Run all linting checks (matches CI exactly)
make lint
# Run unit tests to ensure nothing is broken
make test-unit
# Commit your changes (must follow Conventional Commits — see above)
git add .
git commit -m "feat(scope): your descriptive commit message"
# Push and create a PR (branch must follow Conventional Branches — see above)
git push origin feature/your-feature
Adding Testing
Adding at least 1 test is a hard requirement for all PRs.
Where to Add Tests
Add your tests to the tests/test_litellm/ directory.
- This directory mirrors the structure of the
litellm/directory - Only add mocked tests - no real LLM API calls in this directory
- For integration tests with real APIs, use the appropriate test directories
File Naming Convention
The tests/test_litellm/ directory follows the same structure as litellm/:
litellm/proxy/caching_routes.py→tests/test_litellm/proxy/test_caching_routes.pylitellm/utils.py→tests/test_litellm/test_utils.py
Example Test
import pytest
from litellm import completion
def test_your_feature():
"""Test your feature with a descriptive docstring."""
# Arrange
messages = [{"role": "user", "content": "Hello"}]
# Act
# Use mocked responses, not real API calls
# Assert
assert expected_result == actual_result
Running Tests and Checks
Running Unit Tests
Run all unit tests (uses parallel execution for speed):
make test-unit
If you're running broader test suites, proxy tests, or anything that touches PostgreSQL-backed fixtures/plugins, install the full local test environment first:
make install-test-deps
This syncs the locked test environment used across the repo, including psycopg v3 plus psycopg-binary (used by pytest-postgresql), psycopg2-binary (used by some proxy E2E tests), and a generated Prisma client for DB-backed proxy tests, so pytest startup matches CI without manual package installs.
Run specific test files:
uv run pytest tests/test_litellm/test_your_file.py -v
Running Linting and Formatting Checks
Run all linting checks (matches CI exactly):
make lint
Individual linting commands:
make format-check # Check Black formatting
make lint-ruff # Run Ruff linting
make lint-mypy # Run MyPy type checking
make lint-any # Fail on Any-typed values on changed lines
make check-circular-imports # Check for circular imports
make check-import-safety # Check import safety
Apply formatting (auto-fixes issues):
make format
Black formatting is enforced in CI. All PRs must pass the Black formatting check.
- AI coding agents (Claude Code, Copilot, Cursor, etc.):
AGENTS.mdandCLAUDE.mdinstruct agents to runpoetry run black .before committing.- VS Code users: Install the Black Formatter extension and enable format-on-save:
{ "[python]": { "editor.defaultFormatter": "ms-python.black-formatter", "editor.formatOnSave": true } }
CI Compatibility
To ensure your changes will pass CI, run the exact same checks locally:
# This runs the same checks as the GitHub workflows
make lint
make test-unit
For exact CI compatibility (pins OpenAI version like CI):
make install-dev-ci # Installs exact CI dependencies
Available Make Commands
Run make help to see all available commands:
make help # Show all available commands
make install-dev # Install development dependencies
make install-proxy-dev # Install proxy development dependencies
make install-test-deps # Install the full local test environment
make format # Apply Black code formatting
make format-check # Check Black formatting (matches CI)
make lint # Run all linting checks
make test-unit # Run unit tests
make test-integration # Run integration tests
make test-unit-helm # Run Helm unit tests
Code Quality Standards
LiteLLM follows the Google Python Style Guide.
Our automated quality checks include:
- Black for consistent code formatting
- Ruff for linting and code quality
- MyPy for static type checking
- Circular import detection
- Import safety validation
All checks must pass before your PR can be merged.
Common Issues and Solutions
1. Linting Failures
If make lint fails:
- Formatting issues: Run
make formatto auto-fix - Ruff issues: Check the output and fix manually
- MyPy issues: Add proper type hints
- Circular imports: Refactor import dependencies
- Import safety: Fix any unprotected imports
2. Test Failures
If make test-unit fails:
- Check if you broke existing functionality
- Add tests for your new code
- Ensure tests use mocks, not real API calls
- Check test file naming conventions
3. Common Development Tips
- Use type hints: MyPy requires proper type annotations
- Write descriptive commit messages: Help reviewers understand your changes
- Keep PRs focused: One feature/fix per PR
- Test edge cases: Don't just test the happy path
- Update documentation: If you change APIs, update docs
Building and Running Locally
LiteLLM Proxy Server
To run the proxy server locally:
# Install proxy dependencies
make install-proxy-dev
# Start the proxy server
uv run litellm --config your_config.yaml
Docker Development
If you want to build the Docker image yourself:
# Build using the non-root Dockerfile
docker build -f docker/Dockerfile.non_root -t litellm_dev .
# Run with your config
docker run \
-v $(pwd)/proxy_config.yaml:/app/config.yaml \
-e LITELLM_MASTER_KEY="sk-1234" \
-p 4000:4000 \
litellm_dev \
--config /app/config.yaml --detailed_debug
UI Development
1. Setup Your Local UI Development Environment
# Clone the repo (if you haven't already)
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm
# Navigate to the UI dashboard directory
cd ui/litellm-dashboard
# Install dependencies
npm install
# Start the development server
npm run dev
2. Adding UI Tests
If you are adding a new component or new logic, you must add corresponding tests.
3. Running UI Unit Tests
npm run test
4. Building the UI
Ensure the UI builds successfully before submitting your PR:
npm run build
Submitting Your PR
- Push your branch:
git push origin your-feature-branch - Create a PR: Go to GitHub and create a pull request
- Fill out the PR template: Provide clear description of changes
- Wait for review: Maintainers will review and provide feedback
- Address feedback: Make requested changes and push updates
- Merge: Once approved, your PR will be merged!
Getting Help
If you need help:
- 💬 Join our Discord
- 💬 Join our Slack
- 📧 Email us: ishaan@berri.ai / krrish@berri.ai
- 🐛 Create an issue
What to Contribute
Looking for ideas? Check out:
- 🐛 Good first issues
- 🚀 Feature requests
- 📚 Documentation improvements
- 🧪 Test coverage improvements
- 🔌 New LLM provider integrations
Thank you for contributing to LiteLLM! 🚀