The absence of a CI signal is indistinguishable from a passing one, and
that shape has now produced several independent holes: whole test
directories no job runs, and shipped images no job builds. Nothing was
watching for either, so each was found by accident.
assert_ci_coverage.py enumerates every test_*.py under tests/ and every
Dockerfile in the repo, then credits only what the workflows and the
CircleCI config actually invoke. Paths filters and lint steps that merely
name a directory do not count as coverage, because crediting a mention is
the same mistake one level up. Anything neither invoked nor listed in
.github/ci-coverage-allowlist.yml with a written reason fails the job.
The guard found 275 uncovered test files and 6 unbuilt Dockerfiles. Fixed
here: the root Dockerfile, the primary published image, now gets an
image-scan leg that builds it and runs the offline migration check against
it, and 8 tests/test_litellm subdirectories join the shards that already
enumerate their siblings. Everything else is allowlisted per file, so a
new file cannot inherit an exemption, and the remaining decisions are
tracked rather than invisible.
The job reports but is not in branch protection, so it does not block
merges; promoting it is a separate change once the allowlist has survived
contact with a few pull requests.
The unit workflows trigger on pull_request only, so a commit that
actually lands on a gated branch ends up with no unit-test check runs at
all. The commit status API reports success for those commits, which a
release gate reads as "nothing failed" rather than "never tested". PR
checks also only ever ran against the merge preview, not the commit that
landed, so two branches that are each green can still land broken
together
Add a push trigger on the two gated branches to the twelve unit
workflows and to the code-quality workflow, so every landed commit gets
check runs addressable by its SHA
test-linting.yml is deliberately left on pull_request only; six of its
steps gate on a diff against github.event.pull_request.base.sha, which
is empty outside a pull request, and a "what did this branch add" check
has no meaning on a merge commit
Also key the concurrency group on github.sha and restrict
cancel-in-progress to pull_request. The previous group was stable across
pushes to a branch, so consecutive merges would cancel the in-flight run
for the earlier commit and leave that SHA without a result, which is the
same blind spot this change is meant to close
The Headroom guardrail sent every message to /v1/compress, including the
system prompt and the user's current instruction. On an agentic /v1/messages
request the live turn is the largest compressible blob, so it came back as a
hash marker; the model then called headroom_retrieve and got its own
instruction returned in a tool_result block, which reads as data it fetched
rather than a request to act on, so it described the content instead of doing
the work.
litellm already owns the policy for what a compressor may never rewrite:
get_protected_indices covers the system rows, the last user row and the last
assistant row, and compress() expands it over whole tool exchanges. Headroom
now consults it (promoted from a private name and given tests) and expands it
the same way, so the trailing tool result cannot come back as a marker
standing in for the result of the call the model just made. Protected rows are
withheld from the payload rather than pinned afterwards, so their tokens are
not reported as savings that are never applied; the write-back discards a
compressed system prompt outright, so that saving never existed. The cost is
that a query-aware service no longer sees the newest user message.
A response whose row count differs from what was sent can no longer be
interleaved with the withheld rows, so it goes through the configured fail
policy instead of being adopted. Fail-open now returns the caller's own inputs
object: translation handlers detect a rewrite by identity, so a rebuilt copy
sent an unchanged request through the Anthropic write-back for nothing.
That write-back rebuilt the request with one anthropic_messages_pt call, which
merges every run of consecutive user/tool rows, so a tool_result turn and the
user turn after it arrived fused. Converting a row at a time would separate
them but breaks tool pairing: with modify_params on, an assistant row whose
results are converted separately reads as an orphaned tool call and the
sanitizer answers it with a synthetic "tool execution skipped" result while
dropping the real one. Conversion is now grouped by tool_call_id ownership,
which satisfies both, and the same grouping decides which rows headroom
protects, so the two agree by construction.
The CCR follow-up also dropped any text the model wrote alongside its tool
call, and echoed tool calls it had no results for. Both are fixed by reusing
compresr's extraction helper, now shared instead of duplicated.
Resolves LIT-5018
* ci: skip unit test workflows when only docs or ui files change
Mirror the CircleCI backend path filter (.circleci/scripts/classify_changes.sh)
in the GitHub Actions unit test workflows by adding paths-ignore for ui/**,
docs/**, *.md and *.mdx to every test-unit-*.yml pull_request trigger
* ci: drop docs/** from unit test paths-ignore since the folder no longer exists
* test(batches): add 1:1 test file scaffold for batches component paths
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add harness test for create batch endpoint
* Add retrieve endpoint harness tests
* Add list endpoint harness tests
* Add cancel endpoint harness tests
* Add cancel endpoint harness tests
* Add test for litellm/batches/main.py
* Add test for litellm/tests/test_litellm/batches/test_batch_utils.py
* Add handler and transformation tests for all providers
* Fix: run batches tests in cicd
* fix(tests): remove azure/__init__.py that shadowed azure namespace package
Adding __init__.py to tests/test_litellm/llms/azure/ caused pytest to
insert tests/test_litellm/llms/ into sys.path[0], making our empty
azure/ dir shadow the real azure-identity namespace package. Any test
that patched azure.identity.* would then fail with AttributeError.
* style(tests): apply ruff format to test_batch_utils.py
Base migrated the formatter from black to ruff format (#31317); reformat the
batches scaffold test file to match.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Keep only video test files and CI workflow entries; drop unrelated
production code and non-video test changes from this branch.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): point OSS contributor workflows to litellm_oss_staging
Workflow triggers and guard error messages incorrectly referenced litellm_oss_branch; update them to the branch we actually use for external contributions.
* fix(ci): include test-rust.yml in litellm_oss_staging rename
Missed test-rust.yml when updating OSS contributor target branch references.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(readme): add Deploy on AWS/GCP with Terraform section
Adds a quickstart for the two published Terraform modules on the public
registry (BerriAI/litellm/aws and BerriAI/litellm/google). Copy-paste
main.tf for each cloud, the one-time GCP Artifact Registry remote-repo
command, and pointers to the registry pages for the full input surface.
Sits inside the Get Started section, between the gateway/SDK table and
Run in Developer Mode -- where someone scanning the README for "how do I
deploy this" will land.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(readme): add 1-click deploy buttons for AWS + GCP
GCP gets the real 1-click: Open in Cloud Shell badge that clones the repo
and walks through `terraform apply` via the existing DeployStack
tutorial (already shipped at terraform/litellm/gcp/examples/default/
TUTORIAL.md). User just picks a project.
AWS gets a soft 1-click: a Launch in AWS CloudShell badge that opens an
in-browser, already-authenticated shell. User runs four commands
(clone + cd + cp tfvars + terraform apply) once inside. There's no
native AWS deeplink that pre-clones a repo + runs a tutorial -- CFN
"Launch Stack" + CodeBuild would be needed for that, and that's a
separate piece of work.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs(readme): move AWS + GCP deploy buttons next to Render button
* docs(readme): unify deploy button sizes and badge styles
* docs(readme): bump deploy button height to 48 to match Render/Railway
* docs(readme): bump AWS/GCP badge height to compensate for SVG padding
* docs(readme): bump AWS/GCP badge height to 72
* docs(readme): bump AWS/GCP badge height to 84
* fix(readme): make deploy buttons same height (48px)
https://claude.ai/code/session_01MxQRMHSDXbqJh74rF86UBc
* docs(readme): flag GCP project ID substitution in image_registry
* docs(readme): equalize deploy button heights and fix Cloud Shell button font
GitHub rewrites an image's height attribute to "height: auto; max-height: Npx", which only caps and never stretches, so each image renders at its intrinsic height. The AWS/GCP shields badges are intrinsically 28px while the Render/Railway buttons are 40px, leaving the row uneven regardless of the height="48" we set. Replace the two shields badges with committed 40px PNGs so all four header buttons render at the same 40px.
Also swap the Cloud Shell button from open-btn.svg to open-btn.png. The SVG renders its label as live text with font-family "Roboto, Sans" and no generic fallback; since neither font exists in GitHub's render environment, the text fell back to a serif (Times New Roman). The PNG bakes in the correct typeface.
* docs(readme): collapse Railway deploy anchor to a single line
The Railway button wrapped its img across indented lines, so the anchor contained leading and trailing whitespace. GitHub underlines link content, rendering that whitespace as a small blue underline beside the button. Put the anchor on one line like the other three buttons so there is no inner whitespace to underline.
* Add Claude Fable 5 cost map entries as a data-only hotfix
Backports only the model map changes from #30064 so deployments on
released litellm versions pick up Fable 5 pricing, context window, and
the adaptive thinking flag through the hosted cost map fetch without
upgrading. Includes the supports_sampling_params flag on the 28
Fable 5 / Opus 4.7 / Opus 4.8 entries (ignored by released code, read
by the gating that ships with the next release) and the matching
one-line schema declaration so the map validation test passes.
https://claude.ai/code/session_01MZarYYT3aS7DxaNjoax6Gm
* Add litellm rust workspace with mistral ocr bridge
* address greptile rust ocr feedback
* Simplify rust ocr entrypoint
* rust(core): add Auth/Http/Network error variants
* rust: add reqwest (rustls-tls) workspace dependency
* rust(providers): depend on reqwest
* rust(mistral): add complete_url + resolve_api_key helpers
* rust(providers): end-to-end run_ocr orchestrator with shared client + timeout
* rust(bridge): depend on litellm-core
* rust(bridge): add GIL release accounting
* rust(bridge): end-to-end ocr() + gil_stats(), GIL released for HTTP
* ocr: add minimal Rust bridge (use_litellm_rust + rust_ocr)
* ocr: route mistral to Rust when enabled; keep bare-str file rejection
* litellm: export use_litellm_rust()
* test(ocr): cover Rust OCR routing + toggle
* rust: stop ignoring Cargo.lock
* rust: commit Cargo.lock for reproducible builds
* ci(rust): build with --locked to enforce the lockfile
* Potential fix for pull request finding 'CodeQL / Module-level cyclic import'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for pull request finding 'CodeQL / Module-level cyclic import'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* ocr: lazily import rust bridge inside ocr() to break the import cycle the CodeQL autofix mangled
* ocr: guard OCRResponse under TYPE_CHECKING so the annotation resolves
* ocr: modernize rust_bridge typing (PEP 604, drop typing.Any/Dict) to satisfy strict-rule gate
* ci: re-trigger checks
* ci: re-trigger checks
* Potential fix for pull request finding 'CodeQL / Cyclic import'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for pull request finding 'CodeQL / Cyclic import'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* ocr: make rust_bridge a leaf (return raw dict, no litellm import) so the CodeQL autofix stops re-breaking it
* ocr: wrap rust bridge dict into OCRResponse at the call site
* test(ocr): assert rust_ocr returns the raw bridge dict
* test(interactions): add budget_exceeded to expected status enum (Google updated the published spec)
* ocr: resolve mistral key via get_secret_str before the rust path (secret-manager parity)
* test(ocr): assert rust path resolves key via secret manager
* rust(mistral): document that secret-manager resolution happens on the Python side
* fix(ocr): honor timeout, logging, and missing-bridge fallback on Rust OCR path
- Forward the caller's timeout into the Rust bridge so the fixed 600s client
ceiling no longer overrides shorter deadlines or the library default.
- Run update_from_kwargs and pre_call before invoking the Rust shortcut so
observability, callbacks, and spend tracking match the Python path.
- Fall back to the Python OCR path when litellm_python_bridge isn't importable
instead of raising ImportError to callers.
- Truncate upstream Mistral OCR error bodies before they cross the host
boundary to avoid leaking document or prompt contents in CoreError::Http.
* fix(ocr): log resolved api_base and headers on Rust path
* refactor(ocr): inject the rust bridge via a typed seam, drop the importlib cycle dodge
The rust OCR path was reached through importlib.import_module both for the
bridge module and for probing the native extension, purely to keep CodeQL from
flagging a cyclic import. rust_bridge has no litellm imports, so it is a leaf
and main.py can import it statically without any cycle; the dance is gone
Bridge selection now goes through a typed RustOcr Protocol and a load_rust_ocr()
seam. use_litellm_rust() takes an optional injected bridge, so an embedder (or a
test) can supply an alternative without reaching into sys.modules. The rust-path
body moves into _run_rust_ocr(), which receives its dependencies (the bridge
callable, the logging object, the key resolver) as arguments and is unit-tested
by passing fakes in rather than monkeypatching class methods or module globals
The tests are rewritten around that injection: the bridge is provided via
use_litellm_rust(ocr=...), pre_call is observed through a spy logging object, and
the missing-extension fallback is covered by load_rust_ocr() returning None when
no wheel is built. Types were tightened along the way (a cast for the logging
object, OCRResponse.model_validate for the bridge result) so no basedpyright
per-rule count increases
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(ocr): preserve injected rust bridge across toggle calls
use_litellm_rust() unconditionally assigned the keyword default of None to
_rust_ocr_impl, so any call without ocr= silently dropped a previously
injected bridge. Use a sentinel default so omission preserves the impl
while ocr=None still clears it explicitly.
* ci: run tests/test_litellm/ocr in the misc unit-test group
The OCR test directory was not wired into any CI test group, so its
coverage never uploaded to Codecov and patch coverage failed for new
OCR lines. Add it to the misc group.
* test(ocr): cover compiled-extension load and Python fallback paths
Adds two tests so the Rust bridge module hits 100% and the ocr()
fallback-to-Python branch is exercised:
- load_rust_ocr() returning the compiled extension's ocr callable
- ocr() degrading to the HTTP handler when no bridge is available
* style(ocr): use PEP 604 X | None annotations in rust_bridge
Converts Optional[X]/Union[...] to the X | None form so the new OCR
code stays under the UP045 strict-rule budget gate (lint job). Safe at
runtime — the module already has 'from __future__ import annotations'.
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Required test-unit-* and related workflows only triggered on PRs targeting
main, so feature PRs routed through litellm_internal_staging or
litellm_oss_branch never dispatched the full suite. Branch protection
reported BLOCKED even when CircleCI was green.
Expand pull_request and push branch filters to also match
litellm_internal_staging, litellm_oss_branch, and "litellm_**" (using **
so branch names containing "/" also match).
* Remove redundant matrix unit test workflow
All test paths in test-litellm-matrix.yml are fully covered by the
newer semantic unit test workflows (test-unit-*.yml), making the
matrix workflow redundant CI spend.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add Codecov coverage reporting to semantic unit test workflows
Add coverage collection (--cov) and Codecov OIDC upload to both
reusable base workflows and all 12 caller workflows, replacing the
coverage reporting that was previously only in the matrix workflow.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Move id-token/pull-requests permissions to job level for multi-job workflows
For workflows with multiple jobs (llm-providers, proxy-db), move
id-token: write and pull-requests: write from workflow level to job
level so permissions are scoped to only the jobs that need them.
Removes zizmor inline suppressions that were masking the issue.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rename job keys from generic 'test' to descriptive names (e.g.,
'core-utils', 'proxy-auth', 'router') so GitHub checks display as
'core-utils / run' instead of 'test / test'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace monolithic matrix workflow with individual, descriptively-named
workflow files. Each workflow uses a shared reusable base and follows
least-privilege security: zero secrets, read-only permissions, SHA-pinned
actions, persist-credentials: false, and env-var indirection to prevent
template injection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
