The tightened gate correctly stopped deferring the reservation release for
InteractionsAPIResponses the scheduler will not poll (terminal status, or
in_progress without an id), but the response then fell through into the
generic 'Cost tracking failed' raise and the failed_tracking_alert path.
A create returning failed, cancelled, requires_action, incomplete,
budget_exceeded, or an id-less in_progress without usage therefore released
its reservation as intended and, in the same breath, alerted operators for a
legitimate no-usage response, both creating noise and masking real
cost-tracking failures.
The two gates are now nested under a single 'unbilled interaction response'
outer check, so any InteractionsAPIResponse with no usage takes either the
defer path (pollable, polling on) or the release-and-return path, and none
of them fall through to the generic failure raise. The two regression tests
also now assert failed_tracking_alert is not called, closing the observation
gap the report flagged.
Both leave a background interaction's pre-call reservation open, so the
serving process keeps refusing traffic on the key at the estimated cost
while its recorded spend stays near zero.
A raise from the completion event propagated out with the settlement gate
already claimed, and nothing retries a claim that is set, so the reservation
was never released. Billing now releases it on the way out.
`requires_action` was missing from the terminal set. It is terminal for the
interaction it names: the API has no operation that resumes one, and a caller
answers a tool request by creating a new interaction whose
`previous_interaction_id` points at it. A function-calling background create
that stopped there was polled until the 3600s timeout, losing the tokens it
had already spent producing the tool request and holding its reservation open
for that whole window.
Two review findings, both in the handoff between the create's success
callback and the background poll task.
The callback deferred its budget reservation release for any interactions
response with no usage, but the scheduler only starts a poll task when the
status is in_progress and an id is present. A create that came back
terminal without usage therefore matched the callback's test, got no poll
task, and left its reservation open forever: the pre-call estimate stayed
added to the key, user, team and org spend counters, and the key began
refusing traffic against budget it had never spent. The two conditions now
come from one shared gate so they cannot drift apart again.
Settling a background interaction re-runs the success handlers for a second
result on the same request, and OTEL dedupes span emission on a marker held
in that request's metadata. The in-progress create claimed the marker, so
the completion, the only event carrying usage and cost, was dropped as a
duplicate by OTEL and by every integration deriving from it. Clearing the
success-scoped markers alongside the existing dedup flag lets the cost span
through, leaving failure and guardrail markers untouched.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* test: unshadow the module handles the F811 sweep left behind, and pin the two live tests that went red with it
The F811 sweep in #37878 removed the fixture-local `import litellm` from four
conftests, but the bare `import litellm.proxy.proxy_server` a few lines below
still binds `litellm` as a function local, so `importlib.reload(litellm)` runs
before the name is assigned and every test in those directories errors at
setup. The `hasattr` guard on the line above already proves the module is
loaded, so the import only ever bound the name. Drop it, and enable F823 in
ruff-tests.toml, which flags all four sites at the failing line and would have
blocked the sweep
The same sweep renamed the `check_non_streaming_response` parameter but left
one read of `completion`, which now resolves to `litellm.completion`, and
removed an import whose side effect was the only thing making
`litellm.proxy.proxy_server` reachable in the moderation hook test. That test
already takes `monkeypatch`, so patch the router through it and stop leaking
the router into later tests
`test_content_policy_exception_openai` passed vacuously until #37887 turned it
into a real `pytest.raises`, and OpenAI no longer rejects a lyrics prompt with
a content policy error. Inject an AsyncOpenAI client whose transport answers
with OpenAI's own `content_policy_violation` rejection so the mapping to
ContentPolicyViolationError is exercised every run
`test_async_create_batch` hit a 409 cancelling a batch OpenAI had already
marked failed. The cancel step tolerated a completed batch but not a failed
one. Fold both guards into one helper that tolerates a failed batch only when
OpenAI's recorded error is the org's enqueued token limit, and prints the
batch's errors so the reason is in the log either way
* test: close the injected AsyncOpenAI client after the content policy test
* chore(lint): ratchet TQ005 down by the global mutation this branch cleared
* chore(lint): ratchet TQ005 to 2660 on the merged tree
* chore(lint): ratchet TQ005 to 2561 on the merged tree
* chore(lint): ratchet TQ005 to 2548 on the merged tree
Thirteen tests flipped the flag directly, and an autouse fixture reset it to
True around each of them by hand. monkeypatch.setattr does both jobs, so the
fixture keeps only the part that says what the default is, and each test states
its own override at the point it needs one.
Fifteen tests assigned litellm.callbacks directly and left the conftest global
snapshot to clean up after them. monkeypatch.setattr restores it as part of the
test, so the file no longer depends on that safety net to stay isolated.
Twenty-three tests across eleven files opened with litellm.set_verbose = True
and never put it back, so the flag stayed on for everything that ran after them
in the same process. None of those files read the output it produces: no
caplog, no capsys, no assertion on a log line, so the flag was left over from
debugging. Deleting it beats restoring it, since restoring keeps the noise.
Ten of the eleven stop leaving the flag on. test_volcengine_embedding.py still
ends with it set, from something it exercises rather than from the test itself,
which is worth its own look.
Fifteen tests opened with litellm.set_verbose = True and never put it back, so
the flag stayed on for everything that ran after them in the same process.
Nothing in the file reads the output it produces: there is no caplog, no capsys
and no assertion on a log line, so the flag was left over from debugging.
Deleting it beats restoring it, since restoring keeps the noise.
Seven tests captured litellm.use_legacy_interactions_schema, wrapped their body
in a try, and put it back in a finally. monkeypatch.setattr does that, so the
capture, the try and the finally go and the bodies lose an indentation level.
The remaining hand-rolled restores stay. They hold the flag only across the
iterator's constructor and put it back before the test iterates, so handing
them to monkeypatch would widen that window to the whole test and change what
the streaming assertions run against.
Nine tests in test_http_handler.py captured litellm.disable_aiohttp_transport,
force_ipv4, ssl_ecdh_curve or the request_timeout pair, wrapped their whole body
in a try, and put the value back in a finally. monkeypatch.setattr does all of
that, so the captures, the try and the finally go away and the bodies lose a
level of indentation. The class-scoped restore_request_timeout fixture existed
only for that same bookkeeping and goes with them.
litellm.in_memory_llm_clients_cache is left alone on purpose: the eviction tests
assert a handler is garbage collected, and monkeypatch holds the replaced value
alive until teardown, which keeps the weakref they check from clearing.
Ten tests set litellm.s3_callback_params by hand. Four of them reset it to None
on the last line of the test body, which only runs when the test passes; the
other six wrap the body in try/finally to put the old value back. Raising inside
test_s3_verify_false_handling on the current file leaves the whole callback
config, bucket, endpoint and keys, set in the process for whatever runs next.
monkeypatch.setattr covers both shapes and restores on failure, so the 28 TQ005
violations and the try/finally scaffolding come out together.
51 tests pass, and the wider tests/test_litellm/integrations tree is unchanged.
The five TQ002 mock-echo tests in this file are left alone; those need a
judgement about what S3 logging should assert, not a mechanical sweep.
test_zai_provider.py set LITELLM_LOCAL_MODEL_COST_MAP and litellm.model_cost
directly and never put them back, so every test that ran after it in the same
process saw a local cost map instead of the real one. The two respx tests did
the same to litellm.disable_aiohttp_transport with no restore at all.
Both now go through monkeypatch, which restores on teardown including when the
test fails. The cost-map setup moves into a fixture requested by exactly the
five tests that read the cost map.
The provider is published in lockstep with LiteLLM: every dev, rc and stable
release mirrors terraform/provider/ from the release commit and tags it with
the LiteLLM version, alongside the aws/google modules. The 0.x line ends at
0.4.0, and the CHANGELOG headings no longer drive a release.
RELEASING.md describes the new flow and how to recover a version whose
goreleaser run failed; README gains a Versioning section with the re-pin
note for anyone on `~> 0.4`; CHANGELOG records the change under Unreleased.
goreleaser gets `prerelease: auto` so a v1.99.0-dev.1 / -rc.1 tag in the
mirror is marked as a pre-release instead of becoming the repo's latest
release. The registry ingests it either way.
The mid-conversation system tests prime the prompt cache by re-sending an
identical /v1/messages body until its usage shows the full prefix read back
three times in a row. The e2e stack runs with the litellm response cache on,
so every resend after the first is served from redis with the first call's
usage and the streak can never form; the three unflagged-model tests have
failed on every litellm-e2e build since the consecutive-read check landed.
Send cache: {"no-cache": true} on RichMessagesRequest, as test_cache_control
already does, so each resend reaches the provider.
The two fallback tests sent the same "say hi" / max_tokens=16 body to the
gpt-5.5 fallback, so one empty (finish_reason=length) completion served the
second test from the response cache and failed both. Give each test a unique
prompt and leave gpt-5.5 enough tokens to emit text.
Onyx, prompt security, hiddenlayer, repelloai and deepkeep all write straight to
os.environ and unset again at the bottom of each test. None of the five has a
try/finally, so the moment a test fails it returns to the runner with the keys
still set and whatever runs next in that worker inherits them.
Raising inside test_onyx_guard_with_custom_timeout_from_kwargs on the current
files leaves ONYX_API_BASE and ONYX_API_KEY behind; doing the same in
test_hiddenlayer_config_saas leaves HIDDENLAYER_API_BASE. Both come back clean
after this.
89 raw writes and the hand-rolled deletes become monkeypatch calls. The
class-level setup_method and teardown_method pair in the onyx file, sweeping the
same three keys twice, becomes one autouse fixture. The sys.path.insert lines
and their now-unused imports go too, and litellm.set_verbose = True, which only
turned global debug logging on for whatever ran next, is dropped rather than
restored.
test_onyx_guard_config and test_prompt_security_guard_config asserted nothing at
all, so they could only fail by raising. Each now pins what init_guardrails_v2
produces: exactly one guardrail of the right class on litellm.callbacks,
carrying the configured name, default_on and hook. The zero-assert tests in the
other three are left alone; those are a judgement about each guardrail rather
than a mechanical sweep.
tests/test_litellm/proxy/guardrails passes at 2873.
* test(policy-engine): unwind the callback global the pipeline tests scaffold around
Every one of the 16 tests in this file set litellm.callbacks by hand, each
wrapping its body in a try/finally to put the old value back, and each capturing
that old value with a .copy() first. That is 32 TQ005 violations and about 70
lines of scaffolding to say what monkeypatch.setattr says in one.
The write also sat outside the try, so the block that restores it did not cover
the statement that changed it.
16 tests pass either way, and litellm.callbacks reads restored on both sides,
because the conftest snapshot already lists it. The point is that these tests
stop depending on that snapshot to clean up after them.
* test(realtime): unwind the same callback global in the realtime streaming tests
Same global, same shape as the previous commit. 25 writes to litellm.callbacks,
2 of them wrapped in a try/finally that resets to [] rather than to the old
value, and 12 tests that write it with no protection at all.
monkeypatch.setattr replaces all of them, and the sys.path.insert with its
now-unused os and sys imports goes too.
Both sides read restored here as well, for the same reason as the previous
commit: litellm.callbacks is in the conftest snapshot. What changes is that
these tests no longer lean on it.
101 tests pass in this file, 16 in the policy engine one.
* style(realtime): wrap the one signature the monkeypatch param pushed past 120
Seventeen tests in this file save a litellm module global, open a try, write
it, and restore it in a finally. Four more sit behind autouse fixtures that
reset the flag to a hard-coded False rather than to whatever it was.
monkeypatch.setattr does all of that, so the capture, the try and the finally
come out and the test body loses a level of indentation. The alias-format
fixtures stop guessing the value they are restoring to.
Also drops the sys.path.insert, whose argument resolves four levels above the
repo, so it was never what made the imports work.
TQ003 1077 -> 1076 and TQ005 2836 -> 2796, and the budget ceilings come down
with them. 443 tests pass either way; the conftest snapshot was already
catching these globals, so this is about not needing it.
* test(cost-calc): stop 182 global writes leaking out of the cost-calc suites
Across test_cost_calculator.py and llm_cost_calc/test_llm_cost_calc_utils.py,
58 tests opened by setting LITELLM_LOCAL_MODEL_COST_MAP in os.environ and
replacing litellm.model_cost, and none of them put the env var back. The
second file already had a _local_model_cost_map fixture doing it by hand with
a try/finally, so both idioms sat in the same file.
Keep that fixture, give it monkeypatch, and have every one of those tests ask
for it. The margin and discount tests drop their hand-rolled
copy-then-restore in favour of monkeypatch.setattr, which also puts the
global back when an assertion fails part way through.
Both files also drop a sys.path.insert whose argument resolves outside the
repo, so it was never what made the imports work.
TQ003 1077 -> 1075, TQ004 768 -> 693, TQ005 2836 -> 2731, and the budget
ceilings come down with them.
* fix(test): make the streamed-cost tests load the map they assert against
The local_cost_map fixture set LITELLM_LOCAL_MODEL_COST_MAP but never reloaded
litellm.model_cost, and reading the variable is not what loads the map. So the
three streaming-cost tests billed against whatever map the process happened to
be holding, and their hardcoded prices only held when something else had
already swapped in the checked-in one. This branch stops the cost-calc tests
leaking that map, which left test_main billing at the ambient prices instead.
The fixture now loads the map it names, so the prices these tests assert hold
on their own.
Both datadog test files hand-roll what monkeypatch.setenv already does: read the
old value, write the test value, put the old one back on the way out. The cost
management fixture checks the old value for truthiness rather than for None, so
an operator running the suite with DD_API_KEY set to the empty string gets it
deleted rather than restored. Starting from DD_API_KEY="" and running test_init
leaves it None on the current file, and "" after this.
13 raw os.environ writes become monkeypatch.setenv, the two fixtures stop being
yield fixtures because there is nothing left to do on the way out, and the now
unused os import goes with them.
27 tests pass across the two files, 88 across tests/test_litellm/integrations/datadog.
* test: use monkeypatch.setenv for env writes in tests/test_litellm
`os.environ["X"] = v` inside a test leaks the value into every test that runs
after it in the same worker, so ordering decides the result. 262 of those
writes across 40 files now go through pytest's `monkeypatch` fixture, which
restores the previous value at teardown.
The rewrite skips any test that a mock.patch-family decorator wraps, any test
with defaulted positional parameters, any test whose own name is called
directly elsewhere, and rebinds nothing inside nested defs, because in each of
those cases appending a fixture parameter changes what pytest or mock binds.
Ratchets the TQ004 ceiling from 768 to 506.
* fix(test): delete the key through monkeypatch instead of popping it first
Five tests popped a key straight out of `os.environ`, ran, then restored it with
`monkeypatch.setenv`. By the time monkeypatch saw the name it was already gone,
so it recorded "absent" as the value to go back to and deleted the key at
teardown. On a worker that inherited a real `RESEND_API_KEY`, `SENDGRID_API_KEY`,
`UI_PASSWORD`, `LITELLM_SALT_KEY` or `OPENAI_API_KEY`, every test after the first
one ran without it.
`monkeypatch.delenv(..., raising=False)` removes the key and restores whatever
was there, so the try/finally the manual restore needed goes with it.
* chore(test): leave the two cost-calc files to the PR that rewrites them fully
Both files are also in #37815, which converts the module-global writes as well
as the env writes and folds them into one fixture. Two PRs rewriting the same
lines differently is a conflict nobody benefits from resolving, so this one
drops back to staging on those two and keeps the other 39.
TQ004 clears 200 here instead of 275; the rest moves with #37815.
Fifteen tests assigned litellm.audit_log_callbacks, s3_callback_params or
s3_audit_callback_params directly and leaned on two autouse fixtures to put
them back. monkeypatch.setattr does that at the point of use, so each test now
says what it sets, including the one that swaps the value mid-test to prove the
cache does not serve the stale params.
The fixtures keep only the work monkeypatch cannot do: the per-test empty
callback list, and clearing the logger and audit caches around each test.
Twenty tests in test_request_metadata.py assigned the global directly and
leaned on an autouse fixture to put it back afterwards. monkeypatch.setattr
does both jobs at the point of use, so each test now says what it sets and the
fixture that existed only to undo them goes away.
* fix(ci): stop the mutation report publishing a score it never measured
Run 32475268575 was the first dispatch of this workflow since May. Every setup
step passed and mutmut generated all 48 mutant files, so the suspected
zero-mutants bug is not what stops it. It dies in the stats phase, where mutmut
times the configured test set once up front. That set included
tests/proxy_behavior/management/, a behaviour tier that talks to a real
seeded database, so the run ended having mutated nothing.
Narrow tests_dir to the unit tier that maps to paths_to_mutate. Run 32476663383
proved a Postgres service is not enough on its own: with a schema but no seed
rows the same test fails on a foreign key instead, and a mutation score is only
meaningful against the tests that claim to cover the mutated code.
The second half is the one that matters. With no results at all,
mutation_report.py printed "No surviving mutants, the test suite caught every
mutation" and exited 0, so a run that mutated nothing published a perfect score.
It now separates no survivors from no results, says which it got, and exits 1.
* fix(ci): count mutmut's multi-word verdicts as results
The verdict capture was `\w+`, so it matched only single-word statuses. mutmut's
status_by_exit_code table has four that are not: `no tests`, `not checked`,
`caught by type check` and `check was interrupted by user`. A finished run made
entirely of those parsed as zero results, which is exactly the state this script
now treats as an unfinished run, so it would have failed a run that had in fact
completed.
The regression test asserting `reported == 2` on a three-verdict fixture was
codifying that, and now asserts 3. A second test walks all four multi-word
statuses and checks the report does not call the run unfinished.
Caught by Greptile on #37825.
* fix(ci): keep the saml tests out of the mutmut stats phase
Run 32477695014 got past the database blocker and ran 208 of the configured
tests, then ended on one error: test_saml_sso.py builds an x509 certificate in
a fixture, and inside mutmut's mutants/ sandbox cryptography's hash classes are
imported under a second identity, so .sign() rejects the SHA256 instance with
"Algorithm must be a registered hash algorithm".
That is a property of the sandbox, not of the tests or the code being mutated,
and one erroring test ends the stats phase before a single mutant runs.
* fix(ci): only claim a clean sweep when something was shown to be killed
`mutmut results` skips killed mutants by design, so its silence means either
that everything was killed or that nothing ran. Counting the verdicts it does
print cannot tell those apart, which left the report still able to say the suite
caught every mutation on a run whose mutants were all `no tests` or
`not checked`.
The clean-sweep sentence is now gated on mutmut-cicd-stats.json reporting a
non-zero killed count, which is the only signal that positively distinguishes
the two. Without it the report says so in as many words and main returns 1. A
run with zero kills and a stats file says that too.
The test asserting a non-killed run was not called unfinished was codifying the
same confusion; it is replaced by three that pin each branch.
Caught by Greptile on #37825.
* fix(ci): treat stats that count survivors the report never listed as untrusted
clean_sweep_is_provable passed on any positive kill count, so a stats file
reporting 48 killed and 3 survived, next to a `mutmut results` that listed no
survivors, still published a clean sweep. The two sources contradict each other
there, and neither one is worth believing. It now requires the stats file to
agree that nothing survived, and the report says which disagreement it found.
* fix(ci): refuse a clean sweep while mutants never reached the tests
A run can end with kills, no survivors, and a pile of mutants marked no tests,
skipped, suspicious, timeout or segfault. Those never got put in front of the
suite, so "caught every mutation" says more than the run measured. The verdict
now names which of them it found and withholds the pass, and the status list
those five come from is one constant the summary and the verdict share.
* fix(ci): read anything that is not a kill or a survivor as unresolved
The unresolved statuses were a list of five, so a run ending in a status the
reporter had never met, "check was interrupted by user" among them, still
counted as a clean sweep. The rule is now the other way round: killed, survived
and total are the keys with a meaning here, and every other non-zero count is a
mutant that did not reach the tests, whatever mutmut chose to call it.
The Responses WebSocket path, the pre-call deployment hook and the
per-frame project quota hook are all selected by small predicates that
nothing asserted directly. Mutating those four decisions left 4 of 6
mutants alive against the mapped test file.
Cover them at the boundary: the rust WebSocket path needs both the
openai provider and the rust flag, a plain CustomLogger must not
advertise a pre-call deployment hook while an overriding or inheriting
one must, and only callbacks that actually expose a callable
enforce_project_io_token_quota_for_frame reach the WebSocket loop.
Kill rate on those four decisions goes 2/6 -> 6/6; the file goes 66 -> 75
passing.
Six helpers in `litellm/proxy/utils.py` decide the usage a failed request
records, and none of them is named anywhere in the suite. Two of their
decisions could be reversed with the file still green: a request with nothing
countable in it lifted as a zero-token usage, and a request that never
reached a provider billed for input it never sent.
Twelve cases asserting those contracts directly, plus a canary pinning the
literal no-upstream-call key the module branches on, so a rename cannot pass
silently.
Rebuilding a streamed response and pricing it is the path a spend row comes
from, and nothing asserted it end to end. Reversing either half of the usage
the provider reported left the file green.
Three cases: the rebuilt response bills the usage the last chunk carried,
streaming and not streaming bill the same usage the same, and a stream that
reported no usage is still billed rather than dropped.
The cost is asserted against the catalog prices the run itself reads, with a
non-zero guard in front of it so an all-zeros lookup cannot satisfy it
vacuously. Pinning the dollar figure as a literal would have made a routine
gpt-4o price update fail a test about usage reconstruction.
Eight validators in that module decide what a request body may say, and none
of them was asserted anywhere. Reversing any one of the eight left the file
green.
Cover them at the API boundary: a JWT issuer must pick audience validation or
opt out, a temp budget needs both halves, an empty max budget reads as no
limit, an organization member can only take a role the organization has, an
LLM-backed injection check needs the call it would make, and four server-only
markers are never taken from the caller.
The injection case builds each incomplete body as its own value rather than
deleting a key out of the one it is iterating.
* perf(ci): cache the Rust build the unit shards compile from scratch
Every unit shard installs the workspace, and the root package builds through
maturin, so each of the eleven jobs compiles litellm-rust/crates/python-bridge
in release mode before a single test runs. That step measured 2m40s a shard on
2026-08-21, which is more wall clock than the entire unit tier spends running
tests, and none of it was cached: the uv cache covers wheels it downloads, not
wheels it builds, and a path dependency whose source moves every commit can
never hit that cache anyway.
A composite action now exports CARGO_TARGET_DIR to a fixed workspace path and
caches it alongside the Cargo registry, keyed on Cargo.lock. Cargo rebuilds only
what changed, so a warm job pays for the bridge crate rather than its whole
dependency graph. Measured locally, that is 34s cold against 8s warm, including
after a Python-only or Rust-only edit.
The absolute path matters: uv builds the wheel from its own working directory,
so a relative target directory lands the artifacts where nothing can find them
again.
* perf(ci): cache the Rust build in the other four workflows that sync the workspace
code-quality, mcp, documentation and the schema.d.ts check each install the
workspace and so each compile the bridge from scratch, measured at 138s, 177s,
163s and 154s on 2026-08-21. The lint job pays the same and is left to #37783,
which already owns that file's setup section.
* perf(ci): cache the Rust build in the lint job too
* fix(ci): cache cargo's own target directory instead of redirecting it
uv builds the wheel in place, so cargo already writes to litellm-rust/target,
which test-rust.yml has cached all along. Redirecting CARGO_TARGET_DIR bought
nothing and cost a GITHUB_ENV write that zizmor rejects as a code-execution
path.
* chore(ci): raise the job backstop for the added setup step
The cargo cache is a fifth bounded setup step, so the base's setup ceiling goes
30m to 35m and every job budget follows: 55 to 60, and proxy-server's 95 to 100.
check_workflow_startup_safety enforces exactly this sum, and failed on the first
push without it.
* perf(ci): cache the Rust build in the four remaining workflows that sync
Six workflows were wired; ten install the workspace. The four left out still
compile the pyo3 bridge from scratch.
test-terraform-provider.yml is the one that matters per PR: its endpoint-drift
job triggers on any change under litellm/proxy/**. The other three are a
scheduled load check, a manual mutation run, and the staging-push counts
publisher, whose gate syncs the project inside scripts/type_check_gate.py
rather than in a workflow step, so nothing in the file names the build it pays
for.
completion() aliased the caller's header mapping instead of copying it,
then merged the provider-scoped headers into that same object. The router
shares one header dict across fallback attempts, so the credential written
on an Anthropic attempt was still present when a later Bedrock or Vertex
attempt read the dict, defeating the provider scoping.
Copy the mapping before merging so each attempt sees only its own headers.
get_llm_provider ran in the messages handler and again inside completion,
so a provider/vendor/model id lost its vendor segment and reached upstream
bare. Pass the caller's unresolved model down the bridge instead, and move
the responses marker into the canonical provider/responses/model slot.
Reporting stays provider-local on both bridges: message_start names the id
the provider itself knows, through a shared local_model_name helper.
Fixes#37716
Adversarial review of the new multipart keying turned up collisions where
two different provider requests computed the same replay key, which is the
dangerous failure for a replay harness: the second request silently gets the
first one's response instead of missing loudly.
- a part counts as an upload when it has a filename or declares its own
content type, and the declared content type joins the identity, so two
uploads of the same bytes under the same field no longer collapse
- the uploaded parts contribute a JSON list of [field, filename, type]
triples instead of a "field:filename" string, so a separator inside a
filename can no longer impersonate a field boundary
- repeated field names get a "name[n]" suffix with a literal "[" doubled
first, so a repeated field and a literally indexed one stay distinct
- a field value that is not UTF-8 is stored as a base64 sha256 digest;
base64 rather than hex because the canonicalizer rewrites 64-character
hex runs to <sha256> and folded every binary value onto one key
- a field whose name reads as a credential is stored as <secret>. This
stays key-preserving because the key is recomputed from the stored
request rather than saved beside it, so the live request carrying the
real value still matches its redacted fixture
- the uploaded byte length leaves the key. The canonicalizer absorbs
timestamp and id drift inside a file, and that drift moves the count,
so keeping it there made re-records miss
Also stops a lookalike parameter such as "xboundary=" from being read as
the multipart boundary, and gives the OpenAI batch backend model a single
constant instead of three copies of the literal.
BUNDLE_FORMAT_VERSION goes to 3 because all of this moves recorded keys.
A bundle recorded under the old rules now fails naming both versions
instead of missing on every call.
Chat completions, embeddings, the non-streaming /v1/messages tests, and the
OpenAI batch deployment now register through the provider edge, so
E2E_FIXTURE_MODE=record captures their provider calls and replay serves them
back offline. None of them was wired before, so record was a silent no-op over
these suites and replay quietly went live instead of using the bundle
Multipart uploads now key on their parsed parts: every ordinary form field,
plus the field name, filename, content digest, and length of each file part.
The boundary is envelope rather than content, so it stays out of the digest
instead of changing the key on every run. A body that does not parse as its
declared envelope still has the boundary normalized away before hashing, so
the fallback is at least stable, and it records a name that says why
Binary uploads hash byte for byte. Canonicalizing them first meant decoding
with errors="replace", which collapsed every invalid byte to one U+FFFD and
gave two different PDFs of the same length the same key
Bundles stay out of the repo: they hold verbatim provider response bodies and
expire seven days after recording. Publishing them for CI is LIT-5748, and
streaming fidelity is LIT-5742
* fix(pricing): add undated azure aliases for gpt-audio-mini and gpt-realtime-mini
Azure deployments are commonly created against the undated model name,
and the cost-tracking docs say to set base_model to azure/<model> — but
only the dated -2025-10-06 entries existed for these two models (the
openai provider has undated aliases for both). base_model:
azure/gpt-audio-mini therefore resolved to nothing and, depending on the
fallback path, text tokens billed at $0 while audio tokens billed fine.
Mirror the -2025-10-06 entries as undated aliases, exactly like the
undated openai entries mirror their newest dated variant.
Fixes#33170
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(pricing): assert undated azure audio aliases exactly mirror their dated entries
Review follow-up: COST_FIELDS missed realtime-specific cost keys
(cache_creation_input_audio_token_cost, cache_read_input_token_cost,
input_cost_per_image). Full-entry equality catches drift on every field.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pricing): mirror updated mode=realtime on the undated gpt-realtime-mini alias
Upstream changed the dated entry's mode from chat to realtime after this
branch was cut; the undated alias must stay a byte-for-byte mirror.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pricing): mirror the new deprecation_date onto the undated gpt-audio-mini alias
* test(pricing): use shared local_model_cost_map fixture so get_model_info's lru_cache never crosses maps
---------
Co-authored-by: Mihidum Hettiyahandi <55163074+mihidumh@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(anthropic_messages): gate sampling params on /v1/messages like /chat/completions
/v1/messages forwarded temperature/top_p/top_k raw to models that removed
sampling params (supports_sampling_params: false — Claude 4.7+/Fable 5),
producing provider 400s that router fallbacks mask as silent model
downgrades. The chat path already gates these via
AnthropicModelInfo._apply_sampling_param; reuse it in
get_requested_anthropic_messages_optional_param so both endpoints agree:
drop under drop_params, else raise the clean client-side 400.
Fixes#35053
* test(anthropic): drive new sampling-param tests off the kwarg, not the global
The five tests added here set `litellm.drop_params = True` under a manual
try/finally. That trips TQ005 (module-global mutation, 10 new violations
over the ceiling) and it leaks process-wide if the finally is ever
skipped, which is what the save/restore conftest exists to paper over.
`get_requested_anthropic_messages_optional_param` already takes
`drop_params` as a kwarg, and that is the path /v1/messages actually
uses, so pass it directly. `monkeypatch.setattr` pins the global to
False so each test proves the per-request flag alone is sufficient and
cannot pass on a leaked global.
Verified: TQ gate clean, all 10 tests pass, and the 3 that assert the
new gating still fail with the fix in utils.py reverted.
---------
Co-authored-by: Mihidum Hettiyahandi <55163074+mihidumh@users.noreply.github.com>
* fix(mcp): resolve admin OAuth sessions to the same server set the connect page shows
* fix(mcp): bind admitted admin rows through the entitlement ceiling, not the credential scope clause
add_provider_specific_headers_to_request tagged the client's Authorization header
with the same provider list as anthropic-beta and anthropic-version, so an
sk-ant-oat subscription token was sent to AWS Bedrock and Google Vertex AI as
well. On Bedrock it replaced the SigV4 signature, or the deployment's own API
key, and AWS answered 403 "Invalid API Key format". On Vertex it went out as a
second Authorization header next to the Google one and Google answered 401
ACCESS_TOKEN_TYPE_UNSUPPORTED.
The credential and those API headers need different scopes, so a request can now
carry more than one ProviderSpecificHeader entry. The API headers keep the
provider list they already had and the credential gets its own entry scoped to
anthropic alone. get_provider_specific_headers takes either a single entry or a
sequence and merges only the entries whose provider list matches, so callers that
pass one entry keep working unchanged.
Bedrock SigV4 signing and the deliberate extra_headers Authorization pass-through
in _sign_request are left alone.
