A migration that defines a row-rewriting routine and calls it as
"backfill"() at the top level slipped past the checker, since masking
blanks double-quoted identifiers before the routine-call search runs, so
the call could not be found by name and the body read as uncalled. mask()
now returns those identifier spans and outside_definition puts them back,
so a call written through a quoted identifier reads as the call it is and
the routine's body gets scanned the same as a bare call
The DML scan reconstructed a DO/EXECUTE'd literal with undouble, which collapses
each doubled quote to one character and shrinks the text. Every offset after a
collapsed pair then shifted, so a row-rewrite scanned out of the literal reported
an earlier file line and could miss a data-migration-ok marker placed on its real
line. Reconstruct with defuse_escapes instead, turning each doubled quote into a
quote and a space so the pair keeps its two characters and every offset holds,
while a nested -- or /* still stays inside its string.
The call-detection restore splices into a fixed-position list, so its
.ljust(end - start) holds that length invariant and a test guards it. The
DML-scan recursion instead hands the undoubled literal to a fresh scan_region
as its own region, whose length feeds nothing, so the pad only appends
trailing spaces that shift no keyword and change no reported line. Drop it and
the docstring clause that claimed it kept the offsets landing
Call detection restored a single-quoted DO or EXECUTE payload through
without_comments while its `''` escapes were still doubled. The first quote of
a pair opened an empty string and closed it on the second, leaving a `--` or
`/*` from a nested string bare, so it blanked the real call after it and the
routine read as uncalled: its rewrite body then went unscanned at boot. Undouble
each single-quoted payload before restoring it, and pad it back to the span it
fills so the later offsets still land. Dollar-quoted bodies do not escape quotes
and are left as they were.
A parenthesised VALUES list ended the search for an insert's row source
only when no group followed it, so a RETURNING or an ON CONFLICT DO
UPDATE carrying a subquery was read as the rows the insert copies. A
writable CTE bounded by its own VALUES list was handed the query the
statement ends with for the same reason: the WITH branch read the whole
statement rather than the part holding the insert.
A CREATE FUNCTION or CREATE PROCEDURE body was scanned as if it ran at
boot, but defining a routine only stores it. The body is now read when
the same migration names the routine somewhere else, so a migration that
defines a backfill and then runs it is still caught, and one whose name
needed quoting is read either way since quoting is blanked at the call
sites too.
main() had no test, so neither its exit codes nor the branch the CI gate
reads were pinned; a mutant returning 0 on a violation passed the whole
suite. Its four outcomes now have tests, along with both directions of
each fix above.
Taking the last group at the statement's outermost level assumed the row
source was written there, and an insert is allowed to carry more after it:
`(SELECT ...) ON CONFLICT ("id") DO NOTHING` ends on the conflict target
and `... RETURNING ("id")` on the returning list, so the query supplying
the rows was never reached and a full table copy passed the gate.
Each group is now read on its own terms and the first to name a row source
is the answer, since the others are the column list and the clauses an
insert may carry, none of which names one.
An `INSERT` whose `VALUES` list holds a scalar subquery was reported as a
rewrite whenever that list was not the plain top-level one: joined to
another term by `UNION`, `INTERSECT` or `EXCEPT`, or written inside
parentheses, which Postgres accepts. Both shapes insert a fixed handful of
rows, so the gate was rejecting migrations that do nothing wrong.
A set operation is now split into its terms and each is read on its own,
since the insert is a rewrite when any one term is a query. A row source
kept in parentheses is read on its own terms too. The operators are found
outside every parenthesis, so a set operation written inside a `VALUES`
list does not cut the list in half.
A `JOIN ... USING` inside a subquery that helps build an EXECUTE's command
was taken for the start of its bind values, so anything written after it
went unscanned and a rewrite there was never reported. Only a `USING` with
the parentheses closed can be the bind-values clause.
A `FOR ... LOOP` header carries no semicolon of its own, so the first
statement of the loop body is written into the same semicolon-delimited
run. Reading the pair as one statement let the header's row source stand
in as the keyword for both, which hid whatever the loop repeats: a plain
`UPDATE` in a query-driven loop went unreported, and so did an `EXECUTE`
of one. That is the shape a row-by-row backfill takes, and it is the
shape this gate exists to stop.
Reading every operator let a comparison beside an assignment look like one.
`ok := n = 1 AND stmt = '<dml>'` registered `stmt` as written, which collided
with the `EXECUTE stmt` further down and flagged a block that rewrites nothing.
A statement holds one assignment at most, so the search now stops at the first
operator that reads as one: everything after it is the expression being
assigned, where an `=` only ever compares. Nine shapes were flagged this way,
a cast, a `coalesce`, a `format`, a named-argument arrow and the rest, and all
of them are valid PL/pgSQL that leaves the table untouched.
`INTO` and `USING` no longer count as names an `EXECUTE` runs. Masking blanks a
literal in place, so `EXECUTE '<sql>' INTO n` left `INTO` looking like the name
being run, and an ordinary query reaching the same word collided with it. The
docstring claiming that collision was impossible was wrong, and both words are
now dropped instead.
A loop is a fourth way a literal reaches a variable. `FOR stmt IN SELECT
'<dml>' LOOP EXECUTE stmt` empties the table and the gate passed it, so the
target of a `FOR` or a `FOREACH` is read as assigned too.
Reading each statement once rather than once per operator also drops the cost
of a statement with thousands of them from seconds to milliseconds.
A bare = was found with partition, so a comparison earlier on the line took
the one slot and the assignment after it went unread. INTO targets and the
name an EXECUTE runs are also allowed to sit on the next line now.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* test: unshadow the module handles the F811 sweep left behind, and pin the two live tests that went red with it
The F811 sweep in #37878 removed the fixture-local `import litellm` from four
conftests, but the bare `import litellm.proxy.proxy_server` a few lines below
still binds `litellm` as a function local, so `importlib.reload(litellm)` runs
before the name is assigned and every test in those directories errors at
setup. The `hasattr` guard on the line above already proves the module is
loaded, so the import only ever bound the name. Drop it, and enable F823 in
ruff-tests.toml, which flags all four sites at the failing line and would have
blocked the sweep
The same sweep renamed the `check_non_streaming_response` parameter but left
one read of `completion`, which now resolves to `litellm.completion`, and
removed an import whose side effect was the only thing making
`litellm.proxy.proxy_server` reachable in the moderation hook test. That test
already takes `monkeypatch`, so patch the router through it and stop leaking
the router into later tests
`test_content_policy_exception_openai` passed vacuously until #37887 turned it
into a real `pytest.raises`, and OpenAI no longer rejects a lyrics prompt with
a content policy error. Inject an AsyncOpenAI client whose transport answers
with OpenAI's own `content_policy_violation` rejection so the mapping to
ContentPolicyViolationError is exercised every run
`test_async_create_batch` hit a 409 cancelling a batch OpenAI had already
marked failed. The cancel step tolerated a completed batch but not a failed
one. Fold both guards into one helper that tolerates a failed batch only when
OpenAI's recorded error is the org's enqueued token limit, and prints the
batch's errors so the reason is in the log either way
* test: close the injected AsyncOpenAI client after the content policy test
* chore(lint): ratchet TQ005 down by the global mutation this branch cleared
* chore(lint): ratchet TQ005 to 2660 on the merged tree
* chore(lint): ratchet TQ005 to 2561 on the merged tree
* chore(lint): ratchet TQ005 to 2548 on the merged tree
Thirteen tests flipped the flag directly, and an autouse fixture reset it to
True around each of them by hand. monkeypatch.setattr does both jobs, so the
fixture keeps only the part that says what the default is, and each test states
its own override at the point it needs one.
Fifteen tests assigned litellm.callbacks directly and left the conftest global
snapshot to clean up after them. monkeypatch.setattr restores it as part of the
test, so the file no longer depends on that safety net to stay isolated.
Scanning a statement's own literals for SQL only makes sense when the name
before INTO is a variable the body later executes. INSERT INTO names a table
there, so an insert into a table sharing a variable's name was flagged for
whatever its column values happened to spell. An INSERT that really does
assign reaches INTO through RETURNING, which the preceding word separates.
Also names the scope boundary in the module docstring: the ban is on
row-rewriting DML, not on everything whose cost scales with table size.
Twenty-three tests across eleven files opened with litellm.set_verbose = True
and never put it back, so the flag stayed on for everything that ran after them
in the same process. None of those files read the output it produces: no
caplog, no capsys, no assertion on a log line, so the flag was left over from
debugging. Deleting it beats restoring it, since restoring keeps the noise.
Ten of the eleven stop leaving the flag on. test_volcengine_embedding.py still
ends with it set, from something it exercises rather than from the test itself,
which is worth its own look.
Fifteen tests opened with litellm.set_verbose = True and never put it back, so
the flag stayed on for everything that ran after them in the same process.
Nothing in the file reads the output it produces: there is no caplog, no capsys
and no assertion on a log line, so the flag was left over from debugging.
Deleting it beats restoring it, since restoring keeps the noise.
Seven tests captured litellm.use_legacy_interactions_schema, wrapped their body
in a try, and put it back in a finally. monkeypatch.setattr does that, so the
capture, the try and the finally go and the bodies lose an indentation level.
The remaining hand-rolled restores stay. They hold the flag only across the
iterator's constructor and put it back before the test iterates, so handing
them to monkeypatch would widen that window to the whole test and change what
the streaming assertions run against.
Nine tests in test_http_handler.py captured litellm.disable_aiohttp_transport,
force_ipv4, ssl_ecdh_curve or the request_timeout pair, wrapped their whole body
in a try, and put the value back in a finally. monkeypatch.setattr does all of
that, so the captures, the try and the finally go away and the bodies lose a
level of indentation. The class-scoped restore_request_timeout fixture existed
only for that same bookkeeping and goes with them.
litellm.in_memory_llm_clients_cache is left alone on purpose: the eviction tests
assert a handler is garbage collected, and monkeypatch holds the replaced value
alive until teardown, which keeps the weakref they check from clearing.
Ten tests set litellm.s3_callback_params by hand. Four of them reset it to None
on the last line of the test body, which only runs when the test passes; the
other six wrap the body in try/finally to put the old value back. Raising inside
test_s3_verify_false_handling on the current file leaves the whole callback
config, bucket, endpoint and keys, set in the process for whatever runs next.
monkeypatch.setattr covers both shapes and restores on failure, so the 28 TQ005
violations and the try/finally scaffolding come out together.
51 tests pass, and the wider tests/test_litellm/integrations tree is unchanged.
The five TQ002 mock-echo tests in this file are left alone; those need a
judgement about what S3 logging should assert, not a mechanical sweep.
test_zai_provider.py set LITELLM_LOCAL_MODEL_COST_MAP and litellm.model_cost
directly and never put them back, so every test that ran after it in the same
process saw a local cost map instead of the real one. The two respx tests did
the same to litellm.disable_aiohttp_transport with no restore at all.
Both now go through monkeypatch, which restores on teardown including when the
test fails. The cost-map setup moves into a fixture requested by exactly the
five tests that read the cost map.
The mid-conversation system tests prime the prompt cache by re-sending an
identical /v1/messages body until its usage shows the full prefix read back
three times in a row. The e2e stack runs with the litellm response cache on,
so every resend after the first is served from redis with the first call's
usage and the streak can never form; the three unflagged-model tests have
failed on every litellm-e2e build since the consecutive-read check landed.
Send cache: {"no-cache": true} on RichMessagesRequest, as test_cache_control
already does, so each resend reaches the provider.
The two fallback tests sent the same "say hi" / max_tokens=16 body to the
gpt-5.5 fallback, so one empty (finish_reason=length) completion served the
second test from the response cache and failed both. Give each test a unique
prompt and leave gpt-5.5 enough tokens to emit text.
Onyx, prompt security, hiddenlayer, repelloai and deepkeep all write straight to
os.environ and unset again at the bottom of each test. None of the five has a
try/finally, so the moment a test fails it returns to the runner with the keys
still set and whatever runs next in that worker inherits them.
Raising inside test_onyx_guard_with_custom_timeout_from_kwargs on the current
files leaves ONYX_API_BASE and ONYX_API_KEY behind; doing the same in
test_hiddenlayer_config_saas leaves HIDDENLAYER_API_BASE. Both come back clean
after this.
89 raw writes and the hand-rolled deletes become monkeypatch calls. The
class-level setup_method and teardown_method pair in the onyx file, sweeping the
same three keys twice, becomes one autouse fixture. The sys.path.insert lines
and their now-unused imports go too, and litellm.set_verbose = True, which only
turned global debug logging on for whatever ran next, is dropped rather than
restored.
test_onyx_guard_config and test_prompt_security_guard_config asserted nothing at
all, so they could only fail by raising. Each now pins what init_guardrails_v2
produces: exactly one guardrail of the right class on litellm.callbacks,
carrying the configured name, default_on and hook. The zero-assert tests in the
other three are left alone; those are a judgement about each guardrail rather
than a mechanical sweep.
tests/test_litellm/proxy/guardrails passes at 2873.
* test(policy-engine): unwind the callback global the pipeline tests scaffold around
Every one of the 16 tests in this file set litellm.callbacks by hand, each
wrapping its body in a try/finally to put the old value back, and each capturing
that old value with a .copy() first. That is 32 TQ005 violations and about 70
lines of scaffolding to say what monkeypatch.setattr says in one.
The write also sat outside the try, so the block that restores it did not cover
the statement that changed it.
16 tests pass either way, and litellm.callbacks reads restored on both sides,
because the conftest snapshot already lists it. The point is that these tests
stop depending on that snapshot to clean up after them.
* test(realtime): unwind the same callback global in the realtime streaming tests
Same global, same shape as the previous commit. 25 writes to litellm.callbacks,
2 of them wrapped in a try/finally that resets to [] rather than to the old
value, and 12 tests that write it with no protection at all.
monkeypatch.setattr replaces all of them, and the sys.path.insert with its
now-unused os and sys imports goes too.
Both sides read restored here as well, for the same reason as the previous
commit: litellm.callbacks is in the conftest snapshot. What changes is that
these tests no longer lean on it.
101 tests pass in this file, 16 in the policy engine one.
* style(realtime): wrap the one signature the monkeypatch param pushed past 120
Seventeen tests in this file save a litellm module global, open a try, write
it, and restore it in a finally. Four more sit behind autouse fixtures that
reset the flag to a hard-coded False rather than to whatever it was.
monkeypatch.setattr does all of that, so the capture, the try and the finally
come out and the test body loses a level of indentation. The alias-format
fixtures stop guessing the value they are restoring to.
Also drops the sys.path.insert, whose argument resolves four levels above the
repo, so it was never what made the imports work.
TQ003 1077 -> 1076 and TQ005 2836 -> 2796, and the budget ceilings come down
with them. 443 tests pass either way; the conftest snapshot was already
catching these globals, so this is about not needing it.
* test(cost-calc): stop 182 global writes leaking out of the cost-calc suites
Across test_cost_calculator.py and llm_cost_calc/test_llm_cost_calc_utils.py,
58 tests opened by setting LITELLM_LOCAL_MODEL_COST_MAP in os.environ and
replacing litellm.model_cost, and none of them put the env var back. The
second file already had a _local_model_cost_map fixture doing it by hand with
a try/finally, so both idioms sat in the same file.
Keep that fixture, give it monkeypatch, and have every one of those tests ask
for it. The margin and discount tests drop their hand-rolled
copy-then-restore in favour of monkeypatch.setattr, which also puts the
global back when an assertion fails part way through.
Both files also drop a sys.path.insert whose argument resolves outside the
repo, so it was never what made the imports work.
TQ003 1077 -> 1075, TQ004 768 -> 693, TQ005 2836 -> 2731, and the budget
ceilings come down with them.
* fix(test): make the streamed-cost tests load the map they assert against
The local_cost_map fixture set LITELLM_LOCAL_MODEL_COST_MAP but never reloaded
litellm.model_cost, and reading the variable is not what loads the map. So the
three streaming-cost tests billed against whatever map the process happened to
be holding, and their hardcoded prices only held when something else had
already swapped in the checked-in one. This branch stops the cost-calc tests
leaking that map, which left test_main billing at the ambient prices instead.
The fixture now loads the map it names, so the prices these tests assert hold
on their own.
Both datadog test files hand-roll what monkeypatch.setenv already does: read the
old value, write the test value, put the old one back on the way out. The cost
management fixture checks the old value for truthiness rather than for None, so
an operator running the suite with DD_API_KEY set to the empty string gets it
deleted rather than restored. Starting from DD_API_KEY="" and running test_init
leaves it None on the current file, and "" after this.
13 raw os.environ writes become monkeypatch.setenv, the two fixtures stop being
yield fixtures because there is nothing left to do on the way out, and the now
unused os import goes with them.
27 tests pass across the two files, 88 across tests/test_litellm/integrations/datadog.
* test: use monkeypatch.setenv for env writes in tests/test_litellm
`os.environ["X"] = v` inside a test leaks the value into every test that runs
after it in the same worker, so ordering decides the result. 262 of those
writes across 40 files now go through pytest's `monkeypatch` fixture, which
restores the previous value at teardown.
The rewrite skips any test that a mock.patch-family decorator wraps, any test
with defaulted positional parameters, any test whose own name is called
directly elsewhere, and rebinds nothing inside nested defs, because in each of
those cases appending a fixture parameter changes what pytest or mock binds.
Ratchets the TQ004 ceiling from 768 to 506.
* fix(test): delete the key through monkeypatch instead of popping it first
Five tests popped a key straight out of `os.environ`, ran, then restored it with
`monkeypatch.setenv`. By the time monkeypatch saw the name it was already gone,
so it recorded "absent" as the value to go back to and deleted the key at
teardown. On a worker that inherited a real `RESEND_API_KEY`, `SENDGRID_API_KEY`,
`UI_PASSWORD`, `LITELLM_SALT_KEY` or `OPENAI_API_KEY`, every test after the first
one ran without it.
`monkeypatch.delenv(..., raising=False)` removes the key and restores whatever
was there, so the try/finally the manual restore needed goes with it.
* chore(test): leave the two cost-calc files to the PR that rewrites them fully
Both files are also in #37815, which converts the module-global writes as well
as the env writes and folds them into one fixture. Two PRs rewriting the same
lines differently is a conflict nobody benefits from resolving, so this one
drops back to staging on those two and keeps the other 39.
TQ004 clears 200 here instead of 275; the rest moves with #37815.
Fifteen tests assigned litellm.audit_log_callbacks, s3_callback_params or
s3_audit_callback_params directly and leaned on two autouse fixtures to put
them back. monkeypatch.setattr does that at the point of use, so each test now
says what it sets, including the one that swaps the value mid-test to prove the
cache does not serve the stale params.
The fixtures keep only the work monkeypatch cannot do: the per-test empty
callback list, and clearing the logger and audit caches around each test.
Twenty tests in test_request_metadata.py assigned the global directly and
leaned on an autouse fixture to put it back afterwards. monkeypatch.setattr
does both jobs at the point of use, so each test now says what it sets and the
fixture that existed only to undo them goes away.
* fix(ci): stop the mutation report publishing a score it never measured
Run 32475268575 was the first dispatch of this workflow since May. Every setup
step passed and mutmut generated all 48 mutant files, so the suspected
zero-mutants bug is not what stops it. It dies in the stats phase, where mutmut
times the configured test set once up front. That set included
tests/proxy_behavior/management/, a behaviour tier that talks to a real
seeded database, so the run ended having mutated nothing.
Narrow tests_dir to the unit tier that maps to paths_to_mutate. Run 32476663383
proved a Postgres service is not enough on its own: with a schema but no seed
rows the same test fails on a foreign key instead, and a mutation score is only
meaningful against the tests that claim to cover the mutated code.
The second half is the one that matters. With no results at all,
mutation_report.py printed "No surviving mutants, the test suite caught every
mutation" and exited 0, so a run that mutated nothing published a perfect score.
It now separates no survivors from no results, says which it got, and exits 1.
* fix(ci): count mutmut's multi-word verdicts as results
The verdict capture was `\w+`, so it matched only single-word statuses. mutmut's
status_by_exit_code table has four that are not: `no tests`, `not checked`,
`caught by type check` and `check was interrupted by user`. A finished run made
entirely of those parsed as zero results, which is exactly the state this script
now treats as an unfinished run, so it would have failed a run that had in fact
completed.
The regression test asserting `reported == 2` on a three-verdict fixture was
codifying that, and now asserts 3. A second test walks all four multi-word
statuses and checks the report does not call the run unfinished.
Caught by Greptile on #37825.
* fix(ci): keep the saml tests out of the mutmut stats phase
Run 32477695014 got past the database blocker and ran 208 of the configured
tests, then ended on one error: test_saml_sso.py builds an x509 certificate in
a fixture, and inside mutmut's mutants/ sandbox cryptography's hash classes are
imported under a second identity, so .sign() rejects the SHA256 instance with
"Algorithm must be a registered hash algorithm".
That is a property of the sandbox, not of the tests or the code being mutated,
and one erroring test ends the stats phase before a single mutant runs.
* fix(ci): only claim a clean sweep when something was shown to be killed
`mutmut results` skips killed mutants by design, so its silence means either
that everything was killed or that nothing ran. Counting the verdicts it does
print cannot tell those apart, which left the report still able to say the suite
caught every mutation on a run whose mutants were all `no tests` or
`not checked`.
The clean-sweep sentence is now gated on mutmut-cicd-stats.json reporting a
non-zero killed count, which is the only signal that positively distinguishes
the two. Without it the report says so in as many words and main returns 1. A
run with zero kills and a stats file says that too.
The test asserting a non-killed run was not called unfinished was codifying the
same confusion; it is replaced by three that pin each branch.
Caught by Greptile on #37825.
* fix(ci): treat stats that count survivors the report never listed as untrusted
clean_sweep_is_provable passed on any positive kill count, so a stats file
reporting 48 killed and 3 survived, next to a `mutmut results` that listed no
survivors, still published a clean sweep. The two sources contradict each other
there, and neither one is worth believing. It now requires the stats file to
agree that nothing survived, and the report says which disagreement it found.
* fix(ci): refuse a clean sweep while mutants never reached the tests
A run can end with kills, no survivors, and a pile of mutants marked no tests,
skipped, suspicious, timeout or segfault. Those never got put in front of the
suite, so "caught every mutation" says more than the run measured. The verdict
now names which of them it found and withholds the pass, and the status list
those five come from is one constant the summary and the verdict share.
* fix(ci): read anything that is not a kill or a survivor as unresolved
The unresolved statuses were a list of five, so a run ending in a status the
reporter had never met, "check was interrupted by user" among them, still
counted as a clean sweep. The rule is now the other way round: killed, survived
and total are the keys with a meaning here, and every other non-zero count is a
mutant that did not reach the tests, whatever mutmut chose to call it.
The Responses WebSocket path, the pre-call deployment hook and the
per-frame project quota hook are all selected by small predicates that
nothing asserted directly. Mutating those four decisions left 4 of 6
mutants alive against the mapped test file.
Cover them at the boundary: the rust WebSocket path needs both the
openai provider and the rust flag, a plain CustomLogger must not
advertise a pre-call deployment hook while an overriding or inheriting
one must, and only callbacks that actually expose a callable
enforce_project_io_token_quota_for_frame reach the WebSocket loop.
Kill rate on those four decisions goes 2/6 -> 6/6; the file goes 66 -> 75
passing.
Six helpers in `litellm/proxy/utils.py` decide the usage a failed request
records, and none of them is named anywhere in the suite. Two of their
decisions could be reversed with the file still green: a request with nothing
countable in it lifted as a zero-token usage, and a request that never
reached a provider billed for input it never sent.
Twelve cases asserting those contracts directly, plus a canary pinning the
literal no-upstream-call key the module branches on, so a rename cannot pass
silently.
Rebuilding a streamed response and pricing it is the path a spend row comes
from, and nothing asserted it end to end. Reversing either half of the usage
the provider reported left the file green.
Three cases: the rebuilt response bills the usage the last chunk carried,
streaming and not streaming bill the same usage the same, and a stream that
reported no usage is still billed rather than dropped.
The cost is asserted against the catalog prices the run itself reads, with a
non-zero guard in front of it so an all-zeros lookup cannot satisfy it
vacuously. Pinning the dollar figure as a literal would have made a routine
gpt-4o price update fail a test about usage reconstruction.
Eight validators in that module decide what a request body may say, and none
of them was asserted anywhere. Reversing any one of the eight left the file
green.
Cover them at the API boundary: a JWT issuer must pick audience validation or
opt out, a temp budget needs both halves, an empty max budget reads as no
limit, an organization member can only take a role the organization has, an
LLM-backed injection check needs the call it would make, and four server-only
markers are never taken from the caller.
The injection case builds each incomplete body as its own value rather than
deleting a key out of the one it is iterating.
