* test: add regression coverage for twelve closed issues
Adds targeted regression tests for behavior that was fixed but left ungated,
so the fixes cannot silently regress:
- #33772 openai cache_write_tokens cost
- #34309 Responses API cache cost_breakdown
- #35363 /v1/responses batch spend
- #36619 auto-router api_base/api_key leak on a shared model name
- #35359 batch fallbacks within the owning model group
- #36523 passthrough streamed Responses spend log
- #36646 passthrough embeddings spend log
- #37147 non-object metadata on create_batch is a 400
- #35362 unscoped list files reads the managed-file store
- #33221 gpt-5.6 bridges to Responses on function tools alone
- #34487 LLM complexity classifier runs for every caller metadata shape
- #35124 streamed /v1/messages emits success logging on both bridges
Cost assertions read rates from litellm.model_cost rather than hardcoding
dollar amounts, so they do not drift on repricing.
* fix: stop the new regression tests polluting and tripping over shared global state
Two shard failures, both from global state the new tests share with their
neighbours rather than from the behaviour under test.
test_main.py's local_cost_map pinned litellm.model_cost but left the
get_model_info lru_cache warm, so completion_cost billed at whatever prices
were cached earlier in the process while the assertions read the pinned map.
Clear the cache on both sides of the fixture, matching the local_model_cost_map
fixture in tests/test_litellm/conftest.py.
The anthropic messages streaming tests called GLOBAL_LOGGING_WORKER.flush()
on whatever queue happened to be around. A queue left non-empty by an earlier
test is still bound to that test's loop, so join() either hangs or raises
"bound to a different event loop". Rebind to the running loop before the call
and wait for the captured payload instead of a fixed sleep.
An empty content list, or one holding only opaque blocks, still lets the
provider-bound branch replay the summary text. The inspection path treated
any non-None content as final, so that replayed text stayed invisible to
guardrails and token counting.
The field is declared optional on OpenAIFileObject and its own docstring says it
is absent on every upload guardrails did not touch, but the /v1/files routes have
no response_model, so FastAPI falls through to jsonable_encoder with exclude_none
off and serialises the unset default as an explicit null. Every create and
retrieve response on a proxy with no guardrails configured at all picked up a
litellm_batch_guardrail: null it never had before, and so did every row of a file
list, since those rows are the same object.
A wrap serializer drops the key only when nothing set it, so the populated report
still reaches the wire intact, including a record whose guardrail is null. The
managed-files list route spreads a stored file_object blob rather than the model,
so rows persisted before this lands keep their null until it is dropped there too.
A reasoning input item that carries only summary text is replayed to the
provider as reasoning_content, so inspection-only callers must see that
text too. They used to fall through to the generic content branch, which
reads content and drops a summary-only item, leaving guardrails and token
counters blind to text the model still receives.
litellm_request_total_latency_metric's start_time is set inside
common_processing_pre_call_logic, which only runs after user_api_key_auth
has already succeeded, so the metric silently excluded authentication and
pre-call setup time despite being documented as total request latency. The
sibling litellm_request_queue_time_seconds metric had the same problem:
its arrival_time was captured after auth too, despite its own comment
claiming to track when the request arrived at the proxy.
request.state.litellm_received_at is now stamped unconditionally at the
very first line of user_api_key_auth (previously only when OTEL was
configured), giving a timestamp that precedes all auth work. Both metrics
now derive from it: queue_time_seconds genuinely spans arrival through the
start of pre-call processing, and the total-latency metric adds that
queue time on top of its existing start/end window so it becomes true
end-to-end latency.
queue_time_seconds ends exactly at start_time rather than a separately
captured timestamp, so its window and the total-latency window share a
boundary instead of overlapping and double-counting a few lines of setup
work on every request.
When get_team_object fails, the centralized auth gate rebuilds the team
from the token's own fields. A token whose team row was missing when the
key was read carries team_models=[] and team_blocked=False, and the
model-access check reads an empty model list as every model, so the
rebuilt team grants more than the real team ever did.
get_team_object reported a deleted team and a database that would not
answer as the same 404, so the fallback could not tell a definitive
answer from a degraded read. Raise a TeamNotFoundError subclass, still a
404 with the same detail so every other caller is unaffected, only when
the database answers and the row is absent.
A team that is provably gone now refuses, and no setting overrides that.
Otherwise the grant is merely unknown: a token carrying one may vouch,
since replaying a recorded grant cannot widen it, and a token carrying
none may not. allow_requests_on_db_unavailable still opts back out there,
and is only consulted once the failure is known to be a degraded read.
The Admin UI mints every session key against the UI_TEAM_ID sentinel,
which by design never has a team row, so every UI request hit the new
refusal with no override. Exempt UI_TEAM_ID explicitly so it keeps
reconstructing from the token unconditionally, matching how the MCP
handler and agent_permission_handler already special-case it.
Resolves LIT-5522
* fix(auth): resolve team object_permission independently in the unresolvable-team fallback
When get_team_object fails for a token's team_id, _user_api_key_auth_builder
reconstructs a LiteLLM_TeamTableCachedObj from the token's own cached fields,
carrying team_object_permission_id but leaving object_permission unset. That
silently dropped any vector-store or MCP restriction the team carried,
granting more access than the token's own object_permission_id vouches for.
Resolve the object permission by its id directly via get_object_permission,
independent of the unreadable team row, matching how every other consumer of
a team's object_permission (vector store access checks, MCP tool/server
resolvers) already treats an unresolvable team as "no restriction at this
level" and re-resolves on its own.
* fix(auth): trim ticket references and narrative docstrings per Greptile review
Drop the LIT-5539 ticket id from test names and fixture strings, and shorten
both the new helper's docstring and the regression test docstrings to their
contracts rather than restating the fix's history.
The team roster update, the user.teams update, the team membership
delete, and the team-scoped verification token delete ran as four
sequential writes with no transaction around them, so a failure
between any two left the removal half applied. Thread a single
prisma transaction through all four writes, following the same
tx.<table> pattern /team/member_add and /team/member_update already
use, so either all four land or none do.
The Presidio guardrail masks messages in place inside pre_call_hook, but three
paths independently persisted or emitted the raw pre-guardrail data: the
SpendLogs proxy_server_request body snapshot (taken before the hook runs),
a verbose_proxy_logger.debug dump of the raw request, and logging_only mode's
async_logging_hook, which never masked the model's response before it reached
external logging callbacks.
Resolves LIT-6015
The session lookup reads spend logs straight out of the database, so a
follow-up sent seconds after the turn it chains off found nothing while the
row was still queued in the worker that served it, and the conversation was
dropped without an error. Responses calls now ask the spend-log writer to
flush on its next pass instead of waiting out its poll interval, and the
lookup gives a just-finished turn a short second chance.
Replaying a session also accepted `input` only as a string or a single dict,
so the standard list shape dropped every user turn and left the model with
assistant messages alone.
The standalone migration entrypoint re-runs `prisma generate` after the
migration completes. That refresh writes into the installed prisma package in
site-packages, which an arbitrary non-root uid cannot do, and which no uid can
do under a read-only root filesystem. Both are supported configurations of the
migrations Job: helm/litellm-helm/tests/migrations-job_tests.yaml asserts
runAsNonRoot, runAsUser and readOnlyRootFilesystem all render.
The write has always failed there, but the failure used to be swallowed. Making
migration failures fatal turned it into a hard exit 1, so a Job that applied
every migration correctly now reports Failed and blocks the rollout it was
supposed to gate.
The refresh is redundant in the shipped images: every Dockerfile generates the
client at build time from the same baked schema, copies it into the runtime
stage, and asserts it resolves there. It stays load-bearing only for a source
checkout, where CircleCI runs the entrypoint under `set +e` and ignores the exit
code anyway. So the call stays and only its exit code stops propagating;
migration failures are still fatal.
image-scan never ran on the change that introduced this, because its path filter
did not list the entrypoint it exercises. Add prisma_migration.py and
entrypoint.sh so the non-root offline migration test gates them from now on.
Write the fallback reasoning item id back to the cache so the
reasoning-done path and the completed snapshot cannot drift apart, and
cover the shared delta id and the snapshot alignment with tests.
Guardrails, token counting and rate limiting share the input transform with
the provider path, so moving reasoning onto reasoning_content hid it from
them. Provider-bound callers opt in with replay_reasoning.
The ceiling used to go through `int(... or 3)`, so anything `int()` accepted
worked. Tightening the new shared validator to `isinstance(int)` turned a
config that boots today into a proxy that refuses to start, because
`max_agentic_loops: os.environ/MAX_AGENTIC_LOOPS` is resolved to a string
before it reaches either check, and a YAML-quoted "5" is a string too.
Accept ints, integral floats, and strings that parse to a whole number. Keep
refusing bools, fractional floats, words, and anything below 1.
Align the response.completed item IDs by copying each output item rather than
writing to it in place, and move the regression cases into the existing
completion-response and image-generation test modules.
Streaming /v1/responses over the completion bridge minted a fresh resp_{uuid4}
for every response, while spend tracking stored the inner chat completion id as
request_id. The session lookup queries on request_id, so a follow-up sent with
that response id matched no rows and the prior conversation was silently
dropped. The iterator now pulls the first upstream chunk before emitting
response.created, so created, in_progress and completed all carry the same
encoded chat completion id.
Two more ways the same history went missing:
- The session lookup only read spend logs already written to the DB, so a
follow-up sent inside the batch writer's window found nothing. It now also
reads the rows still queued in memory.
- Input was only accepted as a string or a single dict, so the list shape the
Responses API actually sends dropped every user turn from the reconstructed
history.
* fix(otel): emit LLM Call spans for speech, image, moderation, ocr and transcription
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): log the image request before caller headers are merged in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): map non-chat routes to standard genai operations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop caller image headers aliasing the logged request body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep resolved api_base in async moderation pre_call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): log resolved client endpoint for speech pre_call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(otel): justify mutable request payloads in speech and image pre_call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep caller headers out of the logged speech request body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The ceiling was only checked at the feature level, on
litellm_settings.websearch_interception_params. The per-deployment
litellm_params.max_agentic_loops, which wins over it, went straight into
int(kwargs.get("max_agentic_loops", 3) or 3), so a 0 was swallowed by the
falsy fallback and read as the default 3. Asking for the tightest ceiling
handed you the loosest one. A non-integer booted the proxy and then failed
every request to that model with "invalid literal for int() with base 10".
Both settings now share one validator, which names the field it rejected,
and the per-deployment value is checked while the model list is read at
startup so a bad value stops the proxy rather than surfacing per request.
The check sits in load_config rather than on LiteLLM_Params because the
proxy builds its router with ignore_invalid_deployments=True, where a
validation error drops the deployment silently instead of refusing to
start. This is the same placement the complexity_router_config plugin
check already uses.
Chat completions read the same key through a separate path that turned 0
into 1 and true into a ceiling of 1, so it now shares the validator too
and the key means one thing on both surfaces.
encrypted_content on a reasoning input item is written by LiteLLM's own
_encode_thinking_blocks as a JSON array of Anthropic/Bedrock thinking
blocks, so decode it back and replay the signed blocks on the assistant
message instead of dropping them. Providers without a native
ResponsesAPIConfig now keep the verifiable chain-of-thought across turns,
and prior-turn reasoning stops reaching the provider as visible
assistant text.
The Chat Completions -> Responses bridge stamped the upstream chatcmpl-*
ID onto message output items, so replaying bridged history into native
OpenAI Responses failed with "Expected an ID that begins with 'msg'".
Image generation calls were minted as chatcmpl-*_img_N instead of ig_*,
and reasoning items used a salted hash() that is not stable across
processes.
Streaming minted msg_* for its incremental events but rebuilt the
response.completed snapshot through the same broken transform, so the
snapshot contradicted the events it had just sent and streaming clients
hit the same 400. The snapshot now reuses the IDs already streamed.
Fixes#27333
A capped turn on a streaming request is rebuilt into SSE by
FakeAnthropicMessagesStreamIterator. It emitted content_block_stop for
every block but content_block_start only for text, thinking,
redacted_thinking and tool_use, so a web search turn's server_tool_use
and web_search_tool_result blocks produced stops with no matching start.
Anthropic's SDK accumulator appends on content_block_start and then
indexes content[event.index] on content_block_delta, so the orphan stops
shifted every later index and client.messages.stream() raised IndexError
on the text block. Unknown block types now pass through with a start of
their own, which keeps position equal to index.
Also corrects two claims that said no current caller reaches the loop
with stream=True. AgenticStreamingIterator does, and it keeps raising,
because its events are already on the wire.
Eleven DummyManagedFiles stubs still declared afile_list(self, purpose,
litellm_parent_otel_span). The real hook grew user_api_key_dict, limit and
after, so the doubles no longer stand in for the interface they replace.
Their tests pass today only because every one of them takes a provider
branch that never reaches the hook, which means a stub going stale is
invisible until some later test does reach it and reads a TypeError as a
behavior change.
Signatures only; no test changes behavior.
OpenAIFilesPurpose was missing evals, which OpenAI documents. The upload
route validates against that set, so POST /v1/files with purpose=evals was
already being rejected, and the new listing validator extended the same
rejection to GET /v1/files?purpose=evals, turning a purpose OpenAI accepts
into a hard 400. Nothing branches exhaustively on the type, so widening it
changes no routing.
The managed-file listing test fake only understood a created_by filter. The
OR filter a key carrying both a user_id and a team_id produces, the team_id
filter a service-account key produces, and the empty filter a proxy admin
produces all fell through it and returned every row, so the shapes most real
keys send went uncovered. The fake now applies the filter it is handed, and
the listing is tested against all three, including paging an OR filter
across a cursor.
Two docstrings claimed the continuation chunk bounds what a filtered page
costs. It bounds queries per row scanned; the walk is still linear in the
rows the caller owns.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
Thirteen tests flipped the flag directly, and an autouse fixture reset it to
True around each of them by hand. monkeypatch.setattr does both jobs, so the
fixture keeps only the part that says what the default is, and each test states
its own override at the point it needs one.
Fifteen tests assigned litellm.callbacks directly and left the conftest global
snapshot to clean up after them. monkeypatch.setattr restores it as part of the
test, so the file no longer depends on that safety net to stay isolated.
Twenty-three tests across eleven files opened with litellm.set_verbose = True
and never put it back, so the flag stayed on for everything that ran after them
in the same process. None of those files read the output it produces: no
caplog, no capsys, no assertion on a log line, so the flag was left over from
debugging. Deleting it beats restoring it, since restoring keeps the noise.
Ten of the eleven stop leaving the flag on. test_volcengine_embedding.py still
ends with it set, from something it exercises rather than from the test itself,
which is worth its own look.
Fifteen tests opened with litellm.set_verbose = True and never put it back, so
the flag stayed on for everything that ran after them in the same process.
Nothing in the file reads the output it produces: there is no caplog, no capsys
and no assertion on a log line, so the flag was left over from debugging.
Deleting it beats restoring it, since restoring keeps the noise.
Seven tests captured litellm.use_legacy_interactions_schema, wrapped their body
in a try, and put it back in a finally. monkeypatch.setattr does that, so the
capture, the try and the finally go and the bodies lose an indentation level.
The remaining hand-rolled restores stay. They hold the flag only across the
iterator's constructor and put it back before the test iterates, so handing
them to monkeypatch would widen that window to the whole test and change what
the streaming assertions run against.
Nine tests in test_http_handler.py captured litellm.disable_aiohttp_transport,
force_ipv4, ssl_ecdh_curve or the request_timeout pair, wrapped their whole body
in a try, and put the value back in a finally. monkeypatch.setattr does all of
that, so the captures, the try and the finally go away and the bodies lose a
level of indentation. The class-scoped restore_request_timeout fixture existed
only for that same bookkeeping and goes with them.
litellm.in_memory_llm_clients_cache is left alone on purpose: the eviction tests
assert a handler is garbage collected, and monkeypatch holds the replaced value
alive until teardown, which keeps the weakref they check from clearing.
Ten tests set litellm.s3_callback_params by hand. Four of them reset it to None
on the last line of the test body, which only runs when the test passes; the
other six wrap the body in try/finally to put the old value back. Raising inside
test_s3_verify_false_handling on the current file leaves the whole callback
config, bucket, endpoint and keys, set in the process for whatever runs next.
monkeypatch.setattr covers both shapes and restores on failure, so the 28 TQ005
violations and the try/finally scaffolding come out together.
51 tests pass, and the wider tests/test_litellm/integrations tree is unchanged.
The five TQ002 mock-echo tests in this file are left alone; those need a
judgement about what S3 logging should assert, not a mechanical sweep.
