The LLM classifier capped every prior turn at 200 characters independently, so a
785 character turn was cut even when the whole block it belonged to was 353
characters. A character budget now bounds the block: turns are taken newest first
and quoted whole while they fit, older turns are dropped whole once it runs out,
and only the turn straddling the boundary is cut. The per-turn cap stays as an
optional clamp for operators who set it deliberately, defaulting to unset.
Adds a scheduled GitHub Actions lane on top of the merged record/replay
transport. A Saturday cron records the `replayable` e2e tests against the
real providers and publishes the fixture bundle as a private
`e2e-fixtures-bundle` artifact with a SHA-256 sidecar. Weekday crons pull
that artifact by its pinned digest, verify the checksum before extracting,
and replay it with provider credentials set to bogus values, so a run that
ever reached a real provider fails instead of passing.
An egress sentinel pins the provider hostnames to a local sink for the whole
replay job and counts every connection that reaches them; the job asserts
that count is zero, so hermeticity is proven by measurement. A red Saturday
publishes no bundle, so the next weekday finds nothing fresh and fails loudly
rather than replaying a week-old recording, and the transport's seven-day
freshness gate hard-fails any bundle that has drifted too far. The lane also
runs on demand from the Actions tab with a record/replay `mode` input.
Tests join the lane with `@pytest.mark.replayable`. The streaming Anthropic
test now counts to twenty so its recorded response banks several content
deltas, matching the assertion that the stream arrives incrementally.
A request carrying a session/trace header fans the header value into
litellm metadata as both trace_id and session_id. LangSmith then rejected
the whole ingest batch: a root run's trace_id must equal the run id
embedded in dotted_order (400), and a run-body session_id must reference
an existing tracer session (404/422). Override caller trace_id on runs
that post as roots and drop session_id only when it mirrors trace_id,
so deliberate child-run and valid tracer-session fields still pass through.
The error file now resolves through _provider_output_file_id like the output
file does. Sending the encoded id straight to the provider 404s, and the
swallowed fetch failure would silently report zero failures.
Staging split batch output-line costing into _safe_output_line_stats /
_compute_output_line_stats / _output_line_cost so one uncostable line can no
longer zero a whole batch, and added _provider_output_file_id so model-encoded
output file ids decode before the fetch. This branch's pass/fail counting was
written against the pre-split shape, where every None line meant a provider
failure.
Keep staging's structure and layer the counts on a three-way classification: a
provider-reported failure yields PROVIDER_FAILED, a provider-successful line
litellm cannot price yields UNCOSTABLE and stays in successful_requests billed
at $0. Without that split a litellm-side pricing gap would be reported to the
customer as a failed request and the counts would stop reconciling with the
provider's own request_counts.
Route the error-file fetch through _provider_output_file_id too, and carry the
new dataclass return through the callers staging added after this branch
forked.
A quoted routine call qualified by a schema and sitting inside a CREATE INDEX
expression, ON "Foo" (public."f"(col)), walked its qualifier read-through back
across the opening paren to the ON that introduces the indexed table, so the call
was misread as a relation and dropped from the call set, leaving a rewrite in that
routine unscanned. A word now only introduces the name when nothing but whitespace
and qualifier dots lies between them, so a paren in that gap keeps ON (and any
relation-introducing keyword) from reaching across it and the call stays a call.
The Responses-API to /chat/completions bridge yields ModelResponseStream
chunks that carry choices followed by a trailing event object that has no
choices key. stream_chunk_builder assumed every chunk was subscriptable at
"choices", so assembling those chunks raised KeyError('choices') and was
re-wrapped as a 500 APIError building the streaming usage.
Guard each choices access with .get("choices") so choices-less chunks are
skipped instead of crashing. Behavior is unchanged for chunks that do carry
choices, since .get("choices") is truthy only for a non-empty choices list.
Adds a regression test that assembles content across chunks followed by a
trailing chunk with no choices key.
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
/global/activity/cache_hits now returns an error_breakdown: failed spend
logs bucketed per call_type by error code and error class, read from
metadata->error_information. Clicking a red failed-requests segment on
the cache activity chart opens a per-code bar chart; hovering a bar
lists the error classes behind that code.
The video edit endpoint parsed the multipart body but dropped the uploaded
source video, only normalizing it to an id. When a raw file is uploaded it now
flows through videos.main -> the http handler -> the provider transform, which
emits multipart/form-data with the source video as a file part, matching the
official OpenAI SDK's videos.edit wire format. Edit-by-id still egresses JSON.
OpenAI videos never fetches input URLs or invents download size caps. Drop MAX_VIDEO_MEDIA_* and the fetch-and-inline path. Keep MAX_IMAGE_URL_DOWNLOAD_SIZE_MB for chat image handling. Validate Omni reference URLs, then send them as form fields the way OpenAI videos sends input_reference as a file
The LLM classifier's conversation context cut each prior turn head-only, so a turn
opening with an incident report and closing with the actual request reached the
classifier as the incident report alone. Keeping head and tail costs the same
budget and is what the truncation literature measures as best for classifying
long text.
Suppress the three reportUnknownArgumentType diagnostics with reasons at
the untyped provider-params boundary, collapse the early return, and
assign extra_body via a TypedDict-annotated literal so the file's
basedpyright profile matches the merge base exactly. The user-supplied
extra_body merge is covered end-to-end through get_optional_params.
On mapped pass-through routes, of which /vertex_ai is one,
user_api_key_auth accepts the caller key from a header literally named
litellm_user_api_key and applies it last, so it overrides every other source.
The credential-less filter neither dropped it nor resolved the caller key from
it, so a virtual key there reached Google past a real x-goog-api-key, and a
bring-your-own Authorization could be stripped when auth actually came from that
header. Drop it by name and resolve it at highest precedence.
Follow-up to #38130. The function has no callers in the repo or the docs and is
not exported from `litellm/__init__.py`, and `token_counter` already does the same
job better, so keeping a second entry point only preserves a trap.
That trap is real: Greptile flagged on #38130 that `token_counter` picks the claude
tokenizer only for bare ids. `claude-sonnet-4-5` resolves to huggingface_tokenizer,
while `claude-3-opus-20240229` and `anthropic/claude-sonnet-4-5` fall back to the
OpenAI one, 24 tokens against 27 on the same string. Deleting the wrapper removes
the surface rather than papering over it; the selection gap in `token_counter`
itself is worth its own fix.
BREAKING CHANGE: `from litellm.utils import prompt_token_calculator` no longer
resolves. Use `litellm.token_counter(model=..., text=...)`.
Route hosted_vllm video generation through /v1/videos as multipart form data so Omni extra fields such as width and extra_params reach the server instead of a JSON body Omni rejects
A quoted routine call with a SQL comment between its name and parenthesis,
`"backfill" /* reason */ ()`, is a real call that rewrites rows at boot, but the
call-site check read the raw SQL and stopped at the comment, so the routine was
read as uncalled and its rewrite slipped through. Read the call test from the
masked text instead, where every comment is already blanked to spaces, so a
comment between the name and its parenthesis is skipped exactly as whitespace is,
line, block and nested block comments alike, while a like-named non-call
identifier still opens no call and stays masked
The resolver placed both operator-configured key headers at the top of its
precedence, but user_api_key_auth only overrides with litellm_key_header_name;
a pass_through_endpoints litellm_user_api_key is checked last. So a request that
authenticated via Authorization while also sending a pass-through header could
have the wrong value chosen, leaving the authenticated Authorization key
forwarded. Order the resolver exactly like get_api_key: override first, built-in
headers next, pass-through header last.
A migration that defines an uncalled row-rewriting routine and elsewhere
references a quoted column, table, index, or constraint sharing the routine's
name was wrongly flagged: the guard restored every double-quoted identifier
before the name search, so a like-named identifier read as a call. Restore only
quoted names that open a call, followed by "(", so a routine invoked through a
quoted identifier is still caught while a like-named non-call identifier stays
masked and no rewrite-free migration is rejected
user_api_key_auth also accepts the caller key from a pass_through_endpoints
entry's headers.litellm_user_api_key, not just litellm_key_header_name. Drop
every operator-configured caller-key header by name and treat them as
top-precedence caller-key sources, so a virtual key sent through one is never
forwarded to Google.
LoggingWorker._ensure_queue nulled self._queue on a loop change, discarding every
pending LoggingTask (each an un-awaited spend-logging coroutine) with no counter and
only a debug log. SDK callers using asyncio.run() per request and mixed sync/async
processes rebind the queue's loop and silently lose spend rows and observability events.
Drain the stale queue and move the pending tasks onto a fresh queue bound to the new
loop, warn with the carried-over count, and keep flush()/join() honest since the queue
is no longer thrown away. Adds a regression test that fills the queue before the loop
change and asserts every task survives and still executes.
The LIT-4761 streaming-classification tests passed only the bring-your-own
Google OAuth token in Authorization and mocked get_litellm_virtual_key, a shape
that cannot authenticate in production. The credential-less filter now resolves
the caller key by auth precedence, so a lone Authorization value reads as the
key and is stripped. Send the virtual key in x-litellm-api-key, matching a real
request, so Authorization is preserved and the classification assertions run.
A migration that defines a row-rewriting routine and calls it as
"backfill"() at the top level slipped past the checker, since masking
blanks double-quoted identifiers before the routine-call search runs, so
the call could not be found by name and the body read as uncalled. mask()
now returns those identifier spans and outside_definition puts them back,
so a call written through a quoted identifier reads as the call it is and
the routine's body gets scanned the same as a bare call
the x-litellm-model upload path returns ids wrapped with
encode_file_id_with_model (litellm:<raw>;model,<m> base64'd). chat
completions + /v1/responses forwarded the wrapped id straight to the
provider, breaking openai (file not found / >64 chars), gemini
(unknown mime), etc. wire up the existing get_original_file_id +
is_model_embedded_id helpers in update_messages_with_model_file_ids
and update_responses_input_with_model_file_ids — falls through after
the managed-files path so existing flows are unchanged. 3 new
regression tests + dem proof len 71 -> 26.
The filter's own Bearer-only stripping missed the other schemes
user_api_key_auth accepts, so a virtual key echoed as `Authorization: Basic
<key>` alongside a higher-precedence auth header did not match the caller key
and was forwarded to Google. Reuse the auth module's _get_bearer_token so the
comparison strips exactly what authentication does (Bearer / bearer / Basic /
AWS4-HMAC-SHA256), falling back to the raw value for a bare token.
The DML scan reconstructed a DO/EXECUTE'd literal with undouble, which collapses
each doubled quote to one character and shrinks the text. Every offset after a
collapsed pair then shifted, so a row-rewrite scanned out of the literal reported
an earlier file line and could miss a data-migration-ok marker placed on its real
line. Reconstruct with defuse_escapes instead, turning each doubled quote into a
quote and a space so the pair keeps its two characters and every offset holds,
while a nested -- or /* still stays inside its string.
