* fix(proxy): stop log lines from leaving terminal colors on
* test(proxy): drop source scan for unclosed terminal colors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): ask OpenAI and Azure audio chat models for audio output in the health check probe
* fix(health): check the probed model when health_check_model is set, and mark Azure audio models as audio output
A deployment with an audio base_model and a text health_check_model got the audio fields sent to its text probe model. The Azure gpt-audio and gpt-4o audio preview cost-map entries now set supports_audio_output, so a dated Azure name like azure/gpt-audio-1.5-2026-02-23 gets the audio probe too
* fix(health): read litellm_params.base_model, and drop the stored base_model when Test Connect swaps the model
* fix(health): judge the audio probe on a swapped-in health_check_params model, and keep Test Connect's stored base_model
* fix(health): look up a provider-prefixed base_model under its own provider and keep text-only operator modalities
* fix(vertex_ai): drop the caller's anthropic-version header on Claude /v1/messages
Vertex reads the API version from the body's anthropic_version and answers 400 when a forwarded anthropic-version header names a different one, which anthropic SDK 2.x always does. Strip the header on the Python and LITELLM_RUST Vertex messages routes.
* fix(vertex_ai): carry the caller's anthropic-version into the Claude /v1/messages body
Vertex reads the API version from the body alone, so the caller's
anthropic-version header now becomes the body's anthropic_version instead
of a pinned vertex-2023-10-16. The 2023-01-01 and 2023-06-01 first-party
versions keep vertex-2023-10-16, and the header itself is still never sent.
Same change on the Python proxy and the Rust gateway.
* fix(vertex_ai): follow drop_params for an anthropic-version Vertex does not accept
Only versions Vertex accepts in the body (vertex-2023-10-16, 2026-10-21) are
carried into anthropic_version. 2023-06-01, 2023-01-01, a blank header, or no
header keep vertex-2023-10-16. Any other value drops to vertex-2023-10-16 under
drop_params and answers a 400 naming drop_params without it, on the Python proxy
and the Rust gateway.
* refactor(vertex_ai): return the unsupported anthropic-version error as a value and keep Rust helpers and tests in place
* fix(rust gateway): resolve os.environ drop_params and accept the same flag words as Python
The gateway read drop_params: os.environ/NAME and words like yes as off, so the drop_params opt-out from the new anthropic-version 400 did not work there
* docs(rust router): name Router::from_config in the crate README
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(responses): audit #47445
Integration cells for billing every round of a streamed MCP auto-execute on one spend row, on a native Responses upstream and the chat bridge, across tool rounds, a rejected or broken follow-up, a post-call guardrail block, a client hang-up, the response cache, and the next bridged turn
* test(responses): check a broken follow-up bills its own nonzero usage
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #46997
Add request-path cells for the global proxy spend counter: a key with no user spends toward litellm max_budget, and two proxies sharing Redis add every request to one counter that matches the proxy budget row
* test(proxy): wait for the recorded budget spend and shared counter before checking them
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
OpenAI shut down dall-e-2 and dall-e-3 on 2026-05-12. Drop their image
generation and edit configs, the Azure dall-e-2 create-and-poll flow, and
the silent dall-e-2 fallback when no model is passed.
image_generation without a model or provider now fails locally with
"model is required". image_edit and image_variation leave model out of
the provider request instead of sending dall-e-2, so providers that
ignore model (Topaz) keep working and OpenAI returns its own 400.
* ci(zizmor): run zizmor 1.31.1 without ignores and own the workflow
zizmor-action has no input for --no-ignores, so the workflow now installs the
zizmor 1.31.1 release binary directly, pinned by SHA-256 like grype in
image-scan.yml, and runs it with --no-ignores and --no-config. Inline
`zizmor: ignore` comments and config-file ignores no longer hide findings.
Behavior otherwise matches the old action defaults (regular persona, online
audits with the job token, medium and above, GitHub annotations)
Adds a CODEOWNERS entry for .github/workflows/zizmor.yml
* ci: run spend-logs-guard and cost-map-guard on pull_request
Both guards ran on pull_request_target behind a zizmor ignore comment. They
now run on pull_request and check out the base commit by sha, so the guard
scripts still come from the base branch while the PR's head is only fetched
and diffed. The zizmor ignore comments are gone
Add rustdoc overviews for a new litellm-router-core crate and for
litellm-model-catalog, a module docstring on litellm/router.py, and a
shrink-only allowlist of the Router members and router_utils modules used
outside the router, checked in CI by router_surface_check.py.
Move the router-free helpers out of litellm/router_utils: batch_utils to
litellm_core_utils, and the auto-router naming contract, tuning baseline
and baseline request next to the auto-router plugin. No behaviour change.
* fix(vertex_ai): split Gemini SSE lines on CR/LF only so U+2028 in text keeps the stream
The Gemini and Vertex AI streaming path read the SSE body with httpx
iter_lines()/aiter_lines(). Those follow str.splitlines(), which also breaks
on U+0085, U+2028, U+2029, \x0b, \x0c and \x1c-\x1e. JSON leaves U+0085,
U+2028 and U+2029 unescaped inside strings, so a data: line whose text holds
one of them was cut in two. The tail is not a data: line and the head never
parses, and the accumulate-and-retry fallback then carries the unparsed
prefix forward, so every later event is appended to it and the stream ends
with no more content and no usage.
SSE lines end only at CRLF, LF or CR. The new iter_sse_lines/aiter_sse_lines
split decoded text on exactly those, treating a CR at the end of one chunk and
an LF at the start of the next as one break. The first request and every
long-decode hop now read through them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(streaming): keep merged think tags balanced and split shared SSE lines only on CR/LF
Close an open <think> block on the first content or tool-call delta and at stream end, reopen it when reasoning returns, and route the shared chat completions stream through the SSE line splitter so U+2028 and U+2029 inside JSON no longer drop chunks.
* fix(streaming): exact-match the SSE [DONE] marker in BaseModelResponseIterator
* fix(streaming): split every SSE reader on CR/LF only and keep the logged think block closed
Switch the remaining httpx stream readers (Anthropic chat, Google GenAI,
Interactions, Codestral, openai_like, Predibase, LangGraph, Vertex Agent
Engine, A2A providers, Azure AI agents, sandboxes, the A2A endpoint, and
others) from iter_lines/aiter_lines to iter_sse_lines/aiter_sse_lines, so a
raw U+2028 or U+2029 inside a JSON string no longer splits an event.
Keep the chunk that closes a think block in the logged and cached response
when a stream ends without a finish_reason, close the block before a legacy
function_call, and make sync Codestral FIM streaming send stream=true and use
the text-completion-codestral stream parser.
Test fakes that only mocked iter_lines now stream real httpx bytes and
assert the content they feed in.
* fix(streaming): split buffered SSE parsers only on CR and LF, shard the codestral tests
Buffered SSE and JSONL parsers used str.splitlines(), which also breaks on
U+2028, U+2029 and U+0085 that JSON allows raw inside strings. They now use
split_sse_lines from sse_lines.py, which also gains a str.split fast path.
Adds tests/unit/llms/codestral to the unit workflow shard.
* fix(streaming): keep U+2028 in restamped Anthropic frames and fold CRLF before splitting SSE lines
* test: type the new stream tests, drop a comment, and benchmark the SSE line splitter
* test: type the iterator class parameters in the line-separator stream tests
* fix(responses): stream tool call arguments opened in a text chunk
* fix(responses): hold a tool call that shares a chunk with text until the text ends
* test: type the a2a SSE handler helper and the sandbox fake responses
---------
Co-authored-by: Tim Vink <vinktim@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
* fix(gemini): preserve every grounding web source
* test(gemini): cover all grounding sources per passage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Zabir Raihan <zabirraihan@icloud.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(types): replace Any with proven types in 52 files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(types): use typing_extensions.TypedDict for validated seams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(types): revert strict Anthropic response validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(types): accept a null finish reason from Bedrock Cohere and TwelveLabs Pegasus
* test(bedrock): annotate the Pegasus null finish reason test
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(websearch): strip Claude Code instructional prefix in short-circuit search
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
* fix(websearch): preserve quotes for unwrapped search queries
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
* test(websearch): replace mock with pure unit tests and cover empty queries
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
* style(websearch): fix ruff formatting blank line in handler
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
* refactor(websearch): enforce Final annotations, prevent rebinding, and retain e2e coverage
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
* test(websearch): assert server_tool_use block unconditionally
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
* fix(websearch): search only the query inside Claude Code's web search prompt
Match Claude Code's exact "Perform a web search for the query:" prefix instead of a broad regex family, keep any quotes the user typed so exact-phrase searches still work, and test the search request at the HTTP boundary.
---------
Signed-off-by: agustin18 <agustinsaiz02@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): accept request priority from the x-litellm-priority header
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): only apply x-litellm-priority on scheduler-queued routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply x-litellm-priority only where the router schedules the request
The header now maps onto priority only for chat and text completions that a
router serves, so the Cursor route's responses call and proxies started without
a config never forward it to the provider. /queue/chat/completions now honors
the header too, with the header winning over a body priority.
* test(proxy): use a real ProxyLogging in the priority header routing test
* fix(proxy): ignore a malformed priority header under drop_params instead of dropping the body priority
* fix(proxy): treat an empty priority header as unset and pass a valid one on as an integer
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Adds integration cells for a different key reading the response, message redaction, a rotated, deleted, or team-deleted creator, and a creator whose Langfuse keys resolve to nothing
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(tencent): add Seedream image generation and editing through TokenHub
* feat(tencent): price Seedream image generation and edits from TokenHub usage
* test(tencent): cover Seedream image generation, edits, and pricing through TokenHub
* fix(tencent): keep non-Seedream image models on the OpenAI-compatible route
Only seedream- models use TokenHub's wand route, so Hy-Image and other
tencent image models keep working as on main. The response now carries
TokenHub's image size and format, Pro's flat fallback bills its default
2K size, and the new tests are typed.
* fix(tencent): bill other Tencent image models as before and forward user and TokenHub params on edits
* fix(tencent): keep dropped edit params out and parse form-string watermark and seed
* fix(tencent): send extra_body edit fields to TokenHub as given
* fix(tencent): keep the routed model on Seedream edits and bill a deployment's per-token price
* fix(tencent): accept string drop params on edits, rebuild nested form fields, and honor a per-image deployment price
An image edit with a non-list additional_drop_params answered 500 for every provider after this branch validated it as a list, so a non-list value now drops nothing again, as chat treats it.
Seedream edits sent through the OpenAI SDK arrive as multipart form fields, so TokenHub-only booleans and integers are parsed and bracketed keys like optimize_prompt_options[mode] are rebuilt into objects. A deployment that sets only output_cost_per_image is billed that price, and the TokenHub URL uses the shared api_base helpers.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): route rate limit cache increments through typed helpers
* chore(ui): regenerate schema.d.ts for deployment rpd and tpd fields
* test(types): add rpd and tpd to the owned litellm_params inventory
* fix(router): run daily rpd/tpd checks after the minute checks and increment rpm and rpd in one cache trip
* fix(router): flatten the async rpm reject branch for the ruff strict gate
* fix(router): hand rpm and rpd back on any rate limit reject
A request the rpm, rpd or tpd check rejects no longer keeps its request counts, so retries against a deployment whose rpd is spent cannot fill its rpm and block the in-group fallover with the minute 429.
* test(router): a minute-rejected request no longer keeps its rpm count
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(skills): add a litellm-setup agent skill
* fix(skills): run the quickstart for Docker with Postgres in litellm-setup
Use scripts/quickstart.sh for the Docker plus Postgres path instead of a
hand-rolled Postgres container, keep uv with the user's own Postgres, load
.env as data instead of sourcing it, pin the uv Python range, chmod an
existing .env, list bedrock_converse and vertex_ai rows in the cost map
lookup, ask before editing the shell profile, and stop the health wait
when the proxy exits
* fix(skills): guide empty Postgres, docker sudo, and key fixes in litellm-setup
Ask for a new empty database on the uv path and DIRECT_URL behind a pooler,
add the baseline error to troubleshooting, and drop inherited database
settings in the loader line. Check python3 and lsof, explain Docker's
permission denied, give quickstart commands by the folder's path, and fix
provider keys by recreating the container or deleting the quickstart model.
Drop the structure-only test.
* fix(skills): keep shell profiles untouched when litellm-setup installs uv
The uv installer edits shell profiles unless UV_NO_MODIFY_PATH is set, so the skill now sets it and puts ~/.local/bin on PATH for later commands. It also says litellm --setup runs in a separate terminal and how to retire the shared master-key UI login
* fix(skills): offer litellm --setup only on the uv path
The Docker paths install no host litellm CLI, and the old wording also passed the skill's own step number on to the user
* fix(skills): ask before using an env provider key, print model names config.yaml accepts, probe the port with curl
* fix(skills): state the env key default in the question, chmod the wizard config, keep uv's bin dir on PATH
* fix(skills): keep litellm-setup on its own .env and install when the shell has others
* fix(skills): read the setup wizard's config without printing keys and hand off the exact restart command
* fix(skills): hide the database-access dev skill from end-user skill installs
* fix(skills): give the litellm --setup path a restart command that keeps its config
* docs(skills): point litellm --setup users to the master key in litellm_config.yaml
* fix(skills): hide every non-end-user skill, use Azure deployment names, keep the restart block out of the user's shell
* test(skills): find non-end-user skills in-process instead of through git
* docs(skills): price Azure deployments with base_model and put uv on PATH in the start block
* test(routing): audit #46601
Adds integration cells for the falsy budget 400, the empty and none-only declarations under drop_params, Vertex AI Claude, Databricks Claude, and the Bedrock native output retry
* test(routing): expect the Anthropic default max_tokens on Bedrock Invoke chat
Reads the default from litellm instead of pinning 4096, which #48502 raised to 128000
* test(routing): assert whole sent bodies in the Vertex AI and Bedrock native output cells
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): let the per-model budget field take a model pattern or access group
* feat(proxy): let model_max_budget entries name a model family or access group, shared across replicas
* fix(proxy): open each shared model budget window once for every replica, and match patterns in linear time
Without Redis, the first replica to charge an entry in a new window now opens it in the window spend table and
the others join it, so no flush replaces spend another replica already added. Budget patterns match by splitting
on `*` instead of a regular expression, a refused request names the entry that refused it, and the key form no
longer offers a typed pattern another row already budgets.
* fix(proxy): read the shared budget window from the writer, and keep concurrent first charges on one replica
Opening a shared window now reads the row from the writer, since a lagging read replica could hand a replica its
own start and let a flush replace the others' spend. A charge that waited on the open no longer resets a local
window another charge on the same replica opened meanwhile. Window tests use fixed timestamps.
* fix(proxy): stop a non proxy admin from lifting a team model cap with a looser group or pattern entry
Group and pattern entries now take precedence over other entries, so a team update by anyone but a proxy admin
must keep each new or changed entry within every existing cap it can take precedence over. A pattern can match a
provider-prefixed or wildcard-routed name, so any two patterns count as overlapping, and a model or group entry
counts as outranking every pattern.
* fix(proxy): narrow model budget pattern overlap, keep team caps under key patterns, and accept only typed patterns in the UI
* fix(proxy): keep model family budgets through database blips, team models and key entries
A failed window open no longer restarts the window at this replica's clock, so it cannot wipe shared spend. Spend charged meanwhile stays local and joins the shared window once the database answers
A key entry named after a model replaces only the team entry of that name, so team pattern and group caps still apply. Team-scoped models are gated and charged under their public name, and group lookups see team deployments
The team update guard compares caps by window length and only ranks patterns that can match the same name, including Bedrock vendor patterns behind a bedrock wildcard route
* fix(proxy): refuse team entries that take precedence over a capped model_max_budget entry
A non-proxy-admin could add a carve-out (an exact model, a narrower pattern or
a group) at or under the family cap, and its own counter then multiplied the
team's spend on that family. Any new entry that takes precedence over a capped
entry now needs a proxy admin, and group members include the team's own public
model names so team deployments cannot hide a group overlap.
* test(router): cover team_public_model_names_of
* docs(proxy): name pattern and access group keys in model_max_budget docstrings
* fix(proxy): name the entry holding the cap in the team model budget raise refusal
* fix(proxy): check precedence when a team edit makes an unenforced model_max_budget entry take effect
* fix(proxy): let an org admin give a legacy model_max_budget row a budget_duration at the same or a lower cap
* fix(proxy): keep a legacy model_max_budget row's stored cap when an org admin gives it a budget_duration
* feat(ui): add per-server MCP tool catalog query hook
One query per server keyed by server_id, so the toolset editor can load
tools for many servers in parallel and a failing server surfaces as an
error on its own row instead of an empty list. (LIT-10012)
* feat(ui): add toolset tool search and bulk selection helpers
Pure functions for the toolset editor: filter a server's tools by a
search query, select/deselect many tools without duplicates or dropping
other selections, and compute tri-state selection. (LIT-10012)
* feat(ui): search tools and bulk select in the toolset editor
The toolset editor search now matches tool names and descriptions as
well as server names. Typing fetches every server's tools in parallel
through the per-server query hook, auto-expands matching servers and
hides servers that loaded with no match. Tool rows become checkboxes,
each server header gets a tri-state select-all over its visible tools,
and a bar offers select/deselect all matching tools. Bulk actions only
touch visible tools, so selections on other servers are kept on save.
(LIT-10012)
* feat(ui): clear all and flag unconfirmed tools in the toolset panel
Adds a Clear all action to the Your Toolset panel and labels selected
tools whose server is gone, unreachable, or no longer lists the tool.
Those tools stay selected and are still saved, so an outage never
silently drops part of a toolset. (LIT-10012)
* test(ui): cover toolset editor tool search and bulk selection
Integration tests that drive the real toolset modal against stubbed
networking and assert the saved tools payload: cross-server search by
tool name and description, select/deselect all matching, server
select-all, single-row toggles, Clear all, and that tools on unreachable
or deleted servers survive a save. (LIT-10012)
* fix(ui): pointer cursor on the toolset server select-all checkbox
Base UI renders the checkbox as a span, which the global button cursor
rule does not cover, so the server header select-all showed the default
arrow. Use a pointer, and not-allowed while it is disabled. (LIT-10012)
* fix(ui): stop toolset editor background fan-out and keep tools on failed refetch
The toolset modal stays mounted when closed, so a finished search kept
every server's tool query enabled and refocusing the window refetched
all of them once stale. Fetch only while the editor is open, require
both the live and debounced search before fanning out, and disable
refetch on window focus for these queries. A failed refetch now keeps
the tools that already loaded instead of flagging the server
unreachable, and tools without a name are dropped. Trim comments that
restated the code. (LIT-10012)
* fix(ui): clear toolset search on close and only keep non-empty tool lists on failed refetch
Reopening the editor after a search rendered once with the old query,
which fetched every stale server; clear the search whenever the editor
closes. A failed refetch now keeps the previous list only when it has
tools, so a server that loaded none and then fails reports the error.
(LIT-10012)
* fix(ui): reset toolset editor state on close and avoid mutating selection sets
Closing the editor now also clears expanded servers, so reopening it cannot
refetch them for one render. Build the selection and expanded-server sets
without in-place mutation, drop a comment that restated the code, and use
screen queries in the new tests to stay within the prefer-screen-queries
lint budget. (LIT-10012)
* feat(daily-activity): read guardrail spend per key from the daily guardrail usage table
Adds DailyActivityRepository.guardrail_spend, which sums LiteLLM_DailyGuardrailUsageUnits.cost by date, guardrail and key. A guardrail row counts only when its key has a spend row on that day under the same scope filters as the rest of the daily activity response, so user, key and date filters apply to guardrail cost too
* feat(daily-activity): add a per-guardrail spend breakdown to /user/daily/activity/aggregated
Each day now has breakdown.guardrails[<guardrail_id>] with metrics.spend and an api_key_breakdown carrying each key's spend and metadata. Guardrail entries only report spend, since that is the one metric stored per key. The breakdown is on the user endpoint only, is empty when a model filter is set because guardrail cost is not stored per model, and lists only keys within api_key_limit while the guardrail total keeps every key. spend and budgets are unchanged
Resolves LIT-9785
* feat(daily-activity): add the guardrail breakdown to team and organization activity, with guardrail names
/team/daily/activity/aggregated and /organization/daily/activity/aggregated now return breakdown.guardrails the same way the user endpoint does, since each key belongs to one team and one organization. Each guardrail entry also carries metadata.guardrail_name, read from LiteLLM_GuardrailsTable for guardrails created through the UI or API and falling back to the breakdown key for config.yaml guardrails, whose key is already their name
* fix(daily-activity): keep daily activity up when the guardrail breakdown query fails, and treat an empty model filter as none
A database error from the guardrail spend query now logs a warning and returns the response without guardrails, where it used to fail the whole aggregated response with a 500. An empty model= no longer drops the guardrail breakdown, matching how the rest of the endpoint already ignores it. The daily activity golden files and fixtures gain the empty guardrails key every breakdown now carries
* refactor(daily-activity): fold guardrail breakdown in a worker thread and drop docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(daily-activity): report untracked guardrail units next to spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(migration_data_checks): size btree entries by the bytes Postgres stores (LIT-10244)
* test(migration_data_checks): flag exactly the compressed keys Postgres rejects at the entry limit (LIT-10244)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(aiohttp_transport): keep the owned session open while a response stream is still being read (LIT-10237)
* test(aiohttp_transport): annotate the stream regression test locals with Final and drop its docstring
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(pricing): add databricks priority pay per token prices
* fix(pricing): bill gemini 3.6 flash cache writes at its databricks input rate
* fix(pricing): drop image token priority rate the cost map schema does not carry
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): make hide-secrets honor skip_tool_message_in_guardrail
With the flag on, per guardrail or in litellm_settings, hide-secrets leaves
role: tool messages and Anthropic tool_result blocks as sent, as the
unified guardrails do. Tool call arguments are still masked.
* test(guardrails): type the skip-tool walker test and assert the whole request
* fix(ci): spend logs guard requires the approval to cover the head's spend-log lines
An approving review survives later pushes (dismiss_stale_reviews_on_push is off on main), so a PR approved once could add more LiteLLM_SpendLogs schema or migration changes and the guard kept passing. The guard now reads the review's commit_id and passes only when the approval is on the head, or the head adds no spend-log line the approved commit did not already carry.
* fix(ci): keep approved_by untouched, add latest_approval beside it for the stale-approval check
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): cover owner-scoped file listing under require_managed_files
* fix(files): list only the caller's managed files on every list route under require_managed_files
* fix(files): narrow owner-scoped managed file lists to the named provider or model and page with after_id and before_id
* fix(files): narrow scoped lists the way uploads route, and refuse lists that cannot be scoped
A model name now matches every deployment that serves it, wildcards and
aliases included, and a provider matches deployments by the provider
uploads resolve, so anthropic/* and unprefixed Claude models count. The
OpenAI and Azure lists also return the caller's pass-through uploads.
An unknown provider and different after and after_id cursors answer 400,
and a list with no matching deployment skips the database read.
* fix(files): match stored files by model name and keep flag-off lists as on main
Under require_managed_files, a narrowed list now matches a file's model
mappings by deployment id or model name, so files LiteLLM stores in its own
database show on their model and provider lists. The Anthropic list defaults
to 20 rows when no limit is sent, after_id, before_id and the cursor 400s
apply only under the flag, and an admin keeps the provider list when the
managed files hook is not loaded. List errors are returned as values and
mapped to HTTP errors in one place.
* fix(files): keep admin provider lists and index the owner-scoped walk
Under require_managed_files, an admin keeps the provider list as on main, and
only a non-admin list reads the caller's own and team managed files. A stored
file now also shows on the provider its model name routes to through a
wildcard, with exact model names first, and a list routed to a model takes the
default page size of the provider serving it. A new index on created_by and
created_at keeps each chunk of the walk an index read, and the model check
runs before the file object is parsed.
* fix(files): filter provider and model lists in the query
A non-admin list scoped to a provider or model now asks Postgres only for the
caller's rows that map to one of that scope's deployments or models, or that
live in file storage, instead of reading every owned row and filtering them in
Python. A caller with many files on other providers no longer waits for the
whole set to be read. Team members' files are matched in the same query.
* test(files): type the fake raw query in the files list tests
* refactor(proxy): list managed files through one typed database handle
* fix(files): list alias and routing-group storage uploads on their provider
A Responses API stream that fails partway now bills the usage it already consumed, ends with an error instead of a clean stop, and honors stream_timeout between chunks. Bridged /v1/messages calls report those failures to the proxy's failure hooks, so spend, budgets and logs see them
AriaAvatar pointed at /ui/assets/aria-agent.png, a path that only exists when
the proxy serves the static export under /ui. The dashboard dev server and the
split-chart nginx image 404 it, so the header Aria button and every other
AriaAvatar showed a broken image. Import the PNG statically like the provider
logos (#34125) so the bundle always serves it under /_next/static/media
* fix(streaming): surface upstream abort finish reasons as errors
* fix(streaming): keep SDK chunks with null choices streaming past the abort check
* fix(streaming): treat null choices as empty in the raw SSE abort check
* refactor(streaming): bind raw SSE choices as Final instead of rebinding
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(rust-bridge): require the native extension
_native ships in every wheel and the sdist will not install without it, yet the bridge loaded it optionally and carried None paths that could fail open for fork safety, exception mapping and OTLP errors. Delete loader.py and import _native directly. Tests inject a fake module through NativeBinding(source=...) or with_source() instead of patching the loader
The install smoke checks now verify the compiled module, and tests fail instead of skipping when _native is missing. The interactions spec canary installs the project so it has the extension
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ci): build Rust bridge in Windows smoke tests
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(aiohttp_openai): forward service_tier in transform_response
* chore(aiohttp_openai): mark service_tier assignment rebind-ok for the type gate
* fix(aiohttp_openai): set service_tier with setattr like other providers
ModelResponse has no declared service_tier field, so attribute assignment
added a reportAttributeAccessIssue error to the type gate.
* test(aiohttp_openai): verify serialized served service tier
* fix(aiohttp_openai): copy response when preserving served tier
---------
Co-authored-by: tin-berri <tin@berri.ai>
* test(cli): print-token and whoami let the proxy judge a classic token past the client window
* fix(cli): let the proxy decide when a classic lite login token has expired
* fix(cli): treat 403 and 404 as a rejected token and judge lite up logins the same way
* fix(cli): keep lite up on the client window and bound the proxy check to 3 seconds
* test(cli): prove the proxy check gives up on a silent proxy and pin stale fixtures
Point the timeout test at a local socket that never answers, use fixed
timestamps for stale CLI token fixtures, and correct the print-token docs
for tokens younger than the client window.
* fix(cli): expire a CLI token with no recorded proxy and stop treating 404 as rejected
A token file without base_url cannot be checked, so it keeps the client
window's verdict instead of printing forever. A 404 also comes from a proxy
that does not serve /v1/models, so only 401 and 403 count as a rejected key,
matching the CLI's existing key check. The timeout test now runs in process.
* test(cli): judge the whoami proxy check on a fixed timestamp and name the client window for lite up
