* feat(router): reset_time picks when deployment rpd and tpd daily limits start over
reset_time is "HH:MM" in UTC next to rpd/tpd in litellm_params or model_info. The daily cache keys are dated by now minus that offset and the daily 429 retry-after counts to the next reset, so a free tier whose provider quota resets at 08:00 UTC comes back at 08:00 instead of midnight. Without it the window stays the UTC calendar day.
* chore(ui): regenerate schema.d.ts with reset_time
* refactor(router): rpd, tpd and reset_time are read from litellm_params only
* feat(router): rename reset_time to daily_reset_time, drop the request-sent value, accept only HH:MM
* chore(ui): regenerate schema.d.ts for daily_reset_time
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: add StreamLake provider metadata
* fix(streamlake): complete provider discovery and test request flows
* test(streamlake): assert cost and tool-param invariants instead of pinning vendor prices
* fix(streamlake): accept tools and reasoning on endpoint ids and match the current price page
Endpoint ids have no cost-map row, so tools and reasoning_effort were rejected
before reaching StreamLake. The provider now declares both capabilities, and
named non-reasoning rows opt out explicitly.
The rows now follow StreamLake's 2026-09-17 price page: drop the KAT-Coder and
Qwen3 rows it no longer serves, add the served models that had no price, drop
account-level rpm and tpm, and fill missing context windows. StreamLake is also
registered for streamlake/* wildcards and its messages and responses support.
* fix(streamlake): regenerate the endpoint matrix and type the new tests
The backup endpoint matrix advertised chat only while the root map listed
Messages and Responses, so /public/endpoints hid both. Regenerating with
scripts/generate_provider_endpoints_support.py makes the two files match.
Types the StreamLake tests and sets the Add Model test's API key with
fireEvent.change, per the dashboard test guide.
* fix(streamlake): drop the retired max_tokens field from the StreamLake cost-map rows
* fix(streamlake): price the four served models StreamLake publishes rates for and add the Qwen3.8 cache prices
* fix(streamlake): send token caps as max_completion_tokens and mark image models
StreamLake ignores max_tokens on kimi-k2.6, kimi-k2.7-code, qwen3.6-27b and
qwen3.6-35b-a3b, while every served model honors max_completion_tokens, so
the JSON provider now maps max_tokens to max_completion_tokens instead of the
reverse. Rows whose models read an image in a live probe set supports_vision,
and the Qwen3.8 rows drop the explicit cache write price, which this route
never uses since cache_control is dropped
* fix(streamlake): keep cache_control marks, price explicit cache reads, and use API input limits
StreamLake reads cache_control, so the provider keeps the marks through a preserve_cache_control flag. Qwen3.8 explicit cache hits now use their own read price, rows with a cache read price declare prompt caching, three models StreamLake serves without reasoning drop reasoning_effort, and max_input_tokens follows the limits the API reports.
* fix(utils): warn when a custom_provider_map name matches a built-in provider
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): route rate limit cache increments through typed helpers
* chore(ui): regenerate schema.d.ts for deployment rpd and tpd fields
* test(types): add rpd and tpd to the owned litellm_params inventory
* fix(router): run daily rpd/tpd checks after the minute checks and increment rpm and rpd in one cache trip
* fix(router): flatten the async rpm reject branch for the ruff strict gate
* fix(router): hand rpm and rpd back on any rate limit reject
A request the rpm, rpd or tpd check rejects no longer keeps its request counts, so retries against a deployment whose rpd is spent cannot fill its rpm and block the in-group fallover with the minute 429.
* test(router): a minute-rejected request no longer keeps its rpm count
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(skills): add a litellm-setup agent skill
* fix(skills): run the quickstart for Docker with Postgres in litellm-setup
Use scripts/quickstart.sh for the Docker plus Postgres path instead of a
hand-rolled Postgres container, keep uv with the user's own Postgres, load
.env as data instead of sourcing it, pin the uv Python range, chmod an
existing .env, list bedrock_converse and vertex_ai rows in the cost map
lookup, ask before editing the shell profile, and stop the health wait
when the proxy exits
* fix(skills): guide empty Postgres, docker sudo, and key fixes in litellm-setup
Ask for a new empty database on the uv path and DIRECT_URL behind a pooler,
add the baseline error to troubleshooting, and drop inherited database
settings in the loader line. Check python3 and lsof, explain Docker's
permission denied, give quickstart commands by the folder's path, and fix
provider keys by recreating the container or deleting the quickstart model.
Drop the structure-only test.
* fix(skills): keep shell profiles untouched when litellm-setup installs uv
The uv installer edits shell profiles unless UV_NO_MODIFY_PATH is set, so the skill now sets it and puts ~/.local/bin on PATH for later commands. It also says litellm --setup runs in a separate terminal and how to retire the shared master-key UI login
* fix(skills): offer litellm --setup only on the uv path
The Docker paths install no host litellm CLI, and the old wording also passed the skill's own step number on to the user
* fix(skills): ask before using an env provider key, print model names config.yaml accepts, probe the port with curl
* fix(skills): state the env key default in the question, chmod the wizard config, keep uv's bin dir on PATH
* fix(skills): keep litellm-setup on its own .env and install when the shell has others
* fix(skills): read the setup wizard's config without printing keys and hand off the exact restart command
* fix(skills): hide the database-access dev skill from end-user skill installs
* fix(skills): give the litellm --setup path a restart command that keeps its config
* docs(skills): point litellm --setup users to the master key in litellm_config.yaml
* fix(skills): hide every non-end-user skill, use Azure deployment names, keep the restart block out of the user's shell
* test(skills): find non-end-user skills in-process instead of through git
* docs(skills): price Azure deployments with base_model and put uv on PATH in the start block
* feat(ui): let the per-model budget field take a model pattern or access group
* feat(proxy): let model_max_budget entries name a model family or access group, shared across replicas
* fix(proxy): open each shared model budget window once for every replica, and match patterns in linear time
Without Redis, the first replica to charge an entry in a new window now opens it in the window spend table and
the others join it, so no flush replaces spend another replica already added. Budget patterns match by splitting
on `*` instead of a regular expression, a refused request names the entry that refused it, and the key form no
longer offers a typed pattern another row already budgets.
* fix(proxy): read the shared budget window from the writer, and keep concurrent first charges on one replica
Opening a shared window now reads the row from the writer, since a lagging read replica could hand a replica its
own start and let a flush replace the others' spend. A charge that waited on the open no longer resets a local
window another charge on the same replica opened meanwhile. Window tests use fixed timestamps.
* fix(proxy): stop a non proxy admin from lifting a team model cap with a looser group or pattern entry
Group and pattern entries now take precedence over other entries, so a team update by anyone but a proxy admin
must keep each new or changed entry within every existing cap it can take precedence over. A pattern can match a
provider-prefixed or wildcard-routed name, so any two patterns count as overlapping, and a model or group entry
counts as outranking every pattern.
* fix(proxy): narrow model budget pattern overlap, keep team caps under key patterns, and accept only typed patterns in the UI
* fix(proxy): keep model family budgets through database blips, team models and key entries
A failed window open no longer restarts the window at this replica's clock, so it cannot wipe shared spend. Spend charged meanwhile stays local and joins the shared window once the database answers
A key entry named after a model replaces only the team entry of that name, so team pattern and group caps still apply. Team-scoped models are gated and charged under their public name, and group lookups see team deployments
The team update guard compares caps by window length and only ranks patterns that can match the same name, including Bedrock vendor patterns behind a bedrock wildcard route
* fix(proxy): refuse team entries that take precedence over a capped model_max_budget entry
A non-proxy-admin could add a carve-out (an exact model, a narrower pattern or
a group) at or under the family cap, and its own counter then multiplied the
team's spend on that family. Any new entry that takes precedence over a capped
entry now needs a proxy admin, and group members include the team's own public
model names so team deployments cannot hide a group overlap.
* test(router): cover team_public_model_names_of
* docs(proxy): name pattern and access group keys in model_max_budget docstrings
* fix(proxy): name the entry holding the cap in the team model budget raise refusal
* fix(proxy): check precedence when a team edit makes an unenforced model_max_budget entry take effect
* fix(proxy): let an org admin give a legacy model_max_budget row a budget_duration at the same or a lower cap
* fix(proxy): keep a legacy model_max_budget row's stored cap when an org admin gives it a budget_duration
* feat(ui): add per-server MCP tool catalog query hook
One query per server keyed by server_id, so the toolset editor can load
tools for many servers in parallel and a failing server surfaces as an
error on its own row instead of an empty list. (LIT-10012)
* feat(ui): add toolset tool search and bulk selection helpers
Pure functions for the toolset editor: filter a server's tools by a
search query, select/deselect many tools without duplicates or dropping
other selections, and compute tri-state selection. (LIT-10012)
* feat(ui): search tools and bulk select in the toolset editor
The toolset editor search now matches tool names and descriptions as
well as server names. Typing fetches every server's tools in parallel
through the per-server query hook, auto-expands matching servers and
hides servers that loaded with no match. Tool rows become checkboxes,
each server header gets a tri-state select-all over its visible tools,
and a bar offers select/deselect all matching tools. Bulk actions only
touch visible tools, so selections on other servers are kept on save.
(LIT-10012)
* feat(ui): clear all and flag unconfirmed tools in the toolset panel
Adds a Clear all action to the Your Toolset panel and labels selected
tools whose server is gone, unreachable, or no longer lists the tool.
Those tools stay selected and are still saved, so an outage never
silently drops part of a toolset. (LIT-10012)
* test(ui): cover toolset editor tool search and bulk selection
Integration tests that drive the real toolset modal against stubbed
networking and assert the saved tools payload: cross-server search by
tool name and description, select/deselect all matching, server
select-all, single-row toggles, Clear all, and that tools on unreachable
or deleted servers survive a save. (LIT-10012)
* fix(ui): pointer cursor on the toolset server select-all checkbox
Base UI renders the checkbox as a span, which the global button cursor
rule does not cover, so the server header select-all showed the default
arrow. Use a pointer, and not-allowed while it is disabled. (LIT-10012)
* fix(ui): stop toolset editor background fan-out and keep tools on failed refetch
The toolset modal stays mounted when closed, so a finished search kept
every server's tool query enabled and refocusing the window refetched
all of them once stale. Fetch only while the editor is open, require
both the live and debounced search before fanning out, and disable
refetch on window focus for these queries. A failed refetch now keeps
the tools that already loaded instead of flagging the server
unreachable, and tools without a name are dropped. Trim comments that
restated the code. (LIT-10012)
* fix(ui): clear toolset search on close and only keep non-empty tool lists on failed refetch
Reopening the editor after a search rendered once with the old query,
which fetched every stale server; clear the search whenever the editor
closes. A failed refetch now keeps the previous list only when it has
tools, so a server that loaded none and then fails reports the error.
(LIT-10012)
* fix(ui): reset toolset editor state on close and avoid mutating selection sets
Closing the editor now also clears expanded servers, so reopening it cannot
refetch them for one render. Build the selection and expanded-server sets
without in-place mutation, drop a comment that restated the code, and use
screen queries in the new tests to stay within the prefer-screen-queries
lint budget. (LIT-10012)
* feat(daily-activity): read guardrail spend per key from the daily guardrail usage table
Adds DailyActivityRepository.guardrail_spend, which sums LiteLLM_DailyGuardrailUsageUnits.cost by date, guardrail and key. A guardrail row counts only when its key has a spend row on that day under the same scope filters as the rest of the daily activity response, so user, key and date filters apply to guardrail cost too
* feat(daily-activity): add a per-guardrail spend breakdown to /user/daily/activity/aggregated
Each day now has breakdown.guardrails[<guardrail_id>] with metrics.spend and an api_key_breakdown carrying each key's spend and metadata. Guardrail entries only report spend, since that is the one metric stored per key. The breakdown is on the user endpoint only, is empty when a model filter is set because guardrail cost is not stored per model, and lists only keys within api_key_limit while the guardrail total keeps every key. spend and budgets are unchanged
Resolves LIT-9785
* feat(daily-activity): add the guardrail breakdown to team and organization activity, with guardrail names
/team/daily/activity/aggregated and /organization/daily/activity/aggregated now return breakdown.guardrails the same way the user endpoint does, since each key belongs to one team and one organization. Each guardrail entry also carries metadata.guardrail_name, read from LiteLLM_GuardrailsTable for guardrails created through the UI or API and falling back to the breakdown key for config.yaml guardrails, whose key is already their name
* fix(daily-activity): keep daily activity up when the guardrail breakdown query fails, and treat an empty model filter as none
A database error from the guardrail spend query now logs a warning and returns the response without guardrails, where it used to fail the whole aggregated response with a 500. An empty model= no longer drops the guardrail breakdown, matching how the rest of the endpoint already ignores it. The daily activity golden files and fixtures gain the empty guardrails key every breakdown now carries
* refactor(daily-activity): fold guardrail breakdown in a worker thread and drop docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(daily-activity): report untracked guardrail units next to spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): cover owner-scoped file listing under require_managed_files
* fix(files): list only the caller's managed files on every list route under require_managed_files
* fix(files): narrow owner-scoped managed file lists to the named provider or model and page with after_id and before_id
* fix(files): narrow scoped lists the way uploads route, and refuse lists that cannot be scoped
A model name now matches every deployment that serves it, wildcards and
aliases included, and a provider matches deployments by the provider
uploads resolve, so anthropic/* and unprefixed Claude models count. The
OpenAI and Azure lists also return the caller's pass-through uploads.
An unknown provider and different after and after_id cursors answer 400,
and a list with no matching deployment skips the database read.
* fix(files): match stored files by model name and keep flag-off lists as on main
Under require_managed_files, a narrowed list now matches a file's model
mappings by deployment id or model name, so files LiteLLM stores in its own
database show on their model and provider lists. The Anthropic list defaults
to 20 rows when no limit is sent, after_id, before_id and the cursor 400s
apply only under the flag, and an admin keeps the provider list when the
managed files hook is not loaded. List errors are returned as values and
mapped to HTTP errors in one place.
* fix(files): keep admin provider lists and index the owner-scoped walk
Under require_managed_files, an admin keeps the provider list as on main, and
only a non-admin list reads the caller's own and team managed files. A stored
file now also shows on the provider its model name routes to through a
wildcard, with exact model names first, and a list routed to a model takes the
default page size of the provider serving it. A new index on created_by and
created_at keeps each chunk of the walk an index read, and the model check
runs before the file object is parsed.
* fix(files): filter provider and model lists in the query
A non-admin list scoped to a provider or model now asks Postgres only for the
caller's rows that map to one of that scope's deployments or models, or that
live in file storage, instead of reading every owned row and filtering them in
Python. A caller with many files on other providers no longer waits for the
whole set to be read. Team members' files are matched in the same query.
* test(files): type the fake raw query in the files list tests
* refactor(proxy): list managed files through one typed database handle
* fix(files): list alias and routing-group storage uploads on their provider
AriaAvatar pointed at /ui/assets/aria-agent.png, a path that only exists when
the proxy serves the static export under /ui. The dashboard dev server and the
split-chart nginx image 404 it, so the header Aria button and every other
AriaAvatar showed a broken image. Import the PNG statically like the provider
logos (#34125) so the bundle always serves it under /_next/static/media
Replaces the hand-rolled kbd and code key chips across the dashboard with
the shared Kbd component from components/ui/kbd, so every shortcut hint
renders the same way
* feat(providers): add ZeroGPU as an OpenAI-compatible provider
Registers zerogpu through the JSON provider registry so zerogpu/<model> routes to
https://api.zerogpu.ai/v1/chat/completions with ZEROGPU_API_KEY and an optional ZEROGPU_API_BASE.
Adds LlmProviders.ZEROGPU, the constants.py provider and endpoint lists, cost map entries for the
10 ZeroGPU text generation models, the endpoint matrix, the Add Model form fields, the README row
and the dashboard provider entry
* feat(zerogpu): add capability flags verified against the live API
Live calls to https://api.zerogpu.ai/v1/chat/completions confirmed tools, tool_choice and json_schema
response_format on 8 of the 10 catalog text models, cached token reporting on deepseek-v4.1-flash,
gpt-4.1-mini, gpt-5.4-nano and gpt-5.6-luna, and reasoning_effort on gpt-5.6-luna. LFM2.5-1.2B-Instruct
answers stream requests with plain JSON instead of SSE and LFM2.5-1.2B-Thinking returns 500 on every
request, so both stay out of the cost map until the API serves them correctly
* feat(zerogpu): price LFM2.5-1.2B-Instruct
Adds zerogpu/LFM2.5-1.2B-Instruct to the cost map with the catalog prices and context window. It
gets no capability flags because the live API ignores tools, response_format and
max_completion_tokens for this model, so LiteLLM drops tool params for it instead of sending them.
LFM2.5-1.2B-Thinking stays out of the cost map
* fix(zerogpu): reject transcription requests before calling the upstream API
ZeroGPU only serves chat completions, so drop it from the OpenAI transcription transport set. A zerogpu transcription request now fails locally instead of hitting an /audio/transcriptions endpoint that does not exist
* test(zerogpu): cover the responses and messages bridges
Both endpoints reach ZeroGPU through chat completions, so assert the request each one sends upstream and the reply it returns
* refactor(zerogpu): move zerogpu out of the provider list tails
New upstream providers append to the end of openai_compatible_endpoints, openai_compatible_providers and providers.json, so an entry at the tail conflicts with every one of them. Placing zerogpu next to long-stable entries keeps those merges clean. No behavior change
* fix(zerogpu): price cached input for qwen3-30b-a3b-fp8
The ZeroGPU model catalog (https://docs.zerogpu.ai/docs/model-catalog, checked 2026-10-09) lists cached input for qwen3-30b-a3b-fp8 at $0.05 per 1M tokens. Without cache_read_input_token_cost, cached tokens were billed at the full $0.10 input rate
* revert(zerogpu): leave the audio transcription provider set unchanged
Per review, keep OPENAI_AUDIO_TRANSCRIPTION_PROVIDERS as upstream defines it. This PR only covers chat completions, and ZeroGPU now serves /audio/transcriptions anyway, so the local refusal and its test are removed
* fix(zerogpu): add max_tokens so trim_messages can size ZeroGPU history
trim_messages reads max_tokens as the fallback for max_input_tokens and evaluates it eagerly, so without it the lookup raised, got swallowed, and the full history came back untrimmed. max_tokens matches each model's maxTokens in ZeroGPU's models API (https://api-dashboard.zerogpu.ai/api/models, checked 2026-10-10)
* test(zerogpu): record mocked requests on a recorder object
The mock handler appended to a list the caller passed in, which AGENTS.md
forbids. A recorder now owns the captured requests as a tuple
* fix(zerogpu): cap max_tokens at each model's output limit
Each cap is the max_output_tokens ZeroGPU stores for the model in its production models table (read 2026-10-20), the same column its /v1/models serves as max_length. That gives 65536 for deepseek-v4.1-flash, 32768 for gpt-4.1-mini and gpt-oss-120b, 128000 for gpt-5.4-nano and gpt-5.6-luna, 8192 for LFM2.5-1.2B-Instruct and llama-3.1-8b-instruct-fast, and 16384 for qwen3-30b-a3b-fp8. glm-5.3-flash is stored as 32786, which reads as a typo for 32768, so that row uses 32768
trim_messages sizes history from max_input_tokens, so lowering max_tokens leaves it unaffected
* fix(zerogpu): flag prompt caching on glm-5.3-flash
Repeating the same ~4.3K-token prompt against ZeroGPU on 2026-10-20 returned cached_tokens of 4320 on some glm-5.3-flash calls. gpt-oss-120b and llama-3.1-8b-instruct-fast returned 0 on all nine calls, and qwen3-30b-a3b-fp8 never returned cached_tokens, so they stay unflagged. Their cached prices stay as ZeroGPU publishes them, so any cache hit is still billed at the right rate
* fix(zerogpu): set gpt-5.4-nano context window to 272000
ZeroGPU accepts a 271001-token gpt-5.4-nano prompt and rejects 273000 tokens as over the context window (checked 2026-10-21), so max_input_tokens is 272000 rather than the 400000 its catalog lists
* fix(zerogpu): list zerogpu models for zerogpu/* wildcards
zerogpu had no models_by_provider entry, so get_provider_models("zerogpu") returned None and a zerogpu/* wildcard listed no models. zerogpu_models is now filled from the cost map rows like openinfer_models and registered in models_by_provider
* fix(zerogpu): flag gpt-5.4-nano as a reasoning model
ZeroGPU honors reasoning_effort on gpt-5.4-nano (checked 2026-10-21). Without it the model spends 0 reasoning tokens, and with low or high it spends 32 or more. LiteLLM only treated it as a reasoning model by falling back to OpenAI's bare gpt-5.4-nano row, so get_model_info and /model/info reported supports_reasoning as null. The row now carries the flag itself
* fix(zerogpu): flag deepseek, glm, gpt-oss and qwen3 as reasoning models
ZeroGPU's /v1/models marks these four as reasoning models and accepts reasoning_effort on all of them (checked 2026-10-21), but LiteLLM rejected the param with UnsupportedParamsError because their rows had no supports_reasoning flag. glm reports reasoning_tokens under reasoning_effort, and gpt-oss and qwen3 spend hidden reasoning tokens on every reply. The gpt-oss test case now expects reasoning_effort to reach ZeroGPU
* fix(zerogpu): set gpt-oss-120b context window to 128000
ZeroGPU routes gpt-oss-120b across several upstream backends, and at least one of them returns 413 past a 128000-token window, so max_input_tokens uses that as the safe floor. The backends reached on 2026-10-21 accepted up to about 131K prompt tokens, so this only gives up a little headroom on the larger ones
* feat(zerogpu): call ZeroGPU's Responses API natively
ZeroGPU serves /v1/responses for all nine catalog models, so LiteLLM no longer bridges Responses requests through chat completions. Native calls return ZeroGPU's reasoning output items for glm, gpt-oss and qwen3, which the bridge dropped. Checked live on 2026-10-22 through litellm.responses for every model, non-streaming and streaming, plus a gpt-oss tool call. LFM2.5-1.2B-Instruct streams correctly now that ZeroGPU's text_generation lane emits Responses SSE
* fix(zerogpu): stop inferring the zerogpu provider from api_base alone
Listing https://api.zerogpu.ai/v1 in openai_compatible_endpoints rerouted bare model names on that api_base from openai to zerogpu. A bare gpt-5.6-luna call with verbosity used to return 200 and started raising UnsupportedParamsError. Drop the entry and its api_base-alone test, so zerogpu is only selected through the zerogpu/ prefix
* fix(zerogpu): price gpt-6-luna, llama-3.2-3b, LFM2.5-1.2B-Thinking and qwen3.6
A zerogpu/* wildcard served these four chat models with no cost-map row, so their calls logged $0 spend. Add rows at ZeroGPU's catalog prices from https://api-dashboard.zerogpu.ai/api/models, with context windows and output caps from ZeroGPU's models table. LFM2.5-1.2B-Thinking has no cached price, like LFM2.5-1.2B-Instruct. gpt-6-luna is flagged supports_reasoning because LiteLLM already routes gpt-6 names as reasoning models. Checked live on 2026-10-28, all four now report nonzero response_cost
* fix(zerogpu): drop the retired max_tokens field from the zerogpu rows
Upstream #48471 retired max_tokens from the cost map in favor of max_input_tokens and max_output_tokens, and test_no_row_carries_the_retired_max_tokens_field failed on all 13 zerogpu rows. Every row already carries the same value in max_output_tokens, so nothing is lost. The zerogpu limits test now checks that each row's output limit fits its context window
* chore(zerogpu): regenerate provider endpoint support for zerogpu
Upstream #48619 generates provider_endpoints_support.json from the cost map and checks it in CI, and the hand-written zerogpu entry no longer matched. Regenerated with scripts/generate_provider_endpoints_support.py. Only the zerogpu entry changed, and it now matches what the generator emits for other JSON-registry providers such as openinfer
* fix(zerogpu): flag tools, json_schema and prompt caching on gpt-6-luna
ZeroGPU serves function calling, tool_choice, json_schema response_format and prompt caching on gpt-6-luna, so set supports_function_calling, supports_tool_choice, supports_response_schema and supports_prompt_caching on its row in both cost maps. Checked live on 2026-10-28, a weather tool call came back with the right arguments, json_schema output validated, and repeated prompts reported 3260 of 3263 tokens cached
* fix(zerogpu): set qwen3.6-35b-a3b context to 262144 and flag its tools and reasoning
ZeroGPU accepts up to 262,144 input tokens on qwen3.6-35b-a3b and serves tools, tool_choice, json_schema response_format and reasoning_effort on it. Raise max_input_tokens from 16384 and set supports_function_calling, supports_tool_choice, supports_response_schema and supports_reasoning in both cost maps. Before this, LiteLLM rejected tools and reasoning_effort for the model. Checked live on 2026-10-28, a 209,905-token prompt succeeded, an oversized one was refused with a 262144-token limit, and tool calls, json_schema output and reasoning_effort all worked
* fix(zerogpu): flag vision on gpt-6-luna and qwen3.6-35b-a3b
ZeroGPU reads image inputs on gpt-6-luna and qwen3.6-35b-a3b, so set supports_vision on both rows in both cost maps. Checked live on 2026-10-28, both models named the animal in an image_url photo, and the image showed up in their prompt token counts
* fix(gemini): send Google's current Managed Agents fields on create and forward agent_config on interactions
* fix(gemini): accept id on managed agent get, delete, and list_versions
* fix(gemini): drop unsupported agent fields under drop_params and keep agent_config keyword-only
A model interaction drops agent_config under drop_params, agent create drops wrongly typed optional fields under drop_params and otherwise answers 400 naming the field the caller sent, and agent_config on interactions create is keyword-only so positional response settings stay in place
* fix(gemini): drop a non-object agent_config under drop_params and name the id alias the caller sent
* fix(interactions): reject agent_config on bridged providers and drop a typeless one under drop_params
Bridged providers answer 400 for agent_config without drop_params and drop it with drop_params. A Gemini agent run drops an agent_config without a string type under drop_params. The Gemini agents wire test expects Google's current upstream field names.
* fix(interactions): honor additional_drop_params for agent_config on bridged providers
* fix(agents): drop a clashing instructions alias under drop_params
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): log one line per token count call
Each count call on /v1/messages/count_tokens, /utils/token_counter, /v1/responses/input_tokens and the Gemini countTokens routes now logs one INFO line with the litellm call id, the requested model, the model the count ran against, the count, the count source and the status code, so it joins the inference call that sends the same x-litellm-call-id
* test(google_endpoints): give the count test's fake request a headers list
* fix(proxy): keep token count log lines one per call and trusted in JSON logs, and log Bedrock native counts
Quote log values that carry whitespace, quotes, equals, or control characters so a model name can't split or forge a line. Let JsonFormatter record extras win over a dict parsed from the message. Log the Bedrock native count-tokens route through the same line.
* fix(proxy): escape braces in token count log values, log the Bedrock model AWS counts, and skip the line when there is nothing to count
* fix(proxy): log the deployment id the count routed to on every token count line
* fix(proxy): log count requests rejected before counting with the status they answer
A count route that rejects a request before the shared count function runs now logs the same token count line with its 400. An unsupported content block on the Messages count route becomes its 400 inside the logged wrapper, so its line says 400 instead of 500. A count that ends in a BaseException other than cancellation logs a 500.
* fix(proxy): compute the Bedrock counted model id only after a successful count
* revert(logging): keep the JSON log formatter's field order as on main
The count line already escapes braces in its values, so the formatter never parses a dict from it. Changing the order for every logger is a separate decision.
* test(ui): cover Responses API reasoning events in the playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show reasoning for current Responses API reasoning events
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): prefill the key edit Organization picker from org_id
The Virtual Keys table hands KeyEditView rows from /key/list, which
serializes the organization as org_id, while /key/info calls the same
field organization_id. The edit form read only organization_id, so on a
key opened from the list the Organization picker fell back to its "All
Organizations" placeholder even though the key belongs to an org.
Read both field names through one keyOrganizationId helper, used by the
form value mapper, the edit view's selected organization state, and the
two places in key_info_view that already hand-rolled the same fallback.
* fix(ui): keep the key edit team filter in sync with the organization field
* fix(ui): drop the leftover organization setter calls in the key edit team handler
The previous commit derived the organization from the form and removed the
useState behind it, but handleTeamChange still called the deleted setter, so
the dashboard build failed to type check and picking a team with an
organization threw at runtime.
The form.setValue calls next to them already carry the organization, so the
setter calls only had to go. Adds a test that picks a team with an
organization and asserts the form adopts and submits it.
* fix(ui): drop the list row's stale org_id when a key save returns its organization
* fix(ui): send the key's organization only when the edit form changes it
An untouched save from the key edit form now leaves organization_id out
of the /key/update body, the same way the form already drops unchanged
gated fields. This stops a save from writing a team's fallback
organization onto a key that has none, and keeps the audit log free of
no-op organization updates. The organization tests now use typed inputs
* fix(ui): open team page keys with their own organization in the edit form
* fix(ui): keep every team in the key form's team list for users who cannot change the organization
---------
Co-authored-by: mateo-berri <edgetonmas@gmail.com>
* fix(ui): omit blank expiry from key edit and regenerate payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: credit #45048 author
Co-authored-by: Thomas Scholtz <thomas@quantum-sicarius.za.net>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format regenerate key payload builder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): compare regenerate payloads by fields, not JSON order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Thomas Scholtz <thomas@quantum-sicarius.za.net>
XecGuard AI and Airia AI join the third-party names the SI swap leaves alone, so
their Guardrail Garden cards stop reading "XecGuard SI" and "Airia SI"
The copyable agent instructions go to Claude Code and Codex, so they keep AI the
same way the compliance prompts do
* feat(ui): add LITELLM_UI_USE_SI_TERMINOLOGY to show SI instead of AI in the Admin UI
Setting LITELLM_UI_USE_SI_TERMINOLOGY=true on the proxy makes the Admin UI
say "SI" where its own copy says "AI" and "Super Intelligence" where it says
"Artificial Intelligence". This covers the nav, page headings, settings text,
guardrail garden blurbs, the compliance playground prompts and the playground
sample prompts. Third-party product, provider and law names keep their
spelling (Vertex AI, Cisco AI Defense, EU AI Act, ...), and user data such as
guardrail names, gateway URLs and keys is never rewritten.
The proxy exposes the flag on /.well-known/litellm-ui-config, a
TerminologyProvider at the app root reads it, and components call
useTerminology() on their own strings. The context defaults to a pass-through,
so nothing changes when the flag is unset.
* fix(ui): apply SI terminology to the command palette, page visibility groups and guardrail garden search
* test(ui): cover the flag-off wording on the palette, page visibility and garden search tests
* fix(ui): apply SI terminology to the Aria usage assistant copy
* test(ui): keep the Aria ui-config stub within the inline object lint budget
* fix(ui): keep AI wording in prompts sent to models
* test: remove e2e checks and a CI line count that pin implementation details
Deletes the Auto Router template select anchoring spec, the 390px logs pagination variant, and the call id tooltip and clipboard step in the logs spec. Each one tests library or test helper behavior rather than what users see, and each has been red or flaky on main without a product regression
Also drops the exact line count on the fan-in requirements export in the integration coverage job. It went from 6 to 10 when uv.lock gained a Python 3.15 fork, while the install was still correct. A package the filter drops still fails the job, either at the hash-checked install or at the next step's import
* docs(agents): do not test what shadcn and Base UI already guarantee
Adds the boundary to the dashboard AGENTS.md next to the test tiers, and points the Playwright suite's entry in tests/e2e/AGENTS.md at it
* test: cover request ids with URL special characters in log detail fetches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): percent-encode request id in spend log detail URL
Co-authored-by: Charlie Freeman <243496084+cf3901646@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Charlie Freeman <243496084+cf3901646@users.noreply.github.com>
* test(ui): cover the authorization MCP auth type in create and edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): expose the authorization MCP auth type in create and edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): drop the experimental label from the MCP server and client paths
Move litellm/proxy/_experimental/mcp_server to litellm/proxy/mcp_server and litellm/experimental_mcp_client to litellm/mcp_client, with the mirrored tests moved under tests/unit/proxy/mcp_server and tests/unit/mcp_client. Both old packages stay as shims that warn with DeprecationWarning, re-export the new package's public names and alias every submodule in sys.modules so deep imports keep resolving to the same objects. The OpenAPI docs gate now puts the merge base's enterprise package on PYTHONPATH so its base counts do not mix the checked-out enterprise code with the base litellm tree
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(mcp): keep tests/unit/proxy/_experimental as a sharded root for the shim tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): alias namespace subpackages and bind attributes in the deprecated MCP shims
pkgutil.walk_packages skipped mcp_server/auth because it has no __init__.py, so old imports of litellm.proxy._experimental.mcp_server.auth.* failed, and sys.modules entries alone left litellm.experimental_mcp_client.tools unreachable as an attribute. Walk the package on disk and setattr each alias on its old parent.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show the budget period date range instead of "cycle" on team member spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): derive the budget period start in UTC before showing it in local time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep clock times on sub-day budget periods and restore the total spend lower bound wording
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): budget period tooltip on key, team, user and org spend tiles
Adds a shared BudgetPeriodTooltip to the four detail-view spend tiles showing the current budget period, max budget, reset interval and spend so far, and makes organization create schedule budget_reset_at and organization info return it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): give the budget period tooltip trigger its own aria label
The default SimpleTooltip trigger shares the question-circle label with InheritedBudgetHint, which broke the key budget display tests that count that icon
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): match the proxy's budget window rules in the period tooltip
30d and Nmo now span calendar months like get_budget_window_start, the arithmetic runs in UTC so a viewer's DST change cannot shift the start, the hourly/daily/weekly/monthly aliases the proxy accepts are normalized before parsing, and spend without a duration reads as lifetime spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): simplify auto router setup with deployed model defaults
* fix(ui): recognize catalog Opus deployment formats
* fix(ui): address auto router setup review gaps
* fix(ui): preserve Gemini auto router suggestions
* fix(ui): prefer current aliases and wait for team selection
* fix(ui): default auto routers to classify each user turn
* fix(ui): parse Grok snapshot dates separately from releases
* refactor(ui): use non-mutating auto router sorting
* feat(ui): allow custom models in simple auto router setup
* fix(ui): keep Aria accessible in the dashboard header
* test(ui): wait for every pagination fixture log
* test(ui): update assistant setup for Aria branding
* build(docker): move to newer wolfi-base and nginx base images
Bump the pinned wolfi-base digest in every Wolfi Dockerfile to 360896ef, which
ships brotli 1.2.1-r1 and ca-certificates-bundle 20260909-r3. The previous
digest already carried libbrotli 1.2.0-r6 as an OpenSSL dependency, and
`apk add --no-cache` never upgrades a package the base already has
Move the UI image to nginx 1.31.8-alpine3.24, whose CA bundle is the 20261020
Mozilla set, and upgrade brotli-libs in its runtime stage since no nginx
digest ships 1.2.1 yet
* build(docker): copy only runtime files into the gateway, backend and migrations images
The component runtime stages copied the builder's whole /app, which is the
entire repository plus build output: six lockfiles, helm, terraform, docs,
.env.example and the Rust build tree. Measured at runtime with an audit hook
on a booted container, the gateway reads only gateway/, litellm/ and
enterprise/ from /app, the backend only backend/, litellm/ and enterprise/,
and the migrations job only litellm/ and run.py. The pyproject.toml read is
prisma-python's optional [tool.prisma] lookup, which ours does not define
Copy just those, plus .venv, the entrypoint script and schema.prisma for the
gateway and backend. PYTHONPATH=/app and the in-Dockerfile asserts are
unchanged
* fix(proxy): reconcile team member total_spend with current cycle spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): read /team/list as a list and drain the scripted provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(repositories): compose the membership reset writer and drop SQL text asserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: type the cascade repair test and flatten memberships without a stacked comprehension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: keep the reset job test file's existing typing untouched
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cost): parse 12-hour cache write rates for batch and ultrafast tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): bill 12-hour batch cache writes once when only the 12hr batch key exists
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show "Never resets" instead of "n/a" for an unset budget reset period
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): label unset reset period as Keep current/No change where blank keeps the stored value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show a single Never resets option in create team when no default is set
Without a default team budget_duration, the create team Reset Budget dropdown
offered both "Not set" and "Never resets", and both created a team that never
resets. The explicit Never resets item is now only shown when there is a
default to opt out of, otherwise the empty option itself reads "Never resets"
Also makes the Reset Budget create tests use chooseSelectOption so they no
longer flake on the select popup's pointer-events transition
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(teams): show caller member spend and budget on Your Teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): use member-linked reset dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep type-discipline suppressions on cast lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): preserve empty page list response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep the member reset date inside the spend column
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): match email-only roster members for caller membership
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show team member budget reset date in UTC
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(teams): address review on caller membership enrichment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the caller's team spend line My Spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add OpenCode Go model discovery, protocol routing, session affinity, and published pricing metadata. Cover Chat Completions, Responses, Messages, and Gemini through the existing provider adapters.
Fixes#31568
