* feat(proxy): log one line per token count call
Each count call on /v1/messages/count_tokens, /utils/token_counter, /v1/responses/input_tokens and the Gemini countTokens routes now logs one INFO line with the litellm call id, the requested model, the model the count ran against, the count, the count source and the status code, so it joins the inference call that sends the same x-litellm-call-id
* test(google_endpoints): give the count test's fake request a headers list
* fix(proxy): keep token count log lines one per call and trusted in JSON logs, and log Bedrock native counts
Quote log values that carry whitespace, quotes, equals, or control characters so a model name can't split or forge a line. Let JsonFormatter record extras win over a dict parsed from the message. Log the Bedrock native count-tokens route through the same line.
* fix(proxy): escape braces in token count log values, log the Bedrock model AWS counts, and skip the line when there is nothing to count
* fix(proxy): log the deployment id the count routed to on every token count line
* fix(proxy): log count requests rejected before counting with the status they answer
A count route that rejects a request before the shared count function runs now logs the same token count line with its 400. An unsupported content block on the Messages count route becomes its 400 inside the logged wrapper, so its line says 400 instead of 500. A count that ends in a BaseException other than cancellation logs a 500.
* fix(proxy): compute the Bedrock counted model id only after a successful count
* revert(logging): keep the JSON log formatter's field order as on main
The count line already escapes braces in its values, so the formatter never parses a dict from it. Changing the order for every logger is a separate decision.
* test(ui): cover Responses API reasoning events in the playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show reasoning for current Responses API reasoning events
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): prefill the key edit Organization picker from org_id
The Virtual Keys table hands KeyEditView rows from /key/list, which
serializes the organization as org_id, while /key/info calls the same
field organization_id. The edit form read only organization_id, so on a
key opened from the list the Organization picker fell back to its "All
Organizations" placeholder even though the key belongs to an org.
Read both field names through one keyOrganizationId helper, used by the
form value mapper, the edit view's selected organization state, and the
two places in key_info_view that already hand-rolled the same fallback.
* fix(ui): keep the key edit team filter in sync with the organization field
* fix(ui): drop the leftover organization setter calls in the key edit team handler
The previous commit derived the organization from the form and removed the
useState behind it, but handleTeamChange still called the deleted setter, so
the dashboard build failed to type check and picking a team with an
organization threw at runtime.
The form.setValue calls next to them already carry the organization, so the
setter calls only had to go. Adds a test that picks a team with an
organization and asserts the form adopts and submits it.
* fix(ui): drop the list row's stale org_id when a key save returns its organization
* fix(ui): send the key's organization only when the edit form changes it
An untouched save from the key edit form now leaves organization_id out
of the /key/update body, the same way the form already drops unchanged
gated fields. This stops a save from writing a team's fallback
organization onto a key that has none, and keeps the audit log free of
no-op organization updates. The organization tests now use typed inputs
* fix(ui): open team page keys with their own organization in the edit form
* fix(ui): keep every team in the key form's team list for users who cannot change the organization
---------
Co-authored-by: mateo-berri <edgetonmas@gmail.com>
* fix(ui): omit blank expiry from key edit and regenerate payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: credit #45048 author
Co-authored-by: Thomas Scholtz <thomas@quantum-sicarius.za.net>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format regenerate key payload builder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): compare regenerate payloads by fields, not JSON order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Thomas Scholtz <thomas@quantum-sicarius.za.net>
XecGuard AI and Airia AI join the third-party names the SI swap leaves alone, so
their Guardrail Garden cards stop reading "XecGuard SI" and "Airia SI"
The copyable agent instructions go to Claude Code and Codex, so they keep AI the
same way the compliance prompts do
* feat(ui): add LITELLM_UI_USE_SI_TERMINOLOGY to show SI instead of AI in the Admin UI
Setting LITELLM_UI_USE_SI_TERMINOLOGY=true on the proxy makes the Admin UI
say "SI" where its own copy says "AI" and "Super Intelligence" where it says
"Artificial Intelligence". This covers the nav, page headings, settings text,
guardrail garden blurbs, the compliance playground prompts and the playground
sample prompts. Third-party product, provider and law names keep their
spelling (Vertex AI, Cisco AI Defense, EU AI Act, ...), and user data such as
guardrail names, gateway URLs and keys is never rewritten.
The proxy exposes the flag on /.well-known/litellm-ui-config, a
TerminologyProvider at the app root reads it, and components call
useTerminology() on their own strings. The context defaults to a pass-through,
so nothing changes when the flag is unset.
* fix(ui): apply SI terminology to the command palette, page visibility groups and guardrail garden search
* test(ui): cover the flag-off wording on the palette, page visibility and garden search tests
* fix(ui): apply SI terminology to the Aria usage assistant copy
* test(ui): keep the Aria ui-config stub within the inline object lint budget
* fix(ui): keep AI wording in prompts sent to models
* test: remove e2e checks and a CI line count that pin implementation details
Deletes the Auto Router template select anchoring spec, the 390px logs pagination variant, and the call id tooltip and clipboard step in the logs spec. Each one tests library or test helper behavior rather than what users see, and each has been red or flaky on main without a product regression
Also drops the exact line count on the fan-in requirements export in the integration coverage job. It went from 6 to 10 when uv.lock gained a Python 3.15 fork, while the install was still correct. A package the filter drops still fails the job, either at the hash-checked install or at the next step's import
* docs(agents): do not test what shadcn and Base UI already guarantee
Adds the boundary to the dashboard AGENTS.md next to the test tiers, and points the Playwright suite's entry in tests/e2e/AGENTS.md at it
* test: cover request ids with URL special characters in log detail fetches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): percent-encode request id in spend log detail URL
Co-authored-by: Charlie Freeman <243496084+cf3901646@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Charlie Freeman <243496084+cf3901646@users.noreply.github.com>
* test(ui): cover the authorization MCP auth type in create and edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): expose the authorization MCP auth type in create and edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): drop the experimental label from the MCP server and client paths
Move litellm/proxy/_experimental/mcp_server to litellm/proxy/mcp_server and litellm/experimental_mcp_client to litellm/mcp_client, with the mirrored tests moved under tests/unit/proxy/mcp_server and tests/unit/mcp_client. Both old packages stay as shims that warn with DeprecationWarning, re-export the new package's public names and alias every submodule in sys.modules so deep imports keep resolving to the same objects. The OpenAPI docs gate now puts the merge base's enterprise package on PYTHONPATH so its base counts do not mix the checked-out enterprise code with the base litellm tree
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(mcp): keep tests/unit/proxy/_experimental as a sharded root for the shim tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): alias namespace subpackages and bind attributes in the deprecated MCP shims
pkgutil.walk_packages skipped mcp_server/auth because it has no __init__.py, so old imports of litellm.proxy._experimental.mcp_server.auth.* failed, and sys.modules entries alone left litellm.experimental_mcp_client.tools unreachable as an attribute. Walk the package on disk and setattr each alias on its old parent.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show the budget period date range instead of "cycle" on team member spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): derive the budget period start in UTC before showing it in local time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep clock times on sub-day budget periods and restore the total spend lower bound wording
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): budget period tooltip on key, team, user and org spend tiles
Adds a shared BudgetPeriodTooltip to the four detail-view spend tiles showing the current budget period, max budget, reset interval and spend so far, and makes organization create schedule budget_reset_at and organization info return it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): give the budget period tooltip trigger its own aria label
The default SimpleTooltip trigger shares the question-circle label with InheritedBudgetHint, which broke the key budget display tests that count that icon
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): match the proxy's budget window rules in the period tooltip
30d and Nmo now span calendar months like get_budget_window_start, the arithmetic runs in UTC so a viewer's DST change cannot shift the start, the hourly/daily/weekly/monthly aliases the proxy accepts are normalized before parsing, and spend without a duration reads as lifetime spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): simplify auto router setup with deployed model defaults
* fix(ui): recognize catalog Opus deployment formats
* fix(ui): address auto router setup review gaps
* fix(ui): preserve Gemini auto router suggestions
* fix(ui): prefer current aliases and wait for team selection
* fix(ui): default auto routers to classify each user turn
* fix(ui): parse Grok snapshot dates separately from releases
* refactor(ui): use non-mutating auto router sorting
* feat(ui): allow custom models in simple auto router setup
* fix(ui): keep Aria accessible in the dashboard header
* test(ui): wait for every pagination fixture log
* test(ui): update assistant setup for Aria branding
* build(docker): move to newer wolfi-base and nginx base images
Bump the pinned wolfi-base digest in every Wolfi Dockerfile to 360896ef, which
ships brotli 1.2.1-r1 and ca-certificates-bundle 20260909-r3. The previous
digest already carried libbrotli 1.2.0-r6 as an OpenSSL dependency, and
`apk add --no-cache` never upgrades a package the base already has
Move the UI image to nginx 1.31.8-alpine3.24, whose CA bundle is the 20261020
Mozilla set, and upgrade brotli-libs in its runtime stage since no nginx
digest ships 1.2.1 yet
* build(docker): copy only runtime files into the gateway, backend and migrations images
The component runtime stages copied the builder's whole /app, which is the
entire repository plus build output: six lockfiles, helm, terraform, docs,
.env.example and the Rust build tree. Measured at runtime with an audit hook
on a booted container, the gateway reads only gateway/, litellm/ and
enterprise/ from /app, the backend only backend/, litellm/ and enterprise/,
and the migrations job only litellm/ and run.py. The pyproject.toml read is
prisma-python's optional [tool.prisma] lookup, which ours does not define
Copy just those, plus .venv, the entrypoint script and schema.prisma for the
gateway and backend. PYTHONPATH=/app and the in-Dockerfile asserts are
unchanged
* fix(proxy): reconcile team member total_spend with current cycle spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): read /team/list as a list and drain the scripted provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(repositories): compose the membership reset writer and drop SQL text asserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: type the cascade repair test and flatten memberships without a stacked comprehension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: keep the reset job test file's existing typing untouched
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cost): parse 12-hour cache write rates for batch and ultrafast tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): bill 12-hour batch cache writes once when only the 12hr batch key exists
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show "Never resets" instead of "n/a" for an unset budget reset period
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): label unset reset period as Keep current/No change where blank keeps the stored value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show a single Never resets option in create team when no default is set
Without a default team budget_duration, the create team Reset Budget dropdown
offered both "Not set" and "Never resets", and both created a team that never
resets. The explicit Never resets item is now only shown when there is a
default to opt out of, otherwise the empty option itself reads "Never resets"
Also makes the Reset Budget create tests use chooseSelectOption so they no
longer flake on the select popup's pointer-events transition
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(teams): show caller member spend and budget on Your Teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): use member-linked reset dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep type-discipline suppressions on cast lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): preserve empty page list response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep the member reset date inside the spend column
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): match email-only roster members for caller membership
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show team member budget reset date in UTC
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(teams): address review on caller membership enrichment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the caller's team spend line My Spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add OpenCode Go model discovery, protocol routing, session affinity, and published pricing metadata. Cover Chat Completions, Responses, Messages, and Gemini through the existing provider adapters.
Fixes#31568
* feat(ui): configure the default Usage time range
* fix(ui): retain the selected calendar preset during input sync
* fix(ui): preserve fallback dates when applying across DST
* feat: add OpenAI 12-hour cache write pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: omit empty cache-write response fields
* style: format cache-write fields
* fix(cost): bill openai 30m and 12h cache writes from the usage split, add gpt-5.6 rates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): apply the 12h long-context write rate only above the token threshold
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): key cache write rates by TTL instead of growing the rate tuple
_get_token_base_cost returned a positional 6-tuple and every cache lifetime added a slot to it, to the off-peak helper and to every caller. It now returns a TokenBaseCosts dataclass whose cache write rates live in one mapping keyed by TTL. The TTLs come from the ephemeral_*_input_tokens fields on CacheCreationTokenDetails, so supporting a new lifetime only means adding that usage field and its rate key
A lifetime with no rate of its own now always falls back to the cache creation rate after off-peak pricing is applied. Before this the non-tiered 12h rate and the tiered 1h rate fell back to the rate from before off-peak, so the same request billed differently depending on the pricing path
* refactor(cost): hardcode the cache write TTLs as a Literal instead of parsing usage field names
Parsing ephemeral_*_input_tokens names saved nothing, since a new TTL still needs a usage field and provider parsing code. The TTLs are now a CacheWriteTTL Literal with an explicit rate key per TTL
* refactor(cost): resolve cache write lifetime rates to floats at lookup
Decide the 1h and 12h cost map key in the threshold loop, then resolve each
rate once after off-peak with the base write rate as the default, so no None
flows through the cost code. Add an integration case for a 12h write on a
row without a 12h rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cost): name every cache write TTL, marking 5m and 30m as base rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): list every cache write TTL in one rate key table, None meaning the base write rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): drop base rate entries from the cache write rate table and rely on fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): key cache write rates by plain string TTL
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): fill missing cache write TTL rates with the base write rate after off-peak
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): default missing cache write TTL rates to the base write rate inline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): rename cache write rate key table to make clear it only covers long TTLs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): list every cache write TTL and bill writes of an unknown TTL at the base write rate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): resolve tiered cache write TTL rates after off-peak instead of passing None
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): carry cache write TTL rates on TokenRates and drop TokenBaseCosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(cost): define TokenRates before its first use to drop the string annotation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): build the TokenRates TTL default with default_factory and type tiers as Mapping
MappingProxyType({}) as a dataclass default is rejected on Python 3.11, where
mappingproxy is unhashable, so import litellm failed there. The tier type
moves from dict[str, object] to Mapping[str, object] (LIT001), with tier_rate
accepting a Mapping to match.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): type select_tier_for_input tiers as Mapping so tier_rate callers stay typed
The tier returned as a bare dict made every tier_rate call site a partially
unknown argument once tier_rate took Mapping[str, object]. The range bounds are
now validated with a TypeAdapter instead of indexing an unknown list.
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(chat): support PDF, text, Word, and Excel attachments
* test(chat): match the document attachment recovery banner
---------
Co-authored-by: Ishaan <ishaan@berri.ai>
* feat(proxy): add LITELLM_OFFLINE_MODE to stop every request the proxy makes on its own
* fix(ui): name LITELLM_OFFLINE_MODE where the cost map panel says local mode is forced
* test: isolate the offline-mode tokenizer tests from the local Hugging Face cache
* test: cover the offline policy templates and the cached Kimi K3 tokenizer
* test(integration): count Kimi K3 tokens in the offline-mode egress test
* refactor(token_counter): skip non-bundled tokenizers in offline mode instead of listing downloaded ones
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(chat): port Moyai UI with Model Hub and images
* feat(chat): add Models sidebar with embedded Model Hub
* feat(chat): polish Model Hub with provider cards and logos
* refactor(chat): use a provider dropdown in Model Hub
* feat(chat): keep workspace switching visible with a Chat identity
* Add minimal opt-in Chat usage metrics and prioritize Chat in workspace selector
* fix(chat): enable usage metrics by default only when chat is used
* fix(chat): count requesting users with frontend-only metrics
* feat(chat): send default-on request metrics to litellm-tel
* feat(ui): sort every teams table data column
* fix(ui): order unlimited team limits as infinity and verify org scope
* fix(ui): validate team sort filters without recursion
* feat(ui): sort every users table data column
* fix(ui): order unlimited user budgets as infinity and verify org scope
* fix(ui): cover user sort queries and document pagination bound
* fix(ui): preserve Aria usage token breakdown and viewer model access
* fix(ui): make Aria conversations easier to read and reply to
* fix(ui): wrap long headings in assistant tables
* feat(ui): simplify Aria and add usage charts with PDF answer export
* fix(ui): retain full chart values in Aria PDF exports
