A managed batch whose request lines all failed can reach a terminal provider
status (completed) with output_file_id=None and only an error_file_id. Such a
row matched neither the completed-with-output billing branch nor the
failed/expired/cancelled branch, so batch_processed stayed False and the poller
re-selected it on every cycle for the lifetime of the deployment; output/error
file deletion is also gated on batch_processed, so those files could never be
deleted.
Broaden the terminal handling so a completed/complete/expired batch with an
output file is billed, and any terminal batch with nothing to bill
(failed/cancelled, or completed/expired with no output) is marked terminal
exactly once. Non-terminal statuses (validating/in_progress) are still left for
the next poll, and an expired batch that did produce output is now billed.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A deployment with PTU flat-cost attribution also billed every request per
token, so a team paid for reserved capacity and again for the traffic that
capacity serves. Nothing set the per-token price and an unset price falls
back to the public cost map, which made the double charge the default.
/model/new and /model/{id}/update now store zero for every pricing field the
cost map could otherwise fill, refuse a price the caller supplies alongside
PTU config with a 400 naming the field, zero a price already on the row
rather than rejecting later edits of unrelated fields, and drop the zeros
again when the PTU config goes.
A PTU deployment is no longer read as a free model by the budget checks,
which would have waived every budget for it.
* fix(mcp): expose client HTTP headers to logging callbacks and hooks
MCP protocol tool calls built a synthetic Request with only content-type, so metadata.headers reaching logging callbacks and guardrails was empty while /mcp-rest/tools/call exposed the full set. Rebuild the synthetic request from the connection's raw headers (shared with the sampling path), and pass sanitized headers to the pre-call hook, the MCP to LLM guardrail bridge and the Responses API MCP bridge. Credential headers stay masked and proxy key headers stripped.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): strip custom proxy key and upstream MCP credential headers from logging copies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): make client side auth header name accessor public
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): strip custom proxy key and client redaction opt-out from mcp headers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): drop custom proxy key header in the synthetic request builder
Strips general_settings.litellm_key_header_name in build_synthetic_mcp_request so every caller, including sampling, is covered, and reverts passing general_settings into add_litellm_data_to_request on the tool call path since that also switches on enforced_params.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: shivam <shivam@berri.ai>
tests/proxy_unit_tests/ runs twice on every pull request. The nine
alphabetical shards in test-unit-proxy-legacy.yml cover the same
directory as the twelve semantic shards in test-unit-proxy-db.yml,
and all nine are required checks, so each PR pays for the directory
twice before it can merge.
The semantic shards are a strict superset. Expanding both matrices
against the working tree, the legacy globs collect 58 files while the
semantic shards name all 59: test_model_response_typing is a directory
and matches none of the test_[a-z]*.py patterns, so the legacy lane has
silently skipped it. The semantic workflow also carries its own
assert-shard-coverage guard, which fails if a file under that directory
is not assigned to a shard, so a new file cannot drop out of CI once the
alphabetical fallback is gone.
Verified with .github/scripts/assert_ci_coverage.py: 2380 test files
have a runner both before and after the deletion. Removing
test-unit-proxy-db.yml as well takes the same guard red with 58
orphaned files, which confirms the guard is live and that the semantic
shards, not the legacy ones, are what hold the coverage.
The nine bare contexts this workflow published (auth-and-jwt,
key-generation, proxy-config, proxy-server, proxy-server-extras,
proxy-token-counter, proxy-response-and-misc, proxy-user-auth-and-spend,
proxy-utils) still need pruning from the guard-internal-staging ruleset,
which needs admin rights and is not part of this change
The Admin UI's Authorize & Fetch Token flow stored its pending server in a
module-level dict, so /register, /authorize and /token only succeeded when
every leg happened to land on the process that served /session. On a proxy
with NUM_WORKERS greater than 1, or more than one replica, each click was an
independent draw and failed with a bare 404, which reads as intermittent.
Persist the pending server as a short-lived draft row instead, so any worker
resolves it. The in-memory cache is kept as the fallback for proxies with no
database configured, which keeps single-process deployments working as before.
A session runs under a caller-supplied id only when that id names a server
that really exists, which is the edit form re-authorizing a saved server.
Anything else gets a fresh id, so two concurrent sessions can never share one
draft and silently adopt each other's URL or client credentials. Drafts past
their lifetime are swept on each write so abandoned sessions do not
accumulate, and a lost create race adopts the winner rather than failing a
caller whose session is ready.
Drafts are excluded from listings and never enter the runtime registry. The
exclusion keeps rows whose approval status is NULL, which both short spellings
of the filter drop, silently hiding every server predating the approval
workflow.
Measured on a two-worker proxy against the live GitHub MCP server, 120
concurrent authorize calls per leg: staging 56/120 failures, this branch
0/120, staging again 65/120 as a positive control.
* fix(team): sweep dangling team references and cache on team delete
delete_team drove all of its cleanup off the team's members_with_roles roster, so any
user row referencing the team by another route kept a dangling team id forever and the
deleted team stayed visible on /user/info. Nothing swept LiteLLM_UserTable.teams or
LiteLLM_TeamMembership by team id, schema.prisma declares no relation between the
membership table and the team table so there is no cascade to fall back on, and the
cached team object was never invalidated on delete.
Adds a sweep that runs before the team rows are dropped: it strips the deleted ids from
every user row that still lists them and removes every membership row for those teams.
Adds _delete_cache_team_object in auth_checks and calls it per deleted team so the
team_id:{team_id} entry cannot outlive the team.
The sweep is targeted, not indiscriminate: only the deleted ids are removed and the
other teams on a user record are left intact.
* fix(team): fail member_add when the team is deleted under the row lock
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(team): correct the post-delete sweep note for the member_add lock path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): stop a deselected MCP server keeping its grant on a virtual key
The key editor sent `mcp_tool_permissions` unfiltered, and the MCP resolver
counts a server named only under `mcp_tool_permissions` as entitled, unioning
`tool_perm_servers` into `all_servers` at four sites in
`user_api_key_auth_mcp.py`. Deselecting a server, or removing the access group
that supplied it, therefore left a stale entry that kept the key reaching that
server with its old tool allowlist attached.
Reuse `extractMcpEntitlement`, which already landed for the internal-user
surface, so the key surface drops an entry only once the server is known and no
longer granted, and keeps it whenever a retained access group or toolset could
still supply it. The helper moves to a shared module so the key template does
not import a users page component.
Setting the map unconditionally is part of the same fix: the old
`Object.keys(...).length > 0` guard let the previous map ride through the
`object_permission` spread, which filtering to an empty map would otherwise hit
in exactly the case the fix is for.
* fix(ui): resolve retained MCP groups and toolsets per server when pruning tool permissions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): mock the MCP toolsets hook in the key update suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/ui/chat never rendered a metrics bar. The Responses helper already parses
usage off the response.completed event, but the chat page passed positional
undefined where onTimingData, onUsageData and onTotalLatency sit, ChatMessage
had nowhere to hold them, and ChatMessages never rendered ResponseMetrics.
Thread the three callbacks through, persist the values on the assistant
message, and reuse the playground's ResponseMetrics to show latency, TTFT,
input/output/total tokens and cost. Also map the cost the proxy reports on the
streamed usage object, which only the chat-completions helper did before.
Two behaviour differences the port introduced, both found in browser QA
The antd Select matched a prebuilt pattern on its display_name and its
internal name; a Base UI Combobox only searches itemToStringLabel, so
queries like "amex" and "sg" stopped matching. Restore the second field
with a filter predicate on the Root, covered by a regression test that
searches on a token the visible label does not contain
Base UI portals a popup into a positioner whose "isolate z-50" is fixed
in the primitive, so inside an antd Modal at z-index 1000 the options
were visible but not clickable. antd hid this because its own dropdowns
and tooltips already sat above its Modal. The positioner is not reachable
from the call site, so this needs one app-wide rule keyed on an antd
modal being present, and it becomes deletable when the last one goes
* feat(azure_ai): add Fireworks FW model pricing on Azure AI Foundry
* fix(azure_ai): drop incorrect FW-Kimi-K2.6-Code alias
* test(azure-ai): assert FW max token metadata
* feat(azure_ai): add Inkling and Nemotron 3 Ultra pricing
* fix(cli): hide codex and opencode from the lite command listings
They stay registered and invokable, so existing `lite codex` users keep
working; they just no longer show up in `lite --help` or the interactive
shell's command list.
* feat(cli): make the hidden lite command list configurable
codex and opencode are supported, so hardcoding them as hidden was wrong. Let deployments curate their own listing with `lite config set hidden_commands codex,opencode` instead; nothing is hidden by default and hidden commands stay invokable.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
os.exec* has no process-replacement semantics on Windows, so `lite claude`
printed its routing line and returned to the prompt while Claude Code was left
detached without a usable console. Windows now spawns the agent, waits for it,
and exits with the child's status. Batch shims such as the npm-installed
claude.cmd go through cmd.exe because CreateProcess cannot run them directly,
and that command line is emitted verbatim with every token quoted so a spaced
path or an argument holding a shell metacharacter cannot be re-parsed by the
command processor. POSIX keeps using os.execvpe unchanged.
/team/member_delete dropped the roster entry by matching user_email against
members_with_roles, then built its user-row lookup from that same raw email
instead of from the user_id the roster entry already carries. An email the user
row does not literally hold matched nothing, so the team id stayed in the user's
teams array and the team-membership row was left orphaned while the call still
returned 200.
/team/member_add resolves an email to a user case-insensitively but stores the
caller's casing on the roster, so inviting "Alice@Example.com" for a row holding
"alice@example.com" and removing by that same string is enough to reach it.
_cleanup_members_with_roles now returns the roster entries it removed, and both
the user-row update and the membership delete run against their user ids.
When get_team_object fails, the centralized auth gate rebuilds the team
from the token's own fields. A token whose team row was missing when the
key was read carries team_models=[] and team_blocked=False, and the
model-access check reads an empty model list as every model, so the
rebuilt team grants more than the real team ever did.
get_team_object reported a deleted team and a database that would not
answer as the same 404, so the fallback could not tell a definitive
answer from a degraded read. Raise a TeamNotFoundError subclass, still a
404 with the same detail so every other caller is unaffected, only when
the database answers and the row is absent.
A team that is provably gone now refuses, and no setting overrides that.
Otherwise the grant is merely unknown: a token carrying one may vouch,
since replaying a recorded grant cannot widen it, and a token carrying
none may not. allow_requests_on_db_unavailable still opts back out there,
and is only consulted once the failure is known to be a degraded read.
The chat metrics bar reported In/Out/Reasoning/Total/cost only, so a
playground user had no signal that provider prompt caching worked. The
cached-token counts were already visible in the Logs drawer, which meant
the answer to "does caching work here" lived on a different page.
Adds cacheReadTokens and cacheCreationTokens to TokenUsage and renders
them as two chips, reusing the prompt-cache tooltip wording already
introduced for the Logs drawer so both surfaces say the same thing.
A single helper, extractPromptCacheTokens, normalizes the three usage
shapes the playground consumes: Anthropic Messages
(cache_read_input_tokens / cache_creation_input_tokens), chat
completions (prompt_tokens_details) and the Responses API
(input_tokens_details). All three producers call it instead of parsing
per surface. Counts that are absent, zero or non-finite are dropped, so
providers without prompt caching render exactly what they render today.
CLI session tokens minted by /sso/cli/poll set team_id and team_alias but
never team_models or team_model_aliases, so the token carried a team with
none of that team's grants. /v1/models bails out to "unrestricted" when both
key_models and team_models are empty and listed the whole proxy, and team
model aliases never resolved because both can_team_access_model and the
pre-call rewrite read team_model_aliases off the token.
The team data was not close at hand: _fetch_cli_sso_team_details projected
full team rows down to team_id and team_alias before they reached the mint.
Widen that projection to include the team's models and its joined alias
table, and populate both fields at mint time.
Also stop writing the user's personal allowlist into the key models slot
when a team is bound, matching virtual-key semantics where a team-bound
credential is governed by the team grant.
Because an empty team grant is itself a real value meaning unrestricted, a
team whose grants cannot be resolved must not be minted as empty: that is
the same "unrestricted" bail-out this fix exists to close. The poll now
refuses to mint when the selected team has no complete cached detail.
That refusal is only safe because a login can no longer be pinned to a team
whose grants will never resolve. Deleting an organization drops its team
rows but leaves the memberships behind, so the login now offers only teams
whose rows still exist, and a lookup that fails outright fails the login
rather than caching a session that silently drops every team.
The repository's comment policy keeps reasoning out of the source, so the
notes on the delete dialog, the tooltip wrapper, and the test query choices
move to the pull request instead.
The assertion pinned the Tailwind track string, which jsdom can never
evaluate: it does no layout, so the test could not fail for the reason
that matters, and its pattern accepted any minmax minimum, staying green
if that minimum changed enough to break the layout outright.
Only a viewport-resize browser test can observe this, which belongs in
tests/e2e/ui rather than a route's unit tests. The responsive behaviour
itself is unchanged; the widths it reproduces are recorded in b341a22339
Replaces antd and Tremor with shadcn primitives across the six route-owned,
form-free playground components: the compare view and its panel, message input
and unified selector, plus the realtime playground and the agent builder.
Markup only, no behaviour change. The characterisation tests added in the
previous commit are untouched here and stay green through the swap.
Adds ui/slider.tsx via the shadcn CLI and retires the six antd
no-restricted-imports suppressions the migration made obsolete.
The antd Row and Col the migration replaced never squeezed the summary
cards below their content width; they wrapped onto a second line instead.
Measured on a live dashboard, antd laid out five per row at 1280 and 1024,
then four plus one at 900 and three plus two at 820, never narrower than
about 126px. A fixed grid-cols-5 kept all five on one line and compressed
them to 86px at 820, so the metric values overflowed their cards.
An auto-fit track with a 7rem minimum reproduces antd's wrap points and
card widths exactly at all four measured viewports, and is identical at
1280, so the route's visual baseline is unchanged.
The added test fails against grid-cols-5 and passes against the track
Rewrite the playground tests that reached for antd class names so they
locate controls by role, text, placeholder or lucide icon instead. Add
characterisation suites for AgentBuilderView and RealtimePlayground,
which had none, including tab state that must survive a round trip
through another tab.
Every assertion here passes against the current antd components.
