Semantic cache keys omit the prompt, so every end user behind one virtual key
shares a bucket and can be served another user's semantically similar response.
Add an opt-in cache_params.semantic_cache_scope (key | end_user) that appends the
authenticated end-user id to the tenant scope, read from metadata and
litellm_metadata so /v1/chat/completions, /v1/responses and /v1/messages are all
covered, falling back to the key scope when no end-user id is present. Expose the
setting in the cache settings API and the Admin UI cache settings form
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
DataTable loses the comment that narrated its sticky header classes. The
table scrolling e2e spec now types every management API response it
reads, seeds rows through an immutable reduce instead of pushing into
arrays, and deletes what it seeded in each test's finally block instead
of draining a shared mutable list in afterEach.
Refs LIT-4738
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
Virtual Keys, Teams, Request Logs and Tags now hand DataTable a bounded
flex chain and use fillHeight, so the app shell main stays the only page
scroller, the rows scroll under a pinned header and the pagination footer
sits at the bottom of the page. DataTable keeps the sticky header inside
its own scroller in maxBodyHeight mode too, which is what let the header
scroll away with the rows on Keys, Teams and Models. Model Hub, Vector
Stores and the team detail keys tab drop their 75vh boxes and flow with
the page scroller.
Adds an e2e spec that fails on the merge base for every one of those
pages and passes at this tip.
Refs LIT-4738
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
The search= param on /key/list, /audit, and /spend/logs/ui, plus key_hash= on /key/list, now compare the pasted value verbatim. Only a copied key ID (the hash) matches, so a raw virtual key never needs to travel in a GET query string
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
Virtual Keys and Team Virtual Keys send the search box to the new
key/list search param so a key hash matches. Agents matches agent_id
client-side. Memory sends the box as search so a memory_id matches.
Audit Logs gains a search box. Request Logs sends the box as search so
a session, team, user, key hash, or model id matches without opening
the filter drawer.
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
Always serialize object_permission.agents and agent_access_groups in
the team update payload so removing the last agent in the dashboard
sends an explicit empty array instead of omitting the key, which the
backend merge treats as no change
Resolves LIT-6861
Co-authored-by: yassin <yassin@berri.ai>
* fix: stop a cleared Organization field from failing key creation
Clearing the Organization combobox in the Create Key modal left organization_id set to an empty string, so /key/generate looked up an organization named "" and failed with "Organization doesn't exist in db. Organization=".
OrganizationDropdown now emits null on clear, and GenerateKeyRequest normalizes an empty organization_id or project_id to None the same way it already does for team_id.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop customer-specific docstring from key request normalization test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
* fix(spend_tracking): leave SpendLogs.session_id null when no client session id was established
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(lint): ratchet basedpyright budget after session_id fix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): ignore trace ids as session ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): gate null SpendLogs.session_id behind missing_session_id: omit
Unset, generate and reject keep the legacy trace id fallback. omit records only
metadata.session_id, the key Langfuse reads, so a trace id copied into
litellm_session_id by get_litellm_params never becomes a session.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): stamp the omit decision on the request so a config reload cannot fabricate a session
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): keep omit covering requests the pre-call stamp never reaches
Router-model provider pass-through calls allm_passthrough_route directly and
skips add_litellm_data_to_request, so those requests never run the pre-call
helper and carry no omit stamp. Reading only the stamp made POST
/anthropic/v1/messages write a fabricated uuid into SpendLogs.session_id under
missing_session_id: omit while its Langfuse trace had no session, the exact
divergence the policy exists to remove.
The stamp now only pins omit on, and an unstamped request falls back to the
configured policy, so a config reload still cannot fabricate a session for a
request that was decided pre-call.
* fix(spend_tracking): make the session-omission marker proxy-owned so clients cannot forge it
* fix(spend_tracking): strip the client-sent omission marker from both metadata buckets before they merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): strip the session-omission marker from both metadata buckets
The pre-call policy ran before litellm_metadata is merged into metadata, so a
client that planted the marker in litellm_metadata had it copied back into the
route's own bucket after the strip and still got a null SpendLogs.session_id.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
GET /user/list gains an optional search query param that ORs a case-insensitive contains match over user_id and user_email. The Users page search box now sends that param and reads "Search by email or ID…", the way the Teams page already searches by name or ID. Every existing /user/list param keeps its meaning and the Filters drawer is untouched
Claude-Session: https://claude.ai/code/session_018yW93iDaEMhoQUXcYjus7D
The logs table already rolled up cost per session but the Tokens column only showed the representative call's usage. The per-session aggregate query now also sums prompt, completion and total tokens, and the Tokens cell switches to those sums for multi-call sessions the same way the Cost cell does.
Claude-Session: https://claude.ai/code/session_01CNasFqyjnLN3Rqman25vde
* feat(ui): keyset-paginate request logs by session trace
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep session grouping within type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): ruff format session grouping helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): only group sessions when group_by_session is an explicit true
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): reset session cursor on custom range and live tail toggles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover cursor reset on custom range toggle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): ignore next page clicks while the grouped page is still fetching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): only block next page while grouped placeholder data is shown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
`/team/info` access_group_details now carries mcp_server_ids and agent_ids per
group next to models, so the dashboard can say which group granted a server or
agent. The Object Permissions rows drop the Inherited badge and the row tooltip
reads "Granted via access group <name>. Full ID: <id>", listing every group
when more than one grants the same id and falling back to "an access group"
when the proxy did not say.
Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
Regenerating a key repointed ?key= at the rotated hash with a pushed history
entry, so pressing the browser Back button landed on the hash that had just
been revoked. /key/info answers 404 for it and the page shows "Key not found
in database".
The rotated hash now replaces the current entry instead of pushing a new one,
so Back from a just-regenerated key returns to the key list. Opening a key
from the table still pushes, so Back from a normally opened key is unchanged.
* feat(auto-router): support classifier reasoning effort
* fix(auto-router): harden classifier reasoning effort
* fix(ui): satisfy classifier config lint limits
* refactor(auto-router): simplify classifier effort support
* fix(auto-router): clear frontend-lint and type-discipline gates, trim LOC
---------
Co-authored-by: Tin Chi Lo <tin@berri.ai>
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create
* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side
* fix(proxy): keep access groups in sync when a model is renamed or deleted
* fix(proxy): cap the Request Logs conversation total like the row total
* fix(proxy): judge access group backing by the database for db models
A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.
* fix(ui): keep the conversation badge when an MCP call represents a conversation
A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.
* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type session spend aggregates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): satisfy request logs lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap per-session model aggregation in request logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: ratchet type-discipline budget after staging merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send an explicit null when the key edit form clears the organization
Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep Virtual Keys list state in the URL so it survives leaving the page
The search term, sort, pagination and drawer filters lived in component
state, so navigating away from Virtual Keys and back reset the table to an
unfiltered first page. Move them into query state alongside the existing
?key= deep link, which also makes a filtered view shareable.
* fix(ui): namespace the Virtual Keys filter params and bound page inputs
The unprefixed team_id filter hijacked the /api-keys create-key deep link,
which already takes team_id as a prefill, so ?create=true&team_id=X silently
filtered the list underneath the modal. Prefix the four drawer filters.
Now that page and page_size come from the address bar, clamp them to what
/key/list accepts instead of forwarding 0, negatives or an int64-overflowing
page straight through, and trim filter values arriving from a URL the same
way the drawer already trims them.
* fix(ui): fall back to a sortable column when the URL names an unknown one
A hand-edited or stale sort_by reached /key/list, which 400s it, leaving the
Virtual Keys page on its loading skeleton with no error. Validate it against
the fields the table's own headers can produce, and clear sort_by rather than
blanking it when a sort is reset so the URL stays clean.
Also replaces a default-state URL assertion that ran before any query-state
write could land, so it could not fail for the regression it named.
* fix(ui): use TanStack's functionalUpdate instead of a hand-rolled updater resolver
The local helper narrowed typeof updater === "function" against an
unconstrained T, which TypeScript cannot do because T itself may be a function
type, so next build failed to type check. table-core already exports the same
helper.
* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry
The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.
- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
so proxied traffic stays on async_pre_call_hook (per-key opt-out and
data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
via _process_response/_process_error; opted-out keys and legacy nameless
callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
and /guardrails/ui/provider_specific_params with a config model
Resolves LIT-3548
* Fix hide-secrets passthrough telemetry and JSON config input
* fix(guardrails): validate hide-secrets object config before submit
- apply_guardrail treats empty-string-only texts as no input, so no
false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
until it parses to a JSON object, instead of posting a string to an
object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type
* fix(guardrails): record no hide-secrets telemetry when nothing was inspected
walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
* fix(ui): stop the create team form resetting organization and models
The organization preselect ran in an effect keyed on the organizations
query, so any refetch of that list while the Create Team modal was open
overwrote the user's organization pick, which in turn cleared their
models pick. The models field was also cleared whenever the available
models fetch resolved.
Preselect the organization when the modal opens instead, and clear the
models only when the user picks a different organization. An org admin
whose admin orgs narrow to one while the form is open can still pick,
rather than facing a locked empty field.
* fix(ui): block team create when the picked organization is no longer available
An organization picked in the Create Team form now survives a refetch of
the organization list, so it can outlive the admin's access to it. Refuse
the create with a message on the field rather than letting the request
fail authorization at the proxy.
* fix(ui): keep the team create organization field usable when the pick goes stale
Locking the field on a single admin organization also locked it while it
held a rejected organization, so an admin who lost access could not pick
the one organization left. Lock it only while it holds that organization.
* test(ui): hoist the created team fixture out of the mock call
The inline object pushed the repo past its no-large-inline-object-arg
lint budget, which has no headroom.
The autoRouterPresets mock imported litellm/proxy/public_endpoints/autorouter_presets.json
as a module. That path sits outside ui/litellm-dashboard, the only directory the UI
Dockerfile copies, so `next build` type-checking inside the image failed with
"Cannot find module" and the ui-image job went red on every PR that touched an
image-scan path. Read the file with fs at runtime instead; vitest still derives
expectations from the real bundled catalog.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HqEPCNLDrssxsuezhAaL4j
