* perf(proxy): pipeline spend counter increments into one redis call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply surviving spend increments before raising scope error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): ruff format spend counter helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): settle inner spend counter gathers and fall back per key on pipeline failure
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): suppress BLE001 on pipeline fallback catch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): invalidate all batched spend counters on pipeline failure
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Concurrent usage reads on one worker now share a single spend-log query
instead of each scanning the same window, and a digest that comes back
nameless a second time is remembered for the full ten minutes rather than
thirty seconds, so a key that never resolves costs at most two scans per
worker per window per ten minutes. The first miss still expires after
thirty seconds so a read that lands between the daily spend flush and the
spend-log flush recovers on the next read
The daily spend rows and the spend logs of one batch are written a moment apart, so a usage read landing between them used to remember the session as nameless for ten minutes on that worker. Found identities keep the ten minute entry
The spend-log lookup for permanently unresolvable digests is back to a single DISTINCT ON scan over the requested window, keeping only rows that carry an alias, user, or team so a newer nameless row cannot hide an older named one. Results and misses are cached per worker for ten minutes keyed by digest and window, failed queries are not cached, and JWT rows keyed hashed-jwt-<sha256> now pass the digest gate. Tests cover the JWT gate, cache reuse and partial misses, window changes, error handling, the with-window guard, and the daily activity wiring
CLI session tokens are in-memory only and never get a LiteLLM_VerificationToken
row, so the usage APIs could not resolve key_alias, team_id, or user_email for
their spend rows: the exact join and the reverse-hash recovery both miss. The
owner is written to LiteLLM_SpendLogs.metadata at request time under the same
hashed api_key, so read it back from there for keys still unresolved after the
token-table passes.
The lookup is sha256-gated like the existing reverse-hash recovery and bounded
to the records' startTime window (min date minus one day, max date plus two) so
it stays on the startTime index. No migration.
Also guard the window parser against the date=None rollup rows GROUPING SETS
aggregation emits, which raised TypeError from strptime and turned the
aggregated usage endpoints into HTTP 500s.
Preserve deployment identity through savings calculation, with canonical model fallback only when either ID is absent. Cover negotiated rates, unchanged deployments, alias/base-model cache accounting and missing IDs.
Fixes#38811. Based on the deployment-identity approach proposed by @QuantumBreakz in #38834.
Co-authored-by: Claude Code <noreply@anthropic.com>
Team and service keys often have no user_email after a successful token
join. Treating empty email as a miss hashed every verification token on
routine CloudZero and Focus exports.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Export rows already join user_email from DailyUserSpend.user_id. Recovered
key-owner email must not replace that when only the alias join missed.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Resolving the model from the /bedrock path sends passthrough calls through
optimistic budget reservation, whose tokenizer cannot walk Converse content
blocks and so fell back to the model's max_input_tokens. Count those messages
as text and read inferenceConfig.maxTokens so a budgeted key reserves the
request's cost.
Calling the endpoint without going through FastAPI leaves the new search param set to its Query default object, which is not None, so the grouped-session and request_id lookup tests started taking the search branch
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
The search= param on /key/list, /audit, and /spend/logs/ui, plus key_hash= on /key/list, now compare the pasted value verbatim. Only a copied key ID (the hash) matches, so a raw virtual key never needs to travel in a GET query string
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
GET /key/list?search= matches the key hash (a raw sk- key is hashed
first) or a case-insensitive alias substring, and key_hash= now hashes a
raw sk- value too. GET /v1/memory?search= matches a key prefix or an
exact memory_id. GET /audit?search= matches id, object_id, changed_by,
or changed_by_api_key. GET /spend/logs/ui?search= matches request_id
across all time and api_key, team_id, user, end_user, session_id, or
model_id inside the date window; session grouping is skipped while a
search is active.
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
* fix(spend_tracking): leave SpendLogs.session_id null when no client session id was established
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(lint): ratchet basedpyright budget after session_id fix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): ignore trace ids as session ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): gate null SpendLogs.session_id behind missing_session_id: omit
Unset, generate and reject keep the legacy trace id fallback. omit records only
metadata.session_id, the key Langfuse reads, so a trace id copied into
litellm_session_id by get_litellm_params never becomes a session.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): stamp the omit decision on the request so a config reload cannot fabricate a session
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): keep omit covering requests the pre-call stamp never reaches
Router-model provider pass-through calls allm_passthrough_route directly and
skips add_litellm_data_to_request, so those requests never run the pre-call
helper and carry no omit stamp. Reading only the stamp made POST
/anthropic/v1/messages write a fabricated uuid into SpendLogs.session_id under
missing_session_id: omit while its Langfuse trace had no session, the exact
divergence the policy exists to remove.
The stamp now only pins omit on, and an unstamped request falls back to the
configured policy, so a config reload still cannot fabricate a session for a
request that was decided pre-call.
* fix(spend_tracking): make the session-omission marker proxy-owned so clients cannot forge it
* fix(spend_tracking): strip the client-sent omission marker from both metadata buckets before they merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): strip the session-omission marker from both metadata buckets
The pre-call policy ran before litellm_metadata is merged into metadata, so a
client that planted the marker in litellm_metadata had it copied back into the
route's own bucket after the strip and still got a null SpendLogs.session_id.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
The logs table already rolled up cost per session but the Tokens column only showed the representative call's usage. The per-session aggregate query now also sums prompt, completion and total tokens, and the Tokens cell switches to those sums for multi-call sessions the same way the Cost cell does.
Claude-Session: https://claude.ai/code/session_01CNasFqyjnLN3Rqman25vde
* feat(ui): keyset-paginate request logs by session trace
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep session grouping within type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): ruff format session grouping helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): only group sessions when group_by_session is an explicit true
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): reset session cursor on custom range and live tail toggles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover cursor reset on custom range toggle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): ignore next page clicks while the grouped page is still fetching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): only block next page while grouped placeholder data is shown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create
* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side
* fix(proxy): keep access groups in sync when a model is renamed or deleted
* fix(proxy): cap the Request Logs conversation total like the row total
* fix(proxy): judge access group backing by the database for db models
A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.
* fix(ui): keep the conversation badge when an MCP call represents a conversation
A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.
* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type session spend aggregates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): satisfy request logs lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap per-session model aggregation in request logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: ratchet type-discipline budget after staging merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send an explicit null when the key edit form clears the organization
Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Historical dirty spend on large installs was still unlabeled when the
matching token sat past the first page. Keep scanning until the digest
matches or the table ends.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
The Usage recovery path was adding four BLE001 hits and failing the
strict-rule budget. Soft-fail only on PrismaError so a down token table
still falls through to SpendLogs.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Recovery now resolves the key owner's email from UserTable via the
recovered token user_id, and SpendLogsMetadata keeps user_api_key_user_email
so new batch/export consumers see email without a separate user join.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
