* feat(usage): show subscription-covered providers and account fees separately
* fix(usage): write only the named fee columns and sync the root schema
The fee update now writes just the columns the request names, so two admins changing
different fields at once both land, and the dashboard edit form sends only what changed.
The root schema.prisma carries the new table like the two copies, and a billing period
anchored in the calendar's last month ends on December 31, 9999 instead of failing.
* fix(usage): register the subscription table span and keep provider logic under llms
* fix(subscription_accounts): register the fee routes as management routes and reject an empty account id
* fix(usage): run the new tests in CI and close subscription card review gaps
Adds the subscription test file to its unit shard, sets the migration lock timeout, rejects infinite fees, keeps the card from rewriting login files, lists each deployment once without aliases, shows the provider badge on entity views, and asks before removing a fee
* fix(usage): resolve named-credential ChatGPT accounts and keep admin viewers read-only on the fee card
* fix(usage): count a subscription fee only for accounts behind a current deployment
Usage listed a fee row for an account no deployment uses while another ChatGPT
deployment existed and dropped it when none did. It now lists only accounts behind a
current deployment in both cases. Adds a real proxy and Postgres integration test for
the fee API and for ChatGPT Responses rows on a deployment's own login, and names the
card's multi-component test with the integration suffix.
* fix(usage): keep a stored subscription fee listed after its account's deployments are removed
The usage route lists every account behind a subscription deployment plus every account carrying a stored fee, and reads the fees on every proxy, so a fee stays visible, counted, and editable for past ranges after its last deployment goes. The integration test now polls every day the calls spanned, so a UTC midnight crossing no longer fails it.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): replace the pre-migration row check with a rolled-back dry run of the pending migrations
* feat(proxy): roll back each migration of the dry run on its own and baseline-check databases without a ledger
* test(proxy): cover the dry run CLI and PrismaManager wrapper without a database
* fix(proxy): refuse the no-ledger dry run for a custom migration folder like boot does, and probe the rolled-back indexes directly
* ci(codecov): stop carrying forward the roi-postgres and redis-compat flags
Their carried-forward sessions were measured against older proxy_cli.py revisions (633 and 689 line maps against 710 today) and mark docstring lines of the merged file as missed, which failed codecov/patch on this PR at 70% while every unit shard reports the lines hit
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(migration_data_checks): size btree entries by the bytes Postgres stores (LIT-10244)
* test(migration_data_checks): flag exactly the compressed keys Postgres rejects at the entry limit (LIT-10244)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): cover owner-scoped file listing under require_managed_files
* fix(files): list only the caller's managed files on every list route under require_managed_files
* fix(files): narrow owner-scoped managed file lists to the named provider or model and page with after_id and before_id
* fix(files): narrow scoped lists the way uploads route, and refuse lists that cannot be scoped
A model name now matches every deployment that serves it, wildcards and
aliases included, and a provider matches deployments by the provider
uploads resolve, so anthropic/* and unprefixed Claude models count. The
OpenAI and Azure lists also return the caller's pass-through uploads.
An unknown provider and different after and after_id cursors answer 400,
and a list with no matching deployment skips the database read.
* fix(files): match stored files by model name and keep flag-off lists as on main
Under require_managed_files, a narrowed list now matches a file's model
mappings by deployment id or model name, so files LiteLLM stores in its own
database show on their model and provider lists. The Anthropic list defaults
to 20 rows when no limit is sent, after_id, before_id and the cursor 400s
apply only under the flag, and an admin keeps the provider list when the
managed files hook is not loaded. List errors are returned as values and
mapped to HTTP errors in one place.
* fix(files): keep admin provider lists and index the owner-scoped walk
Under require_managed_files, an admin keeps the provider list as on main, and
only a non-admin list reads the caller's own and team managed files. A stored
file now also shows on the provider its model name routes to through a
wildcard, with exact model names first, and a list routed to a model takes the
default page size of the provider serving it. A new index on created_by and
created_at keeps each chunk of the walk an index read, and the model check
runs before the file object is parsed.
* fix(files): filter provider and model lists in the query
A non-admin list scoped to a provider or model now asks Postgres only for the
caller's rows that map to one of that scope's deployments or models, or that
live in file storage, instead of reading every owned row and filtering them in
Python. A caller with many files on other providers no longer waits for the
whole set to be read. Team members' files are matched in the same query.
* test(files): type the fake raw query in the files list tests
* refactor(proxy): list managed files through one typed database handle
* fix(files): list alias and routing-group storage uploads on their provider
* feat(proxy): refuse migrations that existing rows would make fail, before the first one runs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(migrations): start the serving-proxy preflight case from a database without the api_key index
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type the preflight count rows without casts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): ruff format the db preflight flag
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy-extras): keep the extras files at their existing formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy-extras): type the ledger query and the reader pid so the type gate holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): declare the preflight report type where the extras package may be absent
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): suppress the optional-extras unknown on the preflight report variables
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): suppress the optional-extras unknown on the preflight import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): return a typed preflight report when the extras package is unknown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): sort the preflight import block
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): mark the preflight report binding Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): size the whole btree entry in the row check and name the flag for what it does
The index check now lays out each row's entry the way index_form_tuple does (tuple header, null bitmap, every indexed column at its alignment, rounded to 8) and compares it with BTMaxItemSize for the server's block_size, so a key that passes octet_length > 2704 but still fails the (api_key, startTime) index is caught. --db_preflight becomes --check_migrations_against_existing_data, db_preflight.py becomes migration_data_checks.py and the timeout env var LITELLM_MIGRATION_DATA_CHECK_TIMEOUT
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): unset DB environment variables removed from the row on every worker by row revision
* fix(proxy): order the environment variables revision migration after main's and set its lock timeout
* fix(proxy): snapshot the environment at the first DB sync even when no environment_variables row exists yet
* fix(proxy): close gaps in DB environment variable reconciliation
Grant the revision sequence to every role so a writer role with only
table grants can still save environment_variables. Unset SSO keys that
only the environment_variables row stored on the next SSO sync, while
keeping keys the SSO settings table holds. Skip a revision-less row once
a revision has been applied, so a stale cached row cannot bring a
removed key back. Keep plaintext and undecryptable values when rotating
the master key instead of dropping them from the row.
* fix(proxy): keep env var revisions monotonic across DB copies and restores
Stamp at least the old revision plus one on update so a copied row keeps
rising, re-read a stale cached row from the writer and apply it when the
database itself went back, unset dropped SSO keys when sso_settings are not
loaded from the DB, and keep env values master key rotation cannot decrypt
exactly as stored
* fix(proxy): skip an env var row read before an overlapping sync that already applied a newer one
* test(proxy): cover a read replica back on an older row and an SSO client removed from DB environment_variables
* fix(proxy): keep config reads up when the writer is down, find the revision sequence by table schema, read the writer before rewriting env vars, and add the trigger under db push
* test(proxy): model the writer database in the retry policy config update fake
* fix(proxy): recheck the environment_variables row revision right before applying it
A sync that started later could read a stale environment_variables row
before an earlier sync applied a newer one, then pass the read-order check
and apply the stale row, bringing back keys the newer row removed. The
behind-the-applied-revision check now runs at apply time, re-reading the
writer when the row is behind, and a writer row read while another sync
applied a newer revision is dropped.
The config sync tests now inject a fake database client and use the real
get_config_param and its cache instead of replacing get_config_param.
* test(proxy): keep encryption real in the SSO replica-lag test
* test(proxy): inject a fake Prisma client in the env var replica-lag test
* fix(proxy): keep the applied environment variables when the writer database is out of connections
* test(proxy): cover a writer database out of connections in the env var integration tests
* fix(interactions): keep background interaction bills through crashes, late restarts, and deployment keys
A settlement claim is now a lease, so a row whose settler died between claim and
outcome is claimable again after the lease and boot resumes it. A resumed poll
gets the full poll window and fetches with the serving deployment's credentials
from the live router. The stored create context keeps the request tags and the
request metadata that the creating worker's own spend row used
* fix(interactions): bill a lapsed claim's takeover only when no spend row exists
A settler that billed and then died, or failed to record the outcome, left the row unsettled, so the replica that took over its lapsed claim billed the interaction a second time. A takeover now looks up the spend row keyed by the interaction id and records the row as billed when it finds one.
* fix(interactions): settle Vertex AI and edited deployments, never double bill with spend logs off
A takeover with spend logs off records the interaction unsettled instead of
billing it again, and a release records its outcome before freeing the budget
reservation. A resumed or live fetch carries the deployment's Vertex AI project,
location, and credentials, and a stale deployment id falls back to the model
group only when one same-provider deployment serves it. A poll stops at the
first 400, 401, 403, or 404, string request tags no longer fail the create,
and settled_at gets an index built concurrently.
* fix(interactions): leave a live replica's interaction to it when a resumed poll cannot read it
A replica that boots with credentials unable to read an interaction got a 404 on its resumed poll and settled the row as unsettled at once, so the replica that created it, still polling, found it settled and never billed it. A resumed poll that gets a 400, 401, 403, or 404 now stops fetching and keeps checking the row until its poll window ends, and settles it as unsettled only when no other replica did.
* fix(interactions): keep the creator's metadata on a resumed bill and retry outcome writes inside the claim lease
A bill another replica writes now carries the metadata the creating worker's own
spend row is built from, so a client's metadata.model_group no longer relabels it.
A failed outcome write is retried for half the claim lease before the budget
reservation is left to a takeover, and the creating replica retries every fetch
error for its whole window again, as main does. The resume integration case now
checks that a resumed fetch uses the deployment's own key.
* fix(interactions): resume background interaction polls only after database models load
Boot resume ran before the proxy loaded the models stored in the database,
so a resumed poll for a model added through /model/new or the Admin UI
fetched with env credentials. A 4xx there stops a resumed poll, so the bill
was lost. The settlement now installs after the database models load.
The integration test resumes a database model with an eager first fetch
and fails with the old startup order.
* fix(interactions): renew a settling claim so a slow settle is never taken over and billed twice
A settle runs every success callback in turn, and one slow callback could
hold it past the 5 minute claim lease. A booting replica or a DELETE then
took the live claim over, found no spend row yet, and billed the
interaction a second time. The settler now renews its claim every third
of the lease until it records the outcome, so only a settler that stopped
renewing is taken over.
A takeover also read the spend log after writing its claim, so a failed
read left the row under the taker's own fresh claim, which its poll then
took for another settler's and left unbilled until a reboot. The spend
row is now read before the claim is written, so a failed read takes
nothing and the claim is retried on the next poll.
* test(spend): cover lapsed claim takeovers, rotated deployment keys, late restarts, and resumed bill tags
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): bill a background response once, from the poll job
Reading a completed background response with GET /v1/responses/{id} wrote a
full-cost spend row on every read, on top of the row the background cost poll
writes. Reads of a Responses object (GET, DELETE, cancel) now never bill, and
the poll job is the single biller.
The poll job now bills with the creating request's attribution (key hash, team,
org, user, aliases, tags) read off the managed object row, through a helper it
shares with the batch cost poll. The background create persists the creating
key and request tags so that attribution survives until the poll runs.
* test(proxy): a background response read reports zero cost
* fix(responses): bill a background response once, on the first read or poll that sees it finished
A conditional update moves the stored row out of queued or in_progress so only one worker or pod bills it, through one re-read stamped with the creator's attribution. A failed billing read puts the row back for the next poll
* fix(responses): keep the caller's response id out of the billing failure warning
* fix(responses): keep the creating client's User-Agent tags on the one background charge
* fix(responses): keep managed background reads free and bill each response once
The managed files hook marks reads and cancels of proxy-issued response ids, so only those stay free. SDK and database-less reads of a background response still bill, extra_body background creates are tracked, and the poll's status read no longer cools down a deployment on a 404.
* fix(responses): bill finished background responses from the fetched body and store rows Postgres accepts
* fix(responses): give the background billing event its own spend log row and bill responses the create left untracked
* fix(responses): narrow the managed files hook with a protocol instead of an enterprise import
* fix(responses): leave a read the caller retargets with client-side credentials to the cost poll
* fix(responses): bill background responses on the creator's end user and report the charge on the settling read
The read that settles the bill returns the charge in x-litellm-response-cost, the billed event
carries the creator's end user and the key, team, and user budget fields, and its spend row is
named after the id the client got. Override reads, cancels, and deletes settle from the
configured deployment, a streamed create marks its row billed, stored rows keep no model
output, and model insights counts the billed row's spend without a request.
* test(responses): cover background response billing across proxies, reads and chaos
* fix(responses): settle background billing only for callers allowed to address the response
A delete now settles the bill only after the response id access check passes, a streamed
create is marked billed when it sets background inside extra_body or its completed event
arrives after the last chunk, the billing event keeps the deployment's litellm_params
base_model, a key with no user and no team bills no user instead of its owner scope, and the
enterprise cost check import is optional
* fix(responses): record the bound on the pending status IN filters
* fix(responses): send the billed cost split on the settling read and keep the batch cost event as it was
The read that settles a background response now sends the whole cost header
family from the billed event, so the breakdown headers add up to the charge.
Batch cost events go back to main's attribution (creator id as the user, no
budget fields); only background responses get the end user and budgets.
Integration tests expect the charge on the settling read (or none when the
poll billed first), one charge for a streamed create, and SDK flows that poll
with the id each read returns.
* fix(responses): bill each background response once across upgrades, streams, and failed settles
Rows stored before this billing model are found by the issued id and rekeyed by the
provider id, and the cost poll retires one a read already billed. A streamed background
create is free on its own and bills the finished response through the same managed row
before the client sees response.completed, so a racing read cannot bill it again. A
settle that fails retries in the background, a retargeted read with no row tracks and
bills from the configured deployment, and a delete keeps its billed cost header.
Reverts the unrelated json.dumps narrowing in the completion transformation.
* fix(responses): bill cache-polled background streams and match poll rows by the issued id
* test(responses): cover deletes, failed settles, budgets, base models, and cache-polled background billing
Adds integration cells for the background response billing paths the audit
inventory lists: a DELETE that settles the bill and refuses a stranger, a
settle the database refuses that retries once it recovers, the Prometheus
remaining budget, base_model pricing from model_info and litellm_params, the
daily model usage rollup, the end user, the settling read's cost split, stored
rows without model output, a forged managed marker on the create, streamed
creates with background in extra_body, and the cache-polled create
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Through a transaction-mode pooler Prisma takes pg_advisory_lock(72707369) on one
backend and releases it on another, so the first backend keeps the lock while idle
and the next boot waits on it. Migrations now run with
PRISMA_SCHEMA_DISABLE_ADVISORY_LOCK=1 while LiteLLM holds the coordinator lock and
Prisma's key as transaction-level locks on one pinned connection, so nothing can
outlive the migration
Resolves LIT-9571
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): keep reports in a local table with a persistent instance id and an admin export route
* feat(telemetry): key deployment hashes from a random secret stored in the proxy DB
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): retry a locally stored window under the same id so it is never counted twice
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(telemetry): state the overlap and dedupe contract for paging stored reports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types for the telemetry reports paging note
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): require an explicit LITELLM_TELEMETRY_DESTINATION or saved destination before recording
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types for the explicit telemetry destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): register the attempt logger and create its hash secret only once a sink is built
A proxy with a database that never opts in now carries no telemetry callback
and writes no deployment hash secret to LiteLLM_Config.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Counts failed gateway requests by HTTP status at the ASGI edge
(LiteLLM_DailyGatewayFailedRequests) and rolls failures up per virtual key,
team, user and model group with the status the logging callbacks recorded
(LiteLLM_DailyRequestErrors, fed from the spend writer, flushed like the
gateway counters, directly or through Redis with a pod lease, and on
shutdown). GET /gateway/daily/activity gains by_status_code; the new
admin-only GET /gateway/errors/activity returns per-day totals with
per-status counts, failures by status code, and keys, teams, users and
model groups ranked by failures with their top status. Dashboard schema.d.ts
regenerated.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(credentials): add credential_alias and make credential_name immutable on PATCH
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): query credential rows by role to stay under the no-node-access budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): expect credential_alias in load_credential_list dump
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drive the credential SearchSelect by placeholder and option roles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): pass the decrypted CredentialItem to update_db_credential during master key rotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): trim credential_alias in CredentialModal so whitespace-only input clears the alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(credentials): replace credential_alias with display_name and reject edits to config credentials
credential_name stays the immutable reference key. display_name is a nullable, trimmed, max 255
character label set on POST or PATCH (omit keeps it, null clears it, blank is a 400). Reads return
display_name plus an in-memory source tag (db or config), and PATCH or DELETE on a config-defined
credential answers 400 instead of a misleading 404
* feat(ui): show credential display names everywhere and lock config credentials
The credentials table shows the display name with the credential name beneath it, badges config
credentials and disables their edit and delete actions. The edit modal adds an editable display
name next to the read-only credential name and only sends it when it changed. Every credential
picker and reference (add model, model info, models table, vector store form and info) shows the
label while still submitting credential_name
* feat(cli): set display names on credentials and show them in the list
* refactor(credentials): keep the new credential types inside the lint gate budgets
* refactor(cli): print credential command JSON through one helper
* fix(credentials): treat an empty credential_name on PATCH as omitted and pin the 404 for vanished rows
A blank credential_name never renamed anything, so PATCH accepts it again instead of answering 400. New tests pin a trimmed display_name on PATCH, the repository carrying display_name, and a 404 (not the config-owned 400) for a DB credential another worker already deleted. The hydration helper no longer copies display_name, since none of its callers read it, and the CLI update passes display_name straight through because the exactly-one check already makes it None when clearing.
* fix(ui): keep a model's credential when its picker text is emptied and skip the admin-only list for other roles
Clearing the search text in the model edit form's credential picker used to submit null, silently detaching the model's credential on save. None is now the only way to clear it. The models table fetches /credentials only for proxy admins, since everyone else got a 403 on each page load and falls back to the raw name anyway. Also fixes a type error in the re-use credential dialog and adds tests for the display name surviving a provider switch, the 255 character limit, the request payload, and the vector store None choice.
* fix(client): percent-encode the credential name when updating its display name
A name with ? or # was cut short in the URL, so the PATCH landed on a different credential or 404'd.
* fix(credentials): answer 405 with Allow: GET for PATCH and DELETE on config credentials
A config-defined credential exists (GET returns it) but is read-only through the API, which is what 405 Method Not Allowed means. 400 described a malformed request, and 404 would claim the credential does not exist.
* fix(proxy): ignore a display_name set on config credential_list entries so non-string values cannot fail boot
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): name the database error when the Lens rename check cannot run
The Lens rename check runs before prisma db push and raised a generic
"Cannot verify Lens data safety" RuntimeError on any psycopg error, with
the cause only in the chained exception. proxy_cli prints the message and
exits, so a Postgres at its connection limit read as a Lens problem.
The raised message now ends with the redacted database error text, and
the connection opener is an injected parameter so the unit test drives
the check without a database.
* test(proxy-extras): drop the Lens check test class docstring
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(lens): isolate trace storage and investigation in a Rust service
* fix(lens): include Rust sources in the image build context
* feat(lens): wire service setup, scoped delivery receipts and lease attempts
* fix(lens): complete service routing and reject stale investigation results
* fix(lens): retry key propagation and validate isolated Compose setup
* fix(lens): seed through isolated ingestion and preserve upstream queue fixes
* chore: sync schema.prisma copies from root
* fix(lens): bind nullable due timestamps as text for Prisma
* chore(ui): remove stale lint suppressions
* fix(lens): address CI failures and review findings
* refactor(lens): remove retired Python worker and run evaluations in Rust
* fix(lens): reuse control connections and satisfy review checks
* test(lens): install and upgrade both Helm charts on Kubernetes
* test(lens): run connection reuse coverage as an integration test
* fix(ui): upgrade Next.js to 16.3.8 security release
* fix(lens): fence stale attempts and preserve reviewed evidence
* Revert "fix(ui): upgrade Next.js to 16.3.8 security release"
This reverts commit 2f79a51b25.
* fix(lens): stop failed investigations and stream history excerpts
* test(lens): cover model tool and result contracts
* test(lens): fix retired routes and reuse installation build artifacts
* test(lens): use portable grep in Helm installation smoke
* test(lens): wait for migrations before forwarding Helm services
* fix(lens): keep failed evidence reads retryable
* fix(lens): preserve sandbox output during process exit
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* perf(lens): claim worker jobs from an indexed due queue instead of scanning every lens
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): apply the due_at index concurrently on its own and default legacy rows to due
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(lens): pass repository to claim lifecycle tests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): page past unsupported due lenses and declare the full due_at index
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): claim due lenses in a loop instead of recursion
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(ci): stop deferred pydantic builds leaking caller locals and add missing Lens FK migration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(types): drop narrating comment from caller-locals regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): guard the Lens review FK migration with DO blocks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(types): run the caller-locals regression in-process
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): scope the Lens review FK guards to their table
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): refresh server catalog for each gateway operation
* fix(mcp): reject configuration changes during scoped dispatch
* fix(mcp): refresh shared catalog state for each operation
* fix(mcp): refresh catalog before native alias routing
* docs(mcp): clarify native route resolution order
* test(mcp): align streaming fixture and generated API documentation
* fix(mcp): preserve discovery published during catalog refresh
* fix(mcp): coalesce queued catalog reads without a stale window
* fix(mcp): reconcile concurrent route changes when publishing catalog
* test(mcp): provide catalog scope in post-call hook fixtures
* fix(mcp): retain valid live routes and handlers during refresh
* fix(mcp): keep refreshed OpenAPI operation membership authoritative
* test(mcp): preserve logging fixtures after catalog integration
* fix(mcp): isolate transport test admission state and exhaust OAuth outcomes
* refactor(mcp): narrow catalog consistency change to ticket scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(mcp): gate catalog refresh on a database revision marker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): refresh waiters that observed a newer catalog revision
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): awaitable catalog revision doubles in prisma mocks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): satisfy catalog refresh lint and type gates
* test(mcp): provide catalog scope in toolset fixtures
* test(mcp): exercise temporary OAuth through catalog operations
* fix(mcp): share active catalog snapshots across discovery tasks
* fix(mcp): retain discovered routes across cached catalog operations
* fix(mcp): preserve local tool ownership across discovery
* refactor(mcp): satisfy tightened immutability lint ceiling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): authorize local handlers by registered server ownership
* fix(mcp): check registered handler freshness and isolate fixtures
* fix(mcp): read catalog revisions and snapshots from the writer
* fix(mcp): resolve access groups from the operation catalog
* fix(mcp): preserve empty access group restrictions
* fix(mcp): retain rediscovered routes across concurrent updates
* fix(mcp): enforce registered ownership for local tool dispatch
* fix(mcp): preserve issuer discovery across catalog snapshots
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(lens): add dataset case and size limits
* feat(lens): add dataset, case and build models
* feat(lens): build dataset cases from traces, findings and text
* feat(lens): store dataset revisions insert-only
* feat(lens): add dataset routes for build, save, export and eval cases
* feat(lens): mount the dataset router before lens routes
* feat(lens): add LiteLLM_LensDataset table
* feat(lens): add LiteLLM_LensDataset table to proxy schema
* feat(lens): add LiteLLM_LensDataset table to extras schema
* feat(lens): add migration that creates the dataset table
* test(lens): cover dataset case building, dedupe and limits
* test(lens): cover dataset revisions, conflicts and eval cases
* chore(ui): regenerate API types for lens datasets
* feat(lens): add dataset UI types
* feat(lens): add datasets API client
* feat(lens): add dataset query and mutation hooks
* feat(lens): add case selection and expected edit logic
* test(lens): cover case selection and expected edits
* feat(lens): add the add to dataset dialog
* test(lens): cover saving picked cases from the dialog
* feat(lens): add datasets list
* feat(lens): add dataset detail with revisions and export
* test(lens): cover editing, revisions and export in datasets tab
* feat(lens): expose datasets on the lens API
* feat(lens): add in-memory datasets for demo mode
* feat(lens): wire demo datasets into the demo lens API
* feat(lens): add optional lens API hook
* feat(lens): add optional onboarding hook
* feat(lens): add datasets tab and dataset routing
* feat(lens): show the datasets tab
* feat(lens): add to dataset from the trace header
* feat(lens): add a single turn to a dataset from a step
* feat(lens): add finding evidence to a dataset
* docs(lens): add datasets screenshots for the PR
* test(lens): cover dataset revision storage against Postgres
* test(lens): cover dataset trace paging, findings and route errors
* test(lens): cover dataset build fallbacks and no_content skips
* fix(lens): register the dataset table for postgres span names
* feat(lens): add invalid skip reason for unparseable case lines
* fix(lens): keep valid JSONL cases, provenance and size limits; stop reading past the case cap
* test(lens): cover malformed JSONL, re-import provenance, size fields and early cap
* chore(ui): regenerate API types for the invalid skip reason
* feat(lens): show text for the invalid skip reason
* feat(lens): render datasets in the same inspector table as investigations
* test(lens): open a dataset by clicking its table row
* docs(lens): update the datasets list screenshot
* style(lens): format the datasets table
* fix(lens): normalize JSON text span input and output into messages when building cases
* test(lens): cover JSON text span normalization for dataset cases
* Revert "fix(lens): normalize JSON text span input and output into messages when building cases"
This reverts commit 29c3e34962fa12427dbd516c08f2599db997a905.
* fix(traces): normalize agent assistant summaries into UI messages
* feat(lens): add dataset case view helpers built on the trace parsers
* test(lens): cover dataset case view helpers
* feat(lens): show dataset cases in an inspector table
* feat(lens): open a dataset case in a side panel with trace message cards
* feat(lens): rebuild the dataset page header and layout
* feat(lens): keep the open dataset case in the URL
* test(lens): drive dataset edits through the case table and panel
* docs(lens): update dataset view screenshots
* Revert "test(lens): cover JSON text span normalization for dataset cases"
This reverts commit 30229ae8ae90265c09d755c43c1f1eeaa66050c7.
* fix(lens): import StateMessage from shared in dataset detail
* fix(lens): import StateMessage from shared in datasets list
* fix(lens): give the datasets migration a unique timestamp after review checkpoints
* fix(lens): show investigation findings for agent traces
Replace child tool-error counts in the trace table with distinct investigation findings. Keep unassessed traces separate from completed clean investigations and use the root status for failure filters and timeline counts
* fix(lens): stabilize findings updates and repair UI checks
* fix(lens): allow viewer findings reads and index trace lookups
* test(lens): cover viewer findings reads with Postgres
* feat(proxy): add models column to the end user table
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): enforce the end user models allowlist in model access checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): accept and return models on the customer endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover the customer models allowlist
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): resolve team aliases before the end user model check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): log v1 migration failures at ERROR so LITELLM_LOG=ERROR shows them
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): reuse litellm secret redaction and mask configured DB passwords exactly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy-extras): name the password alternation in _redact_credentials
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy-extras): wrap v1 migration ERROR lines at 120 columns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-extras): integration cells for v1 migration ERROR logging and password redaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-extras): integration cells for component DB env vars, JSON logs, migration Job and v2 resolver
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-extras): give every v1 migration integration cell the 900s timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-extras): bind the recovery forwarder before migrating so the retry cannot race it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-extras): drop the slow P3005 integration cell and bound migration subprocesses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): keep command repr and tolerate non-sequence cmd in migration error logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-extras): double the subprocess boundary in the cmd=None retry test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
* fix(vector_stores): return managed file ids from vector store file list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(vector_stores): cover managed file list route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vector_stores): only map round-trippable managed ids and index flat file ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): build managed file gin index concurrently
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(proxy-extras): move the managed file gin index migration after main's newest
* fix(vector_stores): satisfy lint gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(proxy): drop the stale no-index note on the raw file-id guard
* test(vector_stores): cover managed file ids on the vector store file list end to end
Integration cells for GET /v1/vector_stores/{vs}/files mapping provider file ids back to
the caller's owner-scoped managed ids and decoding managed after and before cursors: raw
httpx, the OpenAI SDK sync and async pagers, the three credential routing modes, the owner
filter branches, raw and unmappable cursors, provider errors, duplicate and non-string ids,
a provider outage mid-burst, a worker SIGKILL mid-burst, and the GIN index migration applied
by the migration entrypoint and by db push
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(interactions): durable cross-pod settlement for background interaction billing
Background interaction billing lived only in the creating replica's memory, so a
DELETE routed to another replica, or a restart of the creating one, never billed
the completed provider work and the budget reservation was refunded at the poll
timeout. The create now registers the billing context in a settlement store
before returning, the proxy installs a Prisma-backed store at boot
(LiteLLM_BackgroundInteractionSettlement, schema-only migration), any replica
claims the row once through a conditional update before billing or releasing,
startup resumes every unclaimed row with its remaining timeout, and a give-up
records an unsettled outcome instead of silently reconciling to zero. The SDK
keeps an in-memory store and behaves as before.
* fix(interactions): survive a settlement install failure at boot and stop carrying request headers
* fix(interactions): drop the stored request context once a settlement row is settled
* fix(interactions): bill the completed response a poll already saw when its claim only answers at the deadline
* fix(interactions): carry a missing model through the settlement context for agent-only background creates
An interaction created with an agent and no model reaches the poll with no model name, exactly as on main. The settlement context now stores that None instead of rejecting the create, which answered the client with a 500 after the provider had already accepted it.
* fix(interactions): leave an unfetchable background interaction to its creating poll when a delete lands elsewhere
The remote pre-delete path fetches with only the delete's credentials, so a fetch it cannot make says nothing about the interaction. It used to claim the settlement row and release the reservation anyway, which stopped the creating replica's poll and lost the bill when the delete then failed the same way. It now returns without claiming; the in-process path keeps releasing on an unfetchable state, since its context carries the create's own credentials.
* fix(interactions): fail a cross-replica delete when its pre-delete fetch fails so the creating poll keeps the bill
* fix(interactions): keep the stored settlement gate when registration raises after landing, and fail resumed-poll deletes closed
A registration that raised after its row committed moved the poll to a private in-memory gate, so the creating worker billed while the stored row stayed unclaimed for another replica's delete or the next boot to bill again. The row is now read back once and, when it landed, the poll claims through it like every other settler.
A worker that resumed the poll after a restart is not the creator, so its delete on a failed pre-delete fetch now fails with the fetch's error instead of releasing and deleting. After a fleet restart every worker holds resumed polls, which left the fail-closed path applying nowhere.
* fix(interactions): settle an unverified registration through the durable claim
A create whose settlement-store write raised no longer bills through a
private in-memory gate that a later boot's resume cannot see. The claim
asks the durable store first and falls back to the local gate only when
the store answers that no row exists, and a missing settlement table reads
as no rows so a replica without the migration still settles in process.
* test(proxy): keep the settlement test where the proxy-infra shard collects it
The merge of main moved test_background_interaction_settlement.py under
tests/unit/proxy/spend_tracking, but the proxy-db shards claim tests/unit/proxy
files one by one in .circleci/scripts/unit_selection.sh, so no CI shard ran it
and codecov/patch dropped. tests/test_litellm/proxy/spend_tracking is collected
whole by the proxy-infra shard, which is where the test ran before the merge.
* fix(interactions): raise on a non-2xx Gemini interaction fetch
AsyncHTTPHandler.get never raises for status and the Gemini GET transform
only raised when the body was not JSON, so a 500 or 404 carrying Gemini's
JSON error body parsed as an interaction with no status. A delete on a
replica other than the creator then claimed the settlement as released and
forwarded the delete instead of failing closed, and the bill was lost. The
transform now raises GeminiError with the vendor's status, as the delete
transform already does; the in-process poll already retries a fetch that
raises
* test(integration): audit durable background interaction settlement across replicas
Twenty-six deterministic cells drive a one-worker creator and a two-worker
settler against an owned scripted Gemini upstream: cross-replica deletes
bill once, failed and cancelled interactions release, a later replica
resumes unclaimed rows, custom deployment pricing bills at the deployment
rate, a fetch the settler cannot make fails the delete closed, odd ids are
refused, a missing settlement table keeps in-process billing, polling
disabled registers nothing, the budget reservation is released by the
settler, an upstream outage mid-burst fails closed and recovers, killed
workers hand their polls to the respawned ones, and concurrent deletes on a
slow upstream settle exactly once. The support upstream gains a scripted
interaction store with per-id GET status and delay, and the process helper
gains an owned upstream a test can stop and restart
* test(integration): refuse a repeated delete in the scripted upstream and pin the settlement budget below one estimate
* chore(ui): regenerate dashboard API types after merging main
* test(integration): accept the 422 budget refusal and a respawned worker's resume
The budget cell pinned a 400 that the proxy stopped answering when budget refusals moved to 422, so it now asserts the status and the budget_exceeded error type the sibling budget tests pin. The later-booting replica cell accepts a claimer that is any worker started after the creates, since uvicorn's supervisor can respawn the creator's worker under load and the respawned worker's boot resume claims the rows by design; the single spend row check is unchanged
* test(integration): delete the pinned key's interaction with a second key
A key whose budget is filled by its own reservation is refused on every route, the DELETE included, so the cell now asserts that 422 and sends the delete with a second key, which is what the reservation release on another replica needs in order to be observable at all
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy-extras): retry P3009 when a peer already recovered the named migration row
* fix(proxy-extras): pin P3009 recovery locals as Final and cover a recovered sibling row
* test(proxy-extras): stop the P3009 ledger fakes shadowing the partition detector cursor
* fix(proxy-extras): annotate the migration ledger reader locals as Final
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(lens): run investigations with configured wildcard models
* fix(lens): validate worker model access and pricing before analysis
* fix(lens): bound worker validation and preserve unrelated edits
The auto-router usage view summed the lifetime savings of every session
overlapping the date range, so it disagreed with the Overall savings view,
which sums daily rollups by request day.
Record auto-routed money per UTC request day and router in one new table,
written in the same statement as the session rollup and corrected in the
same transaction as late baseline estimates. The all-router headline reads
the same daily rows and filters as Overall; savings no router day row
accounts for are reported as unattributed and void the baseline comparison.
Session shape and caching stay whole-session and are labelled so; the
savings-per-session tile is removed.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The migration job runs DatabaseURLSettings.apply_to_env(), which rewrites
DATABASE_SSLMODE=verify-full plus DATABASE_SSLROOTCERT into Prisma's TLS
dialect: sslmode=require&sslcert=<CA>&sslaccept=strict. The request-log
index build then hands that same URL to psycopg, and libpq reads sslcert
as a client certificate, failing with "certificate present, but not
private key file" on every verify-full deployment since #43948.
_strip_prisma_query_params now undoes the Prisma dialect before psycopg
sees the URL. sslaccept=strict (or any value Prisma treats as strict)
becomes sslrootcert=<CA> plus sslmode=verify-full whatever sslmode said,
since strict verifies chain and hostname and libpq only does that in
verify-full; sslmode=disable stays off. Without strict, Prisma verifies
nothing, so the CA is dropped and sslmode is kept as is. A URL that also
carries sslkey is libpq's own client-certificate form and is left alone.
Resolves LIT-9169
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
