* feat(decisions): support LM Studio decision models
Co-Authored-By: Codex <noreply@openai.com>
* fix(ci): resolve OSV dependency findings
Upgrade the extension MSAL dependency and document expiring exceptions for unused Windows filelock APIs while Python 3.10 remains supported.
Co-Authored-By: Codex <noreply@openai.com>
* Clean up ignored vulnerabilities in osv-scanner.tomlchore: remove unrelated dependency and scanner changes
Removed ignored vulnerabilities related to Windows issues and Python version compatibility.
* chore: remove unrelated dependency and scanner changes
* chore: remove unrelated dependency and scanner changes
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: ishaan-j-berri <ishaan.j@berri.ai>
* fix(prometheus): count provider-side 401s in LLM failure and deployment metrics
The invalid virtual key skip no longer applies to an error the LLM provider
returned while the deployment was called with its own credential, so a 401
for a bad deployment credential increments litellm_llm_api_failed_requests_metric,
litellm_deployment_failure_responses and litellm_deployment_total_requests and
marks litellm_deployment_state as a partial outage again. Proxy-gate rejections
and a failing credential that belongs to the caller (a rejected per-user
credential, an api_key sent with the request, a per-user GitHub Copilot
session) keep the skip
* fix(prometheus): keep a caller's own Anthropic OAuth token failure out of the deployment metrics
A provider 401 on an Anthropic OAuth token the caller forwarded through the proxy is the caller's credential
failing, so it stays out of the deployment failure and outage series like a caller-supplied api_key. Adds
integration cells for an Anthropic deployment's own key, the forwarded caller OAuth token and a direct SDK
call that brings its own key
* test(prometheus): pin how a caller's own key or Authorization header on a deployment is counted
An SDK Router call that passes its own api_key is counted against the one-off deployment the router builds for that key, so the operator deployment keeps only its own failure. A provider 401 on an Authorization header the caller sends through the proxy counts against the deployment, as on v1.80.12
* fix(prometheus): keep caller-sent provider credentials and token-exchange 401s out of deployment metrics
A caller can send its own provider credential in the request body, such as
azure_ad_token, azure_scope, the Azure Entra app fields, watsonx token,
gigachat_access_token, OCI keys, AWS keys or Vertex credentials, at the top
level or inside extra_body. A provider 401 on that credential is the caller's
failure, so it stays out of the shared deployment's failure, request,
LLM-failed and state series, as on main. A token-exchange deployment trades the
caller's identity token on every call, so its 401s stay out as well, matching
the router cooldown.
* test(prometheus): Final locals, precise helper types and one for per comprehension in the new test helpers
* test(prometheus): pin that a provider 401 at a caller-sent api_base counts against its one-off deployment
* fix(prometheus): keep a provider 401 on caller-sent headers out of deployment metrics
A caller can override the deployment's credential with headers it sends in the
request body: headers or provider_specific_header, or an Authorization, api-key,
x-api-key, x-goog-api-key or Ocp-Apim-Subscription-Key entry in extra_headers.
A provider 401 on such a request is the caller's failure, so it stays out of the
shared deployment's failure, request, LLM-failed and state series, as on main.
* fix(prometheus): skip a provider 401 only when the caller's headers carry a credential
The proxy now records headers, extra_headers or provider_specific_header in
credential_fields only when they name a provider credential header
(Authorization, api-key, x-api-key, x-goog-api-key, Ocp-Apim-Subscription-Key),
the same way api_key is recorded. A caller that sends a plain header such as a
trace id still has a provider 401 on the deployment's own key counted against
the deployment.
* fix(prometheus): stamp credential_fields after the proxy forwards the caller's headers
A provider credential header the proxy forwards from the caller's HTTP request
(forward_llm_provider_auth_headers + forward_client_headers_to_llm_api) lands in
data["headers"] after proxy_server_request is built, so the stamp missed it and a
401 on that caller credential was counted against the deployment. Compute the
stamp after the header forwarding steps and pin both forwarded cases.
* fix(prometheus): skip a provider 401 when the caller sends any header or an OpenAI organization
The proxy now records headers, extra_headers or provider_specific_header in
credential_fields whenever the caller sends or the proxy forwards any header
there, and records organization when the caller sends one in the body or as a
forwarded OpenAI-Organization header. A forwarded header can be a gateway key
such as x-portkey-api-key or an OpenAI-Project id, and a wrong organization
gets a 401 from OpenAI, so these 401s stay out of the deployment metrics as on
main instead of marking a healthy deployment as a partial outage.
* fix(prometheus): ignore the user information the proxy adds when stamping caller headers
With add_user_information_to_llm_headers on and header forwarding enabled, the
proxy adds x-litellm-user_api_key_* headers to the forwarded headers. Those
carry the proxy's own key and user details, not the caller's, so they no longer
make a request look caller-scoped, and a provider 401 on the deployment's own
key on such a request is counted.
* fix(prometheus): skip a 401 raised before the provider is called or at the caller's own endpoint
A deployment with no key of its own fails with LiteLLM's local "Missing
Anthropic API Key" 401 before any provider call, and a caller who sends
aws_bedrock_runtime_endpoint (or another provider endpoint field) can get a
401 from their own gateway. Both were counted against the shared deployment
and marked it as a partial outage, where main skipped every 401.
Count a 401 only once the provider was called (api_call_start_time is set),
and treat caller-sent provider endpoint fields like caller credential fields.
* fix(prometheus): count the deployment's own 401 on requests that carry only anthropic-version or anthropic-beta
The Anthropic SDK sends anthropic-version on every request, and the proxy copies
anthropic-version and anthropic-beta into provider_specific_header for Anthropic,
Bedrock and Vertex. Both were treated as caller headers, so a 401 on an Anthropic
deployment's own key from any Anthropic SDK client stayed out of the deployment
metrics.
These two headers only pick the API version and beta features. Anthropic answers
a bad anthropic-version or an unknown anthropic-beta with a 400, never a 401, so
they cannot turn the deployment's own key into a 401 and no longer mark the
request as carrying a caller header.
* test(prometheus): read and write the owned proxy's own database in the expired-key and MCP cells
Since #47213 an owned proxy boots on a private database, so the expired-key update and the MCP guardrail row read now go through the gateway's database_url instead of the worker's.
* fix(prometheus): keep credential_fields as main stamps it and give Prometheus its own caller_credential_fields
credential_fields tells the Rust OCR bridge which stripped body fields the
caller sent. Repurposing it for the Prometheus caller-credential check changed
its value (organization and extra_headers added, anthropic-version-only
headers dropped). Restore main's stamp unchanged and stamp the Prometheus
classification under caller_credential_fields, which only Prometheus reads.
* fix(prometheus): skip a provider 401 on a stored credential the caller picks by litellm_credential_name
A request body litellm_credential_name swaps the deployment's stored
credential for one the caller chose, so a 401 on it is the caller's, not the
deployment's. Treat it like the other caller credential fields so it no longer
marks a healthy shared deployment as a partial outage.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(team): keep the team reset schedule when a temp budget increase is set on a member inheriting the team default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expect the temp-only member row to carry the team default cap
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): serve a team's wildcard deployment before a proxy-wide wildcard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): keep the wildcard precedence helpers free of partially typed locals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): call each pattern router once in the wildcard precedence helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ptu): a team's wildcard reservation serves beside a proxy-wide catch-all
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): budget period tooltip on key, team, user and org spend tiles
Adds a shared BudgetPeriodTooltip to the four detail-view spend tiles showing the current budget period, max budget, reset interval and spend so far, and makes organization create schedule budget_reset_at and organization info return it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): give the budget period tooltip trigger its own aria label
The default SimpleTooltip trigger shares the question-circle label with InheritedBudgetHint, which broke the key budget display tests that count that icon
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): match the proxy's budget window rules in the period tooltip
30d and Nmo now span calendar months like get_budget_window_start, the arithmetic runs in UTC so a viewer's DST change cannot shift the start, the hourly/daily/weekly/monthly aliases the proxy accepts are normalized before parsing, and spend without a duration reads as lifetime spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): gate key budget_duration and budget_id updates like max_budget
A key owner or team member could shorten budget_duration or swap budget_id on
/key/update without the budget authorization max_budget gets, so the reset job
cleared spend on a capped key or a bigger tier lifted its cap
Fixes#46916
* test(proxy): cover tier keys and self-serve budget policies for key budget updates
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
OAuthScopeResponse holds a tools/call response's http.response.start so a managed OAuth scope challenge can replace it with a 403. Once the call was allowed, the held start still waited for the next body message, which for a slow tool is the 15s SSE keep-alive ping. Send it as soon as the call is allowed instead.
Fixes#46659
* fix(router): stop concurrent deployment deletes from crashing deployment lookups
delete_deployment and upsert_deployment shrink model_list before they repair
the index maps, so a lookup on another thread in that window read a stale
index: IndexError, or the neighbouring deployment and its credentials. An add
racing a delete could corrupt the index for good.
One RLock now covers model_list and the three index maps. Delete, upsert, add
and the set_model_list reset change them under it, and every lookup that turns
an index into a deployment reads under it.
Fixes#45242
* test(router): release the held delete when the read reaches the index lock
The race tests released the held write after a fixed 0.2 s, so a reader the runner
stalled past it read the repaired list and passed on unfixed code. The read now runs
on its own thread and the write stays held until the read returns or reaches the
router's index lock; the 10 s bound only fails a broken synchronization.
The integration test gains the audit cells: chat, streaming, Responses and
Anthropic messages calls (sync and async, by model name and team public name)
during a held delete, the same for upsert, add and set_model_list, a request for
the deployment being deleted, a provider error, a double delete, and thread and
async bursts during upsert churn.
* test(router): cover the early return of a delete and the team filter of get_model_ids
delete_deployment now returns None from inside the index lock, and get_model_ids by
name reads its deployments through _deployments_named; pin both: deleting an unknown
id returns None and leaves the list alone, and exclude_team_models drops only the
team deployment.
* fix(router): read deployments by name under the index lock outside the Router
The response polling provider check, the complexity router's adaptive pool and the
savings baseline turned the name index into list positions themselves, so a concurrent
delete could still hand them a stale slot. They now read through Router.deployments_named,
which takes the deployment index lock.
* fix(router): type the locked deployment lookups as DeploymentTypedDict
deployments_named and _team_deployments_named returned tuple[Any, ...]. They now return
DeploymentTypedDict rows, which is what model_list holds. The adaptive pool setup in the
Router and the complexity router drop their branches for pydantic Deployment rows, which
these lookups never return, and should_include_deployment accepts the typed row.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(guardrails): exclude litellm_metadata from pass-through guardrail scan text
The pass-through guardrail translation json-dumps the request body as the
scan text when no request_fields are configured. The unified guardrail hook
injects litellm_metadata into that body first, so the dump carried the
caller's key metadata. Exclude it alongside metadata and litellm_logging_obj;
vendors still receive it through request_data.
* test(guardrails): type the pass-through scan-text tests per AGENTS.md
Preserve oversized JSON integers in proxy requests and model rewrites
Parse large bodies in a worker thread to avoid blocking request handling
Fixes#47947
Co-authored-by: Jeevan Babu Gotru <114808955+jeevan1098@users.noreply.github.com>
Co-authored-by: moe-berri <moe@berri.ai>
* fix(proxy): reconcile team member total_spend with current cycle spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): read /team/list as a list and drain the scripted provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(repositories): compose the membership reset writer and drop SQL text asserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: type the cascade repair test and flatten memberships without a stacked comprehension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: keep the reset job test file's existing typing untouched
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(teams): show caller member spend and budget on Your Teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): use member-linked reset dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep type-discipline suppressions on cast lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): preserve empty page list response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep the member reset date inside the spend column
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): match email-only roster members for caller membership
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show team member budget reset date in UTC
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(teams): address review on caller membership enrichment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the caller's team spend line My Spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): bill OpenCode Go requests from the reported list_cost
OpenCode returns usage.opencode.list_cost on every response. Read it into the provider-reported cost the cost calculator checks before the price map, for chat and Responses, streamed and not. Keep the static rows as the fallback, and stop the generic usage.cost path from billing Go requests at 0 (usage.cost is what hit the prepaid balance, 0 inside the plan).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(core): resolve reported cost breakdowns without recursion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(opencode): cover reported list_cost spend through the proxy
Replaces the recursive _reported_cost_value with a flat _cost_number helper
so the code-quality gate passes, and adds an integration test that runs the
proxy against a scripted OpenCode upstream for a non-stream and a stream
chat completion, asserting the response cost header and the spend log row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): bill native /v1/messages from the reported list_cost
The native Anthropic Messages route priced OpenCode Go bodies from the price
rows twice over: the non-stream body is a TypedDict the response-cost header
path priced as-is, and the stream assembled its ModelResponse with the rows
cost already stamped. Both now read usage.opencode.list_cost, the non-stream
body through billable_result and the stream from the final message_delta.
The integration test now covers chat, responses and messages, stream and
non-stream, through the proxy against a scripted OpenCode upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): keep the stream cost stamp inside the lint gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(opencode): read list_cost off dict chunks and trust it only from OpenCode on /v1/messages
Stream builders receive chunks both as mappings and as ModelResponseStream
objects, so the list_cost scan now reads usage through chunk_usage. The
native Messages route only trusts a reported cost from opencode_go, every
other upstream keeps the price map, which also keeps a zero usage.cost from
being billed as free. Served bodies without stop_sequence still price, and
the new code stays inside the basedpyright gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): add disable_pre_routing_error_logs to skip spend rows for pre-routing rejections
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): decide pre-routing rejections by pipeline stage, not exception type
Guardrails such as AIM raise ProxyException(400), the same shape as an unparsable
body, so the status code list skipped their spend rows. The pipeline stamps
litellm_call_id only after auth, body parsing and model resolution, so a failure
reaching the hook without one died before any provider was considered.
_handle_logging_proxy_only_error flags those requests when it mints the call id,
and is_pre_routing_rejection keys off that flag plus ProxyModelNotFoundError.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): mark pre-routing rejections with a server-owned sentinel
request_data is the client body, so a boolean marker or a litellm_call_id in it
can be sent by the client to hide a guardrail block or force a spend row. The
failure handler now stores an object() sentinel whenever it finds no logging
object, and is_pre_routing_rejection checks identity, which JSON cannot forge.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): skip declared text-only deployments for image requests without enable_pre_call_checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): run the vision filter after the cooldown filter so a cooled vision leg falls back to the text leg
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): apply the vision filter after pre-call checks and tidy the new tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(router): mark the remaining locals in the vision filter tests Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(guardrails): add native Reco guardrail provider (pre_call)
Registers guardrail: reco as a first-class vendor integration, built on
the existing generic guardrail API wire contract rather than reimplementing
the HTTP client and payload mapping. Maps reco_tenant_id to the
X-Reco-Tenant-Id header and hardcodes fail_open/fail_on_error since those
should not be customer-configurable. v1 only supports mode: pre_call;
post_call and streaming redaction already work against the existing
generic pilot and are deferred.
Also overrides apply_guardrail to re-raise blocked-request errors with the
guardrail's own configured name: the base class hardcodes a module-level
name in that exception, which would otherwise mislabel every blocked Reco
request in logs.
* feat(guardrails): configurable Reco fail modes
Expose unreachable_fallback and fail_on_error on the Reco guardrail
and keep fail-open as the default. Values under optional_params win.
Top-level litellm_params values apply only when set explicitly, since
their defaults are fail-closed and would otherwise flip the default
Also type RecoGuardrail's forwarded kwargs with an Unpack'd TypedDict
instead of silencing ANN003
Signed-off-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(guardrails): align Reco with main's CI gates
Upstream moved unit tests from tests/test_litellm to tests/unit, so
test_reco.py was invoked by no CI job. Move it next to the other
guardrail tests
Replace RecoGuardrail's forwarded kwargs with explicit keyword
params, read optional_params through a pydantic adapter instead of
untyped dict/getattr access, freeze RecoOptionalParams, and drop
mutable-ok markers that suppress nothing, to stay within main's
type-discipline and basedpyright budgets
Signed-off-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(guardrails): fail closed by default in the Reco guardrail
Matches GenericGuardrailAPI and every other guardrail here: a Reco error or outage now blocks the request unless the admin sets fail_on_error: false. Tests cover the default, explicit fail-open, and top-level vs optional_params precedence
Co-authored-by: RecoLabs-YaronBratspiess <208601538+RecoLabs-YaronBratspiess@users.noreply.github.com>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): honor explicit fail-open on Reco when optional_params is a model
model_dump(exclude_unset=True) so pydantic defaults in optional_params no longer override an explicit top-level fail_on_error. Also drop the duplicate log_guardrail_information decorator, call the parent by keyword, freeze RecoConfigModel, and type the tests
Co-authored-by: RecoLabs-YaronBratspiess <208601538+RecoLabs-YaronBratspiess@users.noreply.github.com>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep Reco config model assignable and the hook decorator the gate requires
frozen=True on RecoConfigModel froze the shared LitellmParams mixin, which the guardrail registry assigns resolved secrets on, so every guardrail failed to load. Dropped it and added a test that LitellmParams stays assignable. Restored log_guardrail_information on the Reco override since tests/code_coverage_tests/check_guardrail_apply_decorator.py requires it on every hook. Test params typed as Mapping for LIT001
Co-authored-by: RecoLabs-YaronBratspiess <208601538+RecoLabs-YaronBratspiess@users.noreply.github.com>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(guardrails): format the Reco test file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Signed-off-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Yaron Bratspiess <yaronb@reco.ai>
Co-authored-by: MickeyShnaiderman-RecoLabs <mickeys@reco.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: MickeyShnaiderman-RecoLabs <77006830+MickeyShnaiderman-RecoLabs@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): buffer tool discoveries in redis and let the leader write LiteLLM_ToolTable in one statement
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): declare the redis family on the tool discovery buffer and drop the blind except and cast
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): satisfy LIT010 and LIT014 in the tool discovery buffer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): mark the aggregation fold Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): requeue drained tool rows to redis when the write provably did not apply
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drain tool discoveries on their own lock and scheduler job, not the spend commit
* chore(proxy): suppress the untyped apscheduler add_job on the tool discovery job
* refactor(proxy): move the LiteLLM_ToolTable bulk upsert into its own module next to the other bulk upserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): name the LiteLLM_ToolTable bulk write span so the prisma span check resolves it
* refactor(proxy): move the tool discovery drain out of DBSpendUpdateWriter into tool_registry_writer
* refactor(proxy): share db_write_failure_is_requeue_safe from exception_handler so the spend writer stays untouched
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): configure the default Usage time range
* fix(ui): retain the selected calendar preset during input sync
* fix(ui): preserve fallback dates when applying across DST
* refactor(typing): narrow Any sources across providers, proxy and guardrails (wave 1)
* refactor(typing): narrow Any sources across hubs, providers and proxy (wave 2)
* refactor(typing): validate RunwayML task JSON and let dotprompt loaders accept exported prompts
* fix: restore pass-through behavior on paths the typing pass tightened
MCP client IP extraction trusts X-Forwarded-For only when use_x_forwarded_for is True, dotprompt skips a malformed JSON prompt entry instead of raising, RunwayML task payloads pass through unvalidated, and a malformed provider body raises the same AttributeError as before at the CompactifAI, Gemini, and mock transport parse sites
* fix(runwayml): keep a malformed 2xx task body on the 500 path
* fix(edenai,cli): keep a non-object body in the error and fail on a malformed migration report
EdenAI's embedding and image generation parse dropped a non-object body from the error
they raised, so the message no longer named what the provider sent. A helper now raises
with the body in the message, and both paths have a regression test.
`lite encryption migrate` exited 0 with no message when the proxy's report was not an
object or its residual_legacy was not a number, where base crashed. It now fails with a
message naming the value.
The CompactifAI and Gemini image-edit non-object body tests accept TypeError or
AttributeError, which both base and this branch raise, so they guard against a
ValidationError instead of pinning one class. RunwayML's async parse gets the same
non-object body test as the sync one.
* fix: keep base's error classes, duck-typed checks and fail-closed guardrails the typing pass changed
* test: annotate the new regression tests' locals with Final
* test: annotate the remaining new test locals with Final
* fix(spend): await the per-request rollup enqueues and report their failures
The daily and entity spend rollups ran in an unreferenced asyncio task that
nothing awaited or drained, so a shutdown after the spend log was enqueued
could drop every rollup for that request, and a malformed metadata payload
dropped the request from all daily tables at debug level. The rollups now run
inline in update_database, each helper failure is logged through
spend_log_error with the request id, and unreadable metadata only costs the
row its metadata-derived fields
Resolves LIT-6027
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): type the rollup failure helpers for the lint gates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): track the rollup enqueue task so shutdown drains it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): shutdown drain waits for an in-flight rollup enqueue
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): register the rollup task without blocking the cost callback
Awaiting the rollups inline made a blocked queue put stall increment_spend_counters, so budget checks could run on stale counters. The task is now only registered via spawn_logging_task; drain_logging_tasks still waits for it on shutdown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): add LITELLM_OFFLINE_MODE to stop every request the proxy makes on its own
* fix(ui): name LITELLM_OFFLINE_MODE where the cost map panel says local mode is forced
* test: isolate the offline-mode tokenizer tests from the local Hugging Face cache
* test: cover the offline policy templates and the cached Kimi K3 tokenizer
* test(integration): count Kimi K3 tokens in the offline-mode egress test
* refactor(token_counter): skip non-bundled tokenizers in offline mode instead of listing downloaded ones
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(chat): port Moyai UI with Model Hub and images
* feat(chat): add Models sidebar with embedded Model Hub
* feat(chat): polish Model Hub with provider cards and logos
* refactor(chat): use a provider dropdown in Model Hub
* feat(chat): keep workspace switching visible with a Chat identity
* Add minimal opt-in Chat usage metrics and prioritize Chat in workspace selector
* fix(chat): enable usage metrics by default only when chat is used
* fix(chat): count requesting users with frontend-only metrics
* feat(chat): send default-on request metrics to litellm-tel
The router plants its deployment fields in litellm_metadata for Interactions
calls, while the proxy put the request's metadata (spend_logs_metadata, tags,
key fields) under metadata. The spend log resolves litellm_metadata and only
carries user_api_key* fields over, so spend_logs_metadata never reached
LiteLLM_SpendLogs. Route /interactions like /responses so both land in
litellm_metadata.
Fixes#47841
Co-authored-by: Igor Tavares <2163472+igorborgest@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(ui): sort every teams table data column
* fix(ui): order unlimited team limits as infinity and verify org scope
* fix(ui): validate team sort filters without recursion
* feat(ui): sort every users table data column
* fix(ui): order unlimited user budgets as infinity and verify org scope
* fix(ui): cover user sort queries and document pagination bound
* fix(ui): preserve Aria usage token breakdown and viewer model access
* fix(ui): make Aria conversations easier to read and reply to
* fix(ui): wrap long headings in assistant tables
* feat(ui): simplify Aria and add usage charts with PDF answer export
* fix(ui): retain full chart values in Aria PDF exports
* feat(ui): consolidate gateway and usage assistants as Aria Agent
* refactor(ui): give Aria an original blue sprite mascot
* feat(ui): prompt shared Aria from usage and retire duplicate usage agent
* fix(ui): keep Aria naming consistent and reachable across dashboard
* feat(ui): open Aria in a floating popup with suggested questions
* fix(ui): simplify the shared Aria Agent title
* feat(proxy): add LITELLM_DISABLE_LATEST_RELEASE_CHECK to stop the release check
The dashboard asks the proxy for the latest LiteLLM release, and the proxy fetches it from api.github.com on every cache miss. Deployments that block or audit outbound traffic had no way to turn that off. With LITELLM_DISABLE_LATEST_RELEASE_CHECK=true the endpoint returns null without making a request, which the dashboard already treats as no upgrade banner.
* fix(proxy): regenerate dashboard types for the release check flag
The endpoint docstring feeds the OpenAPI description, so the ui-api-types
check needs schema.d.ts regenerated. The docstring now says the flag skips
every outbound request, which stays true once the changelog feed is read
too
---------
Co-authored-by: Sujit Nirmal <nirmalsujit891@gmail.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
The Admin UI upgrade notice read GitHub's latest release, which the release
pipeline sets on its own, so admins were offered releases the changelog had
not announced yet and sent to the auto-generated GitHub release page. The
notice now reads the newest stable release from the docs changelog feed and
links to its changelog page. GitHub release notes are still used for the
feature and fix counts of that version's minor line.
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
