1117 commits

Author	SHA1	Message	Date
yassin	2bedefa567	feat(control_plane): service skeleton, settings, license gate, JWT auth and scope RBAC ... Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-14 21:58:52 +00:00
srik-straiker	ff997e6970	feat(guardrails): straiker stream_mode, per-phase logging and prompt-block replies (#51401) ... A Straiker verdict that blocks the prompt now comes back as a reply (HTTP 200, finish_reason content_filter) on chat completions, text completions, messages and responses, the way an answer-side block already does, instead of an HTTP 400 that some clients show as a malformed request. Routes that cannot send a reply, such as embeddings and /guardrails/apply_guardrail, keep the HTTP 400. A blocked answer on those routes is now an HTTP 400 too, where it was a 500. A backend failure still fails closed with an error. HTTP 408, 429 and 500 from Straiker now follow unreachable_fallback once retries are spent, and an attempt that times out is no longer retried. The new stream_mode option picks how a streamed response reaches the client. "buffered", the default, holds it until Straiker has checked the whole response. "streaming" sends chunks as they arrive and checks the assembled response at the end. Each v3 phase now writes its own guardrail log entry with the session, action, turn id and controls, in place of the generic entry that read "allow" even when the guardrail failed open. A claude-cli/ User-Agent is named Claude only for Claude Code entrypoints, so other clients that share the prefix are no longer misnamed.	2026-11-14 13:50:05 -08:00
mrinal-berri	b0c580d391	feat: email key owners before opted-in virtual keys expire (#51482) ... * feat: email owners before opted-in virtual keys expire Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: batch expiry alert queries and use branded email template Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: cast expiry alert query bounds as UTC timestamps Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: satisfy expiry alert CI contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: single-day key expiry alerts with global default and daily check - metadata.key_expiry_alert_days is one integer (1-365) instead of a list - general_settings.key_expiry_alert_days sets a default for every key with an expiry and owner email; key metadata overrides it - LITELLM_KEY_EXPIRY_ALERT_CHECK_INTERVAL_SECONDS defaults to 24 hours * fix: isolate key expiry alert setup and check every 6 hours - build the expiry alert manager in its own startup step that logs and returns None on bad config or setup errors, so later background jobs still register - default LITELLM_KEY_EXPIRY_ALERT_CHECK_INTERVAL_SECONDS to 6 hours * refactor: import litellm_enterprise email helpers directly in key expiry alerts Replace the import_module + Protocol module validation with plain imports, matching the key rotation hook and check_batch_cost. litellm_enterprise is absent from the type-check venv, so the imported names carry the same reasoned pyright ignore used in response_api_endpoints. * fix: apply global key expiry default to keys stored with null metadata Keys created without metadata (for example via /user/new) store JSON null, which failed metadata parsing and skipped the global key_expiry_alert_days default. Read null metadata as empty. * fix: guard litellm_enterprise import in key expiry alert job check_unsafe_enterprise_import requires enterprise imports outside the enterprise package to sit in try/except ImportError. Skip the run when the package is absent, matching the key rotation email hook. * feat: run key expiry alert scan on one pod per cycle, every 12 hours Take the shared PodLockManager lock (litellm_key_expiry_alert_job) like key rotation and expired UI session cleanup, so a single pod scans LiteLLM_VerificationToken each cycle; the per-key Redis claim still guards each email. Default LITELLM_KEY_EXPIRY_ALERT_CHECK_INTERVAL_SECONDS to 12 hours. * fix: scan for key expiry alerts when the job lock cannot be read from Redis acquire_lock returns False both when another pod holds the lock and when Redis errors. Skip the cycle only when another pod's id is on the lock key; otherwise scan without the lock, keeping the per-key claim as the duplicate guard, so a Redis blip cannot drop warnings for a 12-hour cycle. Replace the startup wiring assertion with a behaviour test against the proxy's pod lock. * fix: hold the key expiry alert lock for the whole cycle Each pod's timer fires at its own boot time plus jitter, so releasing the lock after a millisecond scan let every pod run each cycle. Take it non-reentrant for nearly the full interval and never release it, like the weekly spend report, so the first pod to fire runs the cycle and the rest skip. * fix: key the key expiry alert lock by clock window Pods' interval timers fire at their own boot time plus jitter, so a fixed lock name with a TTL shorter than the interval let a late-booted pod run twice in one interval, and a TTL equal to the interval made a pod skip its own next run. Name the lock after the clock window (int(now // interval) * interval, as file_usage_caps buckets its counters) with a one-interval TTL: the first pod in each window runs and the rest skip. Release the window if the scan raises so another pod can retry, and run anyway when the Redis circuit breaker is open. * test: keep key expiry lock tests in-process with a fixed clock Simulate the Redis outage with a disconnected in-process fakeredis server (wired like the router_redis fixture) instead of a socket to a closed port, and give the startup-built job the test clock instead of the wall clock, per tests/unit/AGENTS.md. * fix: declare the key expiry alerts service family on process_alerts process_alerts reads the window lock holder from Redis directly, so test_every_function_that_reaches_redis_directly_declares_its_family requires it to carry its family, as _process_key already does. * refactor: align key expiry alerts with existing litellm surfaces - read general_settings.key_expiry_alert_days on every run via _current_general_settings(), like the performance alert and spend capture jobs, so a UI or API edit applies without a restart; an invalid value is logged and ignored - share the event name as VIRTUAL_KEY_EXPIRING_EMAIL_EVENT in litellm/types, like MODEL_DEPRECATION_EMAIL_EVENT - default the Virtual Key Expiring email event off, like the other scheduled emails - name the Redis family for the claim data (key_expiry_alert_claims) and read the window lock under POD_LOCK_TARGET, mirroring reset_budget_job._lease_is_held - give the per-key check its own validator, run with the logging check through raise_on_invalid_key_metadata * chore: drop two comments that restate the code --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-14 13:47:16 -08:00
tavren27	9063ba6929	fix(proxy): stop wildcard aliases over unknown orgs listing every catalog id (#51805) ... * fix(proxy): stop listing catalog ids under an org the catalog lacks for every provider whose path is its model id * fix(proxy): keep listing Fireworks router and Azure model_router wildcards The catalog cannot price the names these routes call, so the strict org check dropped callable ids. They now list as before, and every exempt provider has a test row that fails if it leaves the list * fix(proxy): check unknown orgs beneath router routes and keep firerouter pools * fix(proxy): keep listing Fireworks routers/firerouter pools and Azure AI speech wildcards * fix(proxy): check unknown orgs under OpenAI and Cohere and drop unreachable Fireworks router ids * fix(proxy): keep the full wildcard listing for deployments behind a custom api_base * fix(proxy): count a provider base URL set in the env or litellm.api_base as a custom endpoint * fix(proxy): match a gateway env var on any part of the provider name * fix(proxy): count a provider's off-pattern base URL env vars as a gateway * test(proxy): cover every off-pattern base URL env var in the gateway table --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 16:34:59 -05:00
norkelbyte	fd6e4fb6a5	fix(sdk): stop global model_fallbacks from recursing on every completion call (#51817) ... * fix(sdk): stop global model_fallbacks from recursing on every completion call With litellm.model_fallbacks set, acompletion and completion read the global chain on every call, including the calls the fallback runner itself makes for each attempt. Each attempt started a new fallback run, so the calls grew into a tree until Python hit RecursionError and no provider was ever reached. The fallback runner now marks every attempt with an internal _litellm_ kwarg, and one resolver in fallback_utils decides the chain for a call. A marked attempt gets no chain. Otherwise a per-call fallbacks value wins, including an empty list to opt out, and the global list is used only when the call sets none. An empty chain skips the runner. The resolved chain is what the runner gets, so completion's sync path passes it explicitly. The marker is dropped from provider params and cache keys like every other _litellm_ kwarg * fix(fallbacks): keep each provider's credentials, typed errors, sync streams and one success log A fallback to another provider drops the original request's credentials and endpoint, and a fallback to another model drops per-call custom pricing. When every attempt fails, the original model's typed error is re-raised. The sync runner calls litellm.completion directly so sync streams work, and the sync wrapper leaves the success log to the attempt inside the runner. Health checks never fall back, and every attempt keeps the caller's litellm_call_id. * fix(fallbacks): set up fallbacks only when tried and keep health checks off the global list in every mode Each fallback is now built only when the runner reaches it, using the provider lookup that never starts a ChatGPT or GitHub Copilot device login, so an unused fallback cannot block a healthy call Health checks run inside a scope that turns the global model_fallbacks off, which also covers modes bridged to chat. The per-site fallbacks=() overrides are removed The sync logging test now forces the inner and outer success handlers to overlap with a barrier instead of sleeping * fix(fallbacks): keep guardrails on sync attempts and keep the global list out of Router calls The sync fallback runner now runs the deployment pre and post call hooks around every attempt, so guardrails a sync call names still run on each model, and it skips the post hook for streams like the async wrapper does Router deployment calls run with the global list turned off, so on the proxy the Router owns fallbacks and a fallback answer is never priced or logged as the failed deployment A fallback to another provider now also drops the request's headers and litellm_credential_name, and an original model whose provider cannot be looked up counts as a switch When every model fails the caller gets main's documented "All fallback attempts failed" error again * fix(fallbacks): keep custom provider settings for unprefixed fallbacks and never cache a rejected answer An unprefixed fallback whose name resolves to no provider on its own now inherits the request's custom_llm_provider, so a fallback on the same OpenAI-compatible server keeps its api_base and key as it did on main Sync fallback attempts no longer write to the cache, since their guardrail runs after the inner call returns. The outer call still caches the answer that passed, so a rejected answer never lands in the cache * fix(fallbacks): run a sync attempt's output guardrail before it is cached or logged The sync runner ran the post-call guardrail after litellm.completion returned, so the attempt's success log already held the raw answer and a rejected answer was logged as a success. The sync wrapper now runs the guardrail for a fallback attempt at the same point the async wrapper does, before the cache write and the success log, which also replaces the no-store workaround * fix(fallbacks): keep search grants, deployment hooks and logs intact Router deployment calls wrap their handler to turn the global list off, which hid asearch from the search fallback grant check. The check now unwraps the handler down to asearch A failed sync fallback attempt runs the post-call failure deployment hook again, as main did through its async detour atext_completion through the global list uses the async runner so it logs one success, and check_valid_key turns the global list off so a fallback answer never makes a bad key look valid * fix(proxy): keep the global fallback list off on direct SDK calls Pass-through models and routes the Router does not serve reach litellm directly, so the global list ran there with no check that the caller's key may use the fallback model. Those calls now turn the global list off, as Router deployment calls already do * fix(fallbacks): price a fallback by its own model and skip the proxy global list A fallback to another model kept the original call's base_model, so its answer was billed at the original model's rates, as on main. base_model now drops with the other per-call pricing params The proxy turns the global fallback list off on every model call, so litellm_settings.model_fallbacks now logs a startup warning that points to router_settings.fallbacks and is not applied --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 16:28:38 -05:00
kirti22jain	73e17cba74	feat(proxy): add per-model project batch enqueued token limits (#51608) ... * feat(proxy): add per-model project batch enqueued token limits Enqueued batch token limits only read key and team metadata, so a deployment that gives each application its own project under a shared team cannot cap one application's queued batch tokens for a given model. The team limit is shared by every project in it, and JWT or custom-auth callers get a new key scope with every token. Read model_batch_enqueued_token_limit from project metadata, keyed by model like model_tpm_limit. Each limited model group the batch runs is reserved against only its own rows' tokens, rows that name no model count toward the routing model, and the reservation records each scope's charge so the terminal-state refund releases exactly that. Only proxy admins may set it, on project create and update as well as on keys and teams, so a team admin cannot lift their own project's cap. * fix(proxy): charge the routing model and keep daily caps for project batch limits Rows can name any model while the batch runs on the routing deployment, for example LiteLLM-executed batches replace every row's model with the run model, so charging only row models let a batch skip a capped deployment's per-model limit. Charge the routing model the whole batch as well as each other limited row model its own rows' tokens. An enqueued-token reservation replaced every batch counter, so a project limit also skipped the key, team and end-user daily token caps. Keep checking those daily caps after reserving, and refund the reservation if one rejects the batch. * fix(proxy): keep per-minute checks for batches a project allowance does not cover A per-model project allowance replaced every per-minute batch check as soon as any row's model had one. A batch routed to an unlimited model could add a single tiny row naming a limited model and skip the routing model's RPM/TPM and project ITPM/OTPM quotas, although every row runs on the routing model. Waive those checks only when the reservation covers the whole batch, through a key or team allowance or a project allowance on the routing model; otherwise reserve the covered models and run the normal checks too. * test(proxy): cover the project per-model batch cap through custom auth on a live proxy * fix(proxy): keep key and team batch allowances authoritative and harden project limits Key and team enqueued allowances again replace the other batch checks. A project allowance on the routing model keeps only the daily caps. Team admins can only be blocked from the project key, invalid project maps answer 400, the edit modal keeps stored limits, aliases share their tightest limit, file-bound batches charge only the routing model, deployment ids resolve to their model group, and reservation records keep project scopes apart so older pods still refund key and team scopes. * fix(proxy): charge row models for model-bound batch uploads Only a managed-file upload rewrites rows to the deployment model. A model-bound upload keeps each row's model, so its rows still count toward the project caps of the models they name. * fix(proxy): skip per-minute batch limits only when the project caps every model in the batch A project cap on the routing model alone used to switch off the project's per-minute input and output limits for the other models a batch file names. The skip now needs a project cap on every one of those models. -latest names are charged under the cap of the model they point to, the limit map only takes JSON integers, and a /project/update that sends only top-level fields keeps the stored metadata instead of replacing it. * fix(proxy): lift per-minute limits off managed batches under a project cap and edit the cap in the Admin UI * fix(ui): keep the project batch cap read-only for team admins and require a whole number above 0 * fix(ui): narrow project metadata directly when keeping the stored batch cap * fix(proxy): charge dated snapshot rows and header or query routed batches to project caps --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 14:36:53 -05:00
tavren27	8fc6789295	chore: move LiteLLM's configs and e2e tests off gpt-4o and gpt-4.1 before their 2027-01-14 shutdown (#51800) ... * chore: move LiteLLM's configs, cookbooks and e2e tests off gpt-4o and gpt-4.1 before their 2027-01-14 shutdown OpenAI shuts down gpt-4o, gpt-4o-mini, gpt-4.1 and gpt-4.1-mini on 2027-01-14. The example proxy configs, cookbook guides and e2e suites now call gpt-5.6, gpt-5.6-luna or gpt-5.4-mini, and the predicted-outputs e2e test is removed because no current OpenAI model accepts prediction * docs(cookbook): name gpt-5.6 instead of gpt-4o as a tool-capable model in the gollem troubleshooting tips * chore(ci): delete the observatory proxy config, which nothing has loaded since its workflow was removed * chore: send max_completion_tokens in batch lines and move README and deploy examples to gpt-5.6 gpt-5.6-luna rejects max_tokens on OpenAI's batch lines, which reach OpenAI unchanged, so the batch JSONL builders send max_completion_tokens. The README quick start, the adaptive router README and the Helm and Terraform example configs move off gpt-4o with the configs * chore: pin output prices next to input prices in the adaptive router example The adaptive router reads only input_cost_per_token from config, and a deployment priced by input alone bills output at $0, so the example and its README pin both at the cost map's current prices * chore: move the configs and e2e tests to gpt-6-luna and gpt-6.1-sol The cost map names gpt-6-luna as the successor of gpt-4o, gpt-4o-mini, gpt-4.1 and gpt-4.1-mini. Cheap roles move to gpt-6-luna and full-size and vision roles to gpt-6.1-sol, the pair main's other configs use. The adaptive router example pins the gpt-6 prices, and the installer quick start and two READMEs drop gpt-4o --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 10:50:59 -08:00
mika-lt	cad4570bbe	fix(scim): skip groups with no team so a PATCH or PUT still deactivates the user (#51829) ... * fix(scim): skip groups with no team so a PATCH or PUT still deactivates the user A group the IdP pushes before LiteLLM has a team for it made the roster write fail, which answered 404 before the user row was written, so the user stayed active and their keys kept serving. #37700 made that failure fatal. Groups with no team row are now logged and left out of the roster write, the persisted teams, and the role resolution. Group ops after a replace now apply in order, as RFC 7644 section 3.5.2 says. * fix(scim): drop the redundant unchanged-groups return and cover the warning and PATCH admin role * fix(scim): strip line breaks from the teamless-group warning * fix(scim): read the PUT role from every requested group, as POST does A PUT naming only a group with no team no longer demotes an admin whose admin group has no team. The new tests also get typed fixtures and Final locals * fix(scim): look up the requested teams on the primary database, as the roster write does --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 12:06:52 -05:00
giladk1221	9d827093c5	feat(github_copilot): support GitHub App authentication with PEM upload (#51427) ... * feat(github_copilot): support GitHub App installation authentication * Add GitHub App authentication to the Copilot credential form * Explain secure private-key setup beside Copilot App fields * Store uploaded Copilot App keys through encrypted named credentials * fix(github_copilot): handle App validation errors and credential replacement * fix(ui): make GitHub App field help accessible * fix(github_copilot): preserve saved App key references on edit * fix(github_copilot): trust litellm's CA settings when minting App tokens and hide App keys from /health * fix(github_copilot): keep device-login headers, mint App tokens off the shared thread pool, cache App responses Device-login requests keep the merge base header behavior, so caller headers still replace the Copilot defaults. Only App requests merge and pin their identity headers. App token refreshes are single-flight per cache key with a shared future. Only the refreshing caller uses a thread, so waiting callers no longer fill the default executor that every provider call shares. A cached App token is served before the key secret is read. Uploaded keys still rotate the token when their contents change. App deployments are deployment-owned, so they share the response cache like shared device-login deployments. App credentials are validated even when credential_info names no provider. * fix(github_copilot): finish an App token refresh even when the caller that started it is cancelled The caller that owns a refresh used to await asyncio.to_thread, so cancelling it while the job was still queued cancelled the job, left the in-flight future unfinished, and every later request for that App waited on it. The refresh now runs in the executor without being tied to the caller, and every caller, the owner included, awaits the shared future * fix(github_copilot): refuse deployments naming an unloaded credential, keep App key references, accept numeric App IDs A Copilot deployment whose litellm_credential_name is not loaded now answers 400 instead of falling back to the shared device login. The router registry keeps github_copilot_app_private_key_ref values as references, a YAML App ID can be an integer, the server-owned field error names Copilot auth for Copilot fields, and the credential edit modal shows the stored provider again * fix(github_copilot): report a deployment naming an unloaded credential as unhealthy The /health check now answers unhealthy with the same 400 the request path returns for a Copilot deployment whose litellm_credential_name is not loaded, instead of running it through the shared device login * fix(github_copilot): keep the base fallback for an unloaded credential and report the App host as api_base A Copilot deployment that names an unloaded credential uses the shared device login again, as on base. A deployment that sets github_copilot_auth_type: github_app still refuses before any GitHub call. get_api_base now returns the Copilot host an App or per-user session calls instead of an admin api_base it ignores. * fix(github_copilot): count a GitHub rejection of the App installation token as a Prometheus deployment failure * fix(github_copilot): keep per-user session api_base reporting as on base and name Copilot in the non-admin 403 Only an App deployment reports its App host as api_base before a configured api_base. Per-user OAuth sessions report the same api_base they did on base. The 403 for a non-admin attaching App credentials now names GitHub Copilot authentication. The token-dir fixture in the App api_base test is applied with usefixtures. * fix(github_copilot): report the configured api_base instead of raising when GitHub App settings are incomplete * test(github_copilot): type the incomplete App settings test parameter --------- Co-authored-by: Gilad Katz <29776916+giladk1221@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 10:33:20 -05:00
Michael Bennett	fd6c00200c	feat(decisions): support Vercel AI Gateway decision models (#50642) ... * feat(decisions): support Vercel AI Gateway decision models * test(decisions): cover Vercel AI Gateway dispatch through the proxy * feat(decisions): add Vercel AI Gateway cost map rows for Jev * fix(decisions): bill Vercel AI Gateway decisions at the charge Vercel reports Vercel returns the charge for every call under provider_metadata.gateway.cost, already net of cache discounts, so unpriced evaluation models and luna cache hits now bill what Vercel charged instead of zero or the full input rate. Jev 1.14.0 and the jev alias also take box questions, as Vercel serves them * fix(decisions): bill Vercel gateway charges with margin and BYOK, forward providerOptions Decisions now records a provider's reported charge as upstream_usage_cost, so the operator's cost discount and margin apply to it like any other provider-billed cost. Vercel calls served on the caller's own provider key bill marketCost on top of the gateway fee, the cache split comes from the upstream usage Vercel nests under provider_metadata, providerOptions reach the gateway, remote images are inlined, and typesafe-ai/jev, which reads text only, refuses or drops images * fix(decisions): stop forwarding caller providerOptions to Vercel and send Jev images Decisions no longer sends a caller's providerOptions to Vercel. Gateway routing and provider keys are set on the Vercel team instead. Vercel serves typesafe-ai/jev images at 768x768 (the earlier 400 came from a 96x96 image TypeSafe rejects for its size), so the Jev rows read images and the per-model image refusal is removed. * fix(decisions): report provider charges on the cost header and read Vercel's cache split from the serving provider Decisions goes back to reporting a provider's own charge through the response cost header, the way main does for OpenRouter. Moving it onto upstream_usage_cost dropped every cost header on free calls, priced base_model deployments from the cost map, and put the margin in the response header but not in the SystemOne body cost. Vercel's cache split now comes from the provider_metadata entry of the provider that served the call, and only when that entry reports input_tokens_details. Any credentialType value is accepted. The Jev 1.13.0 and 1.14.0 rows record their image probes. * test(decisions): type the async image fetch fixture in the Vercel inline image test --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 09:43:08 -05:00
norkelbyte	ac1cc5111c	fix(proxy): run baggage tag mappings and budget checks on /queue/chat/completions (#51753) ... * fix(proxy): run baggage tag mappings and budget checks on /queue/chat/completions * fix(proxy): check the key's max_budget on /queue/chat/completions * fix(proxy): run budget, tag, and request policy checks on queued chat calls * fix(proxy): bill a dropped queued stream and run the trace ID and failure logging checks on queued calls A queued stream now hands its logging object to the stream generator, so a client that leaves mid-stream is billed for the reply it already got, and a stream left before the first model chunk releases its budget reservation. A team's require_trace_id and the failure logging for rejected calls now use the same route check as the other inference checks. * fix(proxy): settle a dropped queued stream's budget hold through the first-chunk step chat uses * fix(proxy): keep the trace ID, call ID, unpriced fallback filter, and JWT admin team header on queued chat calls * fix(proxy): send the call ID on a queued call the provider rejects before streaming * fix(proxy): release a dropped stream's budget hold only when nothing bills it A guardrail that buffers the stream can pull model chunks the client never saw. The early-drop release settled the hold at the input cost first, then the partial bill for those chunks added its full cost on top. The release now runs after partial billing and only when no bill fired * fix(proxy): fold the caller's litellm_metadata into a queued call's metadata A queued call that sent a litellm_metadata bucket, even an empty one, made tag routing and logging read that bucket, so the key and team tags the handler merged into metadata no longer narrowed deployments. The queue handler now folds the bucket into metadata and drops it, as chat does, after stripping the proxy-owned user_api_key_* and internal control fields from both buckets, so a caller can no longer set the queued spend row's organization ID either * fix(proxy): release a queued call's budget hold when the call is cancelled before it answers * fix(proxy): refund a cancelled queued call's hold in full until the model call claims it * fix(proxy): keep a cancelled queued call's hold release running when the call is cancelled again --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 08:54:54 -05:00
tavren27	27c796c412	fix(proxy): count the request's tool_choice on the token count routes (#51133) ... * fix(proxy): count the request's tool_choice on the token count routes /utils/token_counter, /v1/messages/count_tokens and /v1/responses/input_tokens now pass tool_choice into the local estimate, and every DeepSeek model counts it as 0, as its providers bill * fix(token_counter): count a request with no model name again * fix(token_counter): count grok tool_choice the way xAI bills it xAI bills grok-4.3 and grok-4-1-fast-reasoning 21 tokens for "none", 11 for "required" and 12 plus the name for a named tool, where the OpenAI costs counted 1, 0 and 7 plus the name. The grok-4-1 and grok-4.3 chat template tool sections now carry those costs, so a grok count matches its bill for every tool_choice * fix(token_counter): count xAI's grok tool_choice costs only where xAI serves grok Azure AI Foundry bills grok-4-1's tool_choice differently, so grok on other hosts keeps counting it as 0. A tool_choice sent without tools counts nothing, since OpenAI and xAI reject it * fix(token_counter): key xAI's grok tool_choice costs on the provider that serves the deployment The count routes strip the provider prefix before counting, so Azure, Vertex AI and Perplexity grok matched the xAI name pattern. The routes now resolve the serving provider with get_llm_provider and drop xAI's costs when it is not xAI or OpenRouter's x-ai route. The costs move onto the grok tool sections so one pattern decides them * fix(token_counter): count no tool_choice for Gemini and none without tools in the Rust counter Gemini bills the same prompt tokens for every tool_choice, so it skips the OpenAI choice costs the way DeepSeek does. The Rust reservation counter now adds tool_choice tokens only when tools are present, matching the Python counter * fix(proxy): count no tool_choice a deployment drops, and treat an api.x.ai api_base as xAI A deployment that lists tool_choice in additional_drop_params never sends it, so the count routes no longer add its cost. An openai/ model whose api_base is on api.x.ai is served by xAI, so it gets xAI's tool_choice costs like xai/ grok * fix(proxy): count no tool_choice a request drops in its body /v1/messages/count_tokens takes additional_drop_params from the request body, and inference drops tool_choice when it is listed there, so the count no longer adds its cost either * fix(proxy): take additional_drop_params on /utils/token_counter and treat a malformed api_base as not xAI /utils/token_counter now passes a request body's additional_drop_params into the count, so a tool_choice the request drops counts as 0 there like on the other two count routes. An api_base that does not parse falls back to the deployment's provider instead of failing the count with a 500 * fix(proxy): take chatgpt and github_copilot providers from the model name in the count host check Resolving them through get_llm_provider read the saved login, logged a warning and could refresh a token on every count with a tool_choice * fix(proxy): skip a tool_choice or additional_drop_params too deeply nested to validate on the count routes A value nested 300 levels deep failed TokenCountRequest validation, so the count routes answered 422 or 400 where they used to count the request without it * fix(bedrock): count Kimi K3's forced tool_choice message on the exact count path Kimi K3's chat template adds a tool-choice system message when Bedrock gets toolChoice any or a named tool, and Bedrock bills it (170 input tokens unforced, 206 forced). The exact count path now passes tool_choice through the same Converse mapping inference uses and renders that message * test(proxy): type the Kimi K3 count test's tools parameter * perf(count_tokens): validate each kept count option once instead of twice * fix(dot_notation): parse drop paths in linear time when brackets never close * test(count_tokens): benchmark the unclosed-bracket drop path instead of timing it in a unit test --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 07:53:27 -05:00
quietjules	007d1c1b5e	feat(proxy): support budget_limits windows on internal users (#51763) ... * feat(proxy): support internal user budget windows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): make user budget migration repeatable and sync API types Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): bound lock wait for user budget migration Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): omit unset JSON windows when creating users Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(auth): retain personal budget windows for proxy admins Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): give user budget windows their own counters and keep them on the user User window counters move to spend:user_window:<id>:<duration> in Python and Rust, so a user whose id ends in 🪟<duration> can no longer fill another user's window. /user/new stores the windows on the user only, clearing them stores null like keys, edits keep each window's stored reset, and the all-users bulk update goes through the per-user path * fix(proxy): keep windows on team auto keys and write all users' windows in one transaction * fix(proxy): clear spend counters on bulk spend reset and read bulk windows under the lock An all-users bulk update that resets spend now clears each user's spend counters, so users are not left blocked by the old counter. The bulk path reads each user's stored windows after it takes the users lock, so a concurrent edit cannot be overwritten with stale resets. JWT users created from default_internal_user_params get a reset_at on each default window, and the user window error names the spend and the limit like key and team errors do * fix(proxy): give member-add users the default budget windows and bulk-update the users read under the lock Team and org member add now copy default_internal_user_params.budget_limits onto the users they create, each window with its own reset_at, through the same helper the JWT upsert uses The all-users bulk update now writes, evicts and reports the users it read after taking the users lock, so a user created before the lock gets the new limits * test(proxy): check default window resets against the reset helper instead of the wall clock * fix(proxy): return parsed windows from the all-users bulk update and chunk its window writes * fix(proxy): fail an all-users bulk update whole when its transaction fails The per-user fallback after a failed all-users transaction wrote the newest user and then crashed on a missing user_email attribute, so the call answered 500 with one user changed. Racing window bulk updates past the 5s transaction limit reached it. Answer 500 with no user changed instead --------- Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-14 07:44:35 -05:00
paveln83	7d4c0ab91a	fix(proxy): retrieve and delete a managed file when the request also names a model (#51780) ... * fix(proxy): retrieve and delete a managed file when the request also names a model GET and DELETE /v1/files/{id} sent a proxy-managed file id to the provider unchanged whenever the x-litellm-model header or ?model= named a model, so the provider answered 404 and DELETE left the file in place. A managed file id now skips model-based routing and resolves to each deployment's provider file id, as the content route already did. * fix(proxy): keep forwarding a managed file id by the model hint on a proxy without a database A proxy with no database loads no managed files hook, so it cannot resolve a managed file id itself and must keep sending it to the named model, which may be another LiteLLM proxy that owns the file. Skip model routing for a managed id only when the managed files hook is loaded. The tests now run the real managed files hook over a stored row with only the OpenAI file routes faked, and cover the no-database proxy. * test(proxy): type the managed file table helper's delete filter * test(proxy): keep a raw provider file id routed by the model hint when managed files are on A proxy with a database loads the managed files hook, which now skips model routing for its own file ids. This test makes sure a raw provider file id sent with a model hint on that proxy still goes to the hinted deployment with its credentials, so the skip cannot widen to every id unnoticed * test(proxy): cover a model hint that names a model the managed file is not on A hint naming another deployment must still resolve the managed id through the managed files hook, so these cases fail if routing is skipped only when the hint matches the file's own model --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 04:55:36 -05:00
jesus-berri	3988ce2865	fix(proxy): honor cancel_on_disconnect on the rerank endpoints (#51252) ... * fix(proxy): honor cancel_on_disconnect on the rerank endpoints * test(proxy): cover a connected rerank succeeding with cancel_on_disconnect on * test(proxy): cover a connected rerank answer through the cancel watcher in the integration rig * fix(proxy): read cancel_on_disconnect on rerank the same way the chat path does * fix(proxy): set up the rerank request logger before routing so a hang-up after dispatch logs the query's tokens like chat * fix(proxy): keep flag-off rerank logging unchanged and price a cut-off rerank from its query and documents The rerank endpoint now builds its request logger only under cancel_on_disconnect, so a flag-off rerank logs one unbilled 408 row as on main. The failure estimate for rerank counts the query and every document sent, prices the deployment the router picked, and bills one query on a model priced per query. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 03:59:22 -05:00
Yogeshwaran Ravichandran	35b9df1f40	feat(mistral): add native upstream token counting support (#50652) (#50653) ... * feat(mistral): add native upstream token counting support * test(mistral): standardize on mistral-large-4 and add tool calls tests * fix(mistral): unify credential resolution and guard Azure endpoint boundary * fix(mistral): resolve api base from provider env vars and route responses count natively * fix(test): use groq for bridged proxy tests and increase mistral coverage Switch bridged provider tests in proxy token counter from mistral to groq, with comments noting that PR #50653 added native mistral token counting so groq tests the local estimate fallback. Also add test coverage for mistral error responses, invalid json, and system instruction blocks to satisfy codecov. * fix(lint): fix unit package layout, ruff strict rules, and type discipline in mistral token counter * fix(mistral): suppress basedpyright warnings on untyped httpx handler * fix(mistral): keep chat credential resolution and count with the key chat resolves The token counter now resolves its Mistral key and base through the same helper chat uses, so a count goes to the endpoint chat would call. A deployment that would route to Azure, or an Azure-only env key, falls back to the local estimate instead of sending that key to Mistral. Chat key and base precedence is unchanged from main. * fix(mistral): prefix live /v1/models ids with mistral/ so listed ids route * fix(mistral): count follow-up turns, replayed reasoning, and file_id files the way Mistral chat sends them Mistral has no native Responses API in litellm, so /v1/responses inference runs through the chat bridge, which restores previous_response_id history. The count now takes the same bridge instead of forwarding previous_response_id to Mistral's stateless count endpoint, so the separate Responses count method is gone. The chat-format count adds a reasoning item where Mistral chat replays reasoning as a think chunk, and an input_file part for files sent by file_id, so both reach Mistral's count the way inference sends them. Tests now go through the real handler with respx, and the integration Mistral peer serves the count route. * fix(mistral): leave empty assistant turns out of the count, as Mistral chat does Mistral chat drops an assistant turn with no content and no tool calls before sending, and Mistral's count endpoint rejects one. A follow-up whose stored turn spent its whole output budget on reasoning hit that 400 through previous_response_id, so the count now skips the same turns chat skips. * fix(mistral): count what chat sends and keep the bundled Mistral model list A user turn that mixed file_id files with an image or an inline file lost the image or inline file in the count, since the converted content arrives as a tuple and only lists were read. Both content readers now take either shape A /v1/responses count with a json_schema text.format now sends that format to Mistral, so the count matches what the bridged chat call bills Counts for Mistral models that are not chat models (embeddings, OCR, transcription) now fall back to the local estimate instead of returning Mistral's 400 Count calls now use the deployment's timeout when one is set, and a count response without input_tokens falls back to the local estimate MistralConfig.get_models goes back to the bundled model list, so turning on check_provider_endpoint does not change what /v1/models shows for mistral/* The previous_response_id count test now fakes the spend logs table instead of the history lookup, so the real lookup and its query run * fix(mistral): count tool_choice, the magistral reasoning prompt and system messages like chat, and never send the Azure key to Mistral * fix(mistral): answer 408 on a count timeout and use the shared deployment timeout Main's native counters answer a count timeout with 408 and a connection error with 500. The Mistral counter fell back to a local estimate on both, so a hung Mistral endpoint looked like a successful count. It now raises the same count errors, and it reads the deployment timeout with the shared deployment_timeout helper instead of its own copy * fix(mistral): join count text blocks the way Mistral chat does * fix(mistral): count file ids in order, drop the empty turns chat drops, and join tool result text like Mistral chat * fix(mistral): count the thinking chunks Mistral chat replays in assistant history The OpenAI-format count now takes Mistral chat's own replayed turns, so a Magistral assistant turn whose content carries a native thinking chunk is counted as the reasoning chat sends, and a reasoning_content beside that chunk is skipped the way chat skips it --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 03:19:57 -05:00
tavren27	c4c16c7b3f	fix(azure): drop the traceback from stream-cut errors sent to clients and alerts (#51760) ... * fix(azure): drop the traceback from stream-cut errors sent to clients and alerts An Azure chat or text completion stream cut mid-response raised an APIConnectionError whose message carried the full Python traceback, so the in-stream error frame, the /v1/completions 500 body, and every llm_exceptions alert showed server file paths. The Azure no-status branch now keeps the error line alone, and route_failure_proxy_exception redacts the message it answers clients with, the way the chat route already does * fix(proxy): keep route error messages as raised and test the Azure stream cut through the proxy routes Drop the route-level client message redaction, which rewrote valid guardrail and hook 400 messages, and cover the Azure chat and text stream cut end to end through respx * fix(azure): name the error type when a connection error has no message An httpx reset raises ReadError with an empty message, which left the client line with no cause once the traceback was gone * test(proxy): drop the docstring from the Azure stream cut fixture * test(proxy): stop pinning the MidStreamFallbackError prefix in the Azure text stream cut test --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 08:00:58 +00:00
tavren27	d76cdcbb8a	fix(proxy): list key and team aliases on /model_group/info and apply listing callbacks to them (#51754) ... * fix(proxy): list key and team model aliases on /model_group/info * fix(proxy): keep listing-callback and pause filters on model alias rows A name a legacy or V2 listing callback hides stays hidden on /v1/models, /v1/models/{model_id} and /model_group/info even when it is also an alias. V2 Hook.list_models subscribers now limit which target an alias row describes, /model_group/info describes no alias by a paused model, and ?model_group=<name> offers listing callbacks only the names that row reads. * test(proxy): annotate the new alias test locals as Final * fix(proxy): keep a caller alias listed when a router alias of the same name points at a hidden model A key or team alias always wins over a router model_group_alias of the same name, so alias rows are now dropped only for names a listing callback hid itself, not for router aliases whose target it hid * fix(proxy): describe an alias by an unlisted target only when listing callbacks keep it Listing callbacks are now also offered the unlisted model each listed key or team alias sends requests to, and the model a router alias of that name resolves to. The alias row is described by that model only when the callback keeps every one of those names, and a name the callback hides there hides no listed name. Before, any installed listing callback, V2 included, left such an alias with no limits and a 404 lookup * fix(proxy): describe an alias to a team model's public name when the key lists its internal name /model_group/info lists team models by their internal routing name, so an alias to the public name found no listed target and got no row while /v1/models listed it. Alias resolution now reads the listed names by their public names. The listing callback docstring also names litellm.model_alias_map among the alias sources. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 01:10:32 -05:00
zoveki	8973e2dc4f	feat(vertex_ai): label Vertex requests with the caller's key, team, org, and project (#50732) ... * feat(vertex_ai): label Vertex requests with the caller's key, team, org, and project Adds the opt-in litellm.vertex_request_label_fields setting, the Vertex twin of bedrock_request_metadata_fields. When set, Gemini, embeddings, Imagen and Gemini image generation, rerank, and Claude on Vertex (chat and /v1/messages) send the listed proxy identity fields and spend_logs_metadata as request labels, identity first, with the user_api_key_ prefix reserved so a caller cannot forge or evict them. Identity text is rewritten to fit Google's label rules. Gemini image generation now forwards labels at all. * feat(vertex_ai): send the opt-in identity labels on multimodal embeddings, the Google GenAI route, and batches With litellm_settings.vertex_request_label_fields on, the multimodal embedding predict body, the native generateContent and streamGenerateContent body, and the BatchPredictionJob now carry the key's identity labels the same way chat does Caller labels on GenAI are resolved after the extra_body merge, so neither a top-level labels field nor extra_body.labels can write the reserved prefix. Batch labels come only from the proxy's litellm_metadata, never the caller's OpenAI batch metadata. With the setting off every route sends the same body as before * fix(vertex_ai): keep identity labels when a Gemini caller sends extra_body labels * fix(vertex_ai): label Interactions, image edits, Mistral, and the /vertex_ai pass-through, and gate Gemini image labels on the setting * fix(vertex_ai): move body labels into the labels header on Vertex rawPredict pass-through calls * fix(vertex_ai): send Mistral extra_body labels in the labels header instead of the body Labels a caller nests in extra_body now merge over top-level labels before the X-Vertex-AI-Labels header is built, the way the Gemini body merges them, and leave the outbound body. The token-fetch stubs in the Mistral and multimodal embedding tests are scoped to their instance so later tests see the real method * fix(vertex_ai): limit pass-through labels to POST and to publishers that take the labels header rawPredict to custom endpoints and publishers other than Anthropic and Mistral, and PATCH or PUT bodies, now reach Vertex as the caller sent them. The pass-through drops a caller's labels header where it writes body labels, a single field written as text turns the setting on for that field, and an undecodable Claude labels header falls back to the caller's labels * fix(vertex): label Veo video, Gemini transcription, and pass-through predictLongRunning requests * fix(vertex): read the pass-through publisher from the end of the upstream path A project id named publishers shifted the first-match lookup, so a forged X-Vertex-AI-Labels header reached Claude or Mistral untouched * fix(vertex): drop a caller labels header on body-labelled Vertex routes while identity labels are on * test(vertex): script the v1beta1 GenAI target in the labels header integration test * test(vertex): type the transcription test's access token override --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 00:39:01 -05:00
norkelbyte	8decbe4801	fix(bedrock): send s3:// image, document and video URLs to Converse as S3 sources (#50577) ... * fix(bedrock): send s3:// image, document and video URLs to Converse as S3 sources An s3:// URL in an image_url or file part now becomes a Converse s3Location source instead of failing with "Unsupported image type". The format comes from the part's format or the key's extension, an optional bucket_owner is sent as bucketOwner, and an unknown or unsupported format answers 400. An image in an assistant turn no longer crashes the async path. * fix(proxy): limit non-admin s3:// media to allowed buckets and refuse managed prefixes A non-admin key may only send an s3:// media URL whose bucket is listed in general_settings.allowed_client_side_s3_buckets (or when client-side credentials are allowed), and never one under litellm's managed prefixes. Proxy admins are not limited. A bucket_owner on an s3:// URL that is not a non-empty string is a 400, the Responses bridge forwards the owner only for s3:// URLs, audio on S3 is a 400, and video in a tool result is sent to Bedrock. * fix(proxy): check s3:// media in every request field and refuse it in Bedrock batch files The s3:// media check now reads the whole request body instead of a fixed list of fields, and it also reads Gemini file_uri and fileUri, so a /v1beta generateContent request with an s3:// file part gets the same bucket rules as chat, Responses, and Anthropic requests. A Bedrock batch input line with an s3:// media URL is a 400 before upload, since Bedrock batch files do not take s3:// media URLs yet. * fix(bedrock): say that LiteLLM refuses s3:// media in Bedrock batch files The 400 for a Bedrock batch input line with an s3:// media URL now says "LiteLLM does not accept s3:// media URLs in Bedrock batch files", since Bedrock batch takes S3 locations and it is LiteLLM that refuses the line. * fix(proxy): check s3:// URLs only inside media parts and refuse them only on Converse batch lines The s3:// bucket check now reads URL fields only inside the content parts that a Bedrock transform turns into an S3 source, and it uses the same s3:// test as the prompt factory. Values in metadata, annotations, tool arguments and other fields pass as before. Each URL is named once in the 403 and 400 messages Bedrock batch files refuse s3:// media only on the lines that go through the Converse transform, so passthrough-provider lines upload unchanged * test(proxy): check that a media part's non-URI fields never count as s3 media * fix(proxy): check s3 URIs inside encoded file ids and on queued chat completions The s3 media check now decodes model-embedded and unified file ids, since completion() turns them back into the raw s3 URI they carry. The /queue/chat/completions endpoint skips common request processing, so it now runs the same check. The media walk goes level by level instead of recursing, which the recursive_detector code-quality step requires. * fix(proxy): let a key send managed file ids it may read and refuse s3 sources hidden in model_file_id_mapping The s3 media gate refused every LiteLLM managed file id whose upload sits under the managed prefix, even when the key's own user or team created it. It now asks the managed files hook which unified ids the key may read and skips only the objects those ids embed. A forged or unowned id, or a proxy without the hook, still gets a 403. The gate also reads the provider file ids in a client-supplied model_file_id_mapping, so an alias can no longer point a non-admin request at an unlisted or managed object. Markdown documents (.md, .markdown) now map to Bedrock's md document format, so an s3 Markdown source without an explicit format no longer fails with an unknown media type. * fix(proxy): check s3 media on every Responses WebSocket frame and keep inline tool-result video dropped Every response.create frame on WS /v1/responses now runs the same s3 media check as HTTP, so a non-admin key can no longer send an unlisted or managed s3 object over a WebSocket A tool result forwards a video to Bedrock only when it is an s3Location. An inline base64 video is dropped with a warning, as it was before this PR, since Converse rejects video in a toolResult block A non-admin key that sends an AWS identity its deployment admits through configurable_clientside_auth_params (aws_role_name, aws_web_identity_token or aws_profile_name) skips the bucket list, because Bedrock then reads the object with that identity. The managed-prefix 403 still applies. A WebSocket frame's own AWS identity fields never open the bucket list, since the managed WebSocket handler does not forward them and Bedrock reads with the deployment's identity The check now skips the body walk when no string starts with s3:// and no file_id or model_file_id_mapping key is present, which keeps large image bodies cheap * fix(proxy): let a client AWS identity skip the s3 bucket list only when Bedrock signs with it An admitted aws_role_name, aws_web_identity_token or aws_profile_name now skips the bucket list only when it decides the credentials get_credentials picks, on every deployment of the model group. A web identity token without a role and a session name does not, since get_credentials then ignores it. A profile does not when the deployment or the environment sets a role, since the role branch runs first. In those cases Bedrock still reads with the proxy's credentials, so the bucket list applies BaseAWSLLM.identity_choosing_params mirrors the branch order of get_credentials, and both read the same AWS_* environment fallbacks through one helper * fix(proxy): keep the s3 bucket list for non-admin keys that send a client AWS identity A client AWS identity field does not prove that the caller can read the bucket. A role or profile name picks credentials the proxy holds, and Bedrock reads the object with them. So the bucket list now applies to every non-admin s3 media part, whatever AWS identity fields the request sends. allow_client_side_credentials: true still turns it off This removes BaseAWSLLM.identity_choosing_params and the router lookup that only the skip needed * fix(proxy): look up managed file ids only when they can change the s3 refusal * fix(bedrock): check every file id field, skip MCP tool calls, accept registered S3 media types * fix(proxy): refuse unlisted s3 media on the rag query and adapter routes * style(proxy): format the merged collections.abc import * fix(proxy): walk tuples when checking s3 media and skip plain number lists * fix(proxy): skip numbers when walking a request body for s3 media A body with large token-id lists and any s3:// string or file_id key made the full walk build a node per number, taking seconds per request. The walk and its prefilter now skip int, float, bool and None values. * test(proxy): check the s3 refusal through the real request setup * test(proxy): exercise real setup for allowed s3 media Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model-armor): fail closed on remote s3 documents Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): suppress cyclic fallback payload inference Limit the suppression to reportGeneralTypeIssues on the legacy payload assignment. The fallback payload is constructed as a dict and the runtime AST is unchanged Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 21:33:21 -08:00
norkelbyte	dc378de207	fix(cli): let Claude Code renew a --pkce login's key through lite's apiKeyHelper (#51319) ... * fix(cli): let Claude Code renew a --pkce login's key through lite's apiKeyHelper * fix(cli): keep an explicit --api-key over lite's apiKeyHelper and persist its absolute path An explicit --api-key or LITELLM_PROXY_API_KEY is now exported to Claude Code even when lite's renewing helper is configured, since the helper prints the stored login's key instead. The helper command also uses the absolute path of the lite found on PATH, so a relative PATH entry cannot make Claude Code run another repository's bin/lite * fix(cli): keep an explicit key and --pkce when lite's renewing helper is set A --settings that blanks apiKeyHelper now keeps the exported key, and a key passed with --api-key turns lite's helper off for that launch, so Claude Code no longer warns about two credentials or sends the stored login's key beside it The helper names the lite installed beside the Python that ran the login before one found on PATH, so a pyenv shim no longer fails in projects pinned to another Python lite claude signs in again with --pkce when the stored login used it, and print-token's hints name the proxy and --pkce * fix(cli): keep a shell token from beating the renewing helper and reuse --pkce only for the same proxy The --pkce --config-claude login now writes an empty env.ANTHROPIC_AUTH_TOKEN beside the helper, since a token exported in the shell wins over apiKeyHelper lite claude sends any other key through a private settings file that blanks the helper and merges the user's own --settings A re-login keeps --pkce only when the stored login is for the same proxy, else the saved default decides * fix(cli): let any lite install's helper renew for its proxy and name the renewing login in print-token hints lite claude now treats a lite auth print-token helper for this proxy as renewing whichever lite path it names, since every lite reads the same token store Every print-token hint to sign in again, including an expired key and an unreadable keychain, names lite login --pkce with the proxy Shorter docstrings * fix(cli): send a launch key to the proxy it was issued for The private settings file lite claude writes for a key that lite's helper would not renew now also sets ANTHROPIC_BASE_URL. Without it, lite --base-url B --api-key <B key> claude after a --pkce --config-claude login to A sent the B key to A, since the login's settings point Claude Code at A * fix(proxy): preserve caller metadata on auth failures (#51745) * fix(proxy): keep x-litellm-spend-logs-metadata on auth-rejected failure spend logs Budget, invalid-key and other auth-stage rejections are raised before the request headers are merged into metadata, so the persisted failure row lost the caller's spend logs metadata header. The auth failure handler now parses it with the same pre-call helper and stamps it into the metadata bucket it already uses for the requester IP and user agent. * fix(proxy): type the spend logs metadata header parser honestly The parser takes the request headers as a str mapping and returns the parsed JSON as object, which is what successful requests already store. The auth failure path keeps only JSON objects. The precedence regression now also asserts the caller's request body is left untouched. * fix(proxy): stamp the auth failure spend logs metadata into the route's bucket A chat request whose body also carried litellm_metadata got the header in litellm_metadata, which the failure spend row never reads on chat routes. The header now goes into the bucket picked by metadata_variable_name_for_request for the route, and the failure row lifts spend_logs_metadata from that bucket the same way it already lifts the requester IP and user agent. The regression tests now assert on the built failure spend row. * fix(proxy): keep the auth failure spend logs metadata fix inside the auth handler Restores the failure spend row's lifted fields to main, so failures after auth on litellm_metadata routes keep their current logging. The auth handler now writes the parsed header into metadata, which the failure spend row reads on every route, and into the route's own bucket for callbacks on litellm_metadata routes. * fix(proxy): make the spend logs metadata header field read-only and test the real spend writer The header collector now sets spend_logs_metadata when it is built, so the field can be ReadOnly with the same stored value. The auth failure tests run the real DBSpendUpdateWriter and read the queued spend log row from a fake database client instead of an autospec writer. * test(proxy): read auth failure spend rows from a real unconnected PrismaClient Replaces the mocked database client and the class-level queue byte reset. The test now enqueues into a real PrismaClient's own queue and dequeues its row in cleanup, which keeps the shared byte counter balanced. --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> * fix(logging): keep a content-block system prompt in logged messages (#51699) * fix(logging): keep a content-block system prompt in logged messages A /v1/messages request whose top-level system prompt is a list of content blocks never reached the logged messages, because only a plain-text system prompt was prepended. Accept a non-empty list too, so every logger that reads messages sees it. Claude on Vertex also normalizes a plain-text system prompt into blocks, so it was dropped there as well. Fixes #36402 * fix(logging): type the logged system message * fix(logging): type the system message without a mutable annotation, close the test client * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter (#50483) * test(mcp): check the tool dict conversion without patching the MCP handler * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter * refactor(mcp): type the semantic filter's default tool loader Annotate the chosen loader as _OpenAIToolLoader and narrow its return to FunctionToolParam, so a default that returns raw MCP tools fails the type check. Drop the unused patch import. * fix(mcp): give the MCP client exactly the servers left on in the connect page (#50973) * fix(cli): send the launch key under a project apiKeyHelper and give each key its own settings file A `lite login --pkce --config-claude` leaves an empty ANTHROPIC_AUTH_TOKEN in Claude Code's user settings. When a repo's project settings set their own apiKeyHelper, `lite claude` skipped its private settings file, so the empty token hid the exported key and Claude Code sent the project helper's key. `lite claude` now also writes that file whenever the settings files leave ANTHROPIC_AUTH_TOKEN empty Every key launch also wrote the one file `--subscription` uses, so two launches started together could read each other's key and proxy. A key launch's file is now named after a hash of its contents, and every overlay is written to a temp file and renamed so a starting Claude Code never reads half of one * fix(proxy): keep carried end-user spend when a budget reset is retried (#51143) * fix(proxy): keep carried end-user spend when a budget reset is retried End-user pages commit before the budget window advance. When the advance failed, the tier stayed due and the next tick replayed every page, so a rollover-capped row already cut to its carryover (15 -> 5) matched the under-cap zero and lost it, and spend accrued since the first pass was zeroed again. Each end-user reset now stamps the tier window it closes in a new spend_reset_window column and skips rows already stamped with it. Fixes #50898 * fix(proxy): leave never-reset tiers unguarded and test each tier's own window A tier with no budget_reset_at falls back to the old unguarded reset instead of stamping created_at, which an admin clearing the reset time would reuse and silently skip end users. Tests now give tiers distinct windows and cover a second window closing. * test(proxy): give the end-user ledger its own fake prisma client * fix(proxy): pin a reset time on tiers without one before end-user pages commit * fix(proxy): read the reset-time pin from the writer and skip tiers another sweep advanced * fix(proxy): keep linked rows of a tier another sweep advanced out of the rollover reset * fix(proxy): order the end-user reset window migration after upstream's same-timestamp migration * fix(proxy): reset end users again when an admin moves a tier's reset time back to a closed window * fix(proxy): clear end-user reset stamps in the same transaction as the tier's reset time A failed /budget/update used to leave the new reset time written with the stamps still set, so repeating the request found the time unchanged and skipped the clear. The update and the clear now commit together, and a reset time in the future no longer scans the tier's end users * fix(proxy): name each end-user reset window by the tier's last write Stamp end-user resets with the tier's updated_at read on the writer instead of its reset time. A failed window advance leaves updated_at as it was, so the retry still skips the end users already reset, while the advance or any /budget/update moves it and opens a new window. That covers a forced reset on the default end-user tier and drops the stamp clear, so /budget/update is the one-row update it is on main. The writer read also leaves out a tier another sweep already advanced, which a lagging read replica still showed as due * fix(proxy): name end-user reset windows by the tier and its advance count Tiers advanced in one batch share updated_at, so a customer moved between them could skip the new tier's reset, and any unrelated tier edit during a failed advance reopened the window. A budget_reset_count that only the window advance bumps, prefixed with the tier id, names the window instead. * fix(proxy): keep carried spend for customer teams and members on a retried reset Customer team and member pages commit before the window advance like end-user pages, so they now stamp the window they close and a retry skips them too. A member runs on its unexpired override when that has its own duration, else on its template. The window name now joins the tier id, its creation time, its count of closed windows and its reset time, so a duration change or a tier recreated under the same id opens a new window even when the advance failed. * test(integration): list every budget column and relation in the budget update field check --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * test(otel): keep proxy startup out of the per-test timeout in the excluded services matrix (#51759) Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): skip the working directory when the login looks up lite for its helper On Windows, shutil.which searches the current directory before the directories it is given, so a lite.exe planted in the repo where someone ran `lite login --pkce --config-claude` could become their user-level apiKeyHelper The login now checks each directory by full path, this Python's scripts directory first and then each PATH entry, skipping empty entries * fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved (#50516) * fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved Converse used to drop any service_tier outside default, flex, and priority with no error, rewrite auto to default, and drop reserved. Now reserved and standard_only map to Bedrock tiers, auto sends no tier, and any other value answers 400 unless drop_params is on. Nova batch rows honor the row's drop_params. * fix(bedrock): keep a Responses batch row's drop_params and keep it out of passthrough rows * test(bedrock): annotate service tier test locals as Final and drop the row helper's kwargs * fix(bedrock): refuse a service tier on Nova batch rows unless drop_params, and honor the deployment's drop_params * fix(bedrock): map standard and any-case service tiers, and drop tiers from Claude batch rows under the deployment's drop_params * fix(bedrock): forward the tiers OpenAI-format batch rows take, accept auto on Claude rows, and label tiers in lowercase * fix(bedrock): mark the service tier block field read-only * fix(bedrock): read drop_params given as a string on batch rows, and cover the Nova row mapper's drop_params * test(prometheus): sort the service tier label test imports * fix(bedrock): send the Invoke route's service tier as a header and name the accepted tiers in the 400 * fix(bedrock): report and bill the tier Bedrock served on Invoke calls and streams, and send the tier header only to Nova * fix(bedrock): return copies when stamping the served tier, read its header without re-encoding the others, and pin it per stream chunk The Nova Invoke mapper and the event-stream decoder now return model_copy results instead of writing service_tier in place. One shared reader finds the X-Amzn-Bedrock-Service-Tier response header case-insensitively, so a non-ASCII request id no longer breaks the stream decoder, and the passthrough uses the same reader. Tests now vary the served tier in both directions, check every chunk, and cover a stream with no tier header or an empty one * test(bedrock): check the served tier on a stream whose headers come as a plain dict with an upper-case name The non-ASCII request id stream test now also carries X-AMZN-BEDROCK-SERVICE-TIER in a plain dict and asserts every chunk reports that tier, so a case-sensitive header reader or a lookup by the canonical name fails it * refactor(bedrock): drop the passthrough's own service tier header reader that main re-added The passthrough already reads the served tier through the shared case-insensitive reader in common_utils, so the httpx.Headers copy main brought back with the embeddings change has no callers and references a constant this branch removed --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> * fix(cli): delete the key files lite claude leaves on logout and unconfigure lite claude hands its process to Claude Code, so the --settings files holding its key stayed in the Claude config dir. lite logout and lite unconfigure claude now delete them, and Claude Code reads one only at startup, so open sessions keep working --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: joshua-berri <joshua@berri.ai> Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: shrey <shrey@berri.ai> Co-authored-by: Ankit Jha <45779665+PyExtreme@users.noreply.github.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: quietjules <349048734+quietjules@users.noreply.github.com>	2026-11-14 04:04:38 +00:00
zoveki	01de1a686d	feat(opensearch): map vector store search filters to k-NN filters and add hybrid search (#51131) ... * feat(opensearch): map vector store search filters to k-NN filters and add hybrid search * test(code-quality): allow the bounded OpenSearch filter recursion * fix(opensearch): retry an unfilterable engine without filters under drop_params, match strings on keyword subfields, keep per-request field keys Under drop_params, a 400 'does not support filters' from an nmslib or older engine re-runs the search once without filters, as filtered searches behaved before. String eq, ne, in and nin match the key's .keyword subfield, falling back to the key where no subfield is mapped. Only the index name, search pipeline and hybrid flag are registration-only, so opensearch_text_field and opensearch_embedding_field pass through per request again * fix(opensearch): match string filters on the exact keyword field and embed once on the filter retry String filters now read the index mapping and compare the exact keyword field, refusing a key the index cannot match exactly. The drop_params retry reuses the first embedding and also covers OpenSearch before 2.4. An index that refuses filters without drop_params answers litellm's own 400, and a filters value that is not an object stays a 400 under drop_params. The Admin UI store form gains the search pipeline and hybrid search fields * test(ui): expect the OpenSearch search pipeline and hybrid search fields in the provider field list * fix(opensearch): drop string filters under drop_params when the mapping read fails, and skip normalized keyword fields * fix(opensearch): name the permissions a refused index mapping read needs instead of returning OpenSearch's body * fix(opensearch): check the index mapping for every filter, resolve aliases, and keep failed mapping reads from dropping filters * fix(opensearch): keep an alias filter's own name and refuse rewrites that reach an index without the alias * fix(vector_stores): sign the caller's headers as given and reject filters on keys only some OpenSearch indexes map A null or non-str extra_headers value no longer fails Bedrock Knowledge Base, S3 Vectors, or other provider searches before signing. The mapping read gets only the string-valued headers. A filter on a key that some of a store's indexes map and others do not answers 400 (drops under drop_params), since OpenSearch returns its k-NN hits on some calls and none on others. * fix(opensearch): send the caller's headers on the index mapping read as given and name the fix for keys only some indexes map A header httpx cannot send now fails the filtered search on the mapping read, the same way it fails the search, before the query is embedded. Bytes headers now reach the mapping read too. * test(opensearch): pin that a timeout on the index mapping read answers 408 without embedding the query * fix(opensearch): answer 400 for a filter key inside a nested field or naming an object A root-level term query on a field under a nested mapping, or on an object field, matches nothing, so the search answered 200 with no hits. These keys now answer 400 naming the mapping, or drop the filters under drop_params * fix(opensearch): filter on a nested field whose values OpenSearch copies to the root A nested field mapped with include_in_root, or include_in_parent on it and every nested field above it, has its values in the root document, so a root term query matches it. Those filters now search instead of answering 400. A key that names an object or nested field answers that it holds fields rather than values before the nested check, so its remedy no longer leads into a second 400 --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 03:30:55 +00:00
Nate Armstrong	cd2e4b6b60	feat(custom-logger-v2): add the contract for rust callbacks and bind the proxy request scope (#51526) ... * feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) * feat(custom-logger-v2): add the rust contract for native callbacks and bind the proxy request scope * fix(custom-logger-v2): add field projections and keep the request scope across copies * fix(custom-logger-v2): pass the request scope to the async failure attempt_ended line * fix(custom-logger-v2): add v2 callbacks from litellm.callbacks to the success and failure lists _safe_add_callback_to_list dropped CustomLoggerV2 instances, so a V2 callback registered through litellm.callbacks got provider events but never log_prepare or attempt_ended. The manager now adds each V2 instance once by identity, and proxy startup fans V2 callbacks out like CustomLogger ones. * fix(custom-logger-v2): contain a failing v2 callback on logging hooks A V2 log_prepare writer that raised, or returned Abort or Respond, ended the line and escaped async_success_handler, so attempt_ended delivery, sync-for-async scheduling and the Redis flush were skipped. On hooks off the request path a V2 entry's exception or outcome is now counted and logged like a legacy logger's exception, and the line goes on. ContractViolation and BaseException still propagate, and legacy entries are unchanged. * fix(custom-logger-v2): run blocking and awaited v2 callbacks on synchronous logging lines run_plan_sync raised ContractViolation for a V2 log_prepare writer or any awaited V2 subscriber, the logging plane swallowed it, and every legacy logger at that hook was skipped. Legacy entries now always run in order as before. A V2 writer or awaited subscriber runs inline in list order through asyncio.run when the thread has no running event loop, so sync SDK calls apply its patch and finish it before the handler returns. On a thread with a running loop it cannot block, so it is delivered off-path and its patch is not applied. Lines without such entries take the unchanged legacy path. * fix(custom-logger-v2): map a v2 abort to its http error at every proxy hook point Only pre_call turned a V2 Abort into an HTTP error. during_call, post_call_success, post_mcp_call, list_models and the deferred stream hook raised V2Outcome, which reached the client as a 500, and post_call_failure, which never raised before, raised it too. Abort now raises the exception it carries or an HTTPException with its status, message and headers at those points, and at post_call_failure it becomes the transformed error through keep_first. Respond is still not wired and keeps raising V2Outcome. The pre_call routing test is updated to the HTTPException raise_v2_abort already makes. * fix(custom-logger-v2): apply no-log to v2 logging subscribers without a request scope The proxy request scope already carries no-log and caller opt-outs, so a V2 callback that honors no-log is skipped on proxy requests. SDK calls have no scope, and the logging plane ran V2 subscribers with an empty selection, so a no-log call was still delivered to them. Without a bound scope the logging lines now open one with no_log set when the call's litellm_params carry no-log, unless global_disable_no_log_param is on. Callbacks that do not honor no-log, and so every writer or abortable, are unaffected. * fix(custom-logger-v2): deliver a failed attempt to v2 subscribers once per logging object Callers such as MCP tool errors and Responses streaming run both failure handlers without for_async_request, so a V2 attempt_ended subscriber, now present in both failure lists, got the failure twice. The first failure handler to include V2 subscribers claims the delivery on the Logging object and the other skips them. Legacy callbacks are unchanged. * refactor(custom-logger-v2): settle a synchronous v2 run through the shared gather and use aware times in the new tests Drops the blind except the sync line used to capture a V2 callback's exception, reusing _gather_started, and gives the new logging tests timezone-aware times, so the strict ruff counts do not grow. * fix(custom_logger_v2): await a single-entry v2 stage in the caller's context A native handler in its own stage ran inside an asyncio.gather child task, so a ContextVar it set was lost when the stage ended, while a legacy callback's stays visible to later callbacks and the rest of the request. A stage with one v2 entry is now awaited directly, with gather's exception handling kept. Multi-entry parallel stages still gather. * fix(custom-logger-v2): raise a native callback panic as a runtime error * fix(custom-logger-v2): run a synchronous v2 line on a private event loop * fix(custom-logger-v2): deliver an mcp tool failure to v2 subscribers from the async failure handler * fix(custom-logger-v2): raise an abort's carried exception at the deployment hooks and keep the original failure at post_deployment_failure * fix(custom-logger-v2): apply caller opt-outs to v2 logging lines without a bound request scope * fix(custom-logger-v2): dedupe v2 callbacks in the event lists by logger key so a native built twice is delivered once * fix(custom-logger-v2): log an unregistered abort error kind and keep the carried traceback at post_call_failure * fix(callbacks): reject a selection name or alias that is not lowercase * fix(custom-logger-v2): run v2 deployment callbacks on rust bridge deployment hooks * fix(router): type the fallback event kwargs handed to the v2 dispatch * feat(custom_logger_v2): type the contract fields as records and mark derived-only fields Fields that the host derives from a hook's arguments now carry pydantic records on the Python side and matching serde records in litellm-callbacks, with records.json as the shared sample both sides check. Twelve new fields take bits 30 to 41, a write to a derived-only field is a manifest error, and pre_call can write budget_reservation and rate_limit_statuses in place. The payload reference field and the PayloadHost trait give Rust a handle to the logging payload instead of a copy * feat(custom_logger_v2): derive typed fields from hook arguments and always build call facts A slot table maps each typed field to a function over the hook's arguments. A derived slot runs only when a subscriber reads it, once per hook invocation, and off-path readers settle their reads before the hook returns. Auth fields come from the auth object only, budget_plan is claimed once, and budget_reservation and rate_limit_statuses write through to their carriers so the next pre_call subscriber sees them. Call facts are now built on unscoped logging lines and at post_call_success, post_call_failure and response_headers * feat(callbacks-python): hand the logging payload to native callbacks by reference NativeCallback stashes the shallow payload copy it is given and passes Rust a PayloadRef. The Python host renders a handle with a safe_dumps compatible encoder, on the caller's event loop while the hook runs and on the blocking pool after it, and dropping the last reference releases the stash entry. The encoder and its JSON helpers move here from the S3 logging bridge so every port shares one copy * feat(custom_logger_v2): accept every value python accepts for typed members and serve logged key identity Typed members keep the bool, int, float or str python holds through a Scalar type on both sides, so ids, limits and priorities no longer lose values that lax pydantic refused. A member the contract cannot carry is named in rejected so a port can reproduce what python does with it. Logging hooks get logged_key_identity, built from the payload metadata the python loggers read on every route, while key_identity stays on request hooks and keeps reading the auth object. * fix(custom_logger_v2): read the live budget reservation dict and carry non-finite floats both ways budget_reservation read the dict budget_reservation.py builds with a KeyError on callback_bound. Undeclared keys are now ignored, callback_bound is a member, entries keep a non-mapping item as null so capacity's model access group check stays exact, and costs read as the float budgets computes with float(x or 0.0). Float members cross as the raw json marker when they are not finite, and a native verdict returns through json_to_py, so an infinite or NaN cost survives a read and a write instead of turning into null. * feat(callbacks): carry a python value as its str() text and truthiness in scalar members Scalar gains a Stringified variant with the str() text and bool() of a value that is not a bool, int within i128, float or str, so a port can reproduce the label, path or log text python formats and the falsiness of an empty container without confusing it with a real string. Budget reservations say whether python held the entries as a list. * feat(custom_logger_v2): carry non-scalar values and any iterable tags or entries across the boundary A Scalar member now carries a list, dict, set, bytes, object or an int past i128 as its str() text and truthiness instead of naming it in rejected, which stays for values whose str() or bool() raises. request_tags carries any iterable as the items prometheus iterates, and budget reservation entries carry what budgets iterates with entries_is_list for capacity's list check, never written back to the shared dict. * feat(custom_logger_v2): say whether a stringified value hashes and whether it is an int A Stringified scalar now carries hashable, from trying hash() at the boundary, so a port can raise the TypeError capacity's priority lookup and the limiter's descriptor identities raise for a list, dict or set and not for a tuple or object. It also carries integer, so an int past i128 is told apart from any other value whose text is digits. * fix(custom_logger_v2): wrap a non-list guardrail mode as one mode like prometheus does Prometheus treats any guardrail_mode that is not a list as a single mode, so a tuple mode holds no str mode and a missing mode is one None mode. The derivation unpacked tuples and rejected other non-str modes instead. * feat(callbacks): let a native handler await typed host calls the python caller serves * test(callbacks-python): drive a native handler's host calls through the python dispatcher * test(custom_logger_v2): route a native logger with a host through the execution driver * fix(custom_logger_v2): type the wrapped guardrail modes for pyright * feat(custom_logger_v2): carry the python type name and hash error text of a stringified value * feat(custom_logger_v2): serve the model group the logging handlers compute and key remaining limits by it * feat(custom_logger_v2): tell a budget reservation that is not a mapping apart from none * feat(custom_logger_v2): name malformed team model preferences in rejected with the text capacity logs * feat(custom_logger_v2): serve a logging payload that is not a mapping apart from a missing one * fix(callbacks-python): render null keys like a python dict merge, adding a missing key last * feat(custom_logger_v2): carry an explicit output cap past i64 as a scalar * feat(custom_logger_v2): serve the custom metadata labels prometheus derives from the logged payload * feat(custom_logger_v2): serve the input sequence length bucket prometheus picks at attempt end * feat(custom_logger_v2): carry the start and end times prometheus reads from the call details * feat(custom_logger_v2): serve the limiter's body tags, session id, resolved agent and agent policy limits at pre_call * fix(callbacks): export the limiter request record * test(custom_logger_v2): give the type name cases stable ids and sort the record exports * fix(custom_logger_v2): type the new derivations and tests for pyright * fix(custom_logger_v2): drop team model preferences and serve the fallback_on_saturation handoff main's capacity callback reads * style(callbacks): rustfmt the record exports * refactor(custom_logger_v2): rename native callbacks to rust callbacks native.py becomes rust.py, with native_logger, NativeLogger, NativeHandle, NativeState and HostedNative renamed to their rust forms. The pyo3 classes NativeCallback and NativeRequestState become RustCallback and RustRequestState, and the tests, fixtures and messages follow. * refactor(custom_logger_v2): rename slots to field sources slots.py becomes field_sources.py: Slot is FieldSource, SlotFields is SourcedFields, and with_slots, NO_SLOTS and slot_table_problem follow. A table that only derives is named DERIVED_*_FIELDS, and a table that can write back keeps SOURCES (PRE_CALL_AUTH_SOURCES, PRE_CALL_SOURCES). The include_v2 comment moves down to the functions that take it. * refactor(litellm_logging): place the v2 members of Logging beside what they serve v2_scope, v2_auth and v2_failure_delivered join the other per-request class defaults, v2_logging_scope and its opt-out helper follow should_run_callback, v2_call_facts sits by the standard logging payload builder, and claim_v2_failure sits above failure_handler. bind_v2_request_scope checks for a Logging object directly instead of two runtime protocols. * refactor(proxy): rename v2_fields to v2_proxy_fields and say why it is separate * refactor(custom_logger_v2): drop the logged_payload_not_a_mapping field A logged payload that is present but not a mapping now looks the same to a rust callback as a missing one, a parity gap accepted for a case that does not occur. metadata_labels, input_sequence_length and limiter_request move down to bits 44-46, on both sides of the contract. * feat(callbacks): type a rust callback's request state Callback gains an associated State type: request_state returns it and handle and handle_hosted receive Option<&State> beside the input, which no longer carries the state. ErasedCallback is the object-safe form the python wrapper stores; it carries the state type-erased and downcasts it once, returning CallbackError::StateType rather than panicking on a mismatch. handle_hosted's default names its unused host _host. The record sample count follows the dropped logged_payload_not_a_mapping field. * docs(callbacks): say what PayloadRef, PayloadHost and RenderOptions.null_keys are * refactor(callbacks-python): render payloads with python-compat's json writers pyjson.rs reimplemented py_encode_basestring_ascii and json's floatstr. python-compat's json module now exposes write_string, generic over a String or byte sink, and float_text, and the payload renderer uses them. * docs(callbacks): note why Respond, Reroute, call_ended and flush are not wired yet * docs(custom_logger_v2): add a README for the package * fix(custom_logger_v2): import the field sources tests from field_sources * test(callbacks-python): stub the litellm package in each payload test * fix(custom_logger_v2): type the disabled callbacks lookup and the v2 dedupe check	2026-11-14 02:33:27 +00:00
norkelbyte	5849906ddb	fix(proxy): count tokens with the thinking, effort, and tool_choice that inference sends (#51079) ... * fix(proxy): count thinking, tool_choice, and effort the way inference sends them, and count DeepSeek on its own count route /v1/messages/count_tokens dropped the body's thinking, tool_choice, and output_config, so a Claude count with tool_choice any came out lower than the bill. The route now passes those options to the provider counter, which runs them through the deployment's own /v1/messages transform first, so the count sees what inference sends. A body inference would reject now answers the same 400 before any provider call. DeepSeek had no provider counter and fell back to a local estimate that ignores thinking. It now counts on DeepSeek's Anthropic-compatible count_tokens route with the same base, key, tool sanitizer, and reasoning mapping inference uses, and falls back to the estimate only when that route is missing. * perf(proxy): delete count drop paths from one copy of the options additional_drop_params on the count route now removes every path from a single deep copy of the request options, so a long client path list over a large schema stays linear * fix(count_tokens): count deployment thinking defaults and match inference on betas, drops, timeouts, and DeepSeek listing Count routes now merge the deployment's and router's default thinking and output_config under the request's own, the way inference does, so a deployment with thinking disabled counts what it bills Claude counters send the thinking betas inference adds for thinking.display and block_binding, so those counts no longer answer 400 additional_drop_params that is not a list is skipped and non-string entries are ignored, the way inference skips them, instead of turning the count into a 500 The DeepSeek counter passes the deployment timeout to its count call, and DeepSeek model listing reads the bundled model list instead of recursing through get_valid_models * fix(count_tokens): count deployment thinking on Responses and let a request drop list replace the deployment's A Responses count now merges deployment and router defaults under the bridged chat body, so deployment thinking and reasoning_effort reach the count the way they reach chat inference A request's additional_drop_params now replaces the deployment's list for both count drop passes, as inference does The Anthropic, Azure AI, and DeepSeek counters read litellm_params through one typed helper instead of Any * fix(count_tokens): estimate DeepSeek counts when the count route is down, and count deployment thinking on /utils/token_counter and Vertex Claude A timeout, 429, 5xx, or connect error from DeepSeek's count endpoint now falls back to the local estimate instead of failing the count. A 401 or 422 still answers DeepSeek's own status /utils/token_counter now counts the thinking and effort a DeepSeek deployment sets, and Vertex Claude counts run the Anthropic messages transform on the count options instead of sending none * fix(count_tokens): send the Vertex count's thinking betas and count a Responses request's own thinking A Vertex Claude count with thinking display or block binding now sends the beta headers Vertex inference adds, so Vertex no longer rejects a count for a request inference accepts A Responses count now takes the request's own top-level thinking over the deployment's, the order inference uses, since the Responses-to-chat body never carried it * fix(count_tokens): cap DeepSeek count waits, remember missing count routes, and count /utils request thinking A DeepSeek count now waits at most 10 s even when the deployment sets no timeout, a 422 from the count route falls back to the estimate, and a base whose count route answered 404 or 405 is not called again for an hour. /utils/token_counter now passes the request's thinking, reasoning_effort, and tool_choice to the count * fix(count_tokens): count on DeepSeek's endpoint only for api.deepseek.com and send Messages count options to Bedrock Mantle * fix(bedrock): send the thinking beta headers on Mantle counts A Mantle count with thinking display updates or block_binding now sends the same anthropic-beta header Mantle inference sends * fix(count_tokens): drop the region hint on a Mantle count 400 for forwarded count options and unskip the count options wire test * fix(deepseek): count only the tools DeepSeek chat inference sends --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 02:29:16 +00:00
Ankit Jha	5f381ad472	fix(proxy): keep carried end-user spend when a budget reset is retried (#51143) ... * fix(proxy): keep carried end-user spend when a budget reset is retried End-user pages commit before the budget window advance. When the advance failed, the tier stayed due and the next tick replayed every page, so a rollover-capped row already cut to its carryover (15 -> 5) matched the under-cap zero and lost it, and spend accrued since the first pass was zeroed again. Each end-user reset now stamps the tier window it closes in a new spend_reset_window column and skips rows already stamped with it. Fixes #50898 * fix(proxy): leave never-reset tiers unguarded and test each tier's own window A tier with no budget_reset_at falls back to the old unguarded reset instead of stamping created_at, which an admin clearing the reset time would reuse and silently skip end users. Tests now give tiers distinct windows and cover a second window closing. * test(proxy): give the end-user ledger its own fake prisma client * fix(proxy): pin a reset time on tiers without one before end-user pages commit * fix(proxy): read the reset-time pin from the writer and skip tiers another sweep advanced * fix(proxy): keep linked rows of a tier another sweep advanced out of the rollover reset * fix(proxy): order the end-user reset window migration after upstream's same-timestamp migration * fix(proxy): reset end users again when an admin moves a tier's reset time back to a closed window * fix(proxy): clear end-user reset stamps in the same transaction as the tier's reset time A failed /budget/update used to leave the new reset time written with the stamps still set, so repeating the request found the time unchanged and skipped the clear. The update and the clear now commit together, and a reset time in the future no longer scans the tier's end users * fix(proxy): name each end-user reset window by the tier's last write Stamp end-user resets with the tier's updated_at read on the writer instead of its reset time. A failed window advance leaves updated_at as it was, so the retry still skips the end users already reset, while the advance or any /budget/update moves it and opens a new window. That covers a forced reset on the default end-user tier and drops the stamp clear, so /budget/update is the one-row update it is on main. The writer read also leaves out a tier another sweep already advanced, which a lagging read replica still showed as due * fix(proxy): name end-user reset windows by the tier and its advance count Tiers advanced in one batch share updated_at, so a customer moved between them could skip the new tier's reset, and any unrelated tier edit during a failed advance reopened the window. A budget_reset_count that only the window advance bumps, prefixed with the tier id, names the window instead. * fix(proxy): keep carried spend for customer teams and members on a retried reset Customer team and member pages commit before the window advance like end-user pages, so they now stamp the window they close and a retry skips them too. A member runs on its unexpired override when that has its own duration, else on its template. The window name now joins the tier id, its creation time, its count of closed windows and its reset time, so a duration change or a tier recreated under the same id opens a new window even when the advance failed. * test(integration): list every budget column and relation in the budget update field check --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-11-14 01:03:19 +00:00
shrey	68d6db45ef	fix(mcp): give the MCP client exactly the servers left on in the connect page (#50973)	2026-11-13 16:36:49 -08:00
Mateo Wang	c4a72b8fac	refactor(mcp): let the semantic filter take its MCP tool loader as a parameter (#50483) ... * test(mcp): check the tool dict conversion without patching the MCP handler * refactor(mcp): let the semantic filter take its MCP tool loader as a parameter * refactor(mcp): type the semantic filter's default tool loader Annotate the chosen loader as _OpenAIToolLoader and narrow its return to FunctionToolParam, so a default that returns raw MCP tools fails the type check. Drop the unused patch import.	2026-11-13 16:34:00 -08:00
joshua-berri	d11917e89c	fix(proxy): preserve caller metadata on auth failures (#51745) ... * fix(proxy): keep x-litellm-spend-logs-metadata on auth-rejected failure spend logs Budget, invalid-key and other auth-stage rejections are raised before the request headers are merged into metadata, so the persisted failure row lost the caller's spend logs metadata header. The auth failure handler now parses it with the same pre-call helper and stamps it into the metadata bucket it already uses for the requester IP and user agent. * fix(proxy): type the spend logs metadata header parser honestly The parser takes the request headers as a str mapping and returns the parsed JSON as object, which is what successful requests already store. The auth failure path keeps only JSON objects. The precedence regression now also asserts the caller's request body is left untouched. * fix(proxy): stamp the auth failure spend logs metadata into the route's bucket A chat request whose body also carried litellm_metadata got the header in litellm_metadata, which the failure spend row never reads on chat routes. The header now goes into the bucket picked by metadata_variable_name_for_request for the route, and the failure row lifts spend_logs_metadata from that bucket the same way it already lifts the requester IP and user agent. The regression tests now assert on the built failure spend row. * fix(proxy): keep the auth failure spend logs metadata fix inside the auth handler Restores the failure spend row's lifted fields to main, so failures after auth on litellm_metadata routes keep their current logging. The auth handler now writes the parsed header into metadata, which the failure spend row reads on every route, and into the route's own bucket for callbacks on litellm_metadata routes. * fix(proxy): make the spend logs metadata header field read-only and test the real spend writer The header collector now sets spend_logs_metadata when it is built, so the field can be ReadOnly with the same stored value. The auth failure tests run the real DBSpendUpdateWriter and read the queued spend log row from a fake database client instead of an autospec writer. * test(proxy): read auth failure spend rows from a real unconnected PrismaClient Replaces the mocked database client and the class-level queue byte reset. The test now enqueues into a real PrismaClient's own queue and dequeues its row in cleanup, which keeps the shared byte counter balanced. --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-11-13 16:13:50 -08:00
Mateo Wang	ac3277dce4	fix(dynamic_rate_limiter_v3): publish a team's share gauges for requests served while the model is idle (#51709)	2026-11-13 14:58:54 -08:00
moe-berri	62cf355d9f	fix(spend-logs): make token redaction opt-in (#51729)	2026-11-13 14:41:12 -08:00
Kerry Lu	db75214bdc	fix(proxy): keep a customer budget reset from being undone by a stale hierarchy cache entry (#51694) ... * fix(proxy): keep a customer budget reset from being undone by a stale hierarchy cache entry The reset job now evicts the customer_org_alias entries of the organizations it resets, and a hierarchy load that began before an eviction can no longer cache what it read. The cache hands a lease out on a miss and refuses the write when the key was evicted since, on this worker and, with enable_redis_auth_cache, on any worker through a generation key checked in Redis. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): drop auth cache memory before the generation write and skip an empty alias eviction delete cleared memory only after its new Redis round trip, so a deleted entry stayed readable on the handling worker while Redis was slow. The reset job also broadcast an empty alias eviction for a cascade with no customer organizations. * fix(proxy): delete an auth cache entry and void its leases in one script on the bulk path too The bulk delete wrote the generation and sent the DEL as two round trips, so a failed generation write still let the DEL run and an in-flight load could re-cache what it read. * fix(proxy): delete the auth cache entry before the generation write inside the script A Redis at its memory limit with noeviction refused the script at its leading SET, so the DEL never ran and a deleted entry stayed in Redis. With the DEL first the script still deletes it. * test(proxy): mark the new reset cache tests' bindings Final and use rstest for the new Rust test * fix(proxy): refuse a leased auth cache write for a key no generation key can guard * test(proxy): undo the unrelated reformat of the spend ledger parity tests * test(redis-store): mark the updated memory-only auth cache test with rstest * chore(proxy): drop two pointer comments from the cache lease code --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 14:21:20 -08:00
moe-berri	0d422a1b6c	fix(auth): let dashboard sessions use the user's team models (#51663) ... * fix: inherit team models for dashboard sessions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: authorize dashboard models from owner grants Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: preserve dashboard model listing filters Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): attribute dashboard team model spend to granting teams Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Keep dashboard fallbacks on the original billing team Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Preserve admitted dashboard billing through primary model rewrites Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Reject dashboard batches that need team model grants Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Respect wildcard exclusions in dashboard model listings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 17:13:06 -05:00
Nate Armstrong	20b6e855bd	feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214)	2026-11-13 22:08:05 +00:00
tin-berri	7b0a37ba74	feat(ui): add overall customer usage and sortable budgets (#51713) ... * feat(ui): add overall customer usage and sortable budgets * test(ui): complete customer usage metric fixtures * fix(ui): preserve exact customer organization IDs in overview	2026-11-13 22:03:22 +00:00
joshua-berri	88873a70d4	fix(proxy): retain unwritten rollups after database disconnects (#51706) ... * fix(proxy): retain unwritten rollups after database disconnects * fix(proxy): initialize auto-router turns per client --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-11-13 13:36:12 -08:00
moe-berri	9cc808c90b	feat(ui): let admins control Aria access for internal users (#51641) ... * feat: configure Aria access for internal users Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor: keep Aria session access within complexity budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: match Aria role helper to authorized role type Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: type Aria settings and allow permitted team selection Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 16:17:12 -05:00
Kerry Lu	a920b5be6e	fix(proxy): delete a customer team's override budget when the team has no overall budget (#51703) ... Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 13:05:19 -08:00
moe-berri	945ee1ca43	fix(customer-portal): keep existing memberships and scope free tier capacity locks (#51698) ... * fix(customer-portal): preserve memberships and isolate capacity locks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(customer-portal): map model write lock timeouts to retryable failures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 15:51:54 -05:00
mrinal-berri	57c6bbfa12	fix(mcp): route aggregate resources/read to the server that lists the URI (#51462) ... * fix(mcp): resolve aggregate resource reads by unique listed owner Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): discover resource owners concurrently Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): remove obsolete multi-server resource read guard assertion Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): skip blocked stdio, scope bearer and admit candidates in resource owner lookup * fix(mcp): return resource owner outcomes as values and skip blocked stdio candidates * fix(mcp): withhold the caller bearer from every consuming server in aggregate resource reads * fix(mcp): report aggregate resource read failures as MCPError --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 12:50:43 -08:00
tin-berri	b798cf30ab	feat(customer-portal): configure budget groups atomically (#51661) ... * feat(ui): add customer organization budget groups tab Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Add cap policy controls to customer budget groups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Clarify per-group cap policy help text Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix customer budget groups formatting and lint budgets Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(customer-portal): save budget group configuration atomically * refactor(customer-portal): name unchanged budget group state * fix(customer-portal): chunk removed budget reference checks --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 20:29:36 +00:00
tin-berri	b34478b960	fix(customers): align budget displays with recorded reset windows (#51658)	2026-11-13 20:02:16 +00:00
joshua-berri	a166574ba4	fix(sdk): stop requiring redis to import the Messages streaming path (#51685) ... * fix(sdk): stop requiring redis to import the Messages streaming path redis_update_buffer.py and dynamic_rate_limiter_v3.py imported RedisError at module level. Both sit on the import chain that the SDK /v1/messages stream loads, so an SDK install without redis lost every streaming success callback. Import RedisError only where a Redis call can actually raise it. * style(tests): space the optional redis import fixture per ruff format --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-11-13 11:43:53 -08:00
moe-berri	1a682f9dbf	feat(customer-portal): add free tier limits with Enterprise upgrade path (#51687) ... * feat: add customer portal free tier limits Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: narrow customer portal free tier enforcement Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Show customer portal free tier limits in dashboard Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: check customer key scope before organization registration Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(customer-portal): preserve overflow requests and refresh limits after deletion Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 14:43:15 -05:00
Kerry Lu	bad8b1dae8	fix(proxy): drop a customer override expiry sent without an override (#51667) ... * fix(proxy): drop a customer override expiry sent without an override * test(proxy): clear the override before deleting the row so the integration test leaves no budget behind --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 11:24:05 -08:00
Kerry Lu	6549e114cc	fix(proxy): /customer/new budgets reset on the same schedule as other budgets (#51689) ... Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 19:22:14 +00:00
Kerry Lu	367890f958	fix(proxy): refuse a key pinned to another customer org before registering an unknown org alias (#51674) ... * fix(proxy): refuse a key pinned to another customer org before registering an unknown org alias * test(proxy): a pinned key naming an unknown org alias is refused with 403 even when registration would fail --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 11:03:43 -08:00
Kerry Lu	48f23ea1cb	fix(proxy): register a customer member in a customer org with no default budget (#51660) ... * fix(proxy): register a customer member in a customer org with no default budget A request that named a customer org and a customer member the proxy had never seen created the customer org and charged it, then dropped the customer member because a new customer org has no default budget. The customer member is now registered in the default customer team with no budget, so its spend is tracked from the first request and nothing caps it until an admin assigns one. Admin creates still require a budget. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): write only the budget fields a customer member patch names An alias edit wrote back the budget fields it read, so a budget assigned between its read and its write was erased. The update now leaves the budget columns alone unless the patch names them. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-11-13 18:43:51 +00:00
devin-ai-integration[bot]	1defaa6cd1	fix(budgets): invalidate customer hierarchy caches through stored team IDs (#51654) ... * fix(budgets): invalidate customer hierarchy caches through stored team IDs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: restore customer budget alerts after hierarchy schema changes (#51645) * fix(proxy): upsert customer orgs and customer members safely (#51607) * fix(proxy): upsert customer orgs and customer members safely Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): lock customer org snapshots during upsert Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): reject empty customer org upsert identifiers Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): reuse the customer org when its alias is already registered A request for an alias that another worker registered a moment earlier got alias_already_exists back from the create and failed. The registration now treats that the same as an existing id and returns the registered customer org. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai> * test(spend): synchronize concurrent member resets before validation completes (#51655) Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(budgets): invalidate customer hierarchy caches through stored team IDs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(budgets): supply async customer team lookups in caller fixtures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ishaan-j-berri <ishaan.j@berri.ai> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai>	2026-11-13 18:43:38 +00:00
ishaan-j-berri	b2351b8bee	fix(proxy): upsert customer orgs and customer members safely (#51607) ... * fix(proxy): upsert customer orgs and customer members safely Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): lock customer org snapshots during upsert Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): reject empty customer org upsert identifiers Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): reuse the customer org when its alias is already registered A request for an alias that another worker registered a moment earlier got alias_already_exists back from the create and failed. The registration now treats that the same as an existing id and returns the registered customer org. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai>	2026-11-13 18:21:14 +00:00
ishaan-j-berri	7a72c2b17a	fix: restore customer budget alerts after hierarchy schema changes (#51645)	2026-11-13 18:06:53 +00:00