* feat(proxy): publish opted-in pass-through endpoints on the Model Hub
Pass-through endpoints gain display_name and show_in_model_hub. Endpoints that opt in appear on
/public/v1/model_hub and /public/model_hub as rows in mode passthrough, named by their display name
or their path, with the route in pass_through_path. The admin UI exposes both fields on the create
and edit forms and the public hub renders passthrough rows with a route-specific curl example
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): carry pass-through methods onto hub rows and key dashboard rows by route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep health, cost and shell quoting correct for pass-through hub rows
Pass-through rows no longer inherit a router model's health check when they share its name, the public table shows n/a instead of Free for their token costs, and the generated curl example escapes shell characters in the route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): quote the method in the pass-through hub curl example
A configured method string is placed after curl -X inside escaped double quotes so it cannot alter the copied command
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): build pass-through hub fixtures through a helper
Keeps the dashboard lint budget for inline object arguments under its limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit the pass-through Model Hub surface end to end
Cover forwarding, pagination, validation, auth, updates, duplicate names,
config-file precedence, health-name isolation, repeated reads and two chaos
cells (create/delete churn, a killed proxy worker). Build the stored
pass_through_endpoints list in one shot instead of appending to the list
the config reader handed back
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep concurrent pass-through creates from overwriting each other
Creating an endpoint appends to the general_settings list the proxy already holds instead of persisting a private copy, so two admins creating at the same time both land, as on main before this branch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): serialize pass-through endpoint creates behind a process lock
Each create copies the stored list under an asyncio lock before persisting, so simultaneous creates in one worker no longer overwrite each other and a rejected write (config-owned path) never lands in the in-memory settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): collect paginated pass-through rows without mutating a list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): let a pass-through with no stored timeout save from the edit form
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): type the pass-through edit fixture so a null timeout compiles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(proxy): read published pass-throughs from the in-memory settings store
The public Model Hub routes read the pass-through list this worker already
serves from proxy_config.settings instead of querying the config table on
every anonymous hub read. Also drops a mutable-ok suppression that no longer
guards a violation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): list a config file pass-through beside database rows on the hub
Main merges database pass-throughs with config file ones on other paths since
#43962, so a database write beside a config entry is accepted rather than
rejected as config-owned
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): type the monkeypatch fixture in the pass-through hub tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): validate pass-through config entries instead of splatting unknown dicts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): list model hub rows as a tagged union of model groups and pass-throughs
Health enrichment and the no-router check now match on ModelGroupRow or
PassThroughRow instead of testing pass_through_path in each route. Both hub
routes share published_hub_rows, the v1 row helper is sync again, and the
config list route goes back to main's inline merge since the hub no longer
needs configured_pass_through_endpoints
* test(ui): find the pass-through hub row by role to stay under the no-node-access budget
* fix(proxy): keep database pass-through endpoints in the settings store across config reloads
A config reload cleared the stored database row before rebuilding the
pass-through list, so without store_model_in_db the list fell back to the
config file entries alone. The Admin UI triggers that reload on page load
through /get/config/callbacks, which dropped UI-created endpoints from the
Model Hub until the next write
* refactor(proxy): make the hub row matches exhaustive with assert_never
* refactor(proxy): return the exhaustive match fallthrough and build hub test inputs in one shot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): drop the in-process create lock from pass-through endpoints
The lock only covered one worker, and the read-modify-write on the general_settings blob is a separate storage problem tracked as tech debt. Create goes back to main's behavior.
* fix: align pass-through model hub with current lint rules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: share worker state for pass-through hub restart coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: show pass-through curl examples in model hub inspector
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: skip invalid pass-through hub entries and await peer refresh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
* fix(token-counter): count Anthropic server tool history
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(token-counter): bound nested results and update proxy coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(token-counter): include readable web search snippets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When the upstream connection drops or stalls mid-stream on the Rust Messages
path, the caller gets the raw httpx error without litellm_debug_info, so the
llm_exceptions alert names no API base, model group or deployment. Python's
stream iterator attaches that info to the same error, and the Rust host now does
too.
* feat(replicate): add video generation, image generation and text to speech through the predictions API
* fix(replicate): surface poll HTTP errors, keep deployment ssl_verify and native speech input, flatten outputs without recursion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): keep model family mapping for deployments and versions, preserve video id on status, poll until terminal status
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): validate output downloads and poll origin, bind retrieval to the requested model, flatten image extra_body, keep speech input authoritative, use the async image path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): keep the audited prompt authoritative over image extra_body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(replicate): cover extra_body.prompt not replacing the audited prompt
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): honor the request ssl_verify when polling and downloading speech output
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(replicate): build the video usage into VideoObject instead of assigning it afterwards
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): keep the inspected video prompt authoritative over native extra_body prompt
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(replicate): bill native image dimensions and token-metered speech output
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): bound output downloads, accept both token metric spellings and build speech usage at construction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): walk bounded-download redirects iteratively
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): reject dot and empty model path segments and keep the bounded redirect walk annotated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): drop the unused rebind suppression
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): put the rebind suppression on the redirect hop binding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(replicate): run pinned versions through the model's own versions endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Absorb #51347, #51222, #51357 and #51275 with the GMI large-context tier.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(model-catalog): read output_cost_per_billable_unit in the rust registry check
* fix(model-catalog): add billable_unit to the flat key schema pattern
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(dynamic_rate_limiter_v3): split a priority class share equally among the teams active on a model
Adds priority_reservation_settings.priority_class_share_splits_equally_among_active_teams (default false).
When on, the class weights are renormalized over the classes with a team active on the model in the
current window, each class share is divided equally among its active teams, and once the model saturates
each team is held to its own share with a 429 whose rate_limited_by header reads
priority_class_share_saturated. Active teams are one Redis sorted set per model with the window as TTL.
* fix(proxy): keep basedpyright counts flat for the priority class share setting
PriorityReservationSettings is built with model_validate instead of **value, so a new
field no longer adds an unknown-argument diagnostic, and the limiter unit test helpers
type the handler by its implementation class.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): keep team shares usable when Redis, mixed-priority keys, WebSocket turns or tiny shares get in the way
A registry failure falls back to the class pool instead of skipping the model limits, a team's
counter is keyed by model and team so keys of different priorities spend one share, Responses
WebSocket turns are held to the class share like the team model limit, a share is floored at one
request and one token, and the registry declares the rate limits key family.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): catch only Redis and validation errors when the active team registry fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): keys without a team keep their class present, team ids may contain the registry separator, breaker-open counts as a registry failure
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): per-class pool markers that take no team slice, collision-free model and team counter keys, local registry pruned by last-seen time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): settle tokens only on the stash the call owns, keep a configured class named default apart from teams with no priority
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): tag team registry members and count every unenforced request against the team share
A team id can no longer spell a pool marker, since real teams are stored under a team prefix. Below saturation the team share counter is bumped through the atomic increment at the model-wide limit, so the local over-limit short-circuit never skips a request and the counter is complete when the share grows.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): list the caller under its own class and resync the socket stash share on every frame
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): keep the connection share across a registry outage and settle fallback tokens on the served model's team counter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(router): steer flexible teams to a permitted alternate model when their preferred model is saturated
Teams opt in with metadata.model_preferences: [{"model": "luna", "alternates": ["opus"]}].
The dynamic rate limiter v3 pre-call hook reads the saturation cache for the preferred model
and, once it is at or above saturation_threshold, rewrites the request to the first alternate
the team may call that is below the threshold. The response carries
x-litellm-model-steered-from with the preferred model. Rate limits and spend are charged to
the model that served. Teams without model_preferences are untouched.
* fix(dynamic_rate_limiter_v3): type the hook request dicts so the steering rewrite passes the lint gates
* fix(dynamic_rate_limiter_v3): authorize alternates against the key and its model budget, send the steering header on streams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dynamic_rate_limiter_v3): enforce member and project allowlists on alternates, skip the steering header for non Latin-1 names
* fix(dynamic_rate_limiter_v3): check every budget layer for an alternate, fall back to the preferred model when the alternate refuses, merge the served model's guardrails
* fix(dynamic_rate_limiter_v3): never steer agent keys, type the steering unit tests
* fix(dynamic_rate_limiter_v3): hold alternates to customer grants, access group budgets, guardrails and the zero-cost waiver
An alternate now passes every check a budget fallback passes. Customer model grants are enforced, the access groups that grant the alternate are budget checked and stamped on the token and request metadata so spend charges them, a zero-cost preferred model that waived the aggregate budget checks is never steered to a priced alternate, and an alternate whose deployments add model-level guardrails the request did not run is skipped.
* fix(dynamic_rate_limiter_v3): stamp the alternate's groups on every bucket and skip alternates the request cannot safely take
The served model's access groups now land on every metadata bucket and the auth copy inside it even when the preferred model had none, so group spend is charged. An alternate is skipped when a budget reservation already holds a different group, when it does not serve the request's endpoint, when unpriced models are blocked and it has no price, or when its guardrails differ from the request's by name or settings. A failing alternate check or alternate admission leaves the request on the preferred model instead of failing it.
* fix(dynamic_rate_limiter_v3): keep a reserved budget off pricier alternates
While the request holds a budget reservation sized for the preferred model, an alternate that costs more per token is skipped, so fail-closed budget enforcement never admits a call the reservation does not cover.
* fix(dynamic_rate_limiter_v3): an unpriced alternate never takes a reserved budget
While a budget reservation is held, an alternate with no input or output price is skipped rather than treated as free.
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: stop PR runs from writing Actions caches and remove the sccache GitHub cache backend
PR runs were saving their own cache copies and sccache was making one cache
request per compiled file, about 700 to 2000 per Rust build. Together they
pushed the repository past GitHub's cache storage and request limits, so
restores failed with 429 or found nothing and jobs rebuilt from scratch. The
rust-bridge job that gates every unit shard went from 0% cache misses on
Oct 21 to about 65% on Nov 11.
Caches are now saved only from main and restored everywhere else. The Rust DB
cache gets the same save-if guard the other Rust jobs already have. The lint
python and ui-api-types jobs restore the uv cache that the unit workflow saves
on main, because lint only runs on pull requests and could never save a cache
that another pull request can read. The Terraform provider jobs turn off
setup-go's built-in cache, which saves from every ref, and use a new
cache-go-build action that follows the existing restore-everywhere and
save-on-main pattern.
sccache is removed from the unit, Rust and Rust DB workflows. The Cargo target
directory caches they already restore stay in place.
The smoke diagnostics upload no longer fails a job whose checks passed.
* ci(smoke): install the prebuilt Rust bridge and cache wheels built from source
Every smoke job compiled the Rust bridge inside uv sync, about 4.5 minutes each
in 8 jobs per run, and the Python 3.14 and 3.15 jobs also compiled grpcio-tools,
numpy and jiter from source on every run because no wheels exist for those
versions yet. Smoke only ran on pull requests, so nothing it built could be
saved for the next run.
The Linux smoke jobs now restore the bridge that the unit workflow already
builds once and saves on main. The bridge is abi3, so the same build serves
every Python version. When the restore misses, which happens when a pull
request changes a Rust build input, the job restores the Cargo build cache and
compiles as before. The cache key and restore step move into a
restore-rust-bridge action that the unit workflow's producer job also uses, so
the key is defined once.
Smoke now also runs on pushes to main and prunes the uv cache down to wheels
built from source before setup-uv saves it. Pull requests restore that cache,
so 3.14 and 3.15 stop recompiling the same packages.
The windows-sdk job drops its Cargo cache step. Nothing ever saved a Windows
copy, and saving one from main would add about 2 GB to a cache that is already
near its limit.
* feat(rate-limit): team_model_limit_max_borrow_multiple caps how far a team borrows idle model capacity
With team_model_limit_borrows_idle_capacity_from_other_teams on, a team could
use everything up to the model's own tpm and rpm while the model was idle.
Setting team_model_limit_max_borrow_multiple in the team's metadata caps the
team at its model_tpm_limit and model_rpm_limit times that multiple, with a 429
whose rate_limited_by header reads model_per_team_max_borrow_multiple. Unset
keeps the previous behavior. Values below 1 or non-numeric are logged and
ignored.
* fix(rate-limit): reject infinite or oversized team_model_limit_max_borrow_multiple values instead of failing the rate check
* fix(team): make team_model_limit_max_borrow_multiple proxy-controlled team metadata so team admins cannot lift their own cap
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): close strict JSON schemas on the Messages to Responses bridge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(anthropic): copy each strict schema subtree once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): close nullable objects in strict JSON schemas
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fal_ai): give MiniMax H3 Max video endpoints their own resolution profile
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): keep the H3 Max test signature on one line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fal_ai): type the H3 Max test locals and signatures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rate-limit): opt-in team_model_limit_is_floor turns a team's model_tpm_limit into a guaranteed share that borrows idle capacity
With priority_reservation_settings.team_model_limit_is_floor: true, a team's inherited model_tpm_limit on a model the dynamic rate limiter covers stops being a hard cap in the regular limiter. The dynamic limiter tracks it as a team_model_floor descriptor, lets the team past it while the model is below saturation_threshold, and holds the team to it once the model saturates, with rate_limited_by: team_model_floor on the 429. Default false keeps today's behavior. Key-owned model_tpm_limit, model_rpm_limit, the model-wide tpm and customer limits stay hard.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rate-limit): name the setting team_model_limit_mode with hard and soft values
soft replaces the team_model_limit_is_floor flag. The 429 header reads rate_limited_by: team_model_soft_limit and the message says the team soft limit was reached.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rate-limit): team_model_limit_borrows_idle_capacity_from_other_teams covers team model TPM and RPM
Replaces team_model_limit_mode. A team's inherited model_tpm_limit and model_rpm_limit on a model the dynamic
rate limiter covers are enforced by that limiter once the model saturates, so the team borrows idle capacity below
the threshold. The 429 names the limit and reads rate_limited_by: model_per_team_saturated.
Folds in the bot findings: key ownership is checked per requested model, team limits resolve through the model
alias like the regular limiter, the team counter is charged only for requests the pre-call hook tracked, every
response type that carries usage is settled, and tracking below saturation no longer adds a token per request.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rate-limit): keep the type gate flat in the dynamic limiter's team limit paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rate-limit): settle responses usage and null auth metadata, resolve aliases for the borrowing team limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rate-limit): key the borrowing team counter by resolved model group and read usage through get_usage_object
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rate-limit): hold Responses WebSocket turns to the borrowing team limit once the model saturates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): install the Rust gateway and fail forwarded_proxy when Rust is not in front
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): build the Rust gateway only for the management suites that run forwarded_proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Two PRs each added gemini/gemini-decision-1 to the price map, so main defines the key twice and test_price_map_has_no_duplicate_keys fails in the root unit job for every PR that merges main. The two entries hold the same fields and values, so this removes the earlier one and leaves the one JSON parsers already keep. The parsed map is unchanged.
* fix(proxy): reject unknown guardrail names on key and user writes
Key generate, update, regenerate, team bulk update and /user/new now return 400 for a guardrail name that no loaded guardrail answers to and that has no active DB row, instead of storing it so requests run unguarded. Names the key already holds are not rechecked, and the key endpoints keep the enterprise license 403 ahead of the name check.
Resolves LIT-10613
* fix(proxy): read guardrail names from the writer when validating key writes
A guardrail created on another pod may not have reached a lagging read replica yet, so the fallback lookup now reads the writer and a just-created guardrail is not rejected as unknown. Other guardrail table reads keep using the reader.
* feat(proxy): validate guardrail names on team and project writes, reject malformed values loudly
Team and project create and update now return 400 for guardrail names that no loaded guardrail answers to and that have no active DB row, the same check key and user writes use. A metadata.guardrails value that is not a list of names is rejected even when it equals the stored value, with a short message naming the value. The lookup and the license-first wrapper move into common_utils so every entity shares them.
* refactor(proxy): one validate_guardrail_names entry point for guardrail name checks
The name check, its DB lookup and the license-first option now live in litellm/proxy/guardrails/guardrail_name_validation.py and every key, user, team and project write calls validate_guardrail_names directly. The lookup factory, both wrappers and the per-endpoint helpers are gone, and common_utils and guardrail_registry are back to their main versions. Behavior is unchanged.
* fix(proxy): chunk the guardrail lookup and leave legacy dict guardrail settings alone
The guardrail name lookup goes through find_many_in so its IN list has a written bound, and the project test keeps pytest.raises to one statement. A dict-shaped metadata.guardrails, a team's documented {"modify_guardrails": false} or the v1 {name: bool} toggle map, is read at request time, so validation leaves it alone instead of rejecting it.
* fix(proxy): only let the team modify_guardrails permission through as a dict
Stored key, team and project guardrails only run when metadata.guardrails is a list, so a dict such as {"config-guard": true} saved fine and attached nothing. The one stored dict with an effect is a team's documented {"modify_guardrails": bool}, which auth checks read, so that exact shape passes and every other dict gets the Invalid guardrail 400.
* refactor(proxy): move guardrail name validation into guardrail_helpers
validate_guardrail_names and its helpers now live in litellm/proxy/guardrails/guardrail_helpers.py next to the other guardrail helpers, and guardrail_name_validation.py is removed. guardrail_helpers imports LiteLLM_TeamTable and UserAPIKeyAuth from litellm.proxy._types instead of litellm.proxy.proxy_server, so the endpoints can import it at module level without pulling in the proxy server, and its dead sys.path line is dropped. Behavior is unchanged.
* feat(alerts): add default-on monitoring and a five-alert free allowance
* fix(alerts): handle monitoring races and retry-safe migration
* fix(ui): match monitoring upgrade link to status
* feat(ui): restore performance alerts on updated detector
* fix(ui): keep team key detail tabs independent
* fix(ui): preserve team tabs while permissions load
* feat(ui): show concise alerts in the notification bell
* refactor(ui): remove replaced deprecation bell item
* fix(ui): pin notification resource links
* fix(ui): show deprecation failures in notifications
* fix(ui): reset activity state when changing resources
* fix(ui): reset key detail state from every entry point
* feat(alerts): detect daily changes in usage for gateway, models, teams, users and keys
* fix(alerts): drop unused suppressions
* fix(alerts): satisfy strict lint for model usage timing
* fix(alerts): skip only resources that time out or hold invalid data
* fix(alerts): log an unfinished daily sweep at info level
* fix(alerts): keep the deployment-wide run every five minutes
* fix(alerts): record alert and sweep timestamps in UTC
* fix(alerts): repair detector tests and tracing
* refactor(alerts): build the key index in a migration instead of after startup
* refactor(alerts): build the key index after startup again
A concurrent index build inside the boot migration can exceed the 600 second migrate deploy deadline on a large LiteLLM_DailyUserSpend and stop the proxy from starting. This reverts 393db6e4e0.
* fix(alerts): keep keys pending, bound the accounting wait, resume unfinished scopes
Keys waiting for their index no longer mark the day complete. Only deliveries created during the evaluated day hold back its sweep. A day that did not finish restarts the next day from where it stopped instead of a hash boundary that never rotated non-hex IDs.
* refactor(alerts): drop the pending delivery wait, alerts are best effort on flushed usage
* test(alerts): cover detector persistence and recovery
* fix(alerts): recognize Prisma statement timeouts
* feat(proxy): run spend log cleanup from a DB-backed scheduled job table
Spend log retention ran from an in-process APScheduler timer that reset on
every restart and needed Redis for leader election, so pods that restart
more often than the interval never cleaned up at all.
Add LiteLLM_ScheduledJobRun, a generic table with one row per job run.
Every pod polls it, plans the next run idempotently and claims a due run
under a transaction-scoped advisory lock, so exactly one pod runs each
occurrence. Runs carry a lease, a status and a log. Intervals count from
the last finished run and survive restarts. Schedule edits cancel and
replace the pending run. The retention and size-check jobs share a lock
group so they never overlap, and the Redis pod lock is gone.
* fix(proxy): plan an earlier cron edit from now, and cover the job runner
Moving the cleanup cron earlier queued tomorrow's run because the next
occurrence was planned after the pending run being cancelled. Plan from
finished runs only.
Register LiteLLM_ScheduledJobRun as a span table, keep the advisory lock
query inside the tick's span, move the non-string cron startup test to
the schedule parser, and add unit tests for the runner and job wiring.
* test(proxy): annotate new spend log cleanup test locals as Final
* test(proxy): type the parameters of the new spend log cleanup tests
* test(proxy): leave an hour of margin when moving a cron earlier so an hour boundary cannot flake it
* fix(proxy): reject a cron that never fires instead of failing every tick
A crontab like 0 0 31 2 * parses but has no next fire time, so next_run_at raised on every tick and the job was never scheduled again. cron_schedule now returns InvalidSchedule for it, which logs and keeps the pending run like any other bad config
* refactor(proxy): drop the informational log column from LiteLLM_ScheduledJobRun
Nothing read the column back. Run outcomes stay in status, and the reasons
already go to the proxy logs, so JobResult becomes a SUCCEEDED or FAILED
literal and the cleanup returns its RunOutcome instead of a summary report
* feat(gemini): add gemini-decision-1, the Decisions API model priced at $0.10 per 1M input tokens
* fix(cost-map): give gemini-decision-1 its Tier 1 rpm and tpm so test_get_model_info_gemini passes
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(gemini): attribute /gemini batchGenerateContent batches to the calling key
A 200 from models/{model}:batchGenerateContent through the Gemini pass-through now registers a managed
batch row for the calling key, team and tags, so CheckBatchCost prices the output file on completion
instead of the job landing as a $0 pass_through_endpoint row with model unknown.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(gemini): unit regression for pass-through batch registration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): file batch created through /gemini is registered for the calling key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(gemini): bill pass-through batches through the configured gemini deployment
The managed row now points at the gemini/<model> deployment CheckBatchCost fetches the output with, not
the first router id for the model name, and a model with no gemini deployment logs a warning instead of
registering a row that can never be billed. A key without a user keeps user_id unset rather than
default-user. Typed the registration helper and handler so the lint gates stay flat.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
