Router defaults now fill only the keys a deployment leaves unset, so a deployment's own value (vertex_location, api_base, drop_params) beats default_litellm_params. The proxy token counters, drop-params checks, and the Entra cooldown check use the same order
* feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0)
* feat(custom-logger-v2): add the rust contract for native callbacks and bind the proxy request scope
* fix(custom-logger-v2): add field projections and keep the request scope across copies
* fix(custom-logger-v2): pass the request scope to the async failure attempt_ended line
* fix(custom-logger-v2): add v2 callbacks from litellm.callbacks to the success and failure lists
_safe_add_callback_to_list dropped CustomLoggerV2 instances, so a V2 callback registered through litellm.callbacks got provider events but never log_prepare or attempt_ended. The manager now adds each V2 instance once by identity, and proxy startup fans V2 callbacks out like CustomLogger ones.
* fix(custom-logger-v2): contain a failing v2 callback on logging hooks
A V2 log_prepare writer that raised, or returned Abort or Respond, ended the line and escaped async_success_handler, so attempt_ended delivery, sync-for-async scheduling and the Redis flush were skipped. On hooks off the request path a V2 entry's exception or outcome is now counted and logged like a legacy logger's exception, and the line goes on. ContractViolation and BaseException still propagate, and legacy entries are unchanged.
* fix(custom-logger-v2): run blocking and awaited v2 callbacks on synchronous logging lines
run_plan_sync raised ContractViolation for a V2 log_prepare writer or any awaited V2 subscriber, the logging plane swallowed it, and every legacy logger at that hook was skipped. Legacy entries now always run in order as before. A V2 writer or awaited subscriber runs inline in list order through asyncio.run when the thread has no running event loop, so sync SDK calls apply its patch and finish it before the handler returns. On a thread with a running loop it cannot block, so it is delivered off-path and its patch is not applied. Lines without such entries take the unchanged legacy path.
* fix(custom-logger-v2): map a v2 abort to its http error at every proxy hook point
Only pre_call turned a V2 Abort into an HTTP error. during_call, post_call_success, post_mcp_call, list_models and the deferred stream hook raised V2Outcome, which reached the client as a 500, and post_call_failure, which never raised before, raised it too. Abort now raises the exception it carries or an HTTPException with its status, message and headers at those points, and at post_call_failure it becomes the transformed error through keep_first. Respond is still not wired and keeps raising V2Outcome. The pre_call routing test is updated to the HTTPException raise_v2_abort already makes.
* fix(custom-logger-v2): apply no-log to v2 logging subscribers without a request scope
The proxy request scope already carries no-log and caller opt-outs, so a V2 callback that honors no-log is skipped on proxy requests. SDK calls have no scope, and the logging plane ran V2 subscribers with an empty selection, so a no-log call was still delivered to them. Without a bound scope the logging lines now open one with no_log set when the call's litellm_params carry no-log, unless global_disable_no_log_param is on. Callbacks that do not honor no-log, and so every writer or abortable, are unaffected.
* fix(custom-logger-v2): deliver a failed attempt to v2 subscribers once per logging object
Callers such as MCP tool errors and Responses streaming run both failure handlers without for_async_request, so a V2 attempt_ended subscriber, now present in both failure lists, got the failure twice. The first failure handler to include V2 subscribers claims the delivery on the Logging object and the other skips them. Legacy callbacks are unchanged.
* refactor(custom-logger-v2): settle a synchronous v2 run through the shared gather and use aware times in the new tests
Drops the blind except the sync line used to capture a V2 callback's exception, reusing _gather_started, and gives the new logging tests timezone-aware times, so the strict ruff counts do not grow.
* fix(custom_logger_v2): await a single-entry v2 stage in the caller's context
A native handler in its own stage ran inside an asyncio.gather child task, so a ContextVar it set was lost when the stage ended, while a legacy callback's stays visible to later callbacks and the rest of the request. A stage with one v2 entry is now awaited directly, with gather's exception handling kept. Multi-entry parallel stages still gather.
* fix(custom-logger-v2): raise a native callback panic as a runtime error
* fix(custom-logger-v2): run a synchronous v2 line on a private event loop
* fix(custom-logger-v2): deliver an mcp tool failure to v2 subscribers from the async failure handler
* fix(custom-logger-v2): raise an abort's carried exception at the deployment hooks and keep the original failure at post_deployment_failure
* fix(custom-logger-v2): apply caller opt-outs to v2 logging lines without a bound request scope
* fix(custom-logger-v2): dedupe v2 callbacks in the event lists by logger key so a native built twice is delivered once
* fix(custom-logger-v2): log an unregistered abort error kind and keep the carried traceback at post_call_failure
* fix(callbacks): reject a selection name or alias that is not lowercase
* fix(custom-logger-v2): run v2 deployment callbacks on rust bridge deployment hooks
* fix(router): type the fallback event kwargs handed to the v2 dispatch
* feat(custom_logger_v2): type the contract fields as records and mark derived-only fields
Fields that the host derives from a hook's arguments now carry pydantic records on the Python side and matching serde records in litellm-callbacks, with records.json as the shared sample both sides check. Twelve new fields take bits 30 to 41, a write to a derived-only field is a manifest error, and pre_call can write budget_reservation and rate_limit_statuses in place. The payload reference field and the PayloadHost trait give Rust a handle to the logging payload instead of a copy
* feat(custom_logger_v2): derive typed fields from hook arguments and always build call facts
A slot table maps each typed field to a function over the hook's arguments. A derived slot runs only when a subscriber reads it, once per hook invocation, and off-path readers settle their reads before the hook returns. Auth fields come from the auth object only, budget_plan is claimed once, and budget_reservation and rate_limit_statuses write through to their carriers so the next pre_call subscriber sees them. Call facts are now built on unscoped logging lines and at post_call_success, post_call_failure and response_headers
* feat(callbacks-python): hand the logging payload to native callbacks by reference
NativeCallback stashes the shallow payload copy it is given and passes Rust a PayloadRef. The Python host renders a handle with a safe_dumps compatible encoder, on the caller's event loop while the hook runs and on the blocking pool after it, and dropping the last reference releases the stash entry. The encoder and its JSON helpers move here from the S3 logging bridge so every port shares one copy
* feat(custom_logger_v2): accept every value python accepts for typed members and serve logged key identity
Typed members keep the bool, int, float or str python holds through a Scalar
type on both sides, so ids, limits and priorities no longer lose values that
lax pydantic refused. A member the contract cannot carry is named in rejected
so a port can reproduce what python does with it.
Logging hooks get logged_key_identity, built from the payload metadata the
python loggers read on every route, while key_identity stays on request hooks
and keeps reading the auth object.
* fix(custom_logger_v2): read the live budget reservation dict and carry non-finite floats both ways
budget_reservation read the dict budget_reservation.py builds with a KeyError
on callback_bound. Undeclared keys are now ignored, callback_bound is a member,
entries keep a non-mapping item as null so capacity's model access group check
stays exact, and costs read as the float budgets computes with float(x or 0.0).
Float members cross as the raw json marker when they are not finite, and a
native verdict returns through json_to_py, so an infinite or NaN cost survives
a read and a write instead of turning into null.
* feat(callbacks): carry a python value as its str() text and truthiness in scalar members
Scalar gains a Stringified variant with the str() text and bool() of a value
that is not a bool, int within i128, float or str, so a port can reproduce
the label, path or log text python formats and the falsiness of an empty
container without confusing it with a real string. Budget reservations say
whether python held the entries as a list.
* feat(custom_logger_v2): carry non-scalar values and any iterable tags or entries across the boundary
A Scalar member now carries a list, dict, set, bytes, object or an int past
i128 as its str() text and truthiness instead of naming it in rejected, which
stays for values whose str() or bool() raises. request_tags carries any
iterable as the items prometheus iterates, and budget reservation entries
carry what budgets iterates with entries_is_list for capacity's list check,
never written back to the shared dict.
* feat(custom_logger_v2): say whether a stringified value hashes and whether it is an int
A Stringified scalar now carries hashable, from trying hash() at the
boundary, so a port can raise the TypeError capacity's priority lookup and
the limiter's descriptor identities raise for a list, dict or set and not
for a tuple or object. It also carries integer, so an int past i128 is told
apart from any other value whose text is digits.
* fix(custom_logger_v2): wrap a non-list guardrail mode as one mode like prometheus does
Prometheus treats any guardrail_mode that is not a list as a single mode, so a
tuple mode holds no str mode and a missing mode is one None mode. The
derivation unpacked tuples and rejected other non-str modes instead.
* feat(callbacks): let a native handler await typed host calls the python caller serves
* test(callbacks-python): drive a native handler's host calls through the python dispatcher
* test(custom_logger_v2): route a native logger with a host through the execution driver
* fix(custom_logger_v2): type the wrapped guardrail modes for pyright
* feat(custom_logger_v2): carry the python type name and hash error text of a stringified value
* feat(custom_logger_v2): serve the model group the logging handlers compute and key remaining limits by it
* feat(custom_logger_v2): tell a budget reservation that is not a mapping apart from none
* feat(custom_logger_v2): name malformed team model preferences in rejected with the text capacity logs
* feat(custom_logger_v2): serve a logging payload that is not a mapping apart from a missing one
* fix(callbacks-python): render null keys like a python dict merge, adding a missing key last
* feat(custom_logger_v2): carry an explicit output cap past i64 as a scalar
* feat(custom_logger_v2): serve the custom metadata labels prometheus derives from the logged payload
* feat(custom_logger_v2): serve the input sequence length bucket prometheus picks at attempt end
* feat(custom_logger_v2): carry the start and end times prometheus reads from the call details
* feat(custom_logger_v2): serve the limiter's body tags, session id, resolved agent and agent policy limits at pre_call
* fix(callbacks): export the limiter request record
* test(custom_logger_v2): give the type name cases stable ids and sort the record exports
* fix(custom_logger_v2): type the new derivations and tests for pyright
* fix(custom_logger_v2): drop team model preferences and serve the fallback_on_saturation handoff main's capacity callback reads
* style(callbacks): rustfmt the record exports
* refactor(custom_logger_v2): rename native callbacks to rust callbacks
native.py becomes rust.py, with native_logger, NativeLogger, NativeHandle,
NativeState and HostedNative renamed to their rust forms. The pyo3 classes
NativeCallback and NativeRequestState become RustCallback and
RustRequestState, and the tests, fixtures and messages follow.
* refactor(custom_logger_v2): rename slots to field sources
slots.py becomes field_sources.py: Slot is FieldSource, SlotFields is
SourcedFields, and with_slots, NO_SLOTS and slot_table_problem follow. A
table that only derives is named DERIVED_*_FIELDS, and a table that can
write back keeps SOURCES (PRE_CALL_AUTH_SOURCES, PRE_CALL_SOURCES). The
include_v2 comment moves down to the functions that take it.
* refactor(litellm_logging): place the v2 members of Logging beside what they serve
v2_scope, v2_auth and v2_failure_delivered join the other per-request
class defaults, v2_logging_scope and its opt-out helper follow
should_run_callback, v2_call_facts sits by the standard logging payload
builder, and claim_v2_failure sits above failure_handler.
bind_v2_request_scope checks for a Logging object directly instead of two
runtime protocols.
* refactor(proxy): rename v2_fields to v2_proxy_fields and say why it is separate
* refactor(custom_logger_v2): drop the logged_payload_not_a_mapping field
A logged payload that is present but not a mapping now looks the same to a
rust callback as a missing one, a parity gap accepted for a case that does
not occur. metadata_labels, input_sequence_length and limiter_request move
down to bits 44-46, on both sides of the contract.
* feat(callbacks): type a rust callback's request state
Callback gains an associated State type: request_state returns it and
handle and handle_hosted receive Option<&State> beside the input, which no
longer carries the state. ErasedCallback is the object-safe form the python
wrapper stores; it carries the state type-erased and downcasts it once,
returning CallbackError::StateType rather than panicking on a mismatch.
handle_hosted's default names its unused host _host. The record sample
count follows the dropped logged_payload_not_a_mapping field.
* docs(callbacks): say what PayloadRef, PayloadHost and RenderOptions.null_keys are
* refactor(callbacks-python): render payloads with python-compat's json writers
pyjson.rs reimplemented py_encode_basestring_ascii and json's floatstr.
python-compat's json module now exposes write_string, generic over a String
or byte sink, and float_text, and the payload renderer uses them.
* docs(callbacks): note why Respond, Reroute, call_ended and flush are not wired yet
* docs(custom_logger_v2): add a README for the package
* fix(custom_logger_v2): import the field sources tests from field_sources
* test(callbacks-python): stub the litellm package in each payload test
* fix(custom_logger_v2): type the disabled callbacks lookup and the v2 dedupe check
* fix(proxy): count thinking, tool_choice, and effort the way inference sends them, and count DeepSeek on its own count route
/v1/messages/count_tokens dropped the body's thinking, tool_choice, and output_config, so a Claude count with
tool_choice any came out lower than the bill. The route now passes those options to the provider counter, which
runs them through the deployment's own /v1/messages transform first, so the count sees what inference sends.
A body inference would reject now answers the same 400 before any provider call.
DeepSeek had no provider counter and fell back to a local estimate that ignores thinking. It now counts on
DeepSeek's Anthropic-compatible count_tokens route with the same base, key, tool sanitizer, and reasoning
mapping inference uses, and falls back to the estimate only when that route is missing.
* perf(proxy): delete count drop paths from one copy of the options
additional_drop_params on the count route now removes every path from a single deep copy of the request options, so a long client path list over a large schema stays linear
* fix(count_tokens): count deployment thinking defaults and match inference on betas, drops, timeouts, and DeepSeek listing
Count routes now merge the deployment's and router's default thinking and output_config under the request's own, the way inference does, so a deployment with thinking disabled counts what it bills
Claude counters send the thinking betas inference adds for thinking.display and block_binding, so those counts no longer answer 400
additional_drop_params that is not a list is skipped and non-string entries are ignored, the way inference skips them, instead of turning the count into a 500
The DeepSeek counter passes the deployment timeout to its count call, and DeepSeek model listing reads the bundled model list instead of recursing through get_valid_models
* fix(count_tokens): count deployment thinking on Responses and let a request drop list replace the deployment's
A Responses count now merges deployment and router defaults under the bridged chat body, so deployment thinking and reasoning_effort reach the count the way they reach chat inference
A request's additional_drop_params now replaces the deployment's list for both count drop passes, as inference does
The Anthropic, Azure AI, and DeepSeek counters read litellm_params through one typed helper instead of Any
* fix(count_tokens): estimate DeepSeek counts when the count route is down, and count deployment thinking on /utils/token_counter and Vertex Claude
A timeout, 429, 5xx, or connect error from DeepSeek's count endpoint now falls back to the local estimate instead of failing the count. A 401 or 422 still answers DeepSeek's own status
/utils/token_counter now counts the thinking and effort a DeepSeek deployment sets, and Vertex Claude counts run the Anthropic messages transform on the count options instead of sending none
* fix(count_tokens): send the Vertex count's thinking betas and count a Responses request's own thinking
A Vertex Claude count with thinking display or block binding now sends the beta headers Vertex inference adds, so Vertex no longer rejects a count for a request inference accepts
A Responses count now takes the request's own top-level thinking over the deployment's, the order inference uses, since the Responses-to-chat body never carried it
* fix(count_tokens): cap DeepSeek count waits, remember missing count routes, and count /utils request thinking
A DeepSeek count now waits at most 10 s even when the deployment sets no timeout, a 422 from the count route falls back to the estimate, and a base whose count route answered 404 or 405 is not called again for an hour. /utils/token_counter now passes the request's thinking, reasoning_effort, and tool_choice to the count
* fix(count_tokens): count on DeepSeek's endpoint only for api.deepseek.com and send Messages count options to Bedrock Mantle
* fix(bedrock): send the thinking beta headers on Mantle counts
A Mantle count with thinking display updates or block_binding now sends the
same anthropic-beta header Mantle inference sends
* fix(count_tokens): drop the region hint on a Mantle count 400 for forwarded count options and unskip the count options wire test
* fix(deepseek): count only the tools DeepSeek chat inference sends
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): keep carried end-user spend when a budget reset is retried
End-user pages commit before the budget window advance. When the advance
failed, the tier stayed due and the next tick replayed every page, so a
rollover-capped row already cut to its carryover (15 -> 5) matched the
under-cap zero and lost it, and spend accrued since the first pass was
zeroed again. Each end-user reset now stamps the tier window it closes
in a new spend_reset_window column and skips rows already stamped with it.
Fixes#50898
* fix(proxy): leave never-reset tiers unguarded and test each tier's own window
A tier with no budget_reset_at falls back to the old unguarded reset
instead of stamping created_at, which an admin clearing the reset time
would reuse and silently skip end users. Tests now give tiers distinct
windows and cover a second window closing.
* test(proxy): give the end-user ledger its own fake prisma client
* fix(proxy): pin a reset time on tiers without one before end-user pages commit
* fix(proxy): read the reset-time pin from the writer and skip tiers another sweep advanced
* fix(proxy): keep linked rows of a tier another sweep advanced out of the rollover reset
* fix(proxy): order the end-user reset window migration after upstream's same-timestamp migration
* fix(proxy): reset end users again when an admin moves a tier's reset time back to a closed window
* fix(proxy): clear end-user reset stamps in the same transaction as the tier's reset time
A failed /budget/update used to leave the new reset time written with the
stamps still set, so repeating the request found the time unchanged and
skipped the clear. The update and the clear now commit together, and a
reset time in the future no longer scans the tier's end users
* fix(proxy): name each end-user reset window by the tier's last write
Stamp end-user resets with the tier's updated_at read on the writer instead
of its reset time. A failed window advance leaves updated_at as it was, so the
retry still skips the end users already reset, while the advance or any
/budget/update moves it and opens a new window. That covers a forced reset on
the default end-user tier and drops the stamp clear, so /budget/update is the
one-row update it is on main. The writer read also leaves out a tier another
sweep already advanced, which a lagging read replica still showed as due
* fix(proxy): name end-user reset windows by the tier and its advance count
Tiers advanced in one batch share updated_at, so a customer moved between
them could skip the new tier's reset, and any unrelated tier edit during a
failed advance reopened the window. A budget_reset_count that only the
window advance bumps, prefixed with the tier id, names the window instead.
* fix(proxy): keep carried spend for customer teams and members on a retried reset
Customer team and member pages commit before the window advance like end-user
pages, so they now stamp the window they close and a retry skips them too. A
member runs on its unexpired override when that has its own duration, else on
its template.
The window name now joins the tier id, its creation time, its count of closed
windows and its reset time, so a duration change or a tier recreated under the
same id opens a new window even when the advance failed.
* test(integration): list every budget column and relation in the budget update field check
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(mcp): check the tool dict conversion without patching the MCP handler
* refactor(mcp): let the semantic filter take its MCP tool loader as a parameter
* refactor(mcp): type the semantic filter's default tool loader
Annotate the chosen loader as _OpenAIToolLoader and narrow its return to
FunctionToolParam, so a default that returns raw MCP tools fails the type
check. Drop the unused patch import.
* fix(proxy): keep x-litellm-spend-logs-metadata on auth-rejected failure spend logs
Budget, invalid-key and other auth-stage rejections are raised before the
request headers are merged into metadata, so the persisted failure row lost
the caller's spend logs metadata header. The auth failure handler now parses
it with the same pre-call helper and stamps it into the metadata bucket it
already uses for the requester IP and user agent.
* fix(proxy): type the spend logs metadata header parser honestly
The parser takes the request headers as a str mapping and returns the parsed
JSON as object, which is what successful requests already store. The auth
failure path keeps only JSON objects. The precedence regression now also
asserts the caller's request body is left untouched.
* fix(proxy): stamp the auth failure spend logs metadata into the route's bucket
A chat request whose body also carried litellm_metadata got the header in
litellm_metadata, which the failure spend row never reads on chat routes.
The header now goes into the bucket picked by metadata_variable_name_for_request
for the route, and the failure row lifts spend_logs_metadata from that bucket
the same way it already lifts the requester IP and user agent. The regression
tests now assert on the built failure spend row.
* fix(proxy): keep the auth failure spend logs metadata fix inside the auth handler
Restores the failure spend row's lifted fields to main, so failures after auth
on litellm_metadata routes keep their current logging. The auth handler now
writes the parsed header into metadata, which the failure spend row reads on
every route, and into the route's own bucket for callbacks on litellm_metadata
routes.
* fix(proxy): make the spend logs metadata header field read-only and test the real spend writer
The header collector now sets spend_logs_metadata when it is built, so the
field can be ReadOnly with the same stored value. The auth failure tests run
the real DBSpendUpdateWriter and read the queued spend log row from a fake
database client instead of an autospec writer.
* test(proxy): read auth failure spend rows from a real unconnected PrismaClient
Replaces the mocked database client and the class-level queue byte reset.
The test now enqueues into a real PrismaClient's own queue and dequeues
its row in cleanup, which keeps the shared byte counter balanced.
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* fix(proxy): keep a customer budget reset from being undone by a stale hierarchy cache entry
The reset job now evicts the customer_org_alias entries of the organizations it resets, and a
hierarchy load that began before an eviction can no longer cache what it read. The cache hands
a lease out on a miss and refuses the write when the key was evicted since, on this worker and,
with enable_redis_auth_cache, on any worker through a generation key checked in Redis.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop auth cache memory before the generation write and skip an empty alias eviction
delete cleared memory only after its new Redis round trip, so a deleted entry stayed readable on
the handling worker while Redis was slow. The reset job also broadcast an empty alias eviction for
a cascade with no customer organizations.
* fix(proxy): delete an auth cache entry and void its leases in one script on the bulk path too
The bulk delete wrote the generation and sent the DEL as two round trips, so a failed generation
write still let the DEL run and an in-flight load could re-cache what it read.
* fix(proxy): delete the auth cache entry before the generation write inside the script
A Redis at its memory limit with noeviction refused the script at its leading SET, so the DEL
never ran and a deleted entry stayed in Redis. With the DEL first the script still deletes it.
* test(proxy): mark the new reset cache tests' bindings Final and use rstest for the new Rust test
* fix(proxy): refuse a leased auth cache write for a key no generation key can guard
* test(proxy): undo the unrelated reformat of the spend ledger parity tests
* test(redis-store): mark the updated memory-only auth cache test with rstest
* chore(proxy): drop two pointer comments from the cache lease code
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: inherit team models for dashboard sessions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: authorize dashboard models from owner grants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: preserve dashboard model listing filters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): attribute dashboard team model spend to granting teams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Keep dashboard fallbacks on the original billing team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Preserve admitted dashboard billing through primary model rewrites
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Reject dashboard batches that need team model grants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Respect wildcard exclusions in dashboard model listings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: configure Aria access for internal users
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor: keep Aria session access within complexity budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: match Aria role helper to authorized role type
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: type Aria settings and allow permitted team selection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add customer organization budget groups tab
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Add cap policy controls to customer budget groups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Clarify per-group cap policy help text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Fix customer budget groups formatting and lint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(customer-portal): save budget group configuration atomically
* refactor(customer-portal): name unchanged budget group state
* fix(customer-portal): chunk removed budget reference checks
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(sdk): stop requiring redis to import the Messages streaming path
redis_update_buffer.py and dynamic_rate_limiter_v3.py imported RedisError at module
level. Both sit on the import chain that the SDK /v1/messages stream loads, so an SDK
install without redis lost every streaming success callback. Import RedisError only
where a Redis call can actually raise it.
* style(tests): space the optional redis import fixture per ruff format
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* fix(proxy): drop a customer override expiry sent without an override
* test(proxy): clear the override before deleting the row so the integration test leaves no budget behind
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): refuse a key pinned to another customer org before registering an unknown org alias
* test(proxy): a pinned key naming an unknown org alias is refused with 403 even when registration would fail
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): register a customer member in a customer org with no default budget
A request that named a customer org and a customer member the proxy had never
seen created the customer org and charged it, then dropped the customer member
because a new customer org has no default budget. The customer member is now
registered in the default customer team with no budget, so its spend is tracked
from the first request and nothing caps it until an admin assigns one. Admin
creates still require a budget.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): write only the budget fields a customer member patch names
An alias edit wrote back the budget fields it read, so a budget assigned
between its read and its write was erased. The update now leaves the budget
columns alone unless the patch names them.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(budgets): invalidate customer hierarchy caches through stored team IDs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: restore customer budget alerts after hierarchy schema changes (#51645)
* fix(proxy): upsert customer orgs and customer members safely (#51607)
* fix(proxy): upsert customer orgs and customer members safely
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): lock customer org snapshots during upsert
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): reject empty customer org upsert identifiers
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): reuse the customer org when its alias is already registered
A request for an alias that another worker registered a moment earlier got
alias_already_exists back from the create and failed. The registration now
treats that the same as an existing id and returns the registered customer org.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
* test(spend): synchronize concurrent member resets before validation completes (#51655)
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(budgets): invalidate customer hierarchy caches through stored team IDs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(budgets): supply async customer team lookups in caller fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ishaan-j-berri <ishaan.j@berri.ai>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
* fix(proxy): upsert customer orgs and customer members safely
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): lock customer org snapshots during upsert
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): reject empty customer org upsert identifiers
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): reuse the customer org when its alias is already registered
A request for an alias that another worker registered a moment earlier got
alias_already_exists back from the create and failed. The registration now
treats that the same as an existing id and returns the registered customer org.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
* fix(proxy): reset customer org, team and member spend on their budget row's window
The customer tables carried their own budget_reset_at next to a budget_id whose
row already has one. Drop the column from the three tables and let the budget
table cascade zero the customer rows it governs, with a member following its
unexpired override when that override has a duration and its template
otherwise. Customer budget writes stamp budget_reset_at from budget_duration the
way /budget/new does, and the management responses report the governing budget's
window under the same budget_reset_at field.
* feat(budgets): support concurrent customer budget windows
* fix(budgets): resolve compatibility and validation checks
* feat(proxy): name budget rows, give customer teams an override and members a parent org
* fix(budgets): preserve admitted windows during accounting
* chore(schema): allow dropping the unreleased customer budget_reset_at columns
* fix(budgets): apply standard migration lock timeout
* feat(proxy): customer members belong to many teams, charged to the one the request names
* test(migrations): allow the unreleased customer budget_reset_at drop
* style(ui): prettier the customer dashboard files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(reset-job): name the batch bound on the customer budget IN filters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(budgets): preserve mixed limits and resolve CI regressions
* fix(budgets): reuse shared spend verification for customer windows
* refactor(ui): isolate customer budget group creation
Keep only the customer-creation UI on current main. This local patch still depends on the BudgetTemplate.budget_limits API and generated types from the #51510 rebuild.
* fix(proxy): restore Field import and bound the member team lookup after the merge
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(proxy): send parent_team_ids in the budget window customer tests
Co-authored-by: Tin <tin@berri.ai>
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): keep rejecting another organization's template after the budget_template_ids rename
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): reset customer teams and members in pages against each row's own cap, keep a template's clock on resubmitted durations
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* feat(proxy): pause_allow_increase cap policy with customer budget increase requests reviewed like MCP and skill submissions
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(proxy): cover customer budget increase requests and the cap_policy field
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* chore(ui): regenerate API types for cap_policy and customer budget increase requests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): hang the increase request back-reference on the customer member model
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): declare increase request list filters with Annotated Query
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): scope budget increase requests to the key's organization, require a future expiry, one pending request per member, and never revive an expired override on approval
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): write customer budgets without windows as Prisma Json null, run the increase request table through the hierarchy transaction
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): review increase requests exactly once, let an approval replace template windows, pin member budget_id to the org's templates, open the request route to virtual keys
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Tin <tin@berri.ai>
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Datadog answers a request for mcp_all with its granular scopes, so the connect
check never found mcp_all and users looped back to Connect. Reconnecting also
re-asked every granular scope from the old grant, which overflowed the sign-in
cookie. Any non-empty grant now covers mcp_all, and a request for mcp_all does
not re-ask the old grant
* fix(proxy): preserve team aliases in Responses WebSockets
Co-authored-by: sisyphus-dev-ai <sisyphus-dev-ai@users.noreply.github.com>
* test(proxy): drop the pre-call kwargs assertion from the Responses WebSocket routing test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: sisyphus-dev-ai <sisyphus-dev-ai@users.noreply.github.com>
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(interactions): bind Deep Research agents to their deployment and bill the agent
A request that sends an agent with a model group now sends the agent to Google
and is refused with a 400 unless the group's deployment is that agent. Agent
interactions are logged and priced under the agent, including streamed and
background jobs, with a Vertex cost row for deep-research-preview-04-2026.
Background cost polling now waits 7800 seconds by default so a 120-minute job
still settles.
* fix(interactions): refuse extra_body that changes the agent or model
A nested agent or model in extra_body was merged into the request after the agent binding check, so it could run a different agent than the one billed, inject an agent onto a model deployment, or add a second selector. Refuse those before any provider or OAuth call, while identical agent repeats and unrelated extra_body fields still pass through.
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* fix(interactions): route Vertex interaction follow-ups to the selected deployment
GET, cancel and DELETE by interaction id accept a model group or a deployment
id through the model query parameter or the x-litellm-model header, authorize
it against the key and team like any model, and run with that deployment's
project and credentials. Without a selector they still go to Google AI Studio.
Vertex cancel now uses the documented /cancel path.
* test(bedrock): pin the SigV4 clock in the embedding signature test
SigV4Auth.add_auth stamps the request with botocore.auth.get_current_datetime(), overwriting the timestamp the test set on the replay. When the proxy call and the replay landed in different seconds, the signatures differed. Fix the module clock for both signings and drop the ineffective manual timestamp.
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
A request that names a deployment id now goes through the same access-group
filter as its model group, so a key or team limited to access groups cannot
pin a deployment outside them. Wildcard grants that cover the owning group
still reach every deployment of it. Responses reads pinned by response id
share this path.
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* fix(proxy): import soundfile lazily in the transcribe logging handler
The messages streaming path imports the pass-through success handler, which
pulled in soundfile at module load. Installs without soundfile failed on
every Anthropic and Bedrock messages stream. Only media_file_seconds needs it
Fixes#45379
* test(vector_store): pin set_verbose off in the SDK fallback warning test
An ambient litellm.set_verbose=True adds a deprecation warning to the captured records, so the exact one-warning assertion failed depending on test order
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
* feat(proxy): fallback_on_saturation walks key and team fallbacks before a model saturates, replacing team_metadata.model_preferences
Team router_settings.fallbacks now fire on proxy-side rate limits: the pre-call
fallback walk reads router settings Key > Team > Global instead of the key's
column alone, and skips a fallback the caller may not call.
A key or team with router_settings.fallback_on_saturation: true walks that same
fallback list as soon as the requested model reaches saturation_threshold in
dynamic_rate_limiter_v3, and goes back to the requested model when no fallback
admits the request. Responses carry x-litellm-fallback-reason:
rate_limit:model_saturation_fallback.
Removes the team_metadata.model_preferences steering and the
x-litellm-model-steered-from header from #51350, which did the same job through
a second knob.
* fix(proxy): strict bool for fallback_on_saturation, drop a stale import and settle the pre-call tests
fallback_on_saturation only turns on for a real boolean true, so a string like "yes" in router_settings no longer enables it. The saturation unit test reads saturation through the limiter's cache, so it asserts the threshold was reached instead of an exact count. The response polling test now hands the pre-call path an awaitable router_settings lookup, as the pre-call fallback walk reads Key > Team settings before the first call.
* fix(proxy): pre-call fallbacks pass the budget and access-group checks a direct request does
A fallback the pre-call walk picks now has to clear every per-model budget layer and the budgets of the access
groups that grant it, and its spend is charged to those groups instead of the requested model's. At saturation a
fallback that fails for any reason counts as a miss and the request goes back to the requested model. The client
body can no longer opt itself into the saturation walk through router_settings_override, only the saved key or team
router_settings can. The integration test for a forbidden fallback waits for the preferred model's own 429 so it
cannot pass before saturation.
* refactor(proxy): one helper decides which refused fallbacks end the pre-call walk
_pre_call_with_fallbacks had grown past the strict C901 ceiling, so the two except clauses fold into one and _fallback_refusal_ends_the_walk carries the rule: at saturation every refusal is a miss, otherwise only a non-rate-limit failure or a hard per-model 429 reaches the caller.
* fix(proxy): keep the caller's request object through the pre-call fallback walk
Copying the request to drop a client fallback_on_saturation opt-in handed the hooks a different dict than the one the caller holds, so header metadata written into one never reached the other (test_add_litellm_metadata_from_request_headers). drop_client_saturation_opt_in now edits the request in place.
* fix(proxy): pre-call fallbacks keep the pricing and budget-reservation rules a direct request passes
A fallback is skipped when unpriced models are blocked and it has no pricing, when a zero-cost request would land on a priced model, and when the budget auth reserved for the requested model does not fit it (higher per-token prices, or reserved against an access group that does not grant it).
* refactor(proxy): read fallback prices through the router registry entry point
router_surface_check only lets the router surface shrink, so the per-token price lookup goes through registry().group() instead of the deprecated get_model_group_info.
* fix(proxy): a fallback's group budget check skips the groups the request already holds
The auth-time reservation sits in the same counter the plain budget read uses, so a fallback in a group the request already reserved against was refused by its own hold once the group was within one estimate of max_budget. Groups without a hold keep the plain read.
* test(dynamic_rate_limiter_v3): the throttle metric test from main uses fallback_on_saturation instead of the removed steering
A saturation handoff carries no rate_limited_by reason, so it is never counted as a throttle.
* fix(proxy): the saturation handoff only fires under the fallback walk
_pre_call_with_fallbacks arms the request before the hooks run, so routes that call the hooks directly (batches, fine-tuning, polling, images) keep admitting or refusing an opted-in key on a saturated model as before instead of surfacing the handoff 429.
* style(proxy): keep the cast-ok reason within the line length
* fix(proxy): never invent a metadata bucket when arming the saturation fallback walk, take main's limiter metric tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): arm the saturation fallback walk in the request context, refuse fallbacks for agent-owned keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the saturation walk state out of the body, check the end user's allowlist on fallbacks, release a refused fallback's rate limit hold
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): only load the end user for fallback checks when a database is connected
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): grant pre-call fallbacks on the model an alias serves and release the rate limited model's hold before the walk
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): check a pre-call fallback's per-model budgets on its listed name and its served group, as auth does
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert the listed fallback name's budget blocks an aliased fallback, not the order of budget reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): release the dynamic limiter's reservation when the fallback walk leaves a model, and replay an exhausted walk with the requested model
The dynamic rate limiter admits through its own inner v3 limiter, so the walk's hold release never reached its team-share
reservation. It now exposes async_post_call_failure_hook, which refunds that reservation when the proxy refuses a request
after admission, and _release_pre_call_hold calls it next to the registered limiter.
An exhausted saturation walk replayed with the already aliased model name, so chained model_alias_map entries moved the
request on to a further alias. It now replays with the model argument the first attempt saw.
The hold-release test drives the real v3 limiter with one slot per key instead of recording hook calls.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type the dynamic limiter's failure hook request dict for the type-discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(files): match every spelling of a provider file id in the ownership check
enforce_provider_file_access matched the stored provider file id by exact string, and a Gemini output file
can be addressed as files/<id>, files%2F<id>, <id>/ or its full URL, all of which the Gemini file transformation
sends to the same file. Those spellings found no ownership row and fell through the no-row pass-through path.
canonical_provider_file_id collapses the spellings to the bare id; the ownership check looks up both the
spelling the caller sent and the canonical one, and store_batch_output_file_ownership stores the canonical id.
* fix(files): unwrap model-encoded ids and index canonical provider ids in ownership rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): assert the ownership lookup outcome instead of its query shape
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(files): map provider ids back to their unified file by canonical spelling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(files): record the bound on the ownership lookup's in filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(files): reduce any scheme or case wrapping files/<id> the way gemini does before the ownership lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(files): bind the reverse lookup match with a walrus instead of a Final inside the loop
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): assert the provider id lookup covers the requested ids instead of its exact list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit e8fc243a09)
* fix(files): keep non-Gemini storage URLs opaque so an upload name cannot index another key's provider id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): judge the provider id resolver by the mapping it returns only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): name the customer organization by alias with x-litellm-customer-org-alias
A request may carry x-litellm-customer-org-alias next to, or instead of,
x-litellm-customer-member-id. Alone, the alias charges and enforces the
organization level only. Next to the member header it must name the
member's organization, else 403. Unknown alias is 401, an alias shared
by several organizations is 400. Organization alias changes evict the
cached lookup.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for optional customer team and member ids
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* feat(proxy): register unknown customer members in the aliased organization's default team
A request carrying x-litellm-customer-org-alias and an
x-litellm-customer-member-id the proxy has never seen creates the member
under the organization's default team, on the organization's default
budget, and serves the request. Without an alias, or when the
organization has no default budget, the member stays unknown and the
request is still refused.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(proxy): declare the member registrar callable as a TypeAlias
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* feat(proxy): enforce customer org, team and member budgets and limits on the request path
Resolve x-litellm-customer-member-id at auth, walk member -> team -> org, enforce max_budget,
allowed_models, tpm/rpm and per-model limits at every level in the v3 limiter, honor the hierarchy
allowlists in alternate-model steering, and settle spend to the member, team and org rows.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(auth): customer hierarchy survives virtual key auth without a legacy end user
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(auth): keys pinned to a customer organization may only act for that organization's members
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): keep the customer hierarchy spend maps within the line limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(auth): check the key's customer org scope after the key is resolved
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): declare the hierarchy cache key family and keep type counts flat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(auth): resolve the customer hierarchy in the shared authorization gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop a member's cached hierarchy through the declared cache target
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(auth): keep the hierarchy reset within the line limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(auth): type the requested models as a sequence
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(spend): add daily customer usage and budget resets (#51493)
* feat(spend): add daily customer usage and budget resets
* chore(spend): document customer reset query bounds
* fix(spend): chart customer queue metrics and update recovery test
* fix(proxy): read stamped customer ids from either metadata bucket, alias-aware model check, broadcast member evictions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "feat(spend): add daily customer usage and budget resets (#51493)"
This reverts commit ccac47608f.
* fix(proxy): evict cached member hierarchies when team, org or template budgets change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): declare the auth cache dependency alias as a TypeAlias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Reapply "feat(spend): add daily customer usage and budget resets (#51493)"
This reverts commit 7542471e17.
* fix(spend): reconcile the daily customer spend table with main
Keep main's LiteLLM_DailyCustomerMemberSpend shape, make model_group nullable and add the identity unique index the bulk upsert arbitrates on in a new migration, drop the duplicate model and migration, and feed both the member-header hierarchy and the key's customer usage attribution through the shared daily spend projection so neither path double-writes.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(pagerduty): tolerate failure metadata without customer hierarchy ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(accounting): key daily customer spend rows on main's hashed identity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(pagerduty): read customer ids from failure metadata through a typed lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: moe-berri <moe@berri.ai>