* fix(proxy): scope the search free-tier counter to the request's provider
* fix(proxy): read the provider with get so partial spend payloads still get the free tier
* test(proxy): give the grounded spend payload fixture its provider
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): strip caller base_model from spend pricing and reject lone surrogates in multipart metadata
* fix(proxy): type the parsed multipart metadata so basedpyright does not grow reportArgumentType
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): redact Responses instructions and callback output
* fix(logging): avoid copying Responses output twice
* fix(logging): return redacted Responses output without mutation
* fix(logging): type Responses redaction payloads
* fix(logging): redact streamed Responses instructions
* fix(logging): redact request-side Responses instructions in logged params and spend log bodies
* fix(redaction): keep per-callback opt-out off shared instructions and skip redacted replays
A callback that sets message_logging=False no longer strips instructions
from sibling callbacks, the spend log, or the client's response. Dict
logging surfaces keep the reasoning config, and previous_response_id no
longer sends the redacted placeholder upstream as the system prompt.
* fix(redaction): redact instructions in an opted-out callback's own copy
A callback with message_logging=False now gets instructions redacted in its kwargs copy and response object, while sibling callbacks and the spend log keep them
* fix(redaction): give a message_logging false callback its redacted copy on sync calls
The sync log_success_event and log_stream_event branch now hands an opted-out callback the same copy as the async path.
* fix(redaction): redact a message_logging false callback's sync copy and keep failure estimates whole
Give an opted-out callback a copy with messages, input, prompt, the sync stream's
complete_response and its response object redacted, and stop the proxy's failed
request estimate from counting the redaction marker in place of real instructions
* test(redaction): wait on callback events instead of sleep polls
The new redaction tests now wait for each callback to log through an event it sets, so they no longer poll on a timer
* test(redaction): expect redacted follow-ups to replay no instructions
This PR stops bridged previous_response_id follow-ups from reusing
instructions the spend log stored redacted, so the integration test from
#45923 that asserted the replay now asserts that neither the original
instructions nor the redaction sentinel reach the provider
* test(proxy): type the redacted-instructions failure estimate test
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): send batch jobs to a configured control-plane endpoint
Bedrock batch create, retrieve and cancel always built the regional
bedrock.{region} host, so a VPC endpoint or compatible gateway could not
be used. Add aws_bedrock_control_plane_endpoint as a deployment setting
read by the signed HTTP path and the boto3 path, carry it through the
proxy credential filter, and refuse it in request bodies like the other
endpoint overrides. api_base stays out of it because on a Bedrock
deployment it points at the runtime host.
Fixes#45056
* fix(bedrock): read the control-plane endpoint from typed kwargs in job status
The named parameter added one unknown-argument error in litellm/batches/main.py,
which unpacks untyped kwargs into this function
* docs(bedrock): explain batch control-plane endpoint configuration
* fix(bedrock): keep the batch control-plane endpoint out of chat request bodies
A deployment that sets aws_bedrock_control_plane_endpoint also serves chat,
and the unregistered key was swept into the Bedrock request body, so every
chat call on it failed with "Extra inputs are not permitted". Register it
with the other Bedrock batch connection fields so it stays a LiteLLM-owned
kwarg on every route while batches still read it from litellm_params.
* test(bedrock): verify endpoint signatures without internal mocks
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): bill partial Responses API streams when the client disconnects
Streamed /v1/responses requests and the default-route OpenAI and Azure /v1/messages requests behind them logged a 499 with zero spend when the client went away mid-stream, while /v1/chat/completions already billed the chunks it had sent. The Responses API iterator now keeps the response.created payload and can hand the disconnect cleanup a response.incomplete event with counted usage and the served service_tier, and the messages-route byte stream and router wrappers delegate to it.
* fix(proxy): satisfy lint gates for partial Responses API disconnect billing
Make the disconnect helper public so the proxy, router and messages adapter can import it without reportPrivateUsage, type the SSE byte stream as an AsyncGenerator so aclose is known, read litellm_call_id off the logging object, and keep the new tests on Final locals with the recorder injected through dynamic_async_success_callbacks instead of litellm.callbacks.
* fix(router): bill the primary stream's tokens after a mid-stream hop and chat-bridge Responses streams on disconnect
The router's fallback wrapper now folds the primary Responses stream's relayed tokens into the fallback's disconnect event the way _combine_responses_fallback_usage does for a natural end of stream, and LiteLLMCompletionStreamingIterator (chat-only providers served through /v1/responses) builds its disconnect event from the chat completion chunks it collected. stamp_responses_usage_cost is now public so the bridge module can price that event.
* Potential fix for pull request finding 'CodeQL / Log Injection'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Every /v1/responses spend log asked its worker to flush the queue right away so a
chained previous_response_id could find the row. Only the chat-completions bridge
reads that row. Native Responses providers hold the conversation upstream, so their
rows now wait for the batch write like chat rows do
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): size pass drops the oldest spend log partitions first
On a range-partitioned LiteLLM_SpendLogs the size pass drops fully
elapsed partitions oldest first, never the current or DEFAULT one,
remeasures after each drop and falls back to oldest-first row deletes
only while still above the target
* fix(proxy): a partition drop rebases the size state instead of reading as a statistics reset
Bugbot on #47449: dropping a partition removes that child's n_tup_ins and
n_tup_del from the summed probe, which advance_size_state took for a reset
and reseeded from n_dead_tup. The pass now saves the counters of the last
post-drop probe as the state's base and scales the reclaimable count to
the surviving files
* style(proxy): keep the mutable-ok reason on the line it suppresses
* fix(proxy): size pass stops when a post-drop re-measure fails and keeps day partitions ensured
A failed storage probe after a partition drop no longer reads as under target. The pass logs a warning, skips the row step and saves no count, so neither live rows get deleted on the pre-drop numbers nor the next check reads the drop as a statistics reset. The partitioned size path also calls ensure_partitions before dropping, so a size-only setup keeps getting day partitions instead of writing into DEFAULT
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stabilize auth and team delete checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(unit): make loop-blocking and tampered-signature checks deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(unit): isolate model-cost state and synchronize loop progress
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(unit): fail loop-progress handshakes when the blocking call never starts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): opt-in free-tier allowance for Google Search grounding spend
* fix(proxy): declare the search free tier counter's service target
* test(cost_map): allow search_free_requests in the cost map schema
* fix(proxy): skip cache hits and never free more than the billed cost in the search free tier
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds general_settings.maximum_spend_logs_size,
maximum_spend_logs_cleanup_target_size and
maximum_spend_logs_size_check_interval, the spend_log_size_check job,
maximum_spend_logs_cleanup_cron precedence, oldest-first bounded deletes
down to the target, storage fields in the run summary and the
litellm_spend_log_storage_bytes gauge, with a Postgres integration test
covering the post-VACUUM no-refire case
Resolves LIT-9831
Fixes#44589
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
_failure_spend_rows yields the final row plus one re-keyed row per lifted billed_failed_attempts entry (<request_id>-attempt-<position>, the attempt's usage, cost, retry count and deployment id), each written through DBSpendUpdateWriter.update_database so it also deducts from key, team and user spend
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): stop update_cache from resurrecting evicted auth cache entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): update update_cache pins for synchronous write-back
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): wrap long assert in update_cache tag test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover refill race and pin TTL with injected clock
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): count global proxy spend for every request with a user, cached on this worker or not
* fix(proxy): add global proxy spend with an atomic Redis increment that never re-creates a reset counter
The global proxy spend bump read the total, awaited, and wrote old + cost back
from a background task, so a budget reset that deleted the key in between got
the pre-reset total written back, and workers overwrote each other's totals.
RedisCache.async_increment_if_present runs INCRBYFLOAT only when the key
exists, keeping its TTL, and this worker's copy is then set from the shared
total with no await between its read and write. The bump now runs whenever a
proxy max_budget is set, matching the DB writer, and the armed post-call read
and the batch-read declaration it used are removed.
* fix(proxy): report the global spend Redis increment to service metrics and request traces
The global proxy spend increment now emits Redis service success and
failure events under the spend_counters target and passes the request
span through, so it shows as redis.incr spend_counters like the write
it replaced.
The unit tests run the real RedisCache increment against fakeredis
instead of mocks, and a new test checks the span lands under the
request span on success and on a Redis failure.
* test(proxy): keep the spend counter Redis fake off the network
* test(proxy): count increment_if_present as a shared-cache call in the key-family guard
* fix(proxy): keep the newer global spend total when Redis replies arrive out of order
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): resolve AWS region and secret lookups off the event loop
Bedrock request paths that resolve their region, endpoint or bearer token
inside the async handler now do it in the AWS signing executor, so a slow
secret manager read no longer stalls every other request on the worker
* fix(bedrock): type the STS client kwargs and claude platform environment override
Keeps the basedpyright counts at or below the merge base after the off-loop wraps, by typing the STS client kwargs as a TypedDict, the claude platform environment override with concrete dicts, and the realtime region call through the typed getter
* test(bedrock): detect event loop stalls without wall-clock timing
The Vault fake now answers a read only after the test loop runs a callback, so a read made on the loop thread is caught deterministically instead of by a sleep and a heartbeat threshold
* feat(proxy): lift every earlier billed timed-out retry attempt onto the failed request
_billed_earlier_attempts reads the router's previous_models records, keeps the Timeout ones other than the final attempt (matched on deployment_id plus attempted_retries, since the count restarts after a fallback hop and the fallback handler re-logs the exhausting attempt) and prices each at the deployment that served it. post_call_failure_hook lifts the tuple as billed_failed_attempts; nothing writes it yet. previous_models joins the router-reserved metadata fields so a client-planted record never reaches it
* fix(proxy): price earlier timed-out attempts from the prompt estimate only
The final row may lift a mid-stream recovered usage (prompt plus delivered chunks) or a failed_attempts_usage sum; earlier attempts never produced output, so they are priced from _estimated_prompt_usage, and when failed_attempts_usage already bills them into the final row none is lifted again
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(count-tokens): translate chat tool use before Claude token counts
Responses and chat tool use (function tools, function_call items, tool messages) is now converted the way Claude chat inference converts it before the Anthropic, Azure AI and Vertex partner counters call count_tokens. Translation errors answer 400 before any provider call. Anthropic-shaped requests and /v1/messages/count_tokens keep their body as sent.
* test(count-tokens): compare Claude count bodies with inference for Responses tool use
* fix(responses): forward the caller's Responses body to OpenAI token counts
OpenAI deployments now get the caller's /v1/responses/input_tokens body as sent, minus LiteLLM-only fields, so OpenAI's own count and errors come back. previous_response_id is checked for ownership like inference and routes to the deployment that served it
* fix(responses)!: count Claude Responses bodies the way inference sends them
Non-native Responses counts now run the body through the inference bridge, so
text.format, tool_choice, reasoning, built-in tools, and file_search count the
way inference sends them on Anthropic, Azure AI, Vertex, and Bedrock. The
OpenAI native count body uses the drop_params-resolved tool fields.
* fix(responses): honor drop_params and replay previous turns in Claude counts
Claude input_tokens counts now read the same drop_params setting inference merges from the deployment, router defaults, and request, so a forced tool_choice downgrades to auto under drop_params. previous_response_id replays the stored turns through inference's session loader, and chat messages carrying fields beyond role and content go through the provider's chat translation
* fix(responses): keep proxy-only request keys out of OpenAI counts and map chat tools in Bedrock counts
* fix(count-tokens): import on Python 3.11 and keep stored prompts in OpenAI counts under drop_params
* fix(responses): leave tool_choice out of counts when drop_params drops every tool
* fix(count-tokens): count Mantle options, drop cache marks, and reject input inference cannot send
The Bedrock Mantle fallback now counts thinking, tool_choice and output_config
like the runtime count. Claude Responses counts leave out cache_control marks
again, so more than four marks count instead of failing. Counts answer 400
before any provider call for user blocks chat inference refuses and for
Responses tools with no Chat Completions equivalent, instead of dropping the
block or counting a placeholder tool. Adds the wire tests for these paths
* fix(count-tokens): count Responses tools the way the inference bridge sends them
A Claude count now hoists additional_tools input items into tools, strips a malformed tool flag under drop_params and rejects it without, and leaves tools out when the deployment lists tools in additional_drop_params, so the count matches the body Claude inference sends for the same request
* fix(count-tokens): keep Bedrock runtime counts option-free and send OpenAI the provider file id
* test(count-tokens): capture the Anthropic count request at the HTTP layer instead of patching the handler
* fix(count-tokens): ignore a caller-sent file id mapping and keep non-string file ids as sent
* test(responses): billed background responses count toward the creator's project and model budgets
* fix(responses): carry the creator's project and per-model budgets on the billed background response
* fix(responses): count a key's budget tier toward background per-model budgets
* fix(responses): bill a background response's project only while the key stays on its team
* revert(responses): keep billing the creating key's project after a team move
A key can only keep or detach its project, so the billed project always pairs with the team it was created under. This reverts commit ded4f084c5.
* fix(responses): bill a background response under the model name the client sent, with its project alias
* fix(responses): name the team's public model as a billed background response's model group
The model group lookup now passes the creating key's team, so a team model or team wildcard bills under the public name a normal call logs and per-model budgets on that name move
* fix(proxy): let proxy admin keys read raw cloud storage file ids on GET /files/{id}/content
Since #30595 the endpoint answered 400 for every raw gs:// or s3:// id, which broke the
documented unmanaged Vertex batch flow at its last step, while a double-encoded id slipped
past the guard entirely. Raw ids are now readable by proxy admin keys only, every other key
gets 403, and the guard decodes the id fully before deciding, so encoding depth no longer matters
* fix(proxy): decode nested percent-encoded file ids iteratively
An id percent-encoded a thousand or more times sent the recursive decoder past the
interpreter's recursion limit, so a proxy admin key got a 500 instead of a verdict on
the id. The decoder now walks the encodings in a loop, so any depth resolves
* fix(proxy): cap the decode passes the raw cloud storage file id guard makes
Any authenticated key could send a file id wrapped in tens of thousands of percent-encoding layers and the guard would unwrap every one of them, each pass walking the whole id, so a 120 KB request line held a worker for about five seconds and a 2 MB one for minutes. The guard now unwraps at most eight layers, and an id still changing after that is treated as a raw cloud storage id, so it stays admin-only instead of being decoded any further
* refactor(proxy): type the decode step the raw cloud storage file id guard folds over
* test(proxy): return a streaming result from the admin raw id test mock now that vertex_ai file content streams
* test(proxy): send raw cloud ids to the content route the way uvicorn delivers them
Starlette's TestClient decodes the path twice, so the twice-encoded cases reached the route fully decoded and never hit the decode loop. The cases now name the id the route receives, and the helper tests take a typed quote helper and Final constants.
* test(proxy): mark the raw cloud id test locals Final
* fix(responses): keep strong refs to stream and websocket logging tasks
* test(responses): patch _spawn_logging_task instead of asyncio.create_task
The success-logging dispatch in _log_completed_response now goes through
_spawn_logging_task, so tests that patched asyncio.create_task no longer
observed it (and the sync ones hit get_running_loop with no loop).
* test(responses): assert stream and websocket logging tasks survive garbage collection
Adds regression tests that force a collection while the logging task waits
on an otherwise unreferenced future, and types the new test parameters
* test(responses): drive drain and GC tests through a real logging object with gated callbacks
* test(responses): use asyncio.wait_for so the logging tests run on Python 3.10
* test(guardrails): type the spawn_logging_task tracking wrapper
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): map baggage values to budget tags
* fix(proxy): preserve mapped tags in the parsed request cache
* fix(proxy): preserve mapper accounting and tag policy
* fix(proxy): preserve baggage tags across native transports
* fix(proxy): keep mapper rejection and passthrough logging safe
* fix(proxy): scope baggage tag mappings to billed inference and keep them out of tag routing
Mapped tags now apply only to billed LLM API routes, skipping token counting,
model listing, MCP and A2A. They classify spend and budgets but never select
deployments unless a key, team or client set the same tag. WebSocket inference
returns 403 only while a required rule exists. A tag created with /tag/new resets
its spend counter so its budget counts from creation. The General settings editor
keeps a new mapping unsaved until every required field is filled in.
* fix(proxy): keep client-tag rejection identical with baggage, map A2A calls, skip request previews
* fix(proxy): keep mapped tags out of requester metadata, answer 503 on tag lookup errors, map custom pass-through routes, check the key before the WebSocket 403
* fix(proxy): keep caller-sent tags routable, keep mapped tags out of policy and guardrail matching, honor allow_requests_on_db_unavailable, reject control characters in baggage values
* test(proxy): cover baggage tag mappings end to end
Adds integration cells for request_tag_mappings across chat, messages,
responses, A2A and custom pass-through spend, required and budgeted
rules, the WebSocket refusal, unbilled routes, reject_clientside_metadata_tags,
tag routing and policy isolation, a dropped tag read with and without
allow_requests_on_db_unavailable, and the /tag/new spend counter reset
* fix(proxy): keep base tag merges for requests that map nothing, accept JSON-string metadata tags, count litellm_metadata caller tags
Requests with no mapped baggage now reuse the cached body exactly as before, so header and key tag merges still normalize metadata for routing. The client body copy is made only when a tag maps, and the base merges are mirrored onto it. A request that maps nothing no longer gets a 400 for non-object metadata. caller_tags now counts tags sent in both metadata and litellm_metadata. Optional mappings no longer reject JSON-string metadata tags under reject_clientside_metadata_tags.
* fix(router): keep baggage-mapped tags from selecting tag-scoped pre-routing strategies
Auto, complexity, adaptive and quality router selection, the consumed-tags stamp and compression policy lookup now read request tags without baggage-mapped tags, unless the caller sent the tag or the key, team or project carries it, matching deployment tag filtering.
* feat(proxy): parse baggage headers into request tag mappings
* fix(proxy): match plus-sign baggage keys and keep deployment tags in guardrail modes
* fix(proxy): keep baggage-mapped tags behind key, team and deployment tags and serve non-object or invalid DB mapping settings like unset
Mapped tags are added after the key, team, deployment and caller tags, so a tag-mode guardrail picks the same mode with or without baggage. A non-object metadata field maps like empty metadata, and an invalid request_tag_mappings value loaded from the DB is ignored with an ERROR log instead of failing every request
* fix(router): merge deployment tags in linear time and keep malformed client tags as sent under mapped baggage
Merge serving deployment tags into the request tags in one pass, so a large client tag list no longer blocks the event loop. Leave a non-list metadata.tags untouched when baggage maps a tag, so the request gets the same answer as without baggage. Add the C17 integration cell for the team tag's guardrail mode under a mapped deployment tag.
* test(spend): cover string metadata.tags under mapped baggage
* fix(proxy): check mapped tag budgets when metadata.tags is a non-list or holds non-strings
Mapped baggage tags are applied again after the header and key tag merges, so
a header or key tag that replaces a string metadata.tags no longer hides the
mapped tag from the tag budget check. An empty or false metadata.tags is read
like a missing one, and a tags list with non-string items is kept as sent with
the mapped tags added after it, instead of failing auth with a 401
* fix(streaming): restore the token estimate for streams that send no usage
A stream without stream_options got a made-up Usage(0, 0) written into its
final chunk's hidden params. Since #42323 the chunk builder treats that as a
provider-reported zero, so these streams were logged and billed at 0 tokens
and $0 instead of the tokenizer estimate. Only attach the summed usage when
some chunk actually carried usage.
* test(streaming): use a fixed start time and check the estimated cost
* fix(streaming): keep reported usage on the synthesized finish chunk and test it
* test(streaming): cover explicit zero usage reported by the provider
* fix(streaming): set the token estimate on a copy of the final chunk and ignore usage with no counts
* fix(streaming): keep final chunk edits in the logged response when the provider sends no usage
* fix(streaming): mark the unreported usage placeholder so every chunk rebuild falls back to the estimate
* test(streaming): wait on the success callback event instead of sleep-polling
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): drop router_settings_override before routing and apply it on every branch
route_request only popped router_settings_override inside the branch that
runs after the api_key/api_base, comma-separated batch and user_config
branches. Requests taking one of those earlier branches forwarded the field
to the provider, which Anthropic rejects with a 400, and silently lost the
key/team num_retries, fallbacks and timeout. The override is now popped and
merged into the request before any branch is chosen, keeping the existing
precedence where request values win and None values are skipped
Fixes#43582
* fix(proxy): apply key/team fallbacks when the request carries api_key or api_base
The previous commit skipped fallbacks, context_window_fallbacks and
content_policy_fallbacks from router_settings_override whenever the request
had api_key or api_base. That broke the scenario from #43582, where a
trusted pre-call hook sets api_key and the key/team router_settings still
need to apply. All per-request router settings are merged again on every
branch, with None values skipped and request values still winning
The tests now check that fallbacks run on the api_key and api_base paths
and that the provider never receives router_settings_override
* fix(proxy): answer 400 for a non-object router_settings_override and treat null as absent
* fix(proxy): drop a malformed router_settings_override under drop_params
* fix(proxy): route key and team router settings like any request, honor user_config and global drop_params
* fix(proxy): keep Router-only override settings out of direct provider calls
Calls that route_request sends straight to litellm (evals, endpoints that need
no model, pass_through_all_models, the video fallback and the no-router paths)
now leave out context_window_fallbacks, content_policy_fallbacks and
model_group_retry_policy merged from key or team router settings. Only the
Router reads them, and providers reject them as unknown parameters.
* test(proxy): drop a comment from the router settings override test
* fix(proxy): keep key router settings on the model read-through retry
route_request now merges router_settings_override once, before the first
attempt. The retry after a model read-through used to pop the merged
enable_tag_filtering again and recompute an empty Router-only set, so a
model created on another instance was served without the key's tag
filtering
* test(proxy): check the read-through retry keeps key tag filtering through a real router
The regression test now routes through a real Router with two tagged deployments from the database instead of replacing Router.acompletion. Before the fix the retry picks the lower-order untagged-for-request deployment, after it the tagged one
* fix(proxy): keep key Router-format fallbacks out of direct provider calls
Key and team fallbacks use the Router's model-group mapping. litellm.acompletion reads each mapping as a fallback config, so a direct call retried the same model with the mapping merged into the provider body. The override's fallbacks now stay out of direct calls like the other Router-only settings, while num_retries and timeout still pass
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): schedule default_priority requests once and reject a non-integer priority
A router configured with default_priority failed every request that carried no
priority with a 500 naming a missing 'priority' argument, and an explicit
priority re-entered acompletion and hit the same error. acompletion now resolves
the priority once (the request's integer, else the default, with 0 kept as a
real priority) and schedules through _schedule_factory directly, while
schedule_acompletion delegates to acompletion so the two paths share one queue
pass. A non-integer priority answers 400 naming the param, or is dropped under
drop_params, instead of leaking to the provider or crashing the scheduler.
* fix(router): honor router-level drop_params and schedule prompt-management models under a priority
* fix(router): drop a non-integer priority when the model's deployments set drop_params
* fix(router): resolve the drop_params deployment scope for team routes and deployment ids
* fix(router): read the request and router drop_params flags without a loop-assigned Final
* refactor(router): type the prompt-management wrapper's return and mock the provider in its test
* perf(router): consult drop_params for the priority only when it is not an integer
* test(integration): assert the new priority contract for non-integer and zero priorities
* docs(router): describe default_priority as the priority every request without one takes
* fix(router): parse default_priority once and apply it to text completions and the queue route
A digit string default is taken and any other non-integer default is ignored with a warning instead of failing every request. atext_completion and /queue/chat/completions resolve the priority the way acompletion does: a request without one takes the default, an integer queues once, and a non-integer answers 400 or takes the default under drop_params. Integration cells cover every default-priority surface on an owned two-instance rig.
* fix(router): type the priority dispatch so the lint budgets hold
* fix(router): read priority like the scheduler did, require a priority on the queue route, and keep litellm's drop_params order
* fix(router): drop a priority named in additional_drop_params and report the queued priority on streamed queue responses
* fix(router): serve deployment-id requests without waiting behind the queue and ignore a boolean default_priority
A request whose model is a deployment id now counts that deployment as healthy in the
scheduler, since routing by id already ignores cooldown. Before, the scheduler found no
healthy deployment for an id, so the request waited behind every entry ahead of it.
A boolean default_priority (true, or YAML off) is ignored at startup with a warning,
like any other value that is not a whole number.
The scheduler integration tests now share worker state for the defaulted pair, cool a
group through /v1/responses so no stale in-memory queue is left behind, and use "high"
instead of "1" for the non-integer cases. The concurrent burst test no longer asserts
that the queue drains, matching the explicit-priority burst test, since the scheduler's
read-modify-write queue can race across workers.
* test(router): keep the queue drain check as a BUG-skipped integration test
* fix(router): queue default_priority chat requests once instead of failing them
* test(router): open with the default_priority regression tests alone
* fix(router): honor deployment drop_params for a malformed priority on a request routed by provider model name
* fix(router): scope tag routing to prefixed tags once tag_routing_prefix is set
With router_settings.tag_routing_prefix configured, an unprefixed caller tag
(for example a spend tag such as user_id:234) no longer drives tag routing, so
it can no longer refuse a model group whose deployments are untagged.
Key/team/project tags (metadata.inherited_tags) keep applying with or without
the prefix. Rows of batches the proxy runs itself now carry those tags as
inherited_tags too, so they keep routing on them once the prefix is set.
Takes over #39903.
Co-authored-by: Deepanshu Lulla <deepanshu.lulla@gmail.com>
* test(integration): pin that proxy-run batch rows keep key/team tag protection
* refactor(router): filter routing tags from the raw request tags; deterministic tests and key-tag cells
* test(integration): audit cells for tag_routing_prefix: project tags, forged inherited_tags, odd tag values, outage, fallback, user-agent routing, concurrent burst
* test(integration): proxy-run batch rows keep key and project tags too
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
Co-authored-by: Deepanshu Lulla <deepanshu.lulla@gmail.com>
* feat(context_management): honor clear_at_least, exclude_tools and clear_tool_inputs in the clear_tool_uses polyfill
The clear_tool_uses_20250919 polyfill honored only trigger and keep and
ignored the other three fields with a warning. It now follows the Anthropic
API, as measured against it:
- keep counts clearable tool uses only; uses of exclude_tools are never
cleared and count toward neither keep nor clear_at_least.
- clear_at_least clears oldest first in batches of at least that many
tokens and leaves an incomplete last batch, so the cleared prefix only
grows between requests and each batch costs one prompt-cache miss.
- clear_tool_inputs (true or a list of tool names) also empties the input
of the cleared tool uses.
Invalid values are ignored with a <field>_invalid warning on the applied
edit.
* refactor(context_management): build the clear_tool_uses selection without mutable state
* test(context_management): cover a history made only of excluded tools
* fix(context_management): leave tiny tool results out of clear_tool_uses like the API
The API neither counts nor clears a tool use whose result is 1 to 46 bytes
of text, whatever the tool or error flag; empty results still count. Probed
against the API with crafted histories. Test data under 46 bytes is
lengthened so those tests keep exercising clearing
* fix(context_management): count tiny tool results when clear_tool_inputs covers their tool
The API leaves a use whose result is 1 to 46 bytes out of clear_tool_uses
only while its input is kept. When clear_tool_inputs is true or names the
use's tool, the API counts it toward keep and clears it like any other use.
* fix(context_management): count cleared inputs and images toward clear_at_least, answer malformed fields with a 400
* fix(context_management): validate clear_tool_uses edits before any editor runs, drop only the malformed edit under drop_params
* fix(context_management): drop only the malformed clear_tool_uses fields under drop_params
A malformed clear_at_least, exclude_tools or clear_tool_inputs no longer drops the whole edit under drop_params. The edit still runs with its well-formed fields, so the clearing that worked before is kept. Also types the new tests and swaps the hand-rolled spy router for an AsyncMock
* test(context_management): pin the 46 byte cutoff in UTF-8 bytes and a batch closing at exactly clear_at_least
* fix(context_management): leave a non-string tool_use id for the provider, size lone surrogates, and read drop_params strings
* fix(context_management): read a drop_params string the same way on count_tokens and both /v1/messages paths
count_tokens still checked drop_params with is True, so a body drop_params of "true" answered 400 there while /v1/messages dropped the malformed field. Both routes and the sync path now use drops_unsupported_params.
---------
Co-authored-by: AlonSoko <25619863+AlonSoko@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): expose supported_endpoints on /model_group/info
* feat(router): route batch creates to mode batch deployments in mixed model groups
* fix(router): prefer batch deployments after eligibility filters and in group-name credential lookups
* fix(router): keep group-name credential lookups on the first deployment and prefer batch deployments before cooldown
Model-encoded ids carry only the group name, so preferring the batch deployment there moved ids created before the upgrade onto its credentials. Batch creates now pick batch-capable deployments before health checks and cooldown, routing and group info share one batch-capable check, the cost map's /v1/batch is dropped only for Azure, and /model/info applies the same per-deployment rule
* test(router): map _listed_endpoints to its /model/info tests in the router coverage check
* fix(router): skip only Azure chat deployments for batch creates and list batch support per caller team
Narrow the batch create filter to Azure and azure_ai deployments that are not mode batch, so OpenAI groups keep
their chat deployments during cooldown and under tag filters. Mark batch deployments by mode batch only, since
stored endpoint lists from older Admin UI saves copy the cost map. Leave /model/info unchanged so the Admin UI
never saves a computed endpoint list, and pass the caller's team into /model_group/info.
* test(proxy): let the model hub router fakes take the caller's team_id
* fix(router): gate batch-create routing on deployments the caller's team may use and list only /v1/batch for Azure batch deployments
* fix(router): skip a malformed model_info.supported_endpoints instead of failing group info and chat
* fix(router): apply the batch create filter after cooldown, managed files, and tags
The batch filter ran before health checks and cooldown, so a batch create on a mixed
Azure group answered with a router 429 or 401 when cooldown, tags, or a managed file's
targets left no batch deployment, where main sends the create on and returns Azure's
400. The filter now runs right before order filtering, after every eligibility filter,
so those cases answer as main does and creates still skip Azure chat deployments
whenever a batch deployment is left
* test(router): annotate the batch routing test assignments with Final
* fix(videos): keep model_id in proxy video ids so per-model keys resolve on status/content
Provider transforms encode returned video ids with an empty model_id, so
GET /v1/videos/{id} and /content cannot resolve the deployment (and its
per-model api_key) and fail with invalid_api_key.
Re-stamp the returned VideoObject id with the resolved model_id in
video_generation, video_status, video_remix, video_edit and
video_extension.
Fixes#33423
* fix(videos): return a copy instead of mutating the response, read hidden params via helper
* test(videos): fully type the new video id re-stamping tests
Annotate every parameter and return added by this PR (hidden_params,
harness, response, -> None), drop Any from the test helper, mark locals
Final, and narrow the endpoint/helper results with isinstance(VideoObject)
instead of reading .id off an untyped value. Route the mocked processor
response through a typed Harness.respond_with, read NotRequired
DecodedVideoId keys with .get, and name the rule and reason on the
_hidden_params suppressions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(videos): fill only an empty model in returned video ids
Keep an id that already names a model as is, so the asked-for name,
routing group and served group in it still drive access checks and
routing. Fill an empty model from the model the caller asked for
before the routed deployment id, and read the request model through a
typed view so the type gate does not grow
* fix(videos): route video calls to the deployment that created the video
Video ids now carry the id of the deployment that created them. Status,
content, remix, edit and extension calls go to that deployment when it
belongs to the requested group, so a group spread over two provider
accounts no longer polls the wrong one. A provider model shared by
several groups resolves to the creating deployment's group, and the
asked-for model is written into an id only when the router can route it.
* fix(videos): pin video calls only to deployments the caller may use
Pin a video id's creating deployment at deployment selection, and only when the caller may use that
deployment in the requested group, so aliases, routing groups and access groups still apply and the
spend label and fallbacks keep the requested group.
A status poll keeps the deployment its id names and takes the deployment that answered when the id
names none. An alias that routes to the creating deployment's group is kept on status and content.
* test(router): reset module fallback globals between unit tests
* fix(videos): check and route polls by the name the router serves
A poll on a video created through a provider wildcard is checked against the name the wildcard serves, a team model is checked by its public name, a poll stamps the deployment that answered, and the sync Router pins video calls the same way the async path does
* fix(videos): pin polls only among the deployments routing allows
Video calls route by group again and the creating deployment is picked only from the deployments left after access, team, health, cooldown, budget and tag filters, so a pinned poll no longer skips them. Wildcard groups now pin too. Adds tests for cooldown, wildcard pinning and character lookups through a provider wildcard, and return types on the auth tests
* fix(videos): scope the video pin to video calls, keep fallback-served videos reachable, skip cooldown on video 404s
* fix(videos): stamp the requested group only when one of its router fallbacks served the create
* fix(videos): follow chained router fallbacks for fallback-served videos, test with a real Router
* test(router): list the video pin helper with its sibling in the router coverage exemptions
* refactor(auth): walk chained router fallbacks without recursion
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(streaming): keep finish_reason and provider usage when rebuilding streamed text completions
stream_chunk_builder_text_completion read finish_reason from chunks[-1] and recounted usage from messages. With include_usage the last chunk is a usage-only trailer, so finish_reason came back None (or raised IndexError when the trailer had no choices), and prompt_tokens was 0 because a text-completion prompt is not in messages. Scan for the last finish_reason and prefer the provider-reported usage, as the chat path already does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(streaming): type the text completion chunk builder so its scans add no basedpyright debt
Read chunk fields as typed attributes instead of untyped subscripts, which
clears the reportUnknownArgumentType breach the lint gate flagged
* fix(streaming): keep the provider's full usage, including token details, when rebuilding text completions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(streaming): rebuild dict-form text completion chunks again
stream_chunk_builder routes dict chunks holding TextChoices to the text
completion builder, which now reads attributes. Convert them to
TextCompletionResponse at the routing point.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(streaming): accept dict chunks in the text rebuild and derive its usage like the chat path
Moves the dict to TextCompletionResponse conversion into stream_chunk_builder_text_completion so direct callers keep working, and builds usage through ChunkProcessor.calculate_usage over the chunks that report counts, so a trailer without total_tokens keeps its prompt and completion counts and the result is a fresh Usage
* fix(streaming): read text completion usage through the shared usage extractor so usage dicts on typed chunks rebuild
* fix(streaming): merge text completion logprobs across chunks and rebuild dict text chunks as text completions
* fix(streaming): rebuild one text completion choice per index and keep null logprobs fields null
* fix(streaming): keep dumped chat chunks with a choice-level text on the chat rebuild path
Route a dict chunk to the text completion builder only when its choice has `text` and no `delta`.
The Responses bridge snapshots chat chunks with model_dump(), and Together's chat choices carry `text`
next to `delta`, so streamed /v1/responses calls to Together ended without response.completed.
Group text completion choices by index in one sorted pass, so rebuilding an n>1 stream costs about
the same as rebuilding an n=1 stream of the same size.
* fix(streaming): keep text completion rebuilds working on mixed chunks and unreadable logprobs
A chat chunk inside a text completion stream now converts to a text chunk that keeps its finish_reason and adds no text. A dict chunk whose logprobs fail validation is read again with logprobs dropped, and logprobs that are not a Logprobs object keep the last value sent. Each of these inputs raised or lost data at the previous commit, and each now rebuilds without raising
Also adds a Purview test showing a multi-choice text stream is blocked like the non-streamed response, and Final annotations on the new test locals
* fix(streaming): rebuild text completion choices without an index at index 0
Text completion choices whose index is None now rebuild under index 0, as on the merge base, where the sort by index raised at the previous commit. A chat chunk inside a text completion stream now keeps the usage it reports, and a choice whose logprobs mix a Logprobs object with another shape keeps the last value sent, as on the merge base
* fix(streaming): rebuild text completion choices with a string index under that int
A text completion choice whose index is a string now rebuilds under the int it names, and one whose index cannot be read as an int rebuilds at index 0, as on the merge base. A string index mixed with an int index raised TypeError at the previous commit, and a string-only index came back as a string
* test(streaming): drop a docstring that repeats its test
* fix(streaming): fall back to choice 0 for an infinite text completion index
A float infinity index made int() raise OverflowError, which the index
coercion did not catch, so the rebuild failed where the merge base
returned choice 0.
---------
Co-authored-by: Lai Wei <8022184+roywei@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): cover gzip and zstd request bodies on /v1/systemone
* fix(proxy): decode zstd and gzip request bodies before auth reads them
* fix(proxy): bound request body decompression and keep it inside CORS
Limit the decoded size by an expansion ratio and the operator's max_request_size_mb, pass bodies without the coding's magic number through, answer 415 when zstd is unavailable, and register the middleware innermost so errors carry CORS and security headers
* fix(proxy): budget decoded request bodies per worker and forward plugin bodies as sent
Each worker holds at most MAX_DECOMPRESSED_REQUEST_BODIES_IN_FLIGHT_MB of
decoded bodies and answers 503 past it. /plugin-proxy/ bodies reach the
plugin as sent, a zstd skippable frame counts as zstd, the decoded-size 413
names the decoded limit, and an integer max_request_size_mb stays an integer
in the 413 text.
* fix(proxy): read the magic number before size limits and reserve declared decoded sizes
* test(proxy): pin gzip mtime so parametrized test ids match across xdist workers
* fix(proxy): parse max_request_size_mb the same way in the auth size check
* fix(proxy): pass Claude Code OTLP export bodies through undecoded and skip the size parse without premium
* test(proxy): cover client disconnects, chunked bodies past the cap, and body-less request messages in the decompression middleware
* test(proxy): name the decompression test cases so CI logs stay small
* feat(dashscope): price Qwen implicit context-cache reads separately
* fix(dashscope): normalize implicit cache price strings
* fix(dashscope): price explicit cache hits through an explicit_ override key
Keep cache_read_input_token_cost as the default (implicit) Qwen cache-read
rate, which the cost map already holds, and add
explicit_cache_read_input_token_cost for hits DashScope tags with
cache_type "ephemeral". Untyped and unknown cache types keep the base rate,
so older releases reading the remote cost map keep billing unchanged.
Add explicit cache-read and cache-creation rates to the qwen3.7-max rows.
* fix(types): accept explicit_cache_read_input_token_cost as a constructor keyword
* fix(dashscope): inherit the explicit cache price on custom deployments and keep tier rates all-or-nothing
Deployments that set only input and output prices now inherit
explicit_cache_read_input_token_cost from the cost map, so explicit hits bill
at the explicit rate. A pricing tier without its own explicit rate bills its
own cache_read rate instead of the model-level explicit one, matching how an
off-peak block falls back.
The DashScope calculator now delegates the explicit rate to the shared
resolver, and the register_model warning is back to main's wording.
* fix(dashscope): keep a deployment's own cache-read price for explicit hits and ignore a non-string cache_type
* fix(dashscope): keep cache_type through Responses usage and let a registered cache-read price cover explicit hits
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): keep answering requests while a cold /openapi.json builds
* fix(proxy): build the OpenAPI schema off the event loop so a cold /openapi.json no longer stalls the worker
* perf(proxy): reuse the rendered /openapi.json body until the schema changes
* fix(proxy): scope the openapi build's pydantic patch to the build
The schema build swapped a process-wide pydantic hook, which live requests saw once the build moved to a thread, and overlapping builds could leave it installed. The hook now checks a context variable set only inside the build. The proxy test counts liveliness answers instead of timing them.
* test(proxy): hold the cold openapi build on an event instead of pacing on a clock
* fix(proxy): serve a replaced app.openapi like FastAPI's own docs route
* test(proxy): build the cold-docs test on the proxy module's current app
* test(proxy): freeze the model that holds the cold docs build
* feat(proxy): spend log size settings, storage probe and the reclaimable-bytes count
Adds maximum_spend_logs_size, maximum_spend_logs_cleanup_target_size and
maximum_spend_logs_size_check_interval, size_in_bytes, the pg_stat_user_tables
storage probe, and SizeCleanupState / advance_size_state, the running count of
reclaimable bytes kept in LiteLLM_Config that the size pass (next PR) compares
against the limit and the target. Nothing is scheduled or deleted yet.
* refactor(proxy): keep the spend log size settings out of the state layer
The three general_settings keys, their schema.d.ts rows, the SizeRule
parsing and the scheduler constants move up to the PR that acts on them,
so merging this layer alone changes nothing an operator can set
* test(proxy): cover the storage probe and the size state round trip
The probe, the LiteLLM_Config load and save and SpendLogStorage.after_deleting
had no caller in this layer, so codecov/patch sat at 67 percent
* fix(proxy): a stale storage probe carries the size state forward instead of reseeding
Bugbot on #47449: with zero live and dead rows the reseed made
bytes_per_row the whole file. advance_size_state now keeps the prior
count on a stale probe and, with no prior count, treats the files as
fully reclaimable so nothing is deleted on its say-so
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): match partial wildcards against prefix-stripped model ids (issue #46384)
Refactor model checks to improve filtering logic and type hints.
* test(proxy): partial wildcard listing regression coverage (issue #46384)
* fix(proxy): drop unneeded annotation on suffix list (LIT001)
* fix(proxy): annotate suffix list with reasoned mutable-ok (basedpyright)
Change suffix_appended_wildcard_models to a typed list.
* Add lock timeout before creating telemetry report table
Set local lock timeout to 10 seconds before creating the telemetry report table.
* fix(proxy): list alias wildcards as routable ids, not raw catalog ids
An alias whose suffix collides with a catalog sub-prefix (e.g. public
"cloudflare/@cf/*" over deployment "cloudflare/*") took the partial-filter
branch and listed raw catalog ids. Routing substitutes the captured suffix
into the deployment pattern, so those listed ids routed to targets that do
not exist. Only use filter semantics when the deployment repeats the
wildcard suffix (or no deployment is available); aliases prepend their
prefix to provider-stripped catalog ids so every listed id routes back to
a real catalog model. Addresses Greptile review on #46390.
* fix(proxy): derive partial wildcard listings from the deployment's routing pattern
List an id for a partial wildcard only when routing it through the deployment's
pattern lands on a catalog model, so aliases, catch-alls, and key filters over a
wildcard deployment list ids that answer when called. /model/info expands a bare
* row the same way /v1/models does.
* fix(proxy): drop fragment ids for a catch-all over a partial wildcard and route bare aliases through the deployment pattern
A * catch-all over a partial deployment now lists no catalog ids, a bare alias like my-cf/* over cloudflare/@cf/* names ids by the deployment pattern, an alias whose wildcard count differs from its deployment lists nothing, and /model/info rows show the model each listed id routes to
* fix(proxy): keep listed ids on /model/info alias rows and clean up test-registered models
* fix(proxy): list catalog models through routing-prefix wildcard deployments
* fix(proxy): keep namespace and bare catch-all wildcards on routable listings
A leading deployment segment the model name repeats, or the provider name
itself, is no longer read as a routing segment, so same-name namespace
wildcards and a key filter over `*` -> `openai/*` stop listing ids that
route to no model. A bare `*` deployment under a prefixed name keeps the
legacy listing. `*` over a partial deployment now lists the names that
route through it, dropping empty ids.
* fix(proxy): list nothing for a catch-all over a mid-word wildcard deployment
* fix(proxy): list only wildcard ids that route back to their deployment and catalog model
A listed id now has to round-trip through the alias pattern to the catalog model it names, and the router's wildcard match for it has to pick this deployment, so an id a more specific sibling wildcard serves is no longer listed under the catch-all on /v1/models or /model/info
* fix(proxy): list wildcard aliases that route at a word boundary and ids a sibling serves with the same model
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): price the estimated input tokens on a client-cancelled (499) request's spend row
* fix(proxy): read the provider handoff before the proxy-only error handler stamps it so a pre-handoff 499 is not priced
* fix(proxy): drop the pre-logging handoff read now that pre_call no longer stamps proxy-only failures
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(key_management): apply default_key_generate_params.allowed_routes on key generation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(key_management): avoid mutating default routes payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): apply default allowed_routes to key_type presets it covers
* fix(proxy): keep management and read_only presets, skip service account keys, and show the default routes to custom_key_policy
* fix(proxy): keep Full Access keys out of the default allowed_routes, let owners re-pick a key type, and show custom_key_policy the member key's routes
* fix(proxy): keep owners from changing routes a key got from default_key_generate_params
* docs(proxy): describe key_type when default_key_generate_params.allowed_routes is set, and test custom_key_policy input with the default off
* test(proxy): type the default_key_generate_params allowed_routes tests
* fix(proxy): let key owners send unchanged allowed_routes back on /key/update
* fix(proxy): ignore a route default with a non-string entry and match resent routes exactly
A default_key_generate_params.allowed_routes list with a non-string entry made /key/generate fail at the database, so it is now ignored like a non-list. An owner's resent allowed_routes on /key/update skip the route check only when they equal the stored list in order, so reordered or duplicated lists are checked as a change. Adds integration cells for the default and the owner update paths.
* test(proxy): cover a non-string allowed_routes default in the key integration tests
* test(proxy): read key rows from the owned proxy's database in the default routes integration tests
* fix(proxy): leave allowed_routes out of a key update that resends them unchanged
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
