* fix(gemini): preserve every grounding web source
* test(gemini): cover all grounding sources per passage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Zabir Raihan <zabirraihan@icloud.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds integration cells for a different key reading the response, message redaction, a rotated, deleted, or team-deleted creator, and a creator whose Langfuse keys resolve to nothing
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): route rate limit cache increments through typed helpers
* chore(ui): regenerate schema.d.ts for deployment rpd and tpd fields
* test(types): add rpd and tpd to the owned litellm_params inventory
* fix(router): run daily rpd/tpd checks after the minute checks and increment rpm and rpd in one cache trip
* fix(router): flatten the async rpm reject branch for the ruff strict gate
* fix(router): hand rpm and rpd back on any rate limit reject
A request the rpm, rpd or tpd check rejects no longer keeps its request counts, so retries against a deployment whose rpd is spent cannot fill its rpm and block the in-group fallover with the minute 429.
* test(router): a minute-rejected request no longer keeps its rpm count
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(routing): audit #46601
Adds integration cells for the falsy budget 400, the empty and none-only declarations under drop_params, Vertex AI Claude, Databricks Claude, and the Bedrock native output retry
* test(routing): expect the Anthropic default max_tokens on Bedrock Invoke chat
Reads the default from litellm instead of pinning 4096, which #48502 raised to 128000
* test(routing): assert whole sent bodies in the Vertex AI and Bedrock native output cells
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(daily-activity): read guardrail spend per key from the daily guardrail usage table
Adds DailyActivityRepository.guardrail_spend, which sums LiteLLM_DailyGuardrailUsageUnits.cost by date, guardrail and key. A guardrail row counts only when its key has a spend row on that day under the same scope filters as the rest of the daily activity response, so user, key and date filters apply to guardrail cost too
* feat(daily-activity): add a per-guardrail spend breakdown to /user/daily/activity/aggregated
Each day now has breakdown.guardrails[<guardrail_id>] with metrics.spend and an api_key_breakdown carrying each key's spend and metadata. Guardrail entries only report spend, since that is the one metric stored per key. The breakdown is on the user endpoint only, is empty when a model filter is set because guardrail cost is not stored per model, and lists only keys within api_key_limit while the guardrail total keeps every key. spend and budgets are unchanged
Resolves LIT-9785
* feat(daily-activity): add the guardrail breakdown to team and organization activity, with guardrail names
/team/daily/activity/aggregated and /organization/daily/activity/aggregated now return breakdown.guardrails the same way the user endpoint does, since each key belongs to one team and one organization. Each guardrail entry also carries metadata.guardrail_name, read from LiteLLM_GuardrailsTable for guardrails created through the UI or API and falling back to the breakdown key for config.yaml guardrails, whose key is already their name
* fix(daily-activity): keep daily activity up when the guardrail breakdown query fails, and treat an empty model filter as none
A database error from the guardrail spend query now logs a warning and returns the response without guardrails, where it used to fail the whole aggregated response with a 500. An empty model= no longer drops the guardrail breakdown, matching how the rest of the endpoint already ignores it. The daily activity golden files and fixtures gain the empty guardrails key every breakdown now carries
* refactor(daily-activity): fold guardrail breakdown in a worker thread and drop docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(daily-activity): report untracked guardrail units next to spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(migration_data_checks): size btree entries by the bytes Postgres stores (LIT-10244)
* test(migration_data_checks): flag exactly the compressed keys Postgres rejects at the entry limit (LIT-10244)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): make hide-secrets honor skip_tool_message_in_guardrail
With the flag on, per guardrail or in litellm_settings, hide-secrets leaves
role: tool messages and Anthropic tool_result blocks as sent, as the
unified guardrails do. Tool call arguments are still masked.
* test(guardrails): type the skip-tool walker test and assert the whole request
* test(files): cover owner-scoped file listing under require_managed_files
* fix(files): list only the caller's managed files on every list route under require_managed_files
* fix(files): narrow owner-scoped managed file lists to the named provider or model and page with after_id and before_id
* fix(files): narrow scoped lists the way uploads route, and refuse lists that cannot be scoped
A model name now matches every deployment that serves it, wildcards and
aliases included, and a provider matches deployments by the provider
uploads resolve, so anthropic/* and unprefixed Claude models count. The
OpenAI and Azure lists also return the caller's pass-through uploads.
An unknown provider and different after and after_id cursors answer 400,
and a list with no matching deployment skips the database read.
* fix(files): match stored files by model name and keep flag-off lists as on main
Under require_managed_files, a narrowed list now matches a file's model
mappings by deployment id or model name, so files LiteLLM stores in its own
database show on their model and provider lists. The Anthropic list defaults
to 20 rows when no limit is sent, after_id, before_id and the cursor 400s
apply only under the flag, and an admin keeps the provider list when the
managed files hook is not loaded. List errors are returned as values and
mapped to HTTP errors in one place.
* fix(files): keep admin provider lists and index the owner-scoped walk
Under require_managed_files, an admin keeps the provider list as on main, and
only a non-admin list reads the caller's own and team managed files. A stored
file now also shows on the provider its model name routes to through a
wildcard, with exact model names first, and a list routed to a model takes the
default page size of the provider serving it. A new index on created_by and
created_at keeps each chunk of the walk an index read, and the model check
runs before the file object is parsed.
* fix(files): filter provider and model lists in the query
A non-admin list scoped to a provider or model now asks Postgres only for the
caller's rows that map to one of that scope's deployments or models, or that
live in file storage, instead of reading every owned row and filtering them in
Python. A caller with many files on other providers no longer waits for the
whole set to be read. Team members' files are matched in the same query.
* test(files): type the fake raw query in the files list tests
* refactor(proxy): list managed files through one typed database handle
* fix(files): list alias and routing-group storage uploads on their provider
* fix(gemini): send Google's current Managed Agents fields on create and forward agent_config on interactions
* fix(gemini): accept id on managed agent get, delete, and list_versions
* fix(gemini): drop unsupported agent fields under drop_params and keep agent_config keyword-only
A model interaction drops agent_config under drop_params, agent create drops wrongly typed optional fields under drop_params and otherwise answers 400 naming the field the caller sent, and agent_config on interactions create is keyword-only so positional response settings stay in place
* fix(gemini): drop a non-object agent_config under drop_params and name the id alias the caller sent
* fix(interactions): reject agent_config on bridged providers and drop a typeless one under drop_params
Bridged providers answer 400 for agent_config without drop_params and drop it with drop_params. A Gemini agent run drops an agent_config without a string type under drop_params. The Gemini agents wire test expects Google's current upstream field names.
* fix(interactions): honor additional_drop_params for agent_config on bridged providers
* fix(agents): drop a clashing instructions alias under drop_params
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(azure): audit #47348
Add proxy integration tests for #47348's cached Entra token providers and
off-loop token refresh, and fix the token herd the audit found: once the
fetch moved to a worker thread, concurrent cold requests each fetched their
own token. Every cached bearer token provider now serializes its calls so
the first fetch fills the cache and the rest reuse it
* fix(azure): build each cached Entra token provider once under concurrent first calls
lru_cache does not hold back concurrent misses, so worker threads that miss together each built their own credential and token provider, and each fetched its own token. A lock around each provider cache makes the first caller build it and the rest reuse it
* fix(azure): share Entra token fetches only while no valid token is held
Concurrent callers now share one result (or one error) from the cold
fetch, while callers inside the refresh window keep the still-valid
token. The oidc/azure provider build and the client-assertion exchange
are single-flighted per audience, and the shared helpers no longer call
subscripted generic aliases, which crashed on Python 3.10.
* fix(azure): refresh Entra tokens once while other requests keep the valid token
MSAL treats a token with less than five minutes left as expired, so every
request in that window fetched a new one. Now one request refreshes while
the others keep using the still-valid token.
Requests that waited on a fetch that failed retry once through a fresh
fetch instead of all failing with the first error.
Credentials that only implement get_token keep working, and OIDC token
exchanges are cached per scope.
* fix(azure): bound Entra token waits during an outage
Requests waiting on a failed token fetch now retry only when the fetch
before it succeeded, so during an Entra outage later bursts fail after one
fetch instead of two.
Requests during a refresh only get the held token while it has more than
60 seconds left, and a failed refresh falls back to that token instead of
failing its request.
* fix(azure): share one retry per failed Entra token fetch and raise on a failed refresh
Waiters of a failed fetch now retry the flight they joined, so a fast-failing Entra makes one retry instead of one per waiter. A failed refresh raises again instead of returning the held token, which Postgres token auth read as a fresh one. The DefaultAzureCredential fallback reuses one provider per scope.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(batches): cover the batch cost poller gaps left after #47190
* fix(batches): keep unbillable batches from blocking file deletes and new-batch polling
The file delete check now ignores batches past MANAGED_OBJECT_STALENESS_CUTOFF_DAYS, a provider 404 for the output object retires the batch, each poll page takes the arrival lane first, a batch expires only after the poller visits it past the cutoff, and the compare-and-swap matches batch_processed instead of updated_at
* test(batches): cover the full arrival page and a failed expiry write in the batch cost poller
Old batches still get polled every other cycle while new batches fill the page, old batches that fill a page never push a new batch out after boot, and a failed stale_expired write no longer stops the rest of the page from being polled
* fix(batches): share the poll page between old and new batches and hold output files until billed
Each lane now gets at least half of the poll page, a lane with nothing left gives its half to the other, and a lane read that resumed mid-lane wraps to the start, so neither old nor new batches can starve the other.
The delete guard applies the 7 day cutoff to input files only. Output and error files stay locked until the batch is billed or retired, and stale_expired rows no longer lock anything.
Only a lasting provider failure expires a batch past the cutoff, so a timeout or a 5xx no longer gives up on a batch that may still bill. Past the cutoff a lasting failure, a lasting cost-tracking failure, or an output file id that never appeared retires a finished row instead of polling it forever.
* fix(batches): retry refused finished batches past 7 days, expire unfinished ones on a failed retrieve
A finished batch the provider refuses after 7 days (rotated key, bucket 403, a 404 that does not name the batch) now keeps retrying so its spend is written once the config is fixed, instead of retiring unbilled. A batch the provider still reports running keeps polling. A batch that is not finished in the database becomes stale_expired once a retrieve fails after 7 days, so an unreachable provider is not polled forever. An S3 NoSuchKey for the output object now counts as gone.
* fix(batches): expire an unfinished old batch only after 3 failed retrieves in a row
* fix(batches): keep retrying an unfinished old batch through timeouts, 429 and 5xx
* fix(batches): wait 3 polls before expiring an old batch whose output id lags
* fix(batches): match a resource id in a 404 message followed by punctuation
* fix(batches): keep refusal and pending-output expiry streaks apart, and void a pending-output retirement on a client write
* fix(batches): give up on unsettled batches past twice the staleness cutoff
Replace the per-reason refusal and pending-output streaks with one give-up age, and inject a clock into the poller and the delete check so tests pin exact age boundaries
* fix(batches): keep polling finished batches past the give-up age so their spend stays recoverable
* test(batches): expect a restarted poller's first page to split between old and new batches
* test(batches): pin the delete-guard tests to a fixed clock and type a new journal list
* test(batches): type the provider helper and the storage 404 cases
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
enable_pre_call_checks filtered deployments whose in-memory minute count
reached litellm_params.rpm, undocumented since 0d1cca9aa0. Deployment rpm
and tpm stay routing weights unless optional_pre_call_checks includes
enforce_model_rate_limits, which is now the only deployment rate-limit path.
Resolves LIT-10233
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): audit #47363
Adds integration cells for the input token count paths #47363 changed: the provider id of a managed file and the 403 for another team's file on OpenAI, file_search and additional_tools parity plus cache-mark stripping and untranslatable tools on Claude, and the Bedrock runtime count leaving out the options it refuses.
* fix(count-tokens): reject a lone tool with no chat equivalent before the Claude count call
A Claude input token count whose only tool was mcp or computer_use_preview skipped the pre-call check and went to the provider, so the caller got the provider error instead of the 400 naming the tool. The check now runs whenever a tool would get a placeholder name.
* fix(responses): count a lone Anthropic toolset and name the uncountable tool
The lone-tool 400 now fires only for OpenAI Responses built-in tool types, so a nameless Anthropic toolset still counts. The error says LiteLLM cannot count that tool type for the model. Adds a Bedrock unit cell for the lone-tool 400 and pins the runtime wire test's tool shape per request type.
* fix(responses): take the uncountable tool types from the OpenAI SDK and keep the region hint off LiteLLM's 400
The Responses built-in tool types now come from openai's ToolParam union, so a type the SDK adds is refused before the count call too. The AWS region hint no longer lands on an error LiteLLM raised before calling the provider. Tests cover two more lone types and the hint.
* fix(responses): keep LiteLLM's own list of uncountable tool types and pass a non-string tool type through
* fix(responses): count tool_search and namespace tools the way Claude chat inference sends them
The count refused a lone tool_search with LiteLLM's 400 and sent a lone
namespace raw or squashed it into an empty tool, while Claude chat
inference maps tool_search to Anthropic's tool search tool and unpacks
a namespace into its function tools. Both now go through the
deployment's own map_openai_params, so the count sends what inference
sends, and tool_search leaves the list of uncountable types.
* fix(bedrock): count a Claude request with its deployment's own chat translation
Bedrock invoke and Mantle inference turn a chat tool_search tool into Bedrock's
tool search tool, but the Bedrock counter always translated with the Anthropic
config, so the count answered 400 for a request inference serves.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(count_tokens): keep the framed estimate when a region-less token_count_model cannot count (audit #47441)
* fix(count_tokens): let framing stand in for a shipped token_count_model, keep a configured one's error
The framed local estimate now replaces a 400/404 from any count model the cost map ships, Mantle-pinned rows included, and an admin-set token_count_model keeps its error as the docs say.
* fix(count_tokens): warn with the region when framing stands in for a pinned count model
Test a Mantle row #47441 pinned in its own region, and keep the region hint check on an admin-set count model.
* fix(count_tokens): name the deployment in the framing stand-in warning
* fix(count_tokens): let framing stand in when a cost map count model's region denies the account
A count model the cost map pins to a region can answer 401 or 403 when the
account has not enabled that region or lacks Mantle count access there. With
framing set, the estimate now stands in, as it does for 400 and 404. An admin
set count model and a count model with no pinned region keep the error.
The warning now names the model group, the count model and its region, and
says the proxy falls back to the estimate.
* test(count_tokens): cover framed 401 and 403 from a shipped count model region on the wire
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(images): cover SDK retry budgets and router ownership
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(images): forward max_retries to the OpenAI image generation client
Per-call, deployment and default_litellm_params max_retries were filtered
out before reaching the OpenAI-compatible image handler, which then always
used 2 SDK retries. Forward the caller's value, and pin it to 0 for routed
calls so the Router stays the only retry owner, matching completion()
Co-authored-by: Ericz <yonglzh@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(images): read image max_retries through a typed kwargs view
Co-authored-by: Ericz <yonglzh@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): assert the public invalid retry error mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): preserve the invalid retry status code
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): group the proxy test imports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): discard proxy retry test routers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ericz <yonglzh@gmail.com>
* test(huggingface): cover TEI rerank truncate passthrough and server default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(huggingface): stop forcing truncate=false on TEI rerank and pass truncate params through
Co-authored-by: Luke Schaefer <225965040+luke-e-schaefer@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(huggingface): keep TEI truncate options provider-specific
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(huggingface): use MockTransport for rerank truncation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Luke Schaefer <225965040+luke-e-schaefer@users.noreply.github.com>
* test(scim): audit #47544
Add integration cells for the SCIM group delete that now runs through the team delete's bulk path: an admin group delete revokes its members' admin key, a delete sent with the IdP's SCIM token revokes member keys and audits them under that token, a delete that times out on a locked member row changes nothing so a retry works, and two racing team deletes write one tombstone and one team audit row
* test(scim): read the racing team delete's audit rows after the proxy drains them
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Adds a real-proxy case where the MCP follow-up call fails on every router retry, checking that the one failure row bills the first call of every attempt on chat completions and /v1/responses
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #47352
* test(proxy): audit #47352 lazy load during a cold docs build
* test(proxy): route the overlapping chat cell to the rig upstream
* test(proxy): keep the root path cell from rewriting the packaged UI
* test(proxy): prove the lazy and chat cells overlap the cold docs build
* test(proxy): warm the chat path before the overlapping chat cell
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(nvidia_riva): audit #41021
Adds an owned Riva ASR gRPC server and provider integration cells for the riva_offline request-body ban, the offline and streaming paths, the per-deployment caller opt-in, and the 413 and 408 error mappings
* test(nvidia_riva): check word timing requests and audio samples in the Riva double
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(router): cover silent_model shadow fan-out and streaming end to end
Proxy cells check that each silent_model target gets one request in the
caller's stream mode, that the caller sees only the primary, and that every
shadow lands its own spend row under its own model group. SDK cells check the
sync Router.completion thread path streams each shadow and fires its success
callback.
* test(router): flatten the spend row lookup with itertools.chain
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #47358
Add integration cells for pass-through bodies and settings-endpoint mappings across workers
* test(proxy): wait for every worker to hold the saved mapping before probing
* test(proxy): compare passthrough bodies to the client body and probe each worker on its own connection
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Pin the queue route contracts #47498 changed: the priority header on streamed answers, the 400 for a request with no priority to queue at, a provider bad request keeping its own status, and a deployment-id request served ahead of the queue
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Covers the json_schema envelope edge cases #47478 fixed on the Anthropic json tool path: a key listed only in
`required`, a tool-named wrapper on a stream, non-ASCII and lone-surrogate answers, a stream cut before the tool
block closes (proxy and sync SDK, Anthropic and Bedrock Invoke), and a string `output_format` with and without
`drop_params` on Anthropic and Vertex chat and /v1/messages. The scripted Anthropic peer now refuses a non-object
`output_format` the way Anthropic does
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(anthropic, bedrock): audit #47489 thinking replay across Claude routes and Bedrock Converse
* test(bedrock): compare the whole replayed Converse turn in the merged-message and cache point cells
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(files): audit #40621
Pin the raw cloud storage file id gate on /v1/files content and delete against a running proxy: twice-encoded gs:// ids stay blocked for non-admin keys, proxy admin keys read s3:// objects, and ordinary provider ids keep working
* test(files): check the full SigV4 signature on the admin raw s3 read
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #46815
Add integration tests for every count route #46815 changed, and count a Claude
subscription caller's tokens with their forwarded token when the deployment has
no configured key, which #46815 turned into a 401
* test(proxy): accept the forwarded subscription token in the count endpoint mocks
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
A DeepSeek V4 stream that ends without a usage block now has a spend-row cell proving its prompt_tokens are counted with DeepSeek's tokenizer, the one path #47353's own cells did not reach
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #47418
* test(proxy): hold the stream open past its first chunk and require cost headers in the audit cells
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Adds an integration cell proving a failed /v1/responses request counts its input from the newest compaction item on its failure spend row
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): cover the max_parallel_requests slot a streamed /v1/messages MCP auto-execute keeps
A streamed /v1/messages request that auto-executes MCP tools replays its answer
as a stream after its model rounds already logged success. The stream path frees
the request's max_parallel_requests slot only on a client disconnect, so after
message_stop the slot stays taken and the key's next request answers 429
The test streams such a replay to the end and expects the slot to be free. It
fails until the fix lands
* fix(proxy): free the max_parallel_requests slot when a replayed /v1/messages MCP stream completes
A streamed /v1/messages request that auto-executes MCP tools runs its model
rounds without streaming and replays the final answer as a stream. The first
round claims the request's only success log, and its limiter callback keeps the
slot because the tool call reused it afterwards. The stream path released the
slot only on a client disconnect, so after message_stop the key stayed at its
max_parallel_requests limit and its next request answered 429
Stream cleanup now releases the stashed slot when the stream completed and its
logging object already claimed the success log, which only happens when the
stream replays a response that was logged before it started. A real stream
still releases through its own success callback
* test(mcp): free the key's only parallel slot after a streamed auto-execute ends
Run a streamed MCP auto-execute on a key with max_parallel_requests 1 on
every LLM surface, then send the next request on that key and expect 200.
On main both /v1/messages surfaces answer 429 here, while /v1/responses and
/v1/chat/completions already pass
* test(proxy): keep only the MCP stream regression tests, since #48628 already frees the slot at stream end
* test(proxy): require the MCP stream regression test's response to stream and reach its end event
* test(gemini): audit #47542
Proxy integration cells for the Gemini and Vertex default output budget: the env budget reaches Google on chat and responses, turns on long decode, yields to the caller's own limit, caps per deployment, skips resumes, and a camelCase two-candidate request is never continued
* test(gemini): annotate the budget wire module's pytestmark as Final
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Checks that a SageMaker stream failing after its first chunks reaches /v1/chat/completions, /v1/messages and /v1/responses callers as an error with no success end event, and that each writes a failed spend log row
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Check that cancelling a LocalSandbox run stops the grandchild process it started, using a real child process
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
