* fix(fallback): read stored router_settings from the database on fallback writes
POST /fallback and DELETE /fallback/{model} read the current rules through the
config cache. When Redis refuses DEL, the eviction before that read fails and
Redis keeps serving the old copy, so each new rule dropped the earlier ones
The write path now reads the router_settings row straight from the writer
database and stores only that row plus the changed key, so config-file router
settings no longer get copied into the database row
* fix(fallback): edit the config file's fallback list when it owns the key
A delete of a rule only the config file holds answers the config-owned 400 again instead of 404, and restating a rule the file already has answers 200 without copying the file's list into the database row. Adds a writer-read regression test and an integration test against a Redis whose ACL refuses DEL
* fix(fallback): re-read the config file's fallbacks ownership on every fallback write
* test(fallback): run the real config reload in the fallback write tests
* test(fallback): key the refused-DEL Redis double by cache key
* fix(fallback): keep the config-owned 400 for an unset env reference and replace a non-object stored row
* fix(fallback): replace a stored router_settings row that holds a non-JSON string
* fix(fallback): compare a restated config-file rule against its resolved env value
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): strip beta-gated body fields when a leg drops the anthropic-beta
When a provider leg drops an anthropic-beta value (the chat handler's allowlist
filter, /v1/messages' filtered headers, Bedrock Converse's additionalModelRequestFields),
also drop the body fields that beta gates (thinking.block_binding, thinking.display=updates,
mid-conversation system clear_at, output_config, tool_addition/tool_removal blocks) so the
request degrades on that leg instead of failing with a provider 400. Copy-on-write so a
fallback leg still sends the full request, and each strip is logged at WARNING.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prove a leg that drops a beta sends neither the beta nor clear_at
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): strip beta-gated fields against the final body anthropic_beta list
Converse counts the body anthropic_beta list as active, Invoke strips after
its auto-injected betas are known and injects thinking-binding-controls for
thinking.block_binding, test helpers are fully typed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): strip beta-gated system fields before an older model's system turn becomes a user turn
Models without supports_mid_conversation_system send no mid-conversation beta,
so tool_addition and tool_removal blocks are dropped (and logged) before the
turn is converted instead of riding along as user content. The Invoke
output_config regression test reads the real cost map instead of patching
_bedrock_model_supports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): pin the integration proxy to the bundled beta allowlist and type the helper locals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): annotate every new test local with Final and type the fixtures and returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): strip beta-gated body fields only under drop_params
Without drop_params the request now goes out as sent and the provider
answers its own 400, like any other unsupported param. inline-tools-2026-09-15
also unlocks tool_addition and tool_removal blocks, so a leg that carries it
keeps them.
* fix(anthropic): strip beta-gated fields on Foundry chat and Bedrock chat Invoke under drop_params
Foundry chat and Bedrock chat Invoke now drop beta-gated body fields the same way the other legs do when drop_params is on and the request does not send the gating beta. The mid-conversation system normalizer validates only on the conversion path, and the strip WARNING names the beta the caller can send to keep the field.
* fix(anthropic): send the thinking betas for block_binding and display updates, strip Databricks chat thinking
Every Claude leg now adds the thinking-binding-controls beta when thinking carries block_binding and the display-updates beta for display updates, so those fields go through instead of a 400. Databricks chat rejects both fields with or without the beta, so it drops them under drop_params. A system turn the strip empties stays when it still carries output_config, and the drop warning names both causes.
* fix(anthropic): send the thinking betas on every leg and keep fields where the beta is unknown
Vertex messages and Claude Platform chat now add the thinking betas when thinking carries
block_binding or display updates. Converse sends only the caller's header betas when it has them.
A provider with no row in the beta config forwards beta-gated fields instead of dropping them,
an emptied trailing system turn that would leave an assistant prefill is sent as is, and the
prompt cache baseline body matches the body that goes out. The drop WARNING now names the beta
that unlocks each field
* fix(anthropic): filter Converse body betas, keep Foundry auto betas, read the bundled beta config for row checks
* fix(anthropic): validate messages only when a beta-gated drop runs
A non-object messages entry on a Claude model without mid-conversation
system support raised a raw pydantic ValidationError before the
beta-config and drop_params checks. The strip now passes the messages
through untouched unless it drops something, so the provider's 400
reaches SDK callers as BadRequestError again
* fix(anthropic): keep system blocks with a non-string type instead of raising
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(caching): audit #44542 response cache model group key
* fix(caching): key the response cache by the router's metadata bucket only
* fix(caching): read the router's model group before the caller's metadata
* fix(caching): read caching groups only beside the router's model group
A caller's own metadata.caching_groups on /v1/responses and /v1/messages could
still pick the cache key, so a string group let one model group fill another's
entry. The key now reads caching_groups only from the metadata that holds the
router's model_group.
* test(caching): type the planted caching groups parameter
* fix(caching): read the model group from the metadata that carries the call's deployment
A router call can carry a litellm_metadata it did not write, from the auto router's classifier embedding or an SDK caller, while the router writes its own group into metadata. The cache now picks the metadata source whose model_info matches the call's deployment, then falls back to the previous order.
* fix(caching): keep main's model group lookup for calls outside the router
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): stream a guardrail block on /v1/messages as Anthropic events
* fix(proxy): keep post-call guardrails off the streamed messages block
The streamed block is sent as-is, like the Responses endpoint's block, so post-call guardrails and streaming hooks never scan the proxy's own block message. Tests cover a post-call rail on the stream and the call id on the JSON block
* test(proxy): expect Anthropic events from a streamed messages guardrail block
The legacy guardrail suite pinned the old single bare data: line, which is the
malformed block this branch replaces. The test now checks the six Anthropic
stream events and the block text, so it fails if the bare line comes back
* test(proxy): block messages through a real pre-call guardrail in endpoint tests
The block tests replaced ProxyBaseLLMRequestProcessing.base_process_llm_request on the class to raise the block. They now send a real request through the endpoint with a pre-call guardrail that raises the passthrough block, and check that the x-litellm-call-id header carries the request's call id
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs and logging payloads
* fix(router): keep the mirror flag when litellm_metadata is set and strip a client-sent is_silent_experiment
Spend rows read router metadata from litellm_metadata when a call carries it, so the router now sets the flag there too. The proxy drops a client-sent is_silent_experiment with the other router-owned keys, so real traffic cannot be recorded as mirrored.
* fix(logging): mark mirrored calls with a router-only key so clients cannot set is_silent_experiment
Spend rows and the standard logging payload now derive is_silent_experiment from _litellm_silent_experiment, which only the router writes and the proxy and pass-through routes strip from client metadata. A caller's own is_silent_experiment key stays untouched, so it keeps its current effect on mirroring and on provider metadata, and unmirrored calls log false in both places.
* test(langfuse): expect is_silent_experiment false on every logged generation
* fix(logging): mark mirrored calls with a router-only enum value so no client route can set is_silent_experiment
The router stamps SilentExperimentMarker.MIRRORED, which client JSON can never produce, so batches, Responses WebSocket frames and pass-through need no strip. Langfuse layers the flag after its requester-metadata dedupe.
* fix(logging): set is_silent_experiment only on mirrored calls so real calls keep their payloads
Real calls no longer gain an is_silent_experiment key in spend rows or the standard logging payload, so OTEL, S3, GCS and every other logger receive what they did before. The Langfuse regression test now runs the real prompt helper.
* fix(logging): keep the mirror marker out of callback metadata and read the spend-row flag from the logging payload
Logging now strips the router's mirror marker from the request metadata buckets before any callback or vendor sees it, without touching the router's own dicts, so retries keep it. Spend rows read is_silent_experiment from the standard logging payload, which survives the collector's JSON round trip, so collector-mode rows keep the flag too.
* fix(logging): mark mirrored calls through a context variable so router usage counts them once
* fix(router): leave callback-started calls unmarked and stop client flags from turning mirroring off
Logging handlers now run outside the mirrored-call context, so an LLM call a success or failure callback starts while logging a mirrored call is not flagged as mirrored. The guard against mirroring a mirror reads the same context variable instead of request metadata, so a client-sent is_silent_experiment no longer skips the mirrors. The unused module-level get_standard_logging_metadata stops copying resolved keys from raw metadata.
* fix(router): keep main's client flag mirroring skip and the other logging metadata keys
A client's is_silent_experiment: true skips mirroring again as on main, the module-level get_standard_logging_metadata drops only the flag, and the silent experiment tests wait on a logged-call signal instead of polling a deadline
* fix(logging): take the mirror flag in Prometheus labels and Datadog cost tags only from the router
* test: type the new parametrized tests and keep the recorder from changing its argument
* fix(datadog): keep client request tags from setting the silent experiment cost tag
A request tag is_silent_experiment:true set the Datadog cost tag on a direct call.
Only the router flag sets it now. Also covers the sync failure handler's
mirrored-call reset with a failure function callback test.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): let a Claude Code gateway user choose the session team
The gateway's browser verification page now lists the developer's teams, and the device code grant mints the session for the picked team. A team the developer is not on is refused with a 400, and a form without a team keeps the first-team default.
* fix(proxy): refuse a stale gateway team pick and require an explicit choice
* fix(gateway): list the browser-picked team first so an older replica mid-deploy issues it
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): keep every startup line JSON when json_logs is on
Boot-time messages the proxy printed straight to stdout (banner, feedback
box, model list, cache and callback notices, policy table, migration
notices, CLI launch notes) and the first records of uvicorn- or
granian-spawned workers came out as plain text while json_logs was on,
breaking log pipelines that parse every line as JSON.
- print_startup_message() prints unchanged in plain mode and, with
json_logs on, emits one INFO record on the proxy logger (ANSI
stripped) that skips the logger and handler level checks like the
print it replaces; show_banner() routes to it in JSON mode only
- the CLI exports JSON_LOGS=true when the config sets json_logs so
spawned workers log JSON from their first record
- the --enforce_prisma_migration_check deprecation notice prints after
the config is read so a config-only json_logs covers it
Resolves LIT-1624
* fix(proxy): keep show_banner() SDK-free, pick the JSON banner in initialize()
Since #46803 the `lite` shell calls show_banner() against an uninitialized
litellm package, and reading litellm.json_logs inside it initialized the whole
SDK at the banner (0.14 s to 0.83 s for a bare `lite` here, the seconds that PR
removed on small VMs). show_banner() is back to its main version; the proxy's
initialize() now emits the JSON banner record itself and calls show_banner()
only in plain mode. The banner unit tests move to the initialize() call site.
* fix(proxy): keep the JSON banner when an operator --log_config names the proxy logger
uvicorn applies --log_config after proxy_cli has switched JSON logging on and before
initialize() switches it on again through load_config(). A logging config that names
"LiteLLM Proxy" removes that logger's handlers in the window between the two, which is where
initialize() emits the banner and the feedback box, so at one worker both went nowhere;
two-worker children import litellm after uvicorn's logging setup and were not affected.
print_startup_message() now falls back to a one-off JSON handler, on the stream the JSON
handler uses, when the proxy logger has no handler of its own.
Live, json_logs on plus a log_config naming the proxy logger, one worker: main prints the
banner and the feedback box as plaintext, e2303a3925 prints neither, this head prints both as
one JSON record each. New unit test fails on e2303a3925; new integration cell
test_json_logs_with_operator_log_config_keeps_every_line_json passes at one and two workers.
* fix(proxy): keep a long startup listing whole in its JSON record
print_startup_message ran its record through the proxy logger's filters, so
in JSON mode the required-migration SQL list and the policy list were
middle-truncated at MAX_STRING_LENGTH_STDOUT_LOG (4096 by default) with the
litellm_truncated marker, where the prints they replace showed them whole.
That cap bounds echoed request payloads; a startup message is operator text,
so the record now gets the uncapped processing of a DEBUG line: secret
redaction and base64 collapse still apply, the text cap does not.
Red on 41f92b745a, green here:
- tests/unit/test_logging.py: a listing longer than the cap comes out whole;
a credential in a startup message is still redacted
- tests/integration/configuration/test_json_logs_startup.py: a 60-policy
config boots with json_logs on and the policy listing record (9762 chars)
carries every policy with no truncation marker (4096 chars, marker, on
the unfixed tree)
* fix(proxy): keep the --enforce_prisma_migration_check notice on --health and --test
This PR moved the deprecation notice after the config is read, so that a
json_logs set only in the config applies to it; --health and --test return
before that point and stopped printing it, a visible change main did not
have. The notice is now also emitted before those two returns, through the
same helper, so a deployment running `litellm --health
--enforce_prisma_migration_check` still learns the flag is dead, as on main.
A normal server start emits it once, after the config, as in the previous
commit.
Red on 2d7a2d6844, green here: tests/unit/proxy/test_proxy_cli.py asserts
the notice on --health and on --test.
* fix(proxy): keep the JSON startup records on stdout under an operator handler on the proxy logger
An operator --log_config that gives "LiteLLM Proxy" a handler of its own, a NullHandler
that silences it or a plain WARNING-level handler on stderr, is in force in the window
where the banner, the feedback box and the cache notice are emitted: uvicorn applies it
after the CLI process switched JSON on and before initialize() re-installs the JSON
handler through load_config(). 41f92b745a sent the startup records through the proxy
logger's handlers and fell back to a JSON stdout handler only when there were none, so
under such a config the records landed in the operator's handler, as plaintext on stderr,
or vanished, where the prints they replace reached stdout whatever the logging config
said.
print_startup_message() now writes every startup record through the proxy's own JSON
handler on stdout, filters included, and never through the proxy logger's handlers; the
operator's handler gets nothing from a startup message, as it got nothing from the print.
The normal JSON handler is the same class, formatter and filters, so the records a
pipeline reads are unchanged when no operator config is in play.
Tests: tests/unit/test_logging.py gets the two handler shapes (red on f6f5957d16: the
record went to stderr as plaintext or nowhere); tests/integration/configuration/
test_json_logs_startup.py gets a silencing operator --log_config cell at 1 and 2 workers
(red on f6f5957d16: banner, feedback box and cache notice missing; green here).
* fix(proxy): print the --enforce_prisma_migration_check notice when the config fails to load
f6f5957d16 printed the deprecation notice after the config is read, so a json_logs set
only in the config makes it a JSON record. A boot whose config file is missing or does
not parse then exited before that point and never printed it, where main printed it
before the error (10-20 gauntlet finding).
The notice is printed where main printed it, unless a config file can still switch
json_logs on: then it is held until the config is read and printed right after the
json_logs switch, and a click close callback prints it on the way out if the boot fails
first. functools.cache keeps it to one print either way.
Tests: tests/unit/proxy/test_proxy_cli.py covers a failed config read (red on
75043aed83), a plain config printing it once and a config-only json_logs making it one
JSON record; tests/integration/configuration/test_json_logs_startup.py adds the JSON cell
and a missing and an unparseable config cell (red on 75043aed83, green here), through
refused_boot_log, which now takes extra CLI arguments.
* feat(proxy): add suppress_startup_output to turn off startup prints
* fix(proxy): print startup messages once and without color codes off a terminal
* test(policy-engine): check the policy table keeps colors on a terminal and drops them on a pipe
* fix(proxy): keep startup warnings when suppress_startup_output is on and apply it from CONFIG_FILE_PATH and before other settings
Warnings that name something the operator must fix now print through print_startup_warning, which
skips the suppress gate. The config key is read with the CLI's json_logs handling for --config and
CONFIG_FILE_PATH, applied before the rest of litellm_settings in load_config, and can only turn the
setting on. The Prometheus config panel obeys the setting, and startup prints use the plain log
format's color rule.
* fix(proxy): read CONFIG_FILE_PATH in the CLI only when workers would load it
Workers load CONFIG_FILE_PATH only when it names an existing YAML file, so the CLI's early read of
json_logs and suppress_startup_output skips any other value instead of failing a boot that used to start.
* fix(proxy): read only suppress_startup_output from CONFIG_FILE_PATH in the CLI, without secrets
The CLI now reads CONFIG_FILE_PATH as a plain file with its includes, skips it when a config bucket is set,
and applies only suppress_startup_output, also when --config is passed. json_logs there stays with the
workers as before. Fatal boot errors drop their color codes off a terminal like the startup prints.
* fix(logging): emit startup warnings as WARNING records in JSON mode
* fix(proxy): keep one JSON record per search tool in the startup listing
* fix(proxy): keep booting with an ASCII stdout or an empty litellm_settings
Startup messages now go through click.echo, which writes UTF-8 when stdout is ASCII, as the banner did
before. An empty litellm_settings section reads as no settings in the CLI, as the workers already read it
* test(proxy): boot the CLI tests on the real ProxyConfig config readers
* test(proxy): cover the startup warnings that print with startup output suppressed
* test(proxy): boot the deprecated resolver flag test on the real ProxyConfig
* test(proxy): drop the startup output contract's narrating docstrings
* test(proxy): cover the pgbouncer start failure message off a terminal
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(github_copilot): treat a malformed api-key.json like a missing one
* fix(github_copilot): read the key and the endpoints of api-key.json as separate strict views
* fix(github_copilot): ignore a non-UTF-8 api-key.json like any other malformed file
Read the cached key file as bytes so a UTF-16 file, which Windows
PowerShell 5.1 writes with >, fails JSON validation instead of raising
the decoder's error, and the request gets the refresh or the login hint
* fix(github_copilot): decode UTF-16 key files and refresh over a malformed endpoints field
* fix(github_copilot): decode a UTF-16 access-token file and refuse an undecodable one like a missing one
* fix(github_copilot): use the refreshed key when api-key.json cannot be written
* fix(github_copilot): keep a refreshed key that cannot be saved in memory
* fix(github_copilot): serve a kept key and its endpoint together after the key file is repaired
A key held in memory is now checked before api-key.json, so the token and the endpoint come from the same source, and the map holds one entry per key file with the access token beside it
* test(github_copilot): type the authenticator test fixture and the tests this branch adds
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(gemini): use collection endpoint for custom context cache
* fix(vertex_ai): build the cachedContents collection URL under a custom api_base
* fix(vertex_ai): keep a gateway prefix that contains /publishers/
* test(integration): expect the cachedContents collection URL on a custom Gemini api_base
* fix(vertex_ai): cut cache collection URLs at the base's location, graft a mount prefix
* fix(vertex_ai): cache under a publisher model api_base in that model's location
A Vertex AI api_base that names an endpoint or no model keeps its old cache URL, so
no cache is created that the chat call cannot use. The cache body takes its project and
location from the api_base model, matching the collection URL.
* fix(vertex_ai): cache for the api_base's model and skip caching on a Gemini v1 base
A Vertex api_base naming a publisher model now gives the cache its model as well as its project and location, so the cache matches the model generateContent calls. A Gemini api_base ending in /v1 skips context caching, since the v1 API has no cachedContents and rejects the cachedContent field.
* fix(vertex_ai): check the cache token minimum against the api_base's model
The cache is created for the model a publisher-model api_base names, so the minimum prompt size now comes from that model too. A prompt below its minimum is sent uncached instead of failing at cache create.
* fix(vertex_ai): use the stricter cache minimum and send a Gemini base's 404 uncached
---------
Co-authored-by: paulo037 <paulo.h.carneiro.18@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: stabilize cleanup, lockout, and post-kill timing tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bound the wedged Redis burst p95 by the socket timeout the test sets
The burst p95 under a fixed 0.75s cap tracked how fast the box ran the 30
concurrent writes, so a loaded runner failed it at unchanged code. A handler
that waits on the wedged publish takes at least the coordination Redis socket
timeout, so the test now sets that timeout and asserts the p95 stays under it.
* test: pin the lockout clock and bound the wedged Redis burst by the proxy's 2s wait cap
The lockout tests now set the login throttle's clock instead of reading the real one, so Retry-After is pinned at 77 and a round-down fails on every run
The wedged Redis burst p95 is now bounded by the 2s cap the proxy puts on its own Redis waits, so a burst wait capped under the 5s socket timeout still fails the test
The fresh post-kill clients keep the harness's 2s keep-alive expiry, here and in the two tests #47080 changed
* test: tell ceil from round in the lockout tests and bound the burst by the imported eviction cap
The refused attempt now runs with 76.4s left on the 77s block, where ceil gives 77 but round and int give 76, so
both lockout tests fail if Retry-After stops rounding up.
The burst bound imports _BULK_EVICTION_WAIT_SECONDS instead of copying its 2s value, so lowering the cap moves
the bound with it.
* test: bound the wedged Redis burst by the shorter of the eviction cap and the socket timeout
Any wait the proxy puts on the stopped Redis lasts at least the shorter of the two, so the bound now follows
whichever one a later change lowers.
* test: drive the tool index cleanup test with the manual clock and keep the post-kill block as on main
The post-kill reset is already fixed on main by making kill() wait for the
worker's last thread, so the fresh-client and keep-alive edits go back to
main's version. The tool index cleanup test now advances the injected clock
instead of timing the real one, and the lockout clock helper gets a type hint.
* test: advance the spend log cleanup tests' manual clock through its own method
* test: freeze the throttle clock in the login_utils lockout tests and drive the run budget tests with the manual clock
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): drop a malformed prompt_cache_breakpoint under drop_params on the direct /v1/responses path
* refactor(responses): reuse the shared prompt_cache_breakpoint_for_wire helper in the chat bridge
* fix(responses): read a non-flag deployment drop_params as unset in the marker walk, like the param mapper
* fix(responses): drop every prompt_cache_breakpoint under drop_params when the model has no breakpoints
Also drop a marker set on the message item itself, sanitize before the
cache_control conversion so a client typo no longer erases that breakpoint,
and skip items that carry no marker so unmarked history costs no extra walk
* fix(responses): keep a valid prompt_cache_breakpoint unless the model is known to reject it
A deployment alias the cost map does not know (an Azure deployment name, a fine-tune, a
litellm_proxy alias) kept its valid markers before this PR and must keep them under
drop_params. base_model is read from litellm_params first, then model_info, as chat
completions does.
* fix(responses): drop a valid marker on known non-breakpoint models and keep the tool schema gate unchanged
* fix(responses): judge breakpoint support by model name so cost-map aliases keep valid markers
* fix(responses): keep the caller's request stripped of cache_control as before when a marker is copied
* fix(responses): keep valid prompt cache markers in place and let cache_control fill a dropped one
Under drop_params the marker step now hands back the caller's own objects when every marker is
already wire-safe, so a request with valid markers is no longer copied and walked a second time.
A deployment that turns cache_control into markers keeps the caller's own markers too, whatever
its base_model says, so one request never gets two answers about breakpoint support.
On /v1/messages a malformed raw marker next to cache_control is dropped before the conversion,
so the block gets the cache hint cache_control alone would give, as on /v1/responses.
* test(responses): keep a null marker and the original key order on a model that reads breakpoints
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): scope the signoz, otel conversation id, and series cap proxy fixtures to their module
Session scope booted one 2-worker proxy per xdist worker and kept it until the
worker's session ended, so a CircleCI node holding two of these files piled up
proxies until the VM stopped responding. Module scope lets the harness group
each file's rig tests onto one worker and tears the proxies down after them.
* docs(tests): keep proxy fixtures module scoped in integration tests
* docs(tests): say how the harness groups proxy fixtures in the integration AGENTS.md rule
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): resume background response streams with stream and starting_after
Retrieving a response with stream=true and starting_after=N now returns the
replayed event stream from OpenAI and Azure instead of one JSON object, on the
SDK (get_responses / aget_responses) and on the proxy's GET /v1/responses/{id}
Providers that cannot replay a stored stream drop the params under drop_params
and raise UnsupportedParamsError without it. Malformed values answer 400 with
the param named. A replay that reaches its terminal event settles the
background response's bill once and passes the provider's usage through
without estimating one. An empty replay (cursor at the end) ends normally
* fix(responses): encrypt replayed stream ids and resume Azure AI Foundry streams
A replayed background stream now gets the same response id encryption as a
created one, so the key that made the response can reuse the id. Azure AI
Foundry deployments resume streams like Azure ones, and the chat-bridged
Responses iterator closes its wrapped stream instead of raising.
* fix(responses): drop malformed resume params under drop_params and keep JSON answers to a streamed retrieve
A bad stream flag drops to a plain retrieve and a bad cursor to a full replay when drop_params is on, and a cursor without stream is ignored even when malformed. A streamed retrieve that the upstream answers with JSON returns the response object instead of an empty stream. get_responses forwards only the include keys from extra_query.
* fix(responses): give proxy stream resumes the deployment stream_timeout and document retrieve query params
* fix(responses): keep deployment stream off retrieves and reject oversized cursors
* fix(responses): keep router default stream off plain retrieves
* test(responses): black box coverage for resume refusals, drop_params, stream timeout and billing
* test(responses): replay a real stream in the billing test and type the fake wrapper chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn
* fix(responses): keep tool-call-only stream content empty and drop unreachable replay guards
* fix(responses): reject an assistant phase OpenAI does not define unless drop_params is set
* fix(responses): keep other providers' reasoning and stream output unchanged and fall back on records that do not fit
* fix(responses): label a structured output turn with its returned message's phase and keep lone surrogates from failing the checksum
* fix(responses): leave a turn without a phase when its content message cannot be read
* test(responses): add black box tests for chat turn history replay
* test(responses): expect the replay record on bridged chat replies
* test(responses): expect the replay record in bridged chat translation cases
* fix(streaming): keep the served service tier on the finish chunk the stream adds
* test(responses): read the expected chat message without JSON validation so the replay matcher stays
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(responses): cover Codex tool lists repeated across turns on the Bedrock chat bridge and Mantle
* fix(responses): keep each Codex tool once when its tool list repeats across turns
* test(responses): type the new tool list dedupe unit tests
* test(proxy): cover management and health routes labelling rejections as auth failures
* fix(proxy): name management and health route errors by their status instead of auth_error
* fix(proxy): keep auth_error for 401 and 403 and answer database outages with 503 in route catch-alls
/key/list and /key/aliases now share the catch-all helper too, so they stop
adding the Authentication Error prefix and typing a 400 as internal_server_error
* fix(proxy): answer 503 when the database drops mid-query and prisma fails to name the error
A real outage surfaces from prisma's query engine as a builtin AttributeError, which the
prisma-module check skipped, so the route catch-alls still answered 500 or 400
* fix(proxy): report wrapped and stalled database outages as 503 and label upstream failures by their own status
* fix(proxy): answer the upstream status on the catch-all, keep auth_error on tag denials, report model insert outages as 503
* fix(proxy): answer 503 for database failovers and label provider auth errors like chat
A database that refuses connections while starting up, shutting down, in
recovery, or read-only after a failover now answers 503 no_db_connection
instead of 400. A provider's 401 or 403 keeps the type /chat/completions
gives it, while a 401 or 403 the proxy raises keeps auth_error. An
exception with no message is named by its type.
* fix(proxy): scope failover 503s to management route errors and keep a re-raised error's cause
* test(proxy): expect the OpenAI error shape from management route refusals
Team-not-found 404s now answer invalid_request_error, and every handler-raised refusal answers param null instead of the string None.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): drop the null created_by OpenAI rejects when output items are replayed
* fix(responses): drop a null created_by from replayed input items too
A conversation saved from an older proxy still carries created_by: null on its compaction, apply_patch and shell items, so OpenAI keeps rejecting the replay. Drop the null from input items on the HTTP request transform and the WebSocket response.create rewrite, keeping a created_by that is not null.
* fix(responses): drop a null created_by from counted input items too
* test(responses): cover saved reasoning nulls and re-compaction of a saved conversation
* fix(responses): drop a set created_by from input under drop_params
* fix(types): type the Responses output serializer's parameters
* fix(responses): drop a set created_by on reasoning items and Perplexity presets under drop_params
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(responses): audit #47445
Integration cells for billing every round of a streamed MCP auto-execute on one spend row, on a native Responses upstream and the chat bridge, across tool rounds, a rejected or broken follow-up, a post-call guardrail block, a client hang-up, the response cache, and the next bridged turn
* test(responses): check a broken follow-up bills its own nonzero usage
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #46997
Add request-path cells for the global proxy spend counter: a key with no user spends toward litellm max_budget, and two proxies sharing Redis add every request to one counter that matches the proxy budget row
* test(proxy): wait for the recorded budget spend and shared counter before checking them
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(gemini): preserve every grounding web source
* test(gemini): cover all grounding sources per passage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Zabir Raihan <zabirraihan@icloud.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds integration cells for a different key reading the response, message redaction, a rotated, deleted, or team-deleted creator, and a creator whose Langfuse keys resolve to nothing
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): route rate limit cache increments through typed helpers
* chore(ui): regenerate schema.d.ts for deployment rpd and tpd fields
* test(types): add rpd and tpd to the owned litellm_params inventory
* fix(router): run daily rpd/tpd checks after the minute checks and increment rpm and rpd in one cache trip
* fix(router): flatten the async rpm reject branch for the ruff strict gate
* fix(router): hand rpm and rpd back on any rate limit reject
A request the rpm, rpd or tpd check rejects no longer keeps its request counts, so retries against a deployment whose rpd is spent cannot fill its rpm and block the in-group fallover with the minute 429.
* test(router): a minute-rejected request no longer keeps its rpm count
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(routing): audit #46601
Adds integration cells for the falsy budget 400, the empty and none-only declarations under drop_params, Vertex AI Claude, Databricks Claude, and the Bedrock native output retry
* test(routing): expect the Anthropic default max_tokens on Bedrock Invoke chat
Reads the default from litellm instead of pinning 4096, which #48502 raised to 128000
* test(routing): assert whole sent bodies in the Vertex AI and Bedrock native output cells
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(daily-activity): read guardrail spend per key from the daily guardrail usage table
Adds DailyActivityRepository.guardrail_spend, which sums LiteLLM_DailyGuardrailUsageUnits.cost by date, guardrail and key. A guardrail row counts only when its key has a spend row on that day under the same scope filters as the rest of the daily activity response, so user, key and date filters apply to guardrail cost too
* feat(daily-activity): add a per-guardrail spend breakdown to /user/daily/activity/aggregated
Each day now has breakdown.guardrails[<guardrail_id>] with metrics.spend and an api_key_breakdown carrying each key's spend and metadata. Guardrail entries only report spend, since that is the one metric stored per key. The breakdown is on the user endpoint only, is empty when a model filter is set because guardrail cost is not stored per model, and lists only keys within api_key_limit while the guardrail total keeps every key. spend and budgets are unchanged
Resolves LIT-9785
* feat(daily-activity): add the guardrail breakdown to team and organization activity, with guardrail names
/team/daily/activity/aggregated and /organization/daily/activity/aggregated now return breakdown.guardrails the same way the user endpoint does, since each key belongs to one team and one organization. Each guardrail entry also carries metadata.guardrail_name, read from LiteLLM_GuardrailsTable for guardrails created through the UI or API and falling back to the breakdown key for config.yaml guardrails, whose key is already their name
* fix(daily-activity): keep daily activity up when the guardrail breakdown query fails, and treat an empty model filter as none
A database error from the guardrail spend query now logs a warning and returns the response without guardrails, where it used to fail the whole aggregated response with a 500. An empty model= no longer drops the guardrail breakdown, matching how the rest of the endpoint already ignores it. The daily activity golden files and fixtures gain the empty guardrails key every breakdown now carries
* refactor(daily-activity): fold guardrail breakdown in a worker thread and drop docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(daily-activity): report untracked guardrail units next to spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(migration_data_checks): size btree entries by the bytes Postgres stores (LIT-10244)
* test(migration_data_checks): flag exactly the compressed keys Postgres rejects at the entry limit (LIT-10244)
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): make hide-secrets honor skip_tool_message_in_guardrail
With the flag on, per guardrail or in litellm_settings, hide-secrets leaves
role: tool messages and Anthropic tool_result blocks as sent, as the
unified guardrails do. Tool call arguments are still masked.
* test(guardrails): type the skip-tool walker test and assert the whole request
* test(files): cover owner-scoped file listing under require_managed_files
* fix(files): list only the caller's managed files on every list route under require_managed_files
* fix(files): narrow owner-scoped managed file lists to the named provider or model and page with after_id and before_id
* fix(files): narrow scoped lists the way uploads route, and refuse lists that cannot be scoped
A model name now matches every deployment that serves it, wildcards and
aliases included, and a provider matches deployments by the provider
uploads resolve, so anthropic/* and unprefixed Claude models count. The
OpenAI and Azure lists also return the caller's pass-through uploads.
An unknown provider and different after and after_id cursors answer 400,
and a list with no matching deployment skips the database read.
* fix(files): match stored files by model name and keep flag-off lists as on main
Under require_managed_files, a narrowed list now matches a file's model
mappings by deployment id or model name, so files LiteLLM stores in its own
database show on their model and provider lists. The Anthropic list defaults
to 20 rows when no limit is sent, after_id, before_id and the cursor 400s
apply only under the flag, and an admin keeps the provider list when the
managed files hook is not loaded. List errors are returned as values and
mapped to HTTP errors in one place.
* fix(files): keep admin provider lists and index the owner-scoped walk
Under require_managed_files, an admin keeps the provider list as on main, and
only a non-admin list reads the caller's own and team managed files. A stored
file now also shows on the provider its model name routes to through a
wildcard, with exact model names first, and a list routed to a model takes the
default page size of the provider serving it. A new index on created_by and
created_at keeps each chunk of the walk an index read, and the model check
runs before the file object is parsed.
* fix(files): filter provider and model lists in the query
A non-admin list scoped to a provider or model now asks Postgres only for the
caller's rows that map to one of that scope's deployments or models, or that
live in file storage, instead of reading every owned row and filtering them in
Python. A caller with many files on other providers no longer waits for the
whole set to be read. Team members' files are matched in the same query.
* test(files): type the fake raw query in the files list tests
* refactor(proxy): list managed files through one typed database handle
* fix(files): list alias and routing-group storage uploads on their provider
* fix(gemini): send Google's current Managed Agents fields on create and forward agent_config on interactions
* fix(gemini): accept id on managed agent get, delete, and list_versions
* fix(gemini): drop unsupported agent fields under drop_params and keep agent_config keyword-only
A model interaction drops agent_config under drop_params, agent create drops wrongly typed optional fields under drop_params and otherwise answers 400 naming the field the caller sent, and agent_config on interactions create is keyword-only so positional response settings stay in place
* fix(gemini): drop a non-object agent_config under drop_params and name the id alias the caller sent
* fix(interactions): reject agent_config on bridged providers and drop a typeless one under drop_params
Bridged providers answer 400 for agent_config without drop_params and drop it with drop_params. A Gemini agent run drops an agent_config without a string type under drop_params. The Gemini agents wire test expects Google's current upstream field names.
* fix(interactions): honor additional_drop_params for agent_config on bridged providers
* fix(agents): drop a clashing instructions alias under drop_params
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(azure): audit #47348
Add proxy integration tests for #47348's cached Entra token providers and
off-loop token refresh, and fix the token herd the audit found: once the
fetch moved to a worker thread, concurrent cold requests each fetched their
own token. Every cached bearer token provider now serializes its calls so
the first fetch fills the cache and the rest reuse it
* fix(azure): build each cached Entra token provider once under concurrent first calls
lru_cache does not hold back concurrent misses, so worker threads that miss together each built their own credential and token provider, and each fetched its own token. A lock around each provider cache makes the first caller build it and the rest reuse it
* fix(azure): share Entra token fetches only while no valid token is held
Concurrent callers now share one result (or one error) from the cold
fetch, while callers inside the refresh window keep the still-valid
token. The oidc/azure provider build and the client-assertion exchange
are single-flighted per audience, and the shared helpers no longer call
subscripted generic aliases, which crashed on Python 3.10.
* fix(azure): refresh Entra tokens once while other requests keep the valid token
MSAL treats a token with less than five minutes left as expired, so every
request in that window fetched a new one. Now one request refreshes while
the others keep using the still-valid token.
Requests that waited on a fetch that failed retry once through a fresh
fetch instead of all failing with the first error.
Credentials that only implement get_token keep working, and OIDC token
exchanges are cached per scope.
* fix(azure): bound Entra token waits during an outage
Requests waiting on a failed token fetch now retry only when the fetch
before it succeeded, so during an Entra outage later bursts fail after one
fetch instead of two.
Requests during a refresh only get the held token while it has more than
60 seconds left, and a failed refresh falls back to that token instead of
failing its request.
* fix(azure): share one retry per failed Entra token fetch and raise on a failed refresh
Waiters of a failed fetch now retry the flight they joined, so a fast-failing Entra makes one retry instead of one per waiter. A failed refresh raises again instead of returning the held token, which Postgres token auth read as a fresh one. The DefaultAzureCredential fallback reuses one provider per scope.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(batches): cover the batch cost poller gaps left after #47190
* fix(batches): keep unbillable batches from blocking file deletes and new-batch polling
The file delete check now ignores batches past MANAGED_OBJECT_STALENESS_CUTOFF_DAYS, a provider 404 for the output object retires the batch, each poll page takes the arrival lane first, a batch expires only after the poller visits it past the cutoff, and the compare-and-swap matches batch_processed instead of updated_at
* test(batches): cover the full arrival page and a failed expiry write in the batch cost poller
Old batches still get polled every other cycle while new batches fill the page, old batches that fill a page never push a new batch out after boot, and a failed stale_expired write no longer stops the rest of the page from being polled
* fix(batches): share the poll page between old and new batches and hold output files until billed
Each lane now gets at least half of the poll page, a lane with nothing left gives its half to the other, and a lane read that resumed mid-lane wraps to the start, so neither old nor new batches can starve the other.
The delete guard applies the 7 day cutoff to input files only. Output and error files stay locked until the batch is billed or retired, and stale_expired rows no longer lock anything.
Only a lasting provider failure expires a batch past the cutoff, so a timeout or a 5xx no longer gives up on a batch that may still bill. Past the cutoff a lasting failure, a lasting cost-tracking failure, or an output file id that never appeared retires a finished row instead of polling it forever.
* fix(batches): retry refused finished batches past 7 days, expire unfinished ones on a failed retrieve
A finished batch the provider refuses after 7 days (rotated key, bucket 403, a 404 that does not name the batch) now keeps retrying so its spend is written once the config is fixed, instead of retiring unbilled. A batch the provider still reports running keeps polling. A batch that is not finished in the database becomes stale_expired once a retrieve fails after 7 days, so an unreachable provider is not polled forever. An S3 NoSuchKey for the output object now counts as gone.
* fix(batches): expire an unfinished old batch only after 3 failed retrieves in a row
* fix(batches): keep retrying an unfinished old batch through timeouts, 429 and 5xx
* fix(batches): wait 3 polls before expiring an old batch whose output id lags
* fix(batches): match a resource id in a 404 message followed by punctuation
* fix(batches): keep refusal and pending-output expiry streaks apart, and void a pending-output retirement on a client write
* fix(batches): give up on unsettled batches past twice the staleness cutoff
Replace the per-reason refusal and pending-output streaks with one give-up age, and inject a clock into the poller and the delete check so tests pin exact age boundaries
* fix(batches): keep polling finished batches past the give-up age so their spend stays recoverable
* test(batches): expect a restarted poller's first page to split between old and new batches
* test(batches): pin the delete-guard tests to a fixed clock and type a new journal list
* test(batches): type the provider helper and the storage 404 cases
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
enable_pre_call_checks filtered deployments whose in-memory minute count
reached litellm_params.rpm, undocumented since 0d1cca9aa0. Deployment rpm
and tpm stay routing weights unless optional_pre_call_checks includes
enforce_model_rate_limits, which is now the only deployment rate-limit path.
Resolves LIT-10233
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): audit #47363
Adds integration cells for the input token count paths #47363 changed: the provider id of a managed file and the 403 for another team's file on OpenAI, file_search and additional_tools parity plus cache-mark stripping and untranslatable tools on Claude, and the Bedrock runtime count leaving out the options it refuses.
* fix(count-tokens): reject a lone tool with no chat equivalent before the Claude count call
A Claude input token count whose only tool was mcp or computer_use_preview skipped the pre-call check and went to the provider, so the caller got the provider error instead of the 400 naming the tool. The check now runs whenever a tool would get a placeholder name.
* fix(responses): count a lone Anthropic toolset and name the uncountable tool
The lone-tool 400 now fires only for OpenAI Responses built-in tool types, so a nameless Anthropic toolset still counts. The error says LiteLLM cannot count that tool type for the model. Adds a Bedrock unit cell for the lone-tool 400 and pins the runtime wire test's tool shape per request type.
* fix(responses): take the uncountable tool types from the OpenAI SDK and keep the region hint off LiteLLM's 400
The Responses built-in tool types now come from openai's ToolParam union, so a type the SDK adds is refused before the count call too. The AWS region hint no longer lands on an error LiteLLM raised before calling the provider. Tests cover two more lone types and the hint.
* fix(responses): keep LiteLLM's own list of uncountable tool types and pass a non-string tool type through
* fix(responses): count tool_search and namespace tools the way Claude chat inference sends them
The count refused a lone tool_search with LiteLLM's 400 and sent a lone
namespace raw or squashed it into an empty tool, while Claude chat
inference maps tool_search to Anthropic's tool search tool and unpacks
a namespace into its function tools. Both now go through the
deployment's own map_openai_params, so the count sends what inference
sends, and tool_search leaves the list of uncountable types.
* fix(bedrock): count a Claude request with its deployment's own chat translation
Bedrock invoke and Mantle inference turn a chat tool_search tool into Bedrock's
tool search tool, but the Bedrock counter always translated with the Anthropic
config, so the count answered 400 for a request inference serves.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(count_tokens): keep the framed estimate when a region-less token_count_model cannot count (audit #47441)
* fix(count_tokens): let framing stand in for a shipped token_count_model, keep a configured one's error
The framed local estimate now replaces a 400/404 from any count model the cost map ships, Mantle-pinned rows included, and an admin-set token_count_model keeps its error as the docs say.
* fix(count_tokens): warn with the region when framing stands in for a pinned count model
Test a Mantle row #47441 pinned in its own region, and keep the region hint check on an admin-set count model.
* fix(count_tokens): name the deployment in the framing stand-in warning
* fix(count_tokens): let framing stand in when a cost map count model's region denies the account
A count model the cost map pins to a region can answer 401 or 403 when the
account has not enabled that region or lacks Mantle count access there. With
framing set, the estimate now stands in, as it does for 400 and 404. An admin
set count model and a count model with no pinned region keep the error.
The warning now names the model group, the count model and its region, and
says the proxy falls back to the estimate.
* test(count_tokens): cover framed 401 and 403 from a shipped count model region on the wire
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(images): cover SDK retry budgets and router ownership
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(images): forward max_retries to the OpenAI image generation client
Per-call, deployment and default_litellm_params max_retries were filtered
out before reaching the OpenAI-compatible image handler, which then always
used 2 SDK retries. Forward the caller's value, and pin it to 0 for routed
calls so the Router stays the only retry owner, matching completion()
Co-authored-by: Ericz <yonglzh@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(images): read image max_retries through a typed kwargs view
Co-authored-by: Ericz <yonglzh@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): assert the public invalid retry error mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): preserve the invalid retry status code
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): group the proxy test imports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(images): discard proxy retry test routers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ericz <yonglzh@gmail.com>
* test(huggingface): cover TEI rerank truncate passthrough and server default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(huggingface): stop forcing truncate=false on TEI rerank and pass truncate params through
Co-authored-by: Luke Schaefer <225965040+luke-e-schaefer@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(huggingface): keep TEI truncate options provider-specific
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(huggingface): use MockTransport for rerank truncation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Luke Schaefer <225965040+luke-e-schaefer@users.noreply.github.com>
* test(scim): audit #47544
Add integration cells for the SCIM group delete that now runs through the team delete's bulk path: an admin group delete revokes its members' admin key, a delete sent with the IdP's SCIM token revokes member keys and audits them under that token, a delete that times out on a locked member row changes nothing so a retry works, and two racing team deletes write one tombstone and one team audit row
* test(scim): read the racing team delete's audit rows after the proxy drains them
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Adds a real-proxy case where the MCP follow-up call fails on every router retry, checking that the one failure row bills the first call of every attempt on chat completions and /v1/responses
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): audit #47352
* test(proxy): audit #47352 lazy load during a cold docs build
* test(proxy): route the overlapping chat cell to the rig upstream
* test(proxy): keep the root path cell from rewriting the packaged UI
* test(proxy): prove the lazy and chat cells overlap the cold docs build
* test(proxy): warm the chat path before the overlapping chat cell
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
