With compaction armed and an inference profile tier windowed only through base_model,
escalation now accepts a compaction-owned destination whose window is known, so an
oversized history compacts on the cheapest compactable tier instead of skipping it.
The long-prompt e2e calls get a 180 second client timeout and the coverage registry
points at the moved lines.
* test(integration): update stale expectations to intended provider behavior
Each expectation moved because a merged PR changed behavior on purpose.
- chatgpt string input is wrapped as one user message (#41522)
- Gemini 2.5 tool media sits beside function_response (#48428)
- sampling knobs move to Gemini 3.5 Flash, which keeps them (#47645)
- a body base_model is stripped without allow_client_pricing_override (#47639)
- ai_decide answers without usage return usage null (#48261)
- azure_ai forwards mapped client anthropic-beta values (#48033)
- the OpenCode scripted upstream mints a fresh id per request
* test(e2e-ui): type the guardrail name before deleting it
GuardrailDeleteModal requires the exact name before Delete enables since
#45893. All three guardrail delete flows now go through one helper.
* test(observability): accept any sampled traceresponse flags and expect a 400 for a non-object body
OpenTelemetry 1.45 (#48055) sets the W3C random trace id flag on new traces,
so the gateway's traceresponse now ends in -03. The producer copies the server
span's own trace flags, so the tests now accept any two-digit flags with the
sampled bit set instead of pinning -01
#48438 made a non-object JSON body a 400 "Invalid JSON payload: expected an
object" on purpose, so the [1] body case is renamed non_object_body and
expects that 400
* test(observability): stop waiting on OpenTelemetry's export log line in the langtrace outage test
The outage test waited for "Exception while exporting Span batch" in the proxy
log, which is OpenTelemetry's wording and no longer appears with 1.45. The test
keeps asserting what the gateway guarantees, callers unaffected during the
outage, a live gateway, and every span after recovery delivered once with no
outage span duplicated
* test(guardrails): count the Responses disconnect spend rows #47611 added
Since #47611 a streamed /v1/responses disconnect writes its own spend row, so
the concurrent disconnect test saw 20 rows where it counted 10 by model group.
It now reads each row's call type and expects ten acompletion rows with the
outage verdict and ten aresponses rows with success. /v1/messages disconnects
still write no row on main, so they are left out of the count rather than
pinned as absent
The responses-httpx case of the single disconnect test no longer needs its
LIT-8603 skip and passes
* test(observability): size K4's timeout from the harness boot budget
K4 boots three two-worker proxies in a row. On CircleCI each boot takes 43 to
58 seconds, so a passing run took 179.97s against its fixed 180s cap and 6 of
30 runs hit pytest-timeout. The cap and the restart gate wait now come from
readiness_seconds and graceful_stop_seconds, like other multi-boot tests
* test: remove e2e checks and a CI line count that pin implementation details
Deletes the Auto Router template select anchoring spec, the 390px logs pagination variant, and the call id tooltip and clipboard step in the logs spec. Each one tests library or test helper behavior rather than what users see, and each has been red or flaky on main without a product regression
Also drops the exact line count on the fan-in requirements export in the integration coverage job. It went from 6 to 10 when uv.lock gained a Python 3.15 fork, while the install was still correct. A package the filter drops still fails the job, either at the hash-checked install or at the next step's import
* docs(agents): do not test what shadcn and Base UI already guarantee
Adds the boundary to the dashboard AGENTS.md next to the test tiers, and points the Playwright suite's entry in tests/e2e/AGENTS.md at it
* refactor(mcp): drop the experimental label from the MCP server and client paths
Move litellm/proxy/_experimental/mcp_server to litellm/proxy/mcp_server and litellm/experimental_mcp_client to litellm/mcp_client, with the mirrored tests moved under tests/unit/proxy/mcp_server and tests/unit/mcp_client. Both old packages stay as shims that warn with DeprecationWarning, re-export the new package's public names and alias every submodule in sys.modules so deep imports keep resolving to the same objects. The OpenAPI docs gate now puts the merge base's enterprise package on PYTHONPATH so its base counts do not mix the checked-out enterprise code with the base litellm tree
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(mcp): keep tests/unit/proxy/_experimental as a sharded root for the shim tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): alias namespace subpackages and bind attributes in the deprecated MCP shims
pkgutil.walk_packages skipped mcp_server/auth because it has no __init__.py, so old imports of litellm.proxy._experimental.mcp_server.auth.* failed, and sys.modules entries alone left litellm.experimental_mcp_client.tools unreachable as an attribute. Walk the package on disk and setattr each alias on its old parent.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages
Move the unified /v1/messages code out of a directory called pass_through, since it is not the
passthrough feature. The inner messages sub-package becomes native, and the Vertex AI partner
model experimental_pass_through package becomes messages. The old packages stay as import shims
that warn once and alias every submodule in sys.modules, so old deep imports still resolve
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): keep Final variables out of loops in the rename shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): match the warning text in the rename shim test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): resolve old pass_through imports lazily through a meta path finder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): mark the alias loader spec restore for the type discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): format the renamed package shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): keep the rebind-ok marker on the spec restore line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): carry the new package search path on alias specs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): declare the shim test locals Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
gpt_5_transformation.py becomes gpt_reasoning_transformation.py (OpenAI and Azure),
the Bedrock anthropic_claude3_transformation.py files become anthropic_claude_transformation.py,
anthropic_claude2_transformation.py becomes anthropic_legacy_text_completions_transformation.py,
and vertex_ai_partner_models/llama3 becomes vertex_ai_partner_models/llama. Class and symbol
names are unchanged.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep Aria accessible in the dashboard header
* test(ui): wait for every pagination fixture log
* test(ui): update assistant setup for Aria branding
* test(perf): add small_bodies_30rps and mixed_size_requests_4rps load cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): run the load cases at 50 rps small and 5 rps mixed, the pod's knee
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): ramp the load cases until the pod stops keeping up and report sustained_rps
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): hand the ramp to the observation and widen the mixed ramp to 14 rps
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): floor the small ramp at 20 rps and emit the per-step readings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): emit the ramp's hold bound and each step's unexpected statuses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): read each ramp step's cpu, redis latency and connections from the samplers' windows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): floor mixed_size_requests_ramp at 8 rps from the lane's 10 rps sustained step
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(perf): compare step readings with math.isclose so the e2e basedpyright gate stays clean
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A Bedrock ARN tier with no window of its own now escalates on its base_model's
exact cost-map window. An own window, declared or registered, still wins, and
compaction, pre-call checks and get_router_model_info keep main's behavior.
Covers application inference profile, provisioned and imported model ARNs,
and adds e2e metadata to the new e2e tests.
Merge main into the branch and narrow the change to what main's base_model
window rule leaves open. A Bedrock application inference profile ARN names no
model, so its deployment takes the exact cost-map window of its base_model.
Every other route keeps main's rule, which only widens a known window when
base_model names the same model, and an explicit max_input_tokens still wins.
* fix(ui): show the budget spend limit and rate limits without a collapsed section
The create and edit budget dialogs hid max budget and reset period behind a collapsed Optional Settings section that sat 80px below the last field, and dropped both values from the request unless the section was open on submit
Both dialogs now share BudgetFormFields. Budget ID comes first, then the spend limit as one row ("$500 per month"), then the three rate limits as short fields beside their labels. A secondary Optional badge marks the spend limit and the rate limits group, so people can still tell those fields are optional. Every field is always submitted, and the edit dialog sends the existing spend limit and reset period back unchanged
* fix(ui): send only changed fields when editing a budget
Saving the edit budget dialog resent every field, so an unrelated rate
limit edit rounded a stored sub-cent spend limit to 2 decimals (0.001
became 0) and resent the reset period, which makes the proxy recompute
the next reset for custom durations. The dialog now sends the budget ID
plus the fields the user changed, and the proxy keeps everything else.
Also updates the budgets Playwright spec for the new field labels and
the removed Optional Settings section.
* fix(ui): track budget edit changes from render so a reverted field cannot hide a later one
react-hook-form's formState is a render-time snapshot. Reading dirtyFields
only inside the submit handler meant that after a field was changed and
changed back, a later edit to another field was missing from the first
save. Reading dirtyFields during render keeps the snapshot current.
* refactor(ui): rename the budget components to PascalCase
Renames budget_modal, edit_budget_modal and budget_panel (and their tests) to BudgetModal, EditBudgetModal and BudgetPanel, and drops their filename-pascal-case suppressions.
* test(perf): set every perf ceiling to 1.2x the lane median
* test(perf): derive the upgrade verdict test's samples from the case ceilings
* test(perf): keep memory_growth_megabytes at 100 and 600, the metric swings 60 MB between reruns
* test(perf): keep small litellm_overhead_seconds p99 at 1.2x the lane max, it is the 3rd slowest of 300 requests
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(completion): send prompt_cache_retention to OpenAI instead of silently dropping it
litellm.completion(), Router.completion() and the proxy's /v1/chat/completions
accepted prompt_cache_retention and listed it as a supported OpenAI param, but
completion() never handed it to get_optional_params, and the leftover-kwargs
paths skip known OpenAI params, so it never reached the request body. Make it a
named argument of completion() and acompletion() like prompt_cache_key and pass
it on.
A unit guard sends every OpenAI body param through completion() and fails when a
param added to OPENAI_CHAT_COMPLETION_PARAMS has no case proving it reaches the
wire, the same hole store and prompt_cache_key fell into in #33184. Live e2e
tests capture the body OpenAI receives for the proxy, the SDK and the Router on
both openai/gpt-5.5 and openai/responses/gpt-5.5.
* chore(completion): type the param-forwarding unit test and keep the basedpyright count flat
Every new completion() kwarg adds one unknown-argument error at each call
site that splats an untyped dict into it; typing print_verbose's argument
and Chat's params removes more than prompt_cache_retention added.
* test: freeze the outbound-body model and read captured requests from the mock
Addresses review: the e2e body model is frozen, the backend params are a
tuple, and the unit test reads the request from the respx route instead of
appending to a list.
* test(perf): measure Admin UI journey latency in a browser on the 1TB restore
* test(perf): settle on rendered rows, chart bars and the usage view trigger
* test(perf): cover the UI journey summaries in the e2e harness tests
* test(perf): fail UI journeys on failed samples and p50/p90 ceilings
* test(perf): add Usage, Models and Keys journeys and seed realistic keys, teams, users and models on the disposable restore
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): apply the 16 maxTokens floor to Kimi K3 on Converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): use lane AWS credentials for the Kimi K3 messages test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): bypass the proxy cache and declare test metadata for the Kimi K3 probe
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(perf): gate the gateway's own overhead header, redis latency and postgres load in the perf cases
* test(perf): keep the gateway_added_seconds limits next to the header ones
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
memory_growth_megabytes compared end-of-run RSS with RSS read right after boot, before a worker had freed its startup allocations, so a light case (small_bodies_5rps) read negative growth. The case now replays its first minute of traffic before the baseline read, so the metric is the memory the measured traffic added
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(count_tokens): configurable token_count_framing and token_count_fallback_model, count source in the response
Count endpoints now say where each number came from in litellm_count_source
(provider, fallback_model, local_estimate) and name the fallback model in
litellm_count_fallback_model. When the provider answers 400 or 404 for a model
it cannot count, the proxy counts the same request with token_count_fallback_model.
token_count_framing adds a per-model constant to local estimates. Both settings
ship in the cost map for Bedrock Claude Sonnet/Haiku 5.5, Bedrock Kimi K3, and
Databricks Kimi K3 and DeepSeek V4 Flash, and can be overridden in model_info.
* test(count_tokens): classify the new cost map keys in the schema generator and expect count_source in exact-shape tests
* test(count_tokens): e2e matrix against real inference and integration wire tests for count source, fallback and framing
* fix(count_tokens): keep the provider error when local counting is disabled, name the fallback error, keep settings in cooldown
Moves the cost map lookup off the event loop
* ci(cost map): bound token_count_framing to non-negative and token_count_fallback_model to non-empty
* fix(count_tokens): keep a failed fallback's own status, ship fallbacks only where they count, add system.ai framing
A fallback model that answers 404 returns the requested model's error unchanged, and any other fallback
failure answers with its own status and message. The eu, apac, au and jp Sonnet and Haiku 5.5 entries
no longer ship the Opus 5.5 fallback, which AWS serves only in us-east-1. A configured empty
token_count_fallback_model turns the shipped fallback off. Config framing accepts whole-number floats and
rejects negatives. The Databricks system.ai Kimi K3 and DeepSeek V4 Flash names carry the same framing as
their serving endpoint names, and the e2e matrix runs on Bedrock Haiku 5.5 too.
* test(databricks): list the system.ai Kimi K3 and DeepSeek V4 Flash names with their published DBU rates
* fix(count_tokens): skip a token count fallback model for an hour after it answers 404 on that deployment
A us. or global. Sonnet 5.5 deployment outside us-east-1 paid two extra Bedrock
calls and about 200 ms on every count, since Opus 5.5 cannot count there either.
The Kimi K3 e2e also runs the large system prompt shape now.
* test(count_tokens): use the shared integration port allocator in the fallback wire test
* fix(cost map): keep the Databricks system.ai entries to framing only, so their spend stays as today
* test(count_tokens): cover the fallback 404 skip, inference profile base_model, shipped framing, and deployment-scoped token count settings
Adds integration cells for a fallback model that answers 404, an application inference profile ARN deployment whose base_model carries the shipped fallback, and the shipped Kimi K3 framing with token_count_framing 0 turning it off. Adds unit tests that keep token count settings off the shared backend cost map key and that store a token count setting on PATCH only when the operator changed it
* fix(count_tokens): keep the counter's own 400 and the provider's error when a framed estimate cannot read the body
A 400 the OpenAI or Gemini counter returns before calling the provider is the caller's answer, so it no longer tries the fallback model or a framed estimate. When the framed local estimate cannot read a content block, the client gets the provider's error instead of a 500. Adds /audit round 2's integration tests for the fallback leg's converted body, the Anthropic envelope on fallback failures, and both framing cases.
* fix(count_tokens): answer the provider's error whenever a framed estimate raises
A tool schema the local estimate cannot format, such as a boolean subschema or an array with no items, raised AttributeError or KeyError and answered a 500 on a framed deployment whose provider had already answered 400. Any estimate failure now answers the provider's own error.
* fix(count_tokens): load the custom tokenizer inside the guarded framed estimate
A framed deployment whose custom_tokenizer could not load answered a 500 or 503 where the provider had already answered 400, because the tokenizer was selected outside the estimate's error handling. Selecting it inside answers the provider's error, logs a warning with the estimate's error, and still raises the Rust extension's fork-guard errors.
* fix(count_tokens): look up framing under the deployment's custom_llm_provider
* fix(count_tokens): share one tokenizer download per worker and bound the wait after a provider error
A framed deployment whose custom_tokenizer is not cached locally made every
count wait out the Hugging Face hub's whole retry cycle when the hub could not
be reached, and a burst of those loads held the default thread limiter that
every count's cost map lookup also needs. Counts on a worker now share one
download per minute, a failed download is reused for the rest of that minute,
and after a provider error a count waits at most 5 s before answering the
provider's error. The swallowed-estimate warning logs only the error type.
* fix(count_tokens): keep one tokenizer download per key running and load the local route per request
* test(count_tokens): type the scripted counter and wait on events instead of wall time where possible
* fix(count_tokens): run framed tokenizer loads on their own capped threads and answer a tokenizer panic with the provider error
Framed loads used anyio's default 40 threads, so a hub that drops packets let about 40 hanging loads stall every count on the worker. Loads now run on a per-worker pool of 4, and a count that finds 4 loads running answers the provider's error at once. Any finished load is reused for a minute after it finishes, so a load that fell back to tiktoken is not retried on every count. A pyo3 PanicException from the tokenizer, while loading or counting, now answers the provider's error instead of a plain 500
* feat(proxy): count tokens with token_count_model in its pinned region
token_count_model {model, region} replaces token_count_fallback_model. Bedrock Sonnet and Haiku 5.5 count with Opus 5.5 in us-east-1, or Melbourne for au. profiles, and a failure names the region to enable
* fix(count_tokens): use the renamed count model constant in the e2e test and list token_count_model with the object keys
* fix(proxy): keep the upstream error envelope when adding the region hint
* test(proxy): annotate the token count setting echo test
* ci(cost map): require non-blank token_count_model strings and test the region rules
* fix(count_tokens): answer the token_count_model's error even when framing is set
* fix(count_tokens): keep the provider error for Google contents instead of a framed estimate
* fix(count_tokens): estimate Google contents the route translated to messages
* fix(count_tokens): keep the provider error for Google contents with non-text parts
* test(proxy): drive tokenizer wait and cost map tests by events and a skippable loop clock
* fix(count_tokens): check Google text-only parts without a KeysView comparison basedpyright flags
* feat(proxy): count Bedrock Kimi K3 tokens exactly with its pinned tokenizer
* test(bedrock): keep the Kimi K3 image check in-process
* fix(count_tokens): join the region hint to an upstream message that already ends with a period
Also pins that a blank token_count_model in config falls through to the cost map default
* fix(count_tokens): count Gemini tool-call turns and Responses tools, and stop tokenizer retries from stalling counts
The Google countTokens route estimates contents with functionCall, functionResponse and thoughtSignature parts again, and still answers the provider error for media, thought and unknown parts
The Kimi K3 exact count renders Responses API and Anthropic tools through the chat tool translation the Vertex count already used, now shared from the prompt factory, and falls back to the estimate when the Converse body cannot carry a tool
A failed Kimi K3 tokenizer load is retried in the background without making counts wait, and a load still running after the wait logs a warning
* fix(count_tokens): count Kimi K3 tools the way Bedrock inference sends them
A flat Responses API function or custom tool becomes the chat tool the
Responses route sends to inference, and every other tool goes to the
Converse body as is, so hosted tools Bedrock drops are left out of the
count instead of being counted or turning it into an estimate.
* test(count_tokens): annotate new token counter test locals with Final
* fix(count_tokens): keep shipped count regions inside the deployment's geography
A plain Bedrock Sonnet or Haiku 5.5 deployment in a non-US region no longer sends its count prompt to us-east-1. Global count models and configured token_count_model keep their pinned region. Also expect litellm_count_source in the Vertex Llama count test main added
* fix(count_tokens): resolve count settings off the event loop
* test(mcp): move the MCP OAuth happy path from a live Linear e2e to an integration test
GitHub Actions now runs unit tests only, and the MCP OAuth happy path needed a
captured Linear browser login to click through a real consent page. The OAuth
flows it exercised are already integration tests against an owned
authorization-server double, except one behavior: the stored per-user token
must be served by a proxy process that never saw the consent, with no shared
Redis to fall back on. That case is now in test_mcp_oauth_flows.py
The e2e test, its owned-gateway helper, its workflow, the mcp_oauth_live marker
and the JUnit gate script only that workflow used are removed
* test(mcp): chat completions run an OAuth server tool with the caller's own stored token
* test(e2e): drop the Linear chat-completion MCP OAuth e2e and its helpers
The behavior it checked, a chat completion running an OAuth server's tool with
the caller's stored per-user token, is now an integration test in
test_mcp_llm_endpoints.py that needs no saved Linear browser login
* test(mcp): decode the stored per-user credential as urlsafe base64
The plaintext check used b64decode, which drops the - and _ of a urlsafe value
and raised on roughly one ciphertext in five
* test(mcp): drop the chat OAuth integration test that duplicates test_mcp_slots F2-chat
security/test_mcp_slots.py already sends a per-user OAuth token through
/v1/chat/completions and asserts the peer receives it. Every mutant the new test
killed was also killed by the existing suite, so it added no signal
The 48 MB budget on the third batch kept failing on the x86 release gate with no
leak behind it. Isolated rc/1.107.0 and v1.105.1 stacks both grew about 60 MB on
the third batch with gc-tracked objects flat, and both levelled off to a couple of
MB per batch after twenty batches. Measure batches 3-4 against a 200 MB per-worker
ceiling instead, which a native leak of 400 KB per failed request still trips. The
stored-request size check stays the deterministic catch for the v1.100.0 leak
#46484 moved the litellm_redis_circuit_breaker_* metrics behind service_callback prometheus_system, so the chaos proxy, which only set callbacks prometheus, stopped exposing them and the breaker assertion read 0
* test(e2e): event_loop_stalls charges a hold to the longest gc pass that fits it, not to the sum of every pass since the last hold
* test(e2e): event_loop_stalls lists the pass-through post-stream logging step seen once at 0.105 s
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): warm the memory test up with two batches before the measured one
On a cold worker the second batch of failing requests can still raise the
allocator's high-water mark by about 50 MB while the live Python heap stays
flat, so the 48 MB budget failed about 1 in 10 Buildkite runs without a leak.
The test now runs two warmup batches, measures the third against the same
budget, and records every batch's settled growth per worker as an
rss_batch_growth JUnit property for later calibration
* test(e2e): record each memory batch's growth as soon as it settles
A failure in a later batch no longer drops the growth already measured, and
the worker keys are no longer sorted, which raised on a mix of named and
unnamed hosts
* test(e2e): silence the xunit2 record_property warning on the memory test
* test(e2e): event_loop_stalls perf case reads the gateway's asyncio debug log and fails on a loop hold over 100 ms that is not on its allowlist
The gateway runs the stdlib loop in debug mode (new LITELLM_EVENT_LOOP=asyncio knob next to
PYTHONASYNCIODEBUG=1), the case replays the mixed body mix at 1 rps and groups asyncio's
'Executing <Task ...> took X seconds' lines by coroutine and location. Known holds are listed with
the count they show today; anything else, or a known one past its count, fails the case. Opt-in
through E2E_PERF_EVENT_LOOP=1 so the nightly build skips it
* test(e2e): event_loop_stalls reads the pod list and pod logs through e2e_http instead of a raw httpx client
* test(e2e): event_loop_stalls charges a hold to garbage collection when the gateway's debug-mode gc log accounts for it
The proxy registers a gc.callbacks hook only when the event loop runs in debug mode and logs every
collection over the loop's slow-callback threshold (generation, collected count, seconds). The case
parser pairs such a pass with the hold line that follows it and reports the hold under a bounded
garbage collection bucket instead of the task asyncio happened to blame, which made 7 of 10 local
runs fail on a different task name each time. Holds the collector does not account for keep the
strict allowlist, which gains the prisma first-query warm-up and the second pub/sub subscriber
* test(e2e): event_loop_stalls allowlist caps set at twice the ten-run maximum
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
With E2E_PROVIDER_CACHE=1 the test's Anthropic deployment went through the
shared response cache. Each call's recording expires on its own, so a run
could replay the cache-writing call while a follow-up went live against a
cold provider cache, recording creation instead of a read. That recording
then replayed as a failure for 24 hours. Mark the test provider_live, as
PROVIDER_CACHE.md requires for prompt-caching tests
The cache-creating call now reports cache_creation_input_tokens on top of
Vertex's own prompt count and bills the write at the input rate, as Google
does. The old assertions assumed neither, so the test went red on every run.
Check the cache write, the discounted read, and that a second identical call
reuses the cache instead.
* test(e2e): repair fallback round trip for config-owned fallbacks
The e2e stack config sets router_settings.fallbacks, and since #45860 a
/fallback write to a key the config file owns is refused with a 400. The
round trip now runs on context_window fallbacks, which the stack config
leaves to the database, and also deletes the rule and checks it is gone.
A second test locks the new contract: a /fallback write to the
config-owned fallbacks key returns 400 naming router_settings.fallbacks,
and the config rule keeps being served unchanged.
* test(e2e): assert the refused fallback rule is never served
* fix(ui): preserve key settings drafts during background refreshes
* fix(ui): refresh untouched key settings while preserving edits
* fix(ui): keep team-change MCP clears across key refreshes
clearMcpGrants now marks the MCP fields dirty, so a background refresh with
keepDirtyValues no longer restores the grants a team change just cleared.
Also adapts the draft tests to the onSubmit(values, dirtyFields) signature
on main and asserts a refresh does not mark untouched fields dirty, since the
team admin member-key path rejects saves with non-budget dirty fields.
KeyEditView becomes a one-line arrow to stay under the max-lines limit.
* fix(ui): keep MCP tool defaults out of a key's edited fields
MCPToolPermissions writes non-delete tool defaults for unrestricted servers
on mount. Marking that write dirty made a team admin's budget-only save on a
member key get rejected for editing mcp_tool_permissions. onChange now says
whether a write is a default or a user edit, and the key editor dirties only
edits.
The edit session is now keyed by token or token_id, matching the form's own
fallback, so switching between keys that only carry token_id starts fresh.
* fix(ui): keep a new MCP server's default tool limits across refreshes
Default tool writes for a server the user just added in this edit session
are part of that edit, so they are marked dirty and survive a background
refresh. Defaults written for an already saved server stay clean, so a team
admin's budget-only save on a member key is still accepted.
