* fix(ui): show the budget spend limit and rate limits without a collapsed section
The create and edit budget dialogs hid max budget and reset period behind a collapsed Optional Settings section that sat 80px below the last field, and dropped both values from the request unless the section was open on submit
Both dialogs now share BudgetFormFields. Budget ID comes first, then the spend limit as one row ("$500 per month"), then the three rate limits as short fields beside their labels. A secondary Optional badge marks the spend limit and the rate limits group, so people can still tell those fields are optional. Every field is always submitted, and the edit dialog sends the existing spend limit and reset period back unchanged
* fix(ui): send only changed fields when editing a budget
Saving the edit budget dialog resent every field, so an unrelated rate
limit edit rounded a stored sub-cent spend limit to 2 decimals (0.001
became 0) and resent the reset period, which makes the proxy recompute
the next reset for custom durations. The dialog now sends the budget ID
plus the fields the user changed, and the proxy keeps everything else.
Also updates the budgets Playwright spec for the new field labels and
the removed Optional Settings section.
* fix(ui): track budget edit changes from render so a reverted field cannot hide a later one
react-hook-form's formState is a render-time snapshot. Reading dirtyFields
only inside the submit handler meant that after a field was changed and
changed back, a later edit to another field was missing from the first
save. Reading dirtyFields during render keeps the snapshot current.
* refactor(ui): rename the budget components to PascalCase
Renames budget_modal, edit_budget_modal and budget_panel (and their tests) to BudgetModal, EditBudgetModal and BudgetPanel, and drops their filename-pascal-case suppressions.
* test(perf): set every perf ceiling to 1.2x the lane median
* test(perf): derive the upgrade verdict test's samples from the case ceilings
* test(perf): keep memory_growth_megabytes at 100 and 600, the metric swings 60 MB between reruns
* test(perf): keep small litellm_overhead_seconds p99 at 1.2x the lane max, it is the 3rd slowest of 300 requests
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(completion): send prompt_cache_retention to OpenAI instead of silently dropping it
litellm.completion(), Router.completion() and the proxy's /v1/chat/completions
accepted prompt_cache_retention and listed it as a supported OpenAI param, but
completion() never handed it to get_optional_params, and the leftover-kwargs
paths skip known OpenAI params, so it never reached the request body. Make it a
named argument of completion() and acompletion() like prompt_cache_key and pass
it on.
A unit guard sends every OpenAI body param through completion() and fails when a
param added to OPENAI_CHAT_COMPLETION_PARAMS has no case proving it reaches the
wire, the same hole store and prompt_cache_key fell into in #33184. Live e2e
tests capture the body OpenAI receives for the proxy, the SDK and the Router on
both openai/gpt-5.5 and openai/responses/gpt-5.5.
* chore(completion): type the param-forwarding unit test and keep the basedpyright count flat
Every new completion() kwarg adds one unknown-argument error at each call
site that splats an untyped dict into it; typing print_verbose's argument
and Chat's params removes more than prompt_cache_retention added.
* test: freeze the outbound-body model and read captured requests from the mock
Addresses review: the e2e body model is frozen, the backend params are a
tuple, and the unit test reads the request from the respx route instead of
appending to a list.
* test(perf): measure Admin UI journey latency in a browser on the 1TB restore
* test(perf): settle on rendered rows, chart bars and the usage view trigger
* test(perf): cover the UI journey summaries in the e2e harness tests
* test(perf): fail UI journeys on failed samples and p50/p90 ceilings
* test(perf): add Usage, Models and Keys journeys and seed realistic keys, teams, users and models on the disposable restore
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): apply the 16 maxTokens floor to Kimi K3 on Converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): use lane AWS credentials for the Kimi K3 messages test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): bypass the proxy cache and declare test metadata for the Kimi K3 probe
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(perf): gate the gateway's own overhead header, redis latency and postgres load in the perf cases
* test(perf): keep the gateway_added_seconds limits next to the header ones
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
memory_growth_megabytes compared end-of-run RSS with RSS read right after boot, before a worker had freed its startup allocations, so a light case (small_bodies_5rps) read negative growth. The case now replays its first minute of traffic before the baseline read, so the metric is the memory the measured traffic added
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(count_tokens): configurable token_count_framing and token_count_fallback_model, count source in the response
Count endpoints now say where each number came from in litellm_count_source
(provider, fallback_model, local_estimate) and name the fallback model in
litellm_count_fallback_model. When the provider answers 400 or 404 for a model
it cannot count, the proxy counts the same request with token_count_fallback_model.
token_count_framing adds a per-model constant to local estimates. Both settings
ship in the cost map for Bedrock Claude Sonnet/Haiku 5.5, Bedrock Kimi K3, and
Databricks Kimi K3 and DeepSeek V4 Flash, and can be overridden in model_info.
* test(count_tokens): classify the new cost map keys in the schema generator and expect count_source in exact-shape tests
* test(count_tokens): e2e matrix against real inference and integration wire tests for count source, fallback and framing
* fix(count_tokens): keep the provider error when local counting is disabled, name the fallback error, keep settings in cooldown
Moves the cost map lookup off the event loop
* ci(cost map): bound token_count_framing to non-negative and token_count_fallback_model to non-empty
* fix(count_tokens): keep a failed fallback's own status, ship fallbacks only where they count, add system.ai framing
A fallback model that answers 404 returns the requested model's error unchanged, and any other fallback
failure answers with its own status and message. The eu, apac, au and jp Sonnet and Haiku 5.5 entries
no longer ship the Opus 5.5 fallback, which AWS serves only in us-east-1. A configured empty
token_count_fallback_model turns the shipped fallback off. Config framing accepts whole-number floats and
rejects negatives. The Databricks system.ai Kimi K3 and DeepSeek V4 Flash names carry the same framing as
their serving endpoint names, and the e2e matrix runs on Bedrock Haiku 5.5 too.
* test(databricks): list the system.ai Kimi K3 and DeepSeek V4 Flash names with their published DBU rates
* fix(count_tokens): skip a token count fallback model for an hour after it answers 404 on that deployment
A us. or global. Sonnet 5.5 deployment outside us-east-1 paid two extra Bedrock
calls and about 200 ms on every count, since Opus 5.5 cannot count there either.
The Kimi K3 e2e also runs the large system prompt shape now.
* test(count_tokens): use the shared integration port allocator in the fallback wire test
* fix(cost map): keep the Databricks system.ai entries to framing only, so their spend stays as today
* test(count_tokens): cover the fallback 404 skip, inference profile base_model, shipped framing, and deployment-scoped token count settings
Adds integration cells for a fallback model that answers 404, an application inference profile ARN deployment whose base_model carries the shipped fallback, and the shipped Kimi K3 framing with token_count_framing 0 turning it off. Adds unit tests that keep token count settings off the shared backend cost map key and that store a token count setting on PATCH only when the operator changed it
* fix(count_tokens): keep the counter's own 400 and the provider's error when a framed estimate cannot read the body
A 400 the OpenAI or Gemini counter returns before calling the provider is the caller's answer, so it no longer tries the fallback model or a framed estimate. When the framed local estimate cannot read a content block, the client gets the provider's error instead of a 500. Adds /audit round 2's integration tests for the fallback leg's converted body, the Anthropic envelope on fallback failures, and both framing cases.
* fix(count_tokens): answer the provider's error whenever a framed estimate raises
A tool schema the local estimate cannot format, such as a boolean subschema or an array with no items, raised AttributeError or KeyError and answered a 500 on a framed deployment whose provider had already answered 400. Any estimate failure now answers the provider's own error.
* fix(count_tokens): load the custom tokenizer inside the guarded framed estimate
A framed deployment whose custom_tokenizer could not load answered a 500 or 503 where the provider had already answered 400, because the tokenizer was selected outside the estimate's error handling. Selecting it inside answers the provider's error, logs a warning with the estimate's error, and still raises the Rust extension's fork-guard errors.
* fix(count_tokens): look up framing under the deployment's custom_llm_provider
* fix(count_tokens): share one tokenizer download per worker and bound the wait after a provider error
A framed deployment whose custom_tokenizer is not cached locally made every
count wait out the Hugging Face hub's whole retry cycle when the hub could not
be reached, and a burst of those loads held the default thread limiter that
every count's cost map lookup also needs. Counts on a worker now share one
download per minute, a failed download is reused for the rest of that minute,
and after a provider error a count waits at most 5 s before answering the
provider's error. The swallowed-estimate warning logs only the error type.
* fix(count_tokens): keep one tokenizer download per key running and load the local route per request
* test(count_tokens): type the scripted counter and wait on events instead of wall time where possible
* fix(count_tokens): run framed tokenizer loads on their own capped threads and answer a tokenizer panic with the provider error
Framed loads used anyio's default 40 threads, so a hub that drops packets let about 40 hanging loads stall every count on the worker. Loads now run on a per-worker pool of 4, and a count that finds 4 loads running answers the provider's error at once. Any finished load is reused for a minute after it finishes, so a load that fell back to tiktoken is not retried on every count. A pyo3 PanicException from the tokenizer, while loading or counting, now answers the provider's error instead of a plain 500
* feat(proxy): count tokens with token_count_model in its pinned region
token_count_model {model, region} replaces token_count_fallback_model. Bedrock Sonnet and Haiku 5.5 count with Opus 5.5 in us-east-1, or Melbourne for au. profiles, and a failure names the region to enable
* fix(count_tokens): use the renamed count model constant in the e2e test and list token_count_model with the object keys
* fix(proxy): keep the upstream error envelope when adding the region hint
* test(proxy): annotate the token count setting echo test
* ci(cost map): require non-blank token_count_model strings and test the region rules
* fix(count_tokens): answer the token_count_model's error even when framing is set
* fix(count_tokens): keep the provider error for Google contents instead of a framed estimate
* fix(count_tokens): estimate Google contents the route translated to messages
* fix(count_tokens): keep the provider error for Google contents with non-text parts
* test(proxy): drive tokenizer wait and cost map tests by events and a skippable loop clock
* fix(count_tokens): check Google text-only parts without a KeysView comparison basedpyright flags
* feat(proxy): count Bedrock Kimi K3 tokens exactly with its pinned tokenizer
* test(bedrock): keep the Kimi K3 image check in-process
* fix(count_tokens): join the region hint to an upstream message that already ends with a period
Also pins that a blank token_count_model in config falls through to the cost map default
* fix(count_tokens): count Gemini tool-call turns and Responses tools, and stop tokenizer retries from stalling counts
The Google countTokens route estimates contents with functionCall, functionResponse and thoughtSignature parts again, and still answers the provider error for media, thought and unknown parts
The Kimi K3 exact count renders Responses API and Anthropic tools through the chat tool translation the Vertex count already used, now shared from the prompt factory, and falls back to the estimate when the Converse body cannot carry a tool
A failed Kimi K3 tokenizer load is retried in the background without making counts wait, and a load still running after the wait logs a warning
* fix(count_tokens): count Kimi K3 tools the way Bedrock inference sends them
A flat Responses API function or custom tool becomes the chat tool the
Responses route sends to inference, and every other tool goes to the
Converse body as is, so hosted tools Bedrock drops are left out of the
count instead of being counted or turning it into an estimate.
* test(count_tokens): annotate new token counter test locals with Final
* fix(count_tokens): keep shipped count regions inside the deployment's geography
A plain Bedrock Sonnet or Haiku 5.5 deployment in a non-US region no longer sends its count prompt to us-east-1. Global count models and configured token_count_model keep their pinned region. Also expect litellm_count_source in the Vertex Llama count test main added
* fix(count_tokens): resolve count settings off the event loop
* test(mcp): move the MCP OAuth happy path from a live Linear e2e to an integration test
GitHub Actions now runs unit tests only, and the MCP OAuth happy path needed a
captured Linear browser login to click through a real consent page. The OAuth
flows it exercised are already integration tests against an owned
authorization-server double, except one behavior: the stored per-user token
must be served by a proxy process that never saw the consent, with no shared
Redis to fall back on. That case is now in test_mcp_oauth_flows.py
The e2e test, its owned-gateway helper, its workflow, the mcp_oauth_live marker
and the JUnit gate script only that workflow used are removed
* test(mcp): chat completions run an OAuth server tool with the caller's own stored token
* test(e2e): drop the Linear chat-completion MCP OAuth e2e and its helpers
The behavior it checked, a chat completion running an OAuth server's tool with
the caller's stored per-user token, is now an integration test in
test_mcp_llm_endpoints.py that needs no saved Linear browser login
* test(mcp): decode the stored per-user credential as urlsafe base64
The plaintext check used b64decode, which drops the - and _ of a urlsafe value
and raised on roughly one ciphertext in five
* test(mcp): drop the chat OAuth integration test that duplicates test_mcp_slots F2-chat
security/test_mcp_slots.py already sends a per-user OAuth token through
/v1/chat/completions and asserts the peer receives it. Every mutant the new test
killed was also killed by the existing suite, so it added no signal
The 48 MB budget on the third batch kept failing on the x86 release gate with no
leak behind it. Isolated rc/1.107.0 and v1.105.1 stacks both grew about 60 MB on
the third batch with gc-tracked objects flat, and both levelled off to a couple of
MB per batch after twenty batches. Measure batches 3-4 against a 200 MB per-worker
ceiling instead, which a native leak of 400 KB per failed request still trips. The
stored-request size check stays the deterministic catch for the v1.100.0 leak
#46484 moved the litellm_redis_circuit_breaker_* metrics behind service_callback prometheus_system, so the chaos proxy, which only set callbacks prometheus, stopped exposing them and the breaker assertion read 0
* test(e2e): event_loop_stalls charges a hold to the longest gc pass that fits it, not to the sum of every pass since the last hold
* test(e2e): event_loop_stalls lists the pass-through post-stream logging step seen once at 0.105 s
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): warm the memory test up with two batches before the measured one
On a cold worker the second batch of failing requests can still raise the
allocator's high-water mark by about 50 MB while the live Python heap stays
flat, so the 48 MB budget failed about 1 in 10 Buildkite runs without a leak.
The test now runs two warmup batches, measures the third against the same
budget, and records every batch's settled growth per worker as an
rss_batch_growth JUnit property for later calibration
* test(e2e): record each memory batch's growth as soon as it settles
A failure in a later batch no longer drops the growth already measured, and
the worker keys are no longer sorted, which raised on a mix of named and
unnamed hosts
* test(e2e): silence the xunit2 record_property warning on the memory test
* test(e2e): event_loop_stalls perf case reads the gateway's asyncio debug log and fails on a loop hold over 100 ms that is not on its allowlist
The gateway runs the stdlib loop in debug mode (new LITELLM_EVENT_LOOP=asyncio knob next to
PYTHONASYNCIODEBUG=1), the case replays the mixed body mix at 1 rps and groups asyncio's
'Executing <Task ...> took X seconds' lines by coroutine and location. Known holds are listed with
the count they show today; anything else, or a known one past its count, fails the case. Opt-in
through E2E_PERF_EVENT_LOOP=1 so the nightly build skips it
* test(e2e): event_loop_stalls reads the pod list and pod logs through e2e_http instead of a raw httpx client
* test(e2e): event_loop_stalls charges a hold to garbage collection when the gateway's debug-mode gc log accounts for it
The proxy registers a gc.callbacks hook only when the event loop runs in debug mode and logs every
collection over the loop's slow-callback threshold (generation, collected count, seconds). The case
parser pairs such a pass with the hold line that follows it and reports the hold under a bounded
garbage collection bucket instead of the task asyncio happened to blame, which made 7 of 10 local
runs fail on a different task name each time. Holds the collector does not account for keep the
strict allowlist, which gains the prisma first-query warm-up and the second pub/sub subscriber
* test(e2e): event_loop_stalls allowlist caps set at twice the ten-run maximum
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
With E2E_PROVIDER_CACHE=1 the test's Anthropic deployment went through the
shared response cache. Each call's recording expires on its own, so a run
could replay the cache-writing call while a follow-up went live against a
cold provider cache, recording creation instead of a read. That recording
then replayed as a failure for 24 hours. Mark the test provider_live, as
PROVIDER_CACHE.md requires for prompt-caching tests
The cache-creating call now reports cache_creation_input_tokens on top of
Vertex's own prompt count and bills the write at the input rate, as Google
does. The old assertions assumed neither, so the test went red on every run.
Check the cache write, the discounted read, and that a second identical call
reuses the cache instead.
* test(e2e): repair fallback round trip for config-owned fallbacks
The e2e stack config sets router_settings.fallbacks, and since #45860 a
/fallback write to a key the config file owns is refused with a 400. The
round trip now runs on context_window fallbacks, which the stack config
leaves to the database, and also deletes the rule and checks it is gone.
A second test locks the new contract: a /fallback write to the
config-owned fallbacks key returns 400 naming router_settings.fallbacks,
and the config rule keeps being served unchanged.
* test(e2e): assert the refused fallback rule is never served
* fix(ui): preserve key settings drafts during background refreshes
* fix(ui): refresh untouched key settings while preserving edits
* fix(ui): keep team-change MCP clears across key refreshes
clearMcpGrants now marks the MCP fields dirty, so a background refresh with
keepDirtyValues no longer restores the grants a team change just cleared.
Also adapts the draft tests to the onSubmit(values, dirtyFields) signature
on main and asserts a refresh does not mark untouched fields dirty, since the
team admin member-key path rejects saves with non-budget dirty fields.
KeyEditView becomes a one-line arrow to stay under the max-lines limit.
* fix(ui): keep MCP tool defaults out of a key's edited fields
MCPToolPermissions writes non-delete tool defaults for unrestricted servers
on mount. Marking that write dirty made a team admin's budget-only save on a
member key get rejected for editing mcp_tool_permissions. onChange now says
whether a write is a default or a user edit, and the key editor dirties only
edits.
The edit session is now keyed by token or token_id, matching the form's own
fallback, so switching between keys that only carry token_id starts fresh.
* fix(ui): keep a new MCP server's default tool limits across refreshes
Default tool writes for a server the user just added in this edit session
are part of that edit, so they are marked dirty and survive a background
refresh. Defaults written for an already saved server stay clean, so a team
admin's budget-only save on a member key is still accepted.
* fix(azure): add responses_api_version so the Responses route stops using the deployment chat api_version
* fix(azure_ai): fold Grok reasoning tokens into completion_tokens like the xai provider
* fix(utils): refresh provider model sets after register_model
* fix(cost-map): align azure gpt-6.1-sol reasoning effort flags with the openai row
* test(e2e): cover Azure Responses on a deployment pinned to a dated chat api_version
* fix(utils): keep router deployment ids out of provider model sets
register_model now refreshes the provider sets, so the per-request and
router registrations keyed by a deployment id put those opaque ids into
wildcard model listings. Those callers now pass
add_to_provider_model_sets=False.
* fix(azure): carry responses_api_version through the chat to Responses bridge
completion() dropped responses_api_version from the litellm_params it hands
the Responses bridge, so azure/responses/<model> on a dated chat api_version
still built the 404ing /openai/responses URL. Also cover the auto-router
marker never forwarding the new param.
* fix(azure): stop listing a wrong set of reasoning efforts when none is rejected
The message said only low, medium and high are supported, which is wrong for
models that also take xhigh such as gpt-6.1-sol.
* test(azure): use the latest grok and gpt-5.4 rows in the new azure tests
* fix(proxy): list each wildcard-expanded model once
A provider set can hold a model both bare and provider-prefixed, which now
happens after any priced request registers its openai/<model> key, so the
openai/* listing showed that model twice.
* fix(azure): route Responses past dated GA api_versions by default
A deployment pinned to a dated GA api_version such as 2024-10-21 for chat now sends Responses calls (create, get, delete, cancel, input_items and the azure/responses/ bridge) to the default Responses version, because those dated versions have no Responses route on Azure and 404. responses_api_version still overrides it, -preview, v1, latest and preview versions are kept, an api-version in the api_base query still wins, and chat keeps api_version as before
* chore(azure_ai): mark the grok chunk_parser dict as an override contract
* fix(azure): rewrite only the route's /openai segment when moving a call to v1
The v1 rewrite in the Azure URL builder replaced every "/openai" substring in the path, so a gateway
api_base like https://apim.example.com/openai-gw/team turned into /openai/v1-gw/team/openai/v1/responses.
Dated GA Responses calls now land on the v1 route by default, which made this reachable for them too,
so the rewrite now only touches the last /openai path segment.
The Responses version resolver no longer duplicates the builder's default for an unset api_version.
Tests cover the gateway api_base case, the env AZURE_API_BASE query-string precedence, and the e2e suite
now has one test for the dated GA default and one where only responses_api_version can make the call work
* test(integration): cover Azure Responses api_version, Grok reasoning spend, wildcard listing and reasoning_effort none
Wire tests pin the Azure route and query for chat, Responses and the azure/responses bridge on a dated GA api_version, and check responses_api_version never reaches the request body. Cost cases check azure_ai Grok folds reasoning tokens into completion_tokens and spend, streaming and not. A config-driven proxy checks /v1/models next to an openai/* wildcard lists each id once and never a deployment id. Azure gpt-6.1-sol rejects reasoning_effort none before reaching upstream and drops it with drop_params
* fix(utils): list ai21 chat and vertex code models under their provider wildcard
register_model now files ai21 chat models into ai21_chat_models, which models_by_provider never read, so a registered ai21 chat model dropped out of ai21/* listings. models_by_provider now reads ai21_chat_models and the vertex code chat and code text sets too.
* fix(main): keep caller-chosen model names out of provider wildcard listings
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* feat(ui): add clean-slate playground chat prototype
* feat(ui): replace the playground chat with the redesigned composer and inspector
* fix(ui): close the playground chat gaps found in self-review
Resend only complete text exchanges, so empty replies, MCP direct calls and
transcriptions no longer leak into chat history. A stopped request can no longer
append output, attach its gateway trace to a newer reply or clear the newer
request's loading state. MCP selections are only pruned once both the server and
toolset lists load. The MCP tool picker lists toolsets again and calls the server
that backs the chosen tool, and a saved model the current key cannot use is
cleared when the model list loads, matching the old chat
Also vendors the scroll-fade-x utility the attachment primitive uses
* fix(ui): resolve playground toolset tools per server and drop late session ids
A toolset now lists only its own tools, each resolved on the server the toolset
names, so the args form and the call always target the same tool. A saved
toolset no longer asks for tools with its own id while the toolset list loads,
and a failed tool list says so instead of claiming the server has no tools
A stopped request can no longer overwrite the Responses session id, starting a
new chat drops a request that is still running, embeddings replies are no longer
resent as chat history, and a saved model missing from an already cached model
list is cleared too. A failing toolset list falls back right away instead of
retrying while the MCP pickers show loading
* fix(ui): guard playground sends and label inspector stats
Pressing Enter twice no longer sends the same message twice, an attachment the
new endpoint cannot take is dropped when the endpoint changes, a stopped
Responses request can no longer show its code output on a newer reply, and a
reply that broke mid-stream is no longer resent as history
The inspector's speed, cost and routing stats get the same icons as the reply
footer, and the footer's metrics now line up with its copy and Inspect buttons
* fix(ui): write playground key settings only through the encoded storage path
Key source, virtual key and custom headers now go through their own write loop, separate from the plain settings, so a key value can never take the plain sessionStorage branch
* fix(ui): restore playground BYOK connect and reconnect
Picking a BYOK server for a direct tool call now shows a key button next to the
server picker, so the key can be connected without leaving the MCP endpoint. A
connected BYOK server gets a Replace key action in the MCP dialog, also shown
when its tools fail to load, and saving a key reloads that server's tool list
* revert: restore the single playground settings storage loop
Reverts ca4e742115. The CodeQL finding covers the same encoded sessionStorage handling the old chat used, so it is left as is
* test(ui): type the BYOK credential mock without deep instantiation
* fix(ui): let users back out of replacing a BYOK key
The replace-key form in the MCP dialog now has Cancel, and it resets when another server is picked or the dialog closes, so a working key is never forced to be overwritten
* refactor(ui): name the MCP dialog row focus handler
* fix(ui): accept exact custom models, tags and decimal temperatures in the playground
A typed model or tag that is a prefix of a listed one could not be used,
because Enter picked the partial match and the custom action only showed
when nothing matched. The custom value is now a list option whenever no
exact match exists. Temperature gets an editable input beside the slider
so values finer than the 0.1 step can be sent again.
* fix(ui): stop claiming no guardrails ran when the gateway sent no headers
Direct MCP calls, A2A agents and any request that fails before headers
arrive built their trace from an empty header set, so the inspector said
no guardrails or policies ran. The gateway fields are now left unset in
that case and the inspector says the endpoint does not report them.
* fix(ui): leave policies unknown when the gateway headers are hidden
The proxy exposes x-litellm-applied-guardrails to other origins but not
x-litellm-applied-policies, so a dashboard on another origin read every
request as having run no policies. Policies are now unknown unless the
call id header is readable, and the inspector says so.
* feat(ui): add the 1 hour cache write cost to the add model and model edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): cover both cache write tiers in the add model and edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): hoist the expected pricing payload out of the inline assertion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): share the per-token conversion between both cache write tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): derive the touched pricing field type from the pricing field list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): collapse the duplicated pricing source keys into one map
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): convert add model pricing once for all selected models and describe the 1 hour fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): type the cache write tier deployment read back and fill the 1 hour field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(e2e): keep the cache write tier spec on the dashboard prettier config
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): mixed_size_requests_2rps perf case (coding-agent bodies, JWT and key callers, two guardrails)
* test(e2e_harness): test_observe builds DbCounters with statements and expects the background Postgres line
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): perf datastore limits count what a request waits on, background Postgres and Redis work reported on its own
postgres_calls_per_request and redis_calls_per_request were whole-server totals (pg_stat_database commits, Redis INFO
commands), so spend flushes and config reloads moved them and a new per-request lookup could hide in the noise. They
now count the statements and commands a request waits on before it is forwarded (auth and routing reads, the
rate-limiter script), from pg_stat_statements read before and after the run and a Redis MONITOR connection held
during it. Spend writes, daily aggregates, config reloads and cache invalidations are postgres_background_calls_per_request
and redis_background_calls_per_request. The run output lists the top statements and commands per bucket, and the
counters are read right after the run so the test's own spend sample is not counted
* test(e2e): perf datastore reads only the runtime user's pg_stat_statements rows, other users' rows have a NULL queryid
* test(e2e): perf Redis classifier strips the litellm.caching: namespace the deployed gateway puts on every key
* test(e2e): small_bodies_5rps limits from ten split-metric runs (postgres awaited 1.8, redis awaited 18, postgres background 12.5), spend buffers and cron locks classified as background
* test(e2e): small_bodies_5rps asserts only the awaited Postgres and Redis counts, background lines stay printed
* test(e2e): perf Redis classifier puts the cron lock script's GET and DEL in background, not awaited
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): attach perf case metrics and limits as JUnit properties
The perf case only printed its metrics table, so a green run left no numbers
behind. Each observed metric and its limit now land as <property> entries on
the test's JUnit testcase, named perf_<metric>[_p<percentile>] and
perf_<metric>[_p<percentile>]_limit, so project-releaser can collect them
* test(e2e): compare perf properties exactly instead of through pytest.approx
basedpyright flags pytest.approx as partially unknown in the e2e lint step,
so the harness test now uses metric values that are exact in binary floating
point
* test(ui): close each federation spec API connection so the proxy cannot drop an idle keep-alive socket
* test(ui): send Connection: close on every integration browser spec API call
* test(ui): send Connection: close on every integration browser request from the Playwright config
* test(e2e): perf streams take a reasoning or tool-call delta as the first token, tool-call replies had no time_to_first_token
* test(e2e): perf users start at a random offset and jitter each interval by up to 25%, no more lockstep bursts at the planned rate
* test(e2e): small_bodies_5rps sends from the mixed case's callers (two teams, JWT users, keys in and out of teams) through the pre and post guardrail
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
