The bundled beta map stripped dangerous-tool-use-2026-09-03 for providers
main supports it on, so safeguards requests went upstream without the beta
whenever the proxy used the bundled map. The worker-SIGKILL chaos test needs
OwnedProxy.replacing_worker, which this line's integration harness lacks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agentcore/search and vertex_ai/semantic-ranker-default-004 are priced in main's map, so these tests follow main: the search test asserts the mapped per-query price and the custom-pricing rerank test uses an unmapped deployment name.
aiohttp 3.14.3 -> 3.14.5 (yarl 1.23.0 -> 1.25.1 with it), mlflow 3.15.0 -> 3.17.0 (mlflow-skinny and mlflow-tracing with it), claude-agent-sdk 0.1.44 -> 0.2.165 with its dev pin widened to main's ==0.2.165. Every moved version matches main's uv.lock.
Backport of #47625 to stable/1.106.x.
Cherry-picked from 6564a490f4 (main).
Adapted to the line without #45797 or #47494. The router budget limiter claims the batch charge with the line's duration_in_seconds TTL (budget_window_seconds and the injected clock come from #45797, which is not backported), and its realtime skip keeps the line's existing _aresponses_websocket check. The write_started flag on a budget reservation omits the apply_search_free_tier branch, since that flag is #47494's. Tests: the search-allowance accounting test is dropped for the same reason, and the cost-callback and router tests get the litellm and SimpleNamespace imports the line's test files lack.
Backport of #47419 to stable/1.106.x.
Cherry-picked from 3124797399 (main).
Adapted to the line without #41523. Registered rates are tracked per cost-map key in litellm.utils (_registered_pricing_fields, reset when litellm.model_cost is replaced, so a request-scoped registration does not outlive its catalog) instead of AliasedCostEntry.registered_fields. AliasedCostEntry and get_model_cost_map.py are untouched. The register_model replay now merges successive registrations as on main. An inference-profile ARN model_id is reduced to its profile id inside _invoked_model_id_for_cost rather than backporting the get_model_info ARN handling from #46128. Only #47419's lines are taken in completion_cost; base_model_pricing_provider (#46318) and configured search cost (#46571) are not. Tests: CONVERSE_ANSWER added to the slow_upstream helper (it came with #47162, which is not backported), the isolated_pricing_registry fixture and custom-pricing helpers come from main's test file, and the ignore_stamped_cost assertion uses the line's response_cost_calculator signature.
Prerequisite for #47419 on stable/1.106.x. Hand-ports only the helpers #47419 calls from main: ModelInfoLookupMiss, lookup_model_info_for_registration, model_info_with_pricing_overrides with _without_overridden_price_variants and its rate-table helpers (from 9e9dccb9d4, #41523), and runtime_model_cost_registrations (from 3124797399, #47419). runtime_model_cost_registrations reads the line's recorded register_model payloads instead of AliasedCostEntry.registered_fields, so #41523 itself is not backported: register_model, alias entries, cache-price inheritance and unpriced deployments behave as they did on the line.
Prerequisite for #43935 on stable/1.106.x, whose test pins these rows. Ports only the supports_none_reasoning_effort flips for the six azure and azure_ai gpt-6.1-sol rows from 9cf09c18fe (#46377) on main. The rest of #46377 (Azure Responses api-version routing, Grok reasoning tokens, register_model provider refresh) is not included.
Backport of #43935 to stable/1.106.x.
Cherry-picked from d7620c77a1 (main).
Adapted to the line: responses_api_bridge_check lives in litellm/main.py here (moved to litellm/responses/bridge_check.py on main by #45772), so the reasoning_active change and _chat_gate_refuses_reasoning_effort_none are ported there, importing AzureAIGPT5Config locally. Cost-map flags are replayed per model key. Test hunks that conflicted are applied onto the line's files.
Prerequisite for the #47190 integration test on stable/1.106.x. Copied unchanged from tests/integration/_support/prisma_clients.py on main, added there by #45939.
Backport of #47190 to stable/1.106.x.
Cherry-picked from 0ede0e10e7 (main).
Hand-ported onto the line's poller. ManagedObjectRow lives in check_batch_cost.py on this line, so created_at and updated_at are added to its local _ManagedObjectRow. _finalize_unbilled_terminal_job keeps the line's None return and skips the log when the snapshot no longer matches. Test hunks for TestBatchLineItemsFromThePoller are dropped, and the appended tests do not assert the bool return.
Backport of #47182 to stable/1.106.x.
Cherry-picked from 1212260ab1 (main).
Test conflict resolved by keeping the line's file plus this PR's own tests.
Backport of #47184 to stable/1.106.x.
Cherry-picked from 3fb95c8fe3 (main).
Hand-ported onto the line's poller: the claim now writes status and file_object with batch_processed and the release restores them, and _ManagedObjectRow gains status. The line has no postgres_storable_json, so the NUL-stripping sanitizer (_storable_text, _storable_json_value, postgres_storable_json, from #45703 on main) is added to managed_files.py and the claim stores postgres_storable_json(response). Test hunks for TestBatchLineItemsFromThePoller (from #41691, not on the line) are dropped.
Backport of #46102 to stable/1.106.x.
Cherry-picked from 5b042c0db6 (main).
Adapted to the line: only the anthropic Messages config is ported (a non-null safeguards field adds dangerous-tool-use-2026-09-03), with its unit cases. The azure_ai, claude_platform, deepseek and vertex_ai Messages files, the beta-map provider plumbing, and the Rust integration tests are not part of this backport.
Backport of #43955 to stable/1.106.x.
Cherry-picked from eb5295b20c (main).
Adapted to the line: the existing CLI-beta wire assertion change is dropped since that test is not on the line.
The #47389 adaptation moved the handler's return type to dict[str, object] and kept the Any import, so ruff check reports F401 on this file at 1.106.1 and not at 1.106.0
e46904b546 decided by resolved provider name, so the OpenAI-compatible
/v1/messages passthrough and Bedrock Claude Platform /v1/messages, which
inherit the "anthropic" provider name, and the Bedrock Mantle chat and
/v1/messages configs, which inherit the Bedrock Invoke override, still sent
block_binding where 1.106.0 sent {"type": "adaptive"}. Each in-scope config
now opts in for its own class only, and the Bedrock Invoke display: updates
and clear_thinking rebuilds keep block_binding only when the config carries it
The #46342 pick spread every caller key from a legacy thinking.type=enabled
request into the rebuilt adaptive param, so block_binding and any unknown key
newly reached Bedrock Converse, Vertex AI and Databricks, where 1.106.0 sent
only {"type": "adaptive"}. The rebuild now carries block_binding alone, and
only for Anthropic, Azure AI and Bedrock Invoke. The Bedrock Invoke
display: updates and clear_thinking rebuilds keep block_binding and drop the
other caller keys, as 1.106.0 did
* fix(proxy): authorize the public name of a team-scoped model on batch retrieve/cancel
A unified batch id resolves to the deployment's internal model_name
(model_name_{team_id}_{uuid}), which is in no key or team allowlist, so
authorize_model_for_key always denied and team keys got 403 retrieving or
cancelling their own batches. Authorize against model_info
team_public_model_name when the resolved deployment is team-scoped, falling
back to the resolved name otherwise.
Fixes#46958
* fix(proxy): authorize team-scoped batch models by their public name on both auth paths
* fix(proxy): accept the owning team's public name for its own team deployment on every auth path
Managed batch, file, fine-tuning, vector store, character and video ids resolve to the
team deployment's internal name again, so teams that granted the internal name keep access.
The model access check now also accepts the owning team's public name and its legacy team
aliases for that internal name, and only when the caller's team owns the deployment.
* fix(proxy): match a team deployment's public and alias names against the caller's own grants
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit e979bd0ecf)
* fix(vector-stores): keep proxy context out of provider vector store metadata
The proxy merged its request context (user_api_key_* fields, endpoint,
user_agent, requester_ip_address, litellm_api_version) and the router's
deployment info into the vector store request's metadata. For vector
stores that field is also OpenAI's own metadata, which is capped at 16
keys, so a store created with a team key was full on creation and every
later metadata update returned 400.
Vector store routes now keep that context in litellm_metadata, like
batches, files and responses, and the router does the same for
avector_store_create. Provider-side metadata holds only what the caller
sent.
Adds db_scripts/strip_vector_store_internal_metadata.py, an opt-in
script that removes the proxy-written keys from stores created before
this change. It is a dry run unless --apply is passed.
* fix(vector-stores): page the metadata cleanup script without recursion and tolerate non-string values
Listing every store recursed once per page, which overflows the interpreter stack on large projects. Pages are now walked iteratively. Metadata values are no longer required to be strings, and a malformed response exits 2 with a fixed message instead of a traceback that could echo metadata values.
* fix(vector-stores): read proxy agent id, tags and spend_logs_metadata from litellm_metadata
Vector store routes now keep proxy context in litellm_metadata, but the v3
limiter only read agent_id from metadata, and logging let provider metadata
tags or spend_logs_metadata replace the proxy's values. A string tags value
also crashed get_request_tags.
The limiter now prefers the proxy-resolved agent_id in litellm_metadata.
Logging combines list tags from both buckets (metadata first, deduplicated)
and merges dict spend_logs_metadata with litellm_metadata winning, ignoring
values of the wrong type.
* fix(vector-stores): narrow the logging fallback to mistyped caller values
The tag union and spend_logs_metadata merge changed logged tags and
spend_logs_metadata on every route that has both metadata buckets, deduped
tags quadratically, and stopped background interaction polling from keeping
string tags as sent. Caller values now win exactly as before, and the
proxy's litellm_metadata tags or spend_logs_metadata only replace a caller
value of the wrong type.
The limiter reads agent ids through a pydantic model instead of a cast and
ignores non-string ids.
* fix(vector-stores): log the proxy's full tag list on litellm_metadata routes
On routes that keep proxy context in litellm_metadata, the proxy builds
litellm_metadata.tags from the caller's tags plus key, team and header
tags, but get_request_tags preferred the caller's metadata.tags. A caller
sending metadata.tags on a vector store call dropped the key's tags from
request_tags, which tag spend tracking reads. get_request_tags now prefers
a non-empty litellm_metadata tag list, and falls back to metadata.tags as
before.
* fix(vector-stores): strip LiteLLM-written keys at the provider boundary instead of moving metadata buckets
Moving vector store proxy context into litellm_metadata broke shared readers
(agent limits, request tags, merged logging metadata), and fixing them
widened the change to every litellm_metadata route. This reverts the bucket
move, the router change and the reader fixes, so proxy context stays in
metadata exactly as on main.
Vector store create and update now drop the keys LiteLLM itself writes
(user_api_key*, requester and deployment keys, agent ids) from the metadata
sent to the provider, in the shared optional-param helper both paths use.
The cleanup script reuses the same key list.
* fix(vector-stores): pass non-string metadata keys through the provider metadata filter
* fix(vector-stores): strip LiteLLM-written metadata keys only on proxy and router calls
* fix(vector-stores): send the provider exactly the metadata the caller sent through the proxy
* fix(vector-stores): snapshot caller metadata from the raw request body and keep it out of logged request bodies
* fix(vector-stores): copy the parsed-body metadata when the raw request body cannot be reparsed
(cherry picked from commit 4846cfabc7)
* fix(anthropic): treat adaptive thinking as enabled when mapping response_format
is_thinking_enabled only recognised thinking.type=enabled, so a request that
paired thinking.type=adaptive with response_format took the synthetic JSON
tool path with a forced tool_choice, which Anthropic rejects alongside
thinking. The same check also gates the Bedrock Converse and Databricks
forced tool_choice downgrades, so they now see adaptive thinking too.
* fix(anthropic): keep signed thinking blocks with empty text on chat completions
Models that default to display: omitted return signed thinking blocks whose
text is empty, and Anthropic rejects the next turn when they are not replayed
unchanged. is_unsignable_thinking_block dropped them along with unsigned
blocks, and the inline content path required non-empty text, so a tool-use
turn on Opus 4.7+ or the 5 families lost its thinking blocks and 400ed on the
tool-result turn. Only a missing signature now drops a block, and the
mid-conversation system placement mirrors that rule. The /v1/messages strip
helper is left to #41362.
* fix(anthropic): skip the thinking-strip retry when block_binding asks for an error
The /v1/messages retry loop strips thinking blocks and resends on any
invalid-thinking 400, including the binding mismatch a caller opted into
with thinking.block_binding.prefix_mismatch_behavior: error. The retry
decision now sees the request body and surfaces that 400 instead of
overriding the caller's choice. #42888 covers the litellm_params side.
* fix(bedrock): carry thinking.block_binding through the Invoke thinking rebuilds
Bedrock Invoke rebuilt the thinking param from scratch when display: updates
was requested, on both the /v1/messages and chat completions paths, and the
legacy enabled-to-adaptive translation plus the clear_thinking injection did
the same, so thinking.block_binding never reached Bedrock. Every rebuild now
spreads the caller's thinking and only overrides the keys it owns, and
AnthropicThinkingParam types block_binding. The thinking-binding-controls
beta was already on the Bedrock allowlist.
* fix(anthropic): keep forcing the response_format tool under adaptive thinking
This reverts ad9e861ae7. Anthropic, Bedrock Invoke and Vertex all accept a
forced tool_choice next to thinking.type=adaptive and only reject it with
type=enabled, so counting adaptive as enabled fixed no live failure. It did
drop the forced JSON tool, so Claude answered response_format requests in
prose, and it downgraded a required tool_choice to auto on Bedrock Converse
* test(e2e): drop the adaptive thinking tool round trip case
With adaptive thinking the model often skips thinking on the weather prompt, so
turn 1 carries no signed empty block and the case fails before the replay it is
meant to cover. Anthropic also answers the replay with a 200 whether or not the
block is forwarded, so the case could not tell the fix from main. The factory
unit tests pin the outgoing shape instead
* fix(anthropic): only skip the thinking-strip retry for a binding mismatch 400
prefix_mismatch_behavior: error asks Anthropic to reject a block bound to
another conversation. It says nothing about unrelated bad blocks such as a
null signature, so those still get the strip-and-retry self-heal. Also trims
the empty-thinking docstrings to what the code guarantees and types the new
test parameters
* refactor(anthropic): narrow thinking fields with the mapping validator instead of typing.cast
typing.cast is banned by the strict ruff gate (TID251), so the three new casts pushed that rule one over its merge-base count. The existing pydantic mapping validator gives the same typed view without an unchecked cast, and it now has a generic name since it is no longer only used for Claude Code payloads
* refactor(anthropic): read thinking settings through the validated mapping so basedpyright stays at its merge-base count
The Bedrock clear_thinking rewrite still narrowed with a cast and an untyped dict fallback, which added two reportUnknownVariableType errors. It now uses the same pydantic mapping validator, made public for that reuse, and the retry hook validates the request body itself so the handler passes a typed dict instead of an untyped one
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
(cherry picked from commit f470e418ea)
The request-side sanitizer on /v1/messages dropped every thinking block whose
text is empty, even with a valid signature. Claude Opus 4.8 / Fable 5 / Fable 5.1
return exactly that shape by default (thinking.display omitted): the signature
is the preserved reasoning. Replaying the assistant turn as returned therefore
lost the model's reasoning on every multi-turn request, silently (the provider
answers 200 without it), and left preserved-thinking enforcement
(thinking.block_binding) nothing to bind.
Anthropic accepts an empty thinking block that carries a signature and binds it
to its prefix; only an empty block with no signature (the LIT-6357 case, a turn
a non-Anthropic model served through the bridge) is rejected with "each
thinking block must contain thinking". Use the signature-aware predicate
is_empty_unsigned_thinking_block, which #38809 already applied on the emit
side, for the request side too. The unsigned case is still dropped.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 897bfea38d)
A JSON-RPC tools/call on /{server_name}/mcp (or /mcp with x-mcp-servers naming one server) that uses the upstream's bare tool name answered "Tool not found" on every proxy process that had not yet listed that server, because the per-process tool-name map stays empty for a server whose upstream only lists tools with the caller's own credentials, and mcp SDK 2 no longer lists before an uncached call. call_mcp_tool now hands the single scoped server to execute_mcp_tool as requested_server_id, so the existing REST contract lists the server with the caller's headers and routes the bare name to it, and a bare name shared by two servers goes to the one the path names
Co-authored-by: aayushbaluni <73417844+aayushbaluni@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit ec28acf322)
* fix(responses): fold bridged Responses output into one chat choice
A Chat Completions request bridged to /responses got one choice per
Responses message item, so a reply shaped [message, message,
function_call] put the tool call on choices[1] and clients reading
only choices[0] never saw it. Fold every message text, annotation,
reasoning item and tool call into choices[0]
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): drop mutable annotations and the extra cast in the choice fold
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): remove a cast-ok marker that suppressed nothing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): join folded message texts with a newline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(responses): join folded message texts with no separator so stream and non-stream match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4b7b8efd4c)
Takes the main version of each file, which is the rc copy plus one main commit:
- test_image_generation_e2e.py from #46998 (Nova Canvas reached end of life)
- test_bedrock_converse.py from #45530 (ask an uncached model once more)
- migratedPages.spec.ts from #45863 (/ui/ lands on Home since #45858)
* fix(caching): give RedisClusterCache one async client per event loop (#45784)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit cb73bad48a)
* fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit ca2f99edac)
---------
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): check previous_response_id ownership on /v1/responses/compact
* test(proxy): cover the unmanaged response id escape hatches on every response id route
* test(proxy): audit compact previous_response_id ownership
Add integration cells for the native /v1/responses/compact routes covering owner,
admin, cross-tenant, unmanaged, malformed, duplicated and retained ids, provider
errors, the response cache, id-security settings, an upstream outage, a held
upstream, a worker kill and a proxy restart. Unskip the provider wire cell whose
bug this branch fixes.
* test(proxy): cover the no signing key path on every response id route
Backport of #46408 to rc/1.106.0
Cherry-picked from 389403fede (main)
* feat(typesafe): support Jev 1.14 images, box questions, and long-context headers
* fix(cost): drop the typesafe/jev-1.14.0 entry main already prices
* fix(typesafe): send images only with the images header and box questions only to models that answer them
TypeSafe takes assets and input_image parts only when the request carries
TypeSafe-Danger-Zone-Images: true, so without it they follow drop_params like
any other provider that cannot take images. Box support now comes from a
supports_box_questions cost-map flag set on jev-1.14.0 and jev-preview, so
jev-1.13.0 and jev-latest refuse or drop a box question before the call. A
malformed provider_specific_header answers 400 instead of 500.
* fix(typesafe): allow null box confidence and validate asset references and base64
* fix(typesafe): gate box questions on the images header, drop unreadable assets under drop_params, forward opt-in headers on the pass-through
---------
Backport of #46596 to rc/1.106.0
Cherry-picked from 3fec5075fa (main)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cost): price typesafe/jev-1.14.0
TypeSafe released jev-1.14.0 on 2026-10-14 at the same price as jev-1.13.0
($0.042 per 1M input tokens, output free; https://docs.typesafe.ai/models).
Without an entry, calls through the TypeSafe pass-through log a cost of 0:
TypeSafePassthroughLoggingHandler prices from litellm.model_cost["typesafe/<model>"],
and a deployment's model_info price is registered under its model id only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cost): add max_input_tokens to the typesafe jev rows
---------
Backport of #46594 to rc/1.106.0
Cherry-picked from 9d72bae336 (main)
Co-authored-by: Mike Hyams <241361536+hyamie@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): send images to Cloudflare Clef
OpenAI-format input_image parts and the System One images field now reach Cloudflare clef and clef-flash in its images field, with remote URLs downloaded and inlined first. Providers that cannot take images drop them under drop_params and answer 400 otherwise.
* fix(decisions): follow drop_params for images and use the call timeout for image downloads
* fix(decisions): drop a malformed images entry on its own under drop_params
* fix(decisions): refuse images next to input whatever drop_params says
* fix(decisions): bound clef image fetches, honor drop_params for images, and log caller URLs
Refuse more than 4 images for Cloudflare clef before any download, keep the
caller's timeout as one deadline across image fetch retries, accept images
next to input under drop_params, validate image strings and objects, and pass
the caller's image URLs (not inlined bytes) to logging callbacks.
* fix(decisions): check input_image URLs and clef image sizes
* fix(decisions): accept data URL media type parameters and check clef image types
* fix(decisions): count clef images after dropping unsupported types and flag clef vision
---------
Backport of #45734 to rc/1.106.0
Cherry-picked from 2f3ccbe7c7 (main)
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(decisions): support Clef-omni audio and video
* fix(decisions): accept Clef-omni media types with parameters and cover the proxy routes
The clip patterns now mirror Cloudflare's input schema, so a data URL or
content_type carrying parameters like audio/webm;codecs=opus is accepted.
The four proxy routes get media tests of their own, the backup cost map
is synced with the new Clef-omni rows, and the docs page moves to the
docs repo as a follow-up.
* fix(decisions): refuse parameterized media-type data URLs and refresh the OpenAPI snapshot
Cloudflare answers 422 to a data URL whose media type carries parameters even though its schema only requires the data:audio/ prefix, so LiteLLM now refuses those with a 400 before the call. The object form keeps passing parameters through, which Cloudflare accepts. The lazy OpenAPI snapshot and the dashboard API types are regenerated for the new audio and videos fields
* fix(decisions): gate media by cost-map flags, treat null clips as none, drop unsupported clips before validation
* fix(decisions): keep the SDK's positional order and gate media flags on evaluation-mode models
* fix(decisions): gate media on both decisions model modes and move the audit cells onto the harness helpers
* test(decisions): cover the Fireworks System One router in the media refusal cells
---------
Backport of #45811 to rc/1.106.0
Cherry-picked from a86ad9ebf8 (main)
Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(decisions): rename the evaluation model mode to decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): canonicalize the health check mode without rebinding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): replace canonical_model_mode with is_decisions_model_mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): preserve evaluation mode compatibility
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: drop the rebuilt dashboard bundle from this PR
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(dashboard): format decision mode helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Backport of #45474 to rc/1.106.0
Cherry-picked from 4c532a5713 (main)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(prometheus): skip custom metadata labels a metric already carries
A custom metadata label whose sanitized name matches one of a metric's
built-in labels was appended a second time, so every .labels() call on
that metric raised ValueError. With custom_prometheus_metadata_labels:
[project_alias], the per-project per-model rate limit gauges raised on
every request from a project key with per-model limits, and the rest of
the success event (latency, deployment success, total requests) was
skipped.
* test(prometheus): cover shared custom labels on every endpoint, failure, metadata and outage burst
* test(prometheus): drain the shared upstream at the start of each integration test
* test(prometheus): assert the forwarded route, model and stream per endpoint cell
---------
Backport of #46267 to rc/1.106.0
Cherry-picked from b581ebc405 (main)
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
