mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-15 09:12:18 +00:00
939 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
maciej-berri
|
36edf9c4b9
|
feat(messages): serve azure ai foundry messages on rust only (#51721)
* feat(messages): resolve azure_ai keys and entra tokens like python on rust
The native azure_ai Messages config now authenticates in the order Python's
BaseAzureLLM._base_validate_azure_environment uses. The key comes from the
call, then litellm.api_key, litellm.azure_key, AZURE_OPENAI_API_KEY and
AZURE_API_KEY, each skipped only when empty, so a blank key is sent as given
as Python does. Without a key the call mints an Entra ID token through
litellm-auth-azure from azure_ad_token, AZURE_AD_TOKEN, tenant_id, client_id
and client_secret, the caller's azure_ad_token_provider, OIDC references and
litellm.enable_azure_ad_token_refresh, and sends it as a bearer. Without
either the request goes out unauthenticated and Azure's 401 is mapped as
Python maps it, attributed to azure_ai instead of anthropic.
The bridge reads litellm.api_key, litellm.azure_key and the token refresh
switch from the provider defaults snapshot, and calls the caller's token
provider on the blocking pool so its exception reaches the caller as Python
raises it.
Approved differences that stay: a caller x-api-key or non-blank bearer
replaces the operator key instead of being sent beside it.
Known differences: Python's azure_username/azure_password flow has no
native counterpart, and a token provider combined with an OIDC reference
uses the OIDC token where Python calls the provider.
* fix(messages): let a caller api-key replace the azure_ai operator key on rust
A caller api-key header on an azure_ai Messages call was forwarded next to
the operator's x-api-key, so Azure received both credentials. It now
replaces the operator key and is sent as x-api-key, as Python does for a
lowercase api-key.
Approved difference: Python only matches the lowercase spelling and sends
a mixed-case Api-Key beside the operator key. Rust matches any casing.
* fix(messages): order azure_ai entra id flows exactly as python on rust
Without a key the azure_ai config now walks Python's get_azure_ad_token
order itself instead of letting litellm-auth-azure pick. Every setting is
read from the call and then the secret lookup, as AZURE_API_KEY is.
A static token, the caller's azure_ad_token_provider and client secret run
natively. Username and password, OIDC references, os.environ/ references,
an uncallable provider and token refresh go to Python's own
get_azure_ad_token through the host, as Python does them, so a static token
with refresh on and a provider with an OIDC reference behave as in Python,
and a call Python leaves without a token goes out unauthenticated.
Every azure_ai call now asks the secret lookup for the Entra ID names up
front, where Python asks only when no key is set.
* fix(messages): read azure_ai entra settings where python does on rust
Python's get_azure_ad_token reads AZURE_AD_TOKEN through get_secret_str and
AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_USERNAME,
AZURE_PASSWORD and AZURE_SCOPE with os.getenv
(litellm/llms/azure/common_utils.py:580-588). The azure_ai config now does
the same, reading the process environment only when no key is set, and only
AZURE_AD_TOKEN stays in the secret names, so a value only a secret manager
holds is ignored where Python ignores it.
Approved by the owner, same class as A06-A08: any caller credential header
(x-api-key, api-key or Authorization other than a blank bearer, in any
casing) is the only credential sent, and the Entra ID path never runs for
it. That covers a caller bearer on an Entra-only deployment (A10), which
Python overwrites with the deployment's token, a lowercase authorization
Python joins with the Entra token, and a caller x-api-key Python sends
beside the Entra token.
* fix(messages): look up AZURE_AD_TOKEN only where python does on rust
Python reads AZURE_AD_TOKEN through get_secret_str only when the azure_ai
call has no key, no caller credential and no azure_ad_token param
(litellm/llms/azure/common_utils.py:580). Messages configs can now name
deferred secrets, resolved right before validate_environment and only when
the config names them for that call, so a keyed call or a call carrying its
own credential never asks the secret lookup for the Entra token.
Approved by the owner, same class as A06-A08: a caller Authorization other
than a bearer, such as Basic, replaces the azure_ai operator key instead of
being sent beside it as Python does.
* fix(messages): build azure_ai urls and resolve the base like python on rust
The native azure_ai Messages URL builder now ports Python's
azure_anthropic_messages_url. It cuts at /anthropic in the path only, so a
base whose userinfo starts with "anthropic" no longer sends the key to
http://anthropic/v1/messages (security regression fix). It keeps the base
untrimmed and moves the slash before a query as Python does.
The base resolves as Python's `api_base or AZURE_API_BASE`: a blank call base
is kept, an empty one falls back to the environment, and the environment
value is used as given. With neither, the call raises Python's bare
ValueError ("Missing Azure API Base ..."), flagged as raised by Python and as
a late rejection, so the failure log carries the resolved provider and model
parameters as Python's does.
* fix(messages): send bases the native parser misreads through python's client on rust
A Messages api_base with credentials, no host or a port Python cannot read
now goes through Python's client, as bases without an http(s) scheme or with
whitespace already did. Before, reqwest read such a URL differently from
httpx: for http:///d it sent the request, with the operator key, to host
`d`. This is a security fix for both azure_ai and anthropic; the
anthropic_base_empty_host parity case covers anthropic.
For azure_ai the choice reads the base Python's URL builder reads, falling
back to AZURE_API_BASE, so a padded environment base also goes through
Python's client. Python reads AZURE_API_BASE once: transport_handler returns
the value it read and the bridge seeds the call's secret source with it, so
the native route reuses it rather than reading the secret again. _sent_url
only rewrites anthropic URLs, since the azure_ai builder no longer trims.
Parity: 20 azure_ai and anthropic URL cases added; 65 aspects failed on the
previous extension, and the full messages parity suite now passes.
* fix(messages): look up AZURE_API_BASE only where python does on rust
Python reads AZURE_API_BASE only through the `or` fallback in
`api_base or get_secret_str("AZURE_API_BASE")`. The native azure_ai config
listed it among the secrets it always reads, so a call with its own base
still looked it up through the secret manager. It is now a deferred secret,
read only when the call's base is empty or absent. A blank base is kept
without a lookup, as in Python.
deferred_secret_names now takes the call's api_base, the same base the URL
builder reads. A lookup-counting test covers a call base (0 lookups), an
empty base (1) and no base (1); before this change the call-base case
looked it up once.
* fix(messages): filter azure_ai anthropic-beta through its row like python on rust
Python's handler runs update_headers_with_filtered_beta over the azure_ai row of
anthropic_beta_headers_config.json after the config adds its feature betas, so
unknown and null-mapped betas are dropped whatever the header casing, the
caller's and the auto-added ones alike. Rust forwarded the caller's betas and
every auto-added beta as sent.
A caller anthropic-beta replaces a deployment's extra_headers anthropic-beta
before either lane runs, as Python does, and a Router parity test pins it with
the filter applied to both. The advisor tool already matched, since the bridge
hands it to Python's interceptor and its beta is null in the row, so a parity
case pins it. extra_body, unknown top-level params and the -latest alias also
matched and get parity cases.
* fix(messages): strip beta-gated system fields under drop_params like python on rust
Python's transform runs strip_beta_gated_native_request with the final,
filtered headers, so under drop_params a mid-conversation system turn loses
the clear_at, output_config and tool_addition/tool_removal fields no sent beta
unlocks, and a turn left empty is dropped unless that would end the
conversation on an assistant turn. _normalize_system_role_messages strips a
turn it converts to a user turn again with no betas at all. Rust sent those
fields as they came.
azure_ai now picks its betas from the request as sent and runs the strip,
system-role normalization and the cache_control.scope strip in a new
finalize_request step, once the final header is known. Vertex gets the
conversion-time strip too, since it shares the normalization helper.
The strips and the speed drop log Python's warnings word for word. A
non-object output_format under drop_params is still popped after the headers
were built, so structured-outputs-2025-11-13 is sent without it, as Python
does.
* fix(messages): send the system prompt as text blocks like python on rust
Python's _normalize_system_role_messages always rebuilds the top-level system
as a list of content blocks, so a string system reaches azure_ai and Vertex as
one text block, and the success log, which only lists a string system as a
message, logs the messages as sent. Rust kept the string unless a leading
system turn was hoisted.
Python's billing strip in transform_anthropic_messages_request also pops a
system prompt it leaves empty, an empty string included, as Python does. The
shared helper now does the same for every config that strips billing
metadata: azure_ai, vertex_ai, deepseek, minimax, tencent and edenai, each
with its own test.
* fix(messages): keep non-claude azure_ai messages passthroughs on python
A deployment whose provider has no Messages config of its own, like a
non-Claude azure_ai model, but lists /v1/messages in
model_info.supported_endpoints is forwarded untranslated by Python's
OpenAILikeAnthropicMessagesConfig, with a bearer key to <base>/v1/messages.
Dispatch gave it the MESSAGES target, so a rule that serves azure_ai on Rust
would hand it to the native Anthropic config, which sends x-api-key to
/anthropic/v1/messages and breaks the success log.
Dispatch now targets MESSAGES only when the provider's own config serves
the call, and MESSAGES_PASSTHROUGH for the opt-in passthrough. The catalog
keeps that target on Python next to the chat and Responses bridges, so once
azure_ai is Rust-required these calls resolve to Python and the Rust-only
refusal does not fire for them. Proxy token counting treats both targets as
before.
The parity forced rule now serves a provider natively only for the MESSAGES
target, as a Rust-required rule below the target exclusions does. A pre-request
hook that sets a non-Claude model raises functools.partial's TypeError on both
lanes, as on main, and a parity case pins it.
* fix(messages): raise azure_ai transport and provider errors like python on rust
The bridge mapped a native failure with the whole call as exception_type's
extra_kwargs, so litellm_debug_info gained an API Base line from the named
api_base. Python's entry passes only its **kwargs there, so its debug info
names the model alone. The bridge now passes the same kwargs, for anthropic
and azure_ai alike, and a decoding failure on a stream carries the
deployment's debug info as Python's stream iterator attaches it.
A connect failure to an azure_ai base read from AZURE_API_BASE kept the
native "could not reach the provider" text, since the call had no api_base.
The host now hands Python's failure mapping the secrets it read to send the
call, so the aiohttp and httpx texts name the base the call went to, without
reading AZURE_API_BASE again.
A key or header value with a control character, like a deployment key with
a trailing newline, failed natively as APIConnectionError with the model
stripped of its provider. It is now the InternalServerError 500 Python's
transport raises, with aiohttp's "Forbidden control character detected in
headers" text and the model as called. On the httpx transport Python's text
quotes the header value, which may be the key, so Rust keeps its own text
there.
The parity outcome now compares litellm_debug_info. The caller client's
ReadTimeout from a stalled stream still has none on Rust and is recorded as
a gap.
* fix(messages): send a caller azure_ai api-key without an operator key like python on rust
With no deployment key, Rust renamed a caller api-key header in any casing
to x-api-key. Python's AzureAnthropicMessagesConfig renames only a header
spelled exactly api-key and sends any other casing, like API-Key, as given.
Rust now does the same when no operator key would be sent. With an operator
key the approved behaviour stays, and a caller api-key in any casing replaces
it as x-api-key.
* fix(auth-azure): retry entra token posts like python's azure-identity
A client secret or workload identity token endpoint that keeps answering
500 made the native path POST 9 times over about 80 s, because
azure_identity 1.0 uses the SDK's default exponential retry (8 retries,
200 ms doubling, 60 s cap). Python's azure-identity 1.25.2 posts 4 times
in about 5 s and then raises, so the Rust credential now carries a custom
retry policy with the same numbers.
Python sources in the wt-auth venv
- azure/identity/_internal/pipeline.py:67 builds the token pipeline with
azure-core's RetryPolicy(**kwargs) and no overrides
- azure/identity/_internal/msal_client.py:98 (client secret) and
aad_client.py:77 (client assertion, used by workload identity) run the
token POST with retry_on_methods=["POST"]
- azure/core/pipeline/policies/_retry.py (azure-core 1.41.0) lines 77-95
give retry_status=3, retry_backoff_factor=0.8, retry_backoff_max=120,
retried codes 408, 429, 500, 502, 503, 504 and 506-998. Lines 127-143
sleep 0 before the first retry and factor * 2^(n-1) after that, so
0 s, 1.6 s, 3.2 s. Lines 455-509 sleep exactly Retry-After (then
retry-after-ms, x-ms-retry-after-ms) instead of the backoff when it is
present and non-zero
azure_core only hands the policy a Retry-After value for 429 and 503, so
a Retry-After on another status still falls back to the backoff. Managed
identity keeps the SDK default since Python uses a separate policy there,
and client assertion keeps it because Python mints OIDC tokens through
litellm's own HTTP client. A 401 invalid_client is still one POST.
* fix(auth-azure): post an oidc token exchange once like python
Python exchanges an OIDC assertion through litellm's own HTTPHandler
(_exchange_oidc_token in litellm/llms/azure/common_utils.py), a single
client.post with no retry, so a 503 is one POST. The native client
assertion credential used azure_identity's default retry (9 POSTs over
about 80 s) and now runs with RetryOptions::none().
* fix(auth-azure): honour retry-after on entra token posts like python
azure_core runs a custom RetryPolicy through its own loop, which only
reads Retry-After for 429 and 503 and never retries a status outside the
policy's list. Python's azure-core RetryPolicy (1.41.0,
azure/core/pipeline/policies/_retry.py) retries any 4xx or 5xx that
carries Retry-After (is_retry, lines 210-242) and waits exactly the
Retry-After, retry-after-ms or x-ms-retry-after-ms value when it is
non-zero (get_retry_after in _utils.py:79-96, sleep at lines 455-509).
The client secret and workload identity credentials now run with the
SDK retry turned off and a per-call policy that follows Python's
loop. It keeps separate budgets of 3 for status, connect and read
retries (increment and is_exhausted, lines 265-330), backs off 0 s,
then 0.8 * 2^(n-1) capped at 120 s, and retries 408, 429, 500, 502,
503, 504 and 506-998 for the token POST. Client assertion still posts
once and managed identity keeps the SDK default.
A Retry-After that is neither a number nor an HTTP date now fails the
token request after the first POST with "Invalid HTTP date", as Python
does. azure-core's _parse_http_date (_utils.py:48-57) raises ValueError
from the sleep before the retry, azure-identity's wrap_exceptions
(decorators.py:63-80) turns it into ClientAuthenticationError
"Authentication failed: Invalid HTTP date", and get_azure_ad_token
(common_utils.py:663) raises RuntimeError "Failed to get Azure AD token:
Authentication failed: Invalid HTTP date".
* fix(messages): raise python's entra token errors for native azure_ai flows
On the Messages route every Entra token failure now reaches the caller as
Python's get_azure_ad_token and anthropic_messages raise it, with the same
class, status_code, llm_provider, message and log lines.
A native client secret flow keeps the token endpoint's final answer
(status, headers, body) in a typed EntraTokenFailure from the azure-identity
retry policy. The bridge hands it to the Messages host, which replays it
through azure-identity's own ClientSecretCredential over a transport that
returns that answer, so MSAL and azure-core read it exactly as they read it
off the wire. That covers a rejected secret (EntraRejectionError 401 with
the first AADSTS sentence and GUIDs redacted), an exhausted 5xx ("HTTP
Error: 500"), a malformed Retry-After ("Invalid HTTP date"), non-JSON and
odd JSON bodies, including the content-type dependent decode error. A
transport failure has no answer to replay and keeps Rust's own reason
after "Authentication failed:", so its text differs from requests' text.
The caller's azure_ad_token_provider goes through the helper
get_azure_ad_token now shares (azure_ad_token_failure), so a listed AADSTS
rejection is an AuthenticationError 401 instead of a RuntimeError, and a
non-string token raises the same TypeError. Both log Python's "Error
calling Azure AD token provider" and "returned non-string value" lines.
Every Python exception raised while getting the call's credential,
including the host-minted username/password, OIDC and workload identity
flows, now goes through Python's handler rule: a BaseLLMException is mapped
by exception_type (AuthenticationError 401, BadRequestError 422 with
llm_provider azure_ai), anything else reaches the caller as raised.
The Retry-After date parser stays RFC 7231 only. Python's
email.utils.parsedate_tz also accepts dates without a weekday, two-digit
years, numeric zones and missing seconds, which Rust rejects as an invalid
HTTP date and Python waits on. A NaN or infinite number of seconds now
falls back to the backoff, where Python sleeps 0 for NaN and fails on
infinity.
* fix(auth-azure): wait on retry-after and report connect failures like python
The Entra token retry policy now reads Retry-After the way azure-core
1.41.0 does. Seconds go through Python's float rules, including
underscores between digits, and anything else through a port of
email.utils.parsedate_tz, so a date without a weekday, a two-digit year,
a numeric or named zone, a zone glued to the time, missing seconds, a
dotted time, month before day and RFC 850 dates all wait as Python waits.
A date datetime() rejects (day out of range, an offset of a day or more)
still fails the request after one POST, as do unparseable values.
A wait that time.sleep cannot take (infinity, or 2^63 ns and past) now
fails after one POST as Python does. The host replays that answer with a
sleep that still calls time.sleep for such a wait, so the caller gets
"Authentication failed: timestamp out of range for platform time_t".
A transport failure keeps the token endpoint's host and port. A direct
connect that fails with an OS error, or a failed name lookup glibc words,
reaches Python as urllib3 2 words it after "Authentication failed:", for
example "HTTPSConnection(host='login.microsoftonline.com', port=443):
Failed to establish a new connection: [Errno 111] Connection refused".
Python hits it on MSAL's discovery GET to the same host. Failures through
a proxy, TLS failures, timeouts and resets while reading keep Rust's own
text, since urllib3's wording for them depends on details Rust does not
expose.
* fix(messages): treat a contentless system message as a system turn like python on rust
Python's _normalize_system_role_messages reads only the role, so a role
system message sent without content or with null content is still a system
turn. A leading one is hoisted and adds nothing to the top-level system, and
a later one on a model without mid-conversation system support becomes the
operator note user turn, placed after a tool result it sits before and
dropped under drop_params when its gated field is stripped. Rust failed to
parse the message and sent it raw, which also showed up in the logged
messages. A model that accepts the role still gets it as sent.
* fix(messages): let the provider filter a non-ascii anthropic-beta like python on rust
Python merges every anthropic-beta spelling into one header and the
provider config filters it before the HTTP client encodes it, so on azure_ai
an unknown non-ASCII beta is dropped and the call goes out, with or without
drop_params. Rust held the header back as unencodable before the beta filter
ran and failed the call with the ascii codec error. The beta header now
reaches the provider filter, and the wire check fails the call only when a
value Python's client cannot encode survives it. A value with a lone
surrogate still fails up front since it cannot reach Rust as text.
* fix(messages): log nothing for a stream the caller never read like python on rust
Python's Messages stream iterator only starts on the first read, so a
stream the caller closes or drops unread bills nothing and logs nothing.
Rust billed the empty stream on close and teardown, which logged "Unable to
build complete streaming response" and the read() error from the success
handler.
* fix(messages): log a traceback for failures python raises before the send on rust
Python raises its pre-flight errors, such as the missing azure_ai api_base
ValueError, the missing key AuthenticationError and the Entra token
provider failures, through Python frames, so the failure log and the OTel
span carry a stack trace. Rust built those exceptions without raising them,
so error_information.traceback was empty. The host now raises them through
a Python frame before handing them back.
* fix(messages): return python's streaming response types from native messages streams
Python's Messages handler returns an AnthropicMessagesStreamingResponse for
a stream, so isinstance checks, hidden_params and aclose behave the same for
every caller. The native async entrypoint returned the bare execution
AsyncStream. It now wraps the stream the way the deployment success hook
already did, sharing the stream's hidden params. The entrypoint is shared, so
this moves anthropic as well as azure_ai toward Python.
With a response cache configured Python hands back its caching handler's
types too. A streamed cache hit is now the CachedAnthropicMessagesStreamIterator
Python's handler builds from the cached events, stamped with the metadata
@client stamps, and the native execution closes unread so only that iterator
logs the hit, as Python does. A miss is wrapped in the
AnthropicMessagesStreamCacheWriter Python returns, marked persisted since Rust
stores the stream itself. The parity suite now compares the stream type for
every streamed case.
* fix(messages): read unparsed messages in beta detection and gated strips like python on rust
Python reads beta triggers and gated fields off any message mapping, while
Rust only looked at messages that parsed. A role system message sent without
content, or any message with a content block Rust cannot type, was skipped:
- a message carrying output_config now adds per-turn-control-2026-07-01
- a system message with a tool_reference addition or removal now adds
mid-conversation-tool-changes-2026-07-01
- a signed compaction block now adds compact-2026-09-04
- under drop_params an unparsed system message loses its gated clear_at,
output_config and tool changes, is dropped once emptied, and the
warning names the strip
- an unparsed assistant message counts as the last assistant turn when
deciding whether a strip would leave a prefill
- an unparsed message loses cache_control.scope on its content blocks
* fix(messages): rewrite and normalize unparsed messages and system prompts like python on rust
Python's sanitizer, advisor and encrypted-reasoning strips and system
normalization read any message mapping, while Rust skipped every message
or system block it could not type:
- an unparsed message now gets the sanitizer's rules: a text block whose
text is not a non-blank string and an empty unsigned thinking block go,
a message left with no blocks is dropped, tool ids are sanitized,
replayed web search results are flattened and provider_specific_fields
dropped; a message the rewrite leaves parseable is parsed again
- an unparsed assistant turn loses advisor exchanges, and an unparsed
message loses bridge-tagged reasoning blocks
- a kept redacted_thinking block gets Claude's own data back from a
Bedrock Converse base64 wrapper, parsed or not
- system is kept as sent when it is a list holding a block that does not
parse, or an object; billing blocks and cache_control.scope are still
stripped from such a list
- a leading system turn whose content does not parse is hoisted as sent,
and a later one becomes the operator note user turn as sent, placed
after a tool result turn that did not parse either
* fix(messages): keep anthropic's API Base debug line on rust failures
Mapping a native failure with only the Python entry's **kwargs dropped the
named api_base from exception_type's input. That matches azure_ai, whose
Python entry maps with **kwargs alone, but it removed the API Base line that
main's Rust-only anthropic route has always put in litellm_debug_info when the
caller passes api_base.
Python gives no rule that tells the two apart, because the legacy async entry
maps both providers the same way. The difference is that anthropic has no
Python path, so main's native route is its contract. The bridge now maps the
whole call for anthropic and the entry kwargs for every other provider.
The parity harness still uses the legacy Python entry as anthropic's
reference, so it now expects the API Base line main's anthropic route adds.
New cases cover an anthropic call on an azure_ai model failing with api_base,
unary and stream.
* fix(messages): keep anthropic stream types and header errors as main while azure_ai matches python
By the owner's ruling anthropic keeps main's behaviour for two things the azure_ai port had changed for every
native provider.
A native async stream returns Python's AnthropicMessagesStreamingResponse, and with a response cache the
CachedAnthropicMessagesStreamIterator or AnthropicMessagesStreamCacheWriter, only for azure_ai. The host asks
Python for those types only when the call resolves to azure_ai, so anthropic gets main's execution AsyncStream
again, for a cache hit or miss too.
A header value the native client cannot send, like a key with a control character, is Python's
InternalServerError with aiohttp's "Forbidden control character" text only for azure_ai. Anthropic raises
main's APIConnectionError "invalid request: failed to parse header value" with the bug report link again.
The parity stream_type aspect compares azure_ai with Python and expects main's AsyncStream for anthropic,
in the same way the outcome aspect already expects main's API Base debug line. New azure_ai cache hit and
miss stream cases check the cached types.
* fix(messages): keep anthropic as main for azure_ai parity side effects
By the owner's ruling anthropic, which has no Python path, keeps main's native
route; the azure_ai port's Python-parity changes now apply to azure_ai only.
- An unread stream: only azure_ai's starts on the first read and bills
nothing when closed or dropped unread. The logging adapter learns the
provider from the execution facts; anthropic bills it as main does.
- Failures before the send: the missing-key AuthenticationError and the
unmapped builtin errors are raised through a Python frame only for azure_ai;
anthropic hands them back unraised, without a traceback, as main does.
- A stream's decoding failure carries the deployment's debug info only for
azure_ai; anthropic raises it bare.
- A non-ASCII anthropic-beta is left to the provider filter only for
azure_ai. Anthropic holds it back before the beta merge as main does, so
the error names it ahead of any later unencodable header.
- Unparsed messages: the shared shaping reads messages and system prompts
Rust could not type only when the provider asks for it
(UnparsedMessages::Read), which only the azure_ai config does. Elsewhere
the sanitizer, advisor and encrypted-reasoning strips leave such messages
as sent, their beta triggers add no beta, a redacted_thinking Converse
wrapper is kept as sent, and a system prompt that does not parse is
main's "untagged enum SystemPrompt" request error from messages_body.
Each item has an anthropic test pinned to main's behaviour that fails on the
previous commit, beside the azure_ai cases comparing with Python.
* test(messages): pin the malformed api_base fix for anthropic and azure_ai
|
||
|
tavren27
|
8fc6789295
|
chore: move LiteLLM's configs and e2e tests off gpt-4o and gpt-4.1 before their 2027-01-14 shutdown (#51800)
* chore: move LiteLLM's configs, cookbooks and e2e tests off gpt-4o and gpt-4.1 before their 2027-01-14 shutdown OpenAI shuts down gpt-4o, gpt-4o-mini, gpt-4.1 and gpt-4.1-mini on 2027-01-14. The example proxy configs, cookbook guides and e2e suites now call gpt-5.6, gpt-5.6-luna or gpt-5.4-mini, and the predicted-outputs e2e test is removed because no current OpenAI model accepts prediction * docs(cookbook): name gpt-5.6 instead of gpt-4o as a tool-capable model in the gollem troubleshooting tips * chore(ci): delete the observatory proxy config, which nothing has loaded since its workflow was removed * chore: send max_completion_tokens in batch lines and move README and deploy examples to gpt-5.6 gpt-5.6-luna rejects max_tokens on OpenAI's batch lines, which reach OpenAI unchanged, so the batch JSONL builders send max_completion_tokens. The README quick start, the adaptive router README and the Helm and Terraform example configs move off gpt-4o with the configs * chore: pin output prices next to input prices in the adaptive router example The adaptive router reads only input_cost_per_token from config, and a deployment priced by input alone bills output at $0, so the example and its README pin both at the cost map's current prices * chore: move the configs and e2e tests to gpt-6-luna and gpt-6.1-sol The cost map names gpt-6-luna as the successor of gpt-4o, gpt-4o-mini, gpt-4.1 and gpt-4.1-mini. Cheap roles move to gpt-6-luna and full-size and vision roles to gpt-6.1-sol, the pair main's other configs use. The adaptive router example pins the gpt-6 prices, and the installer quick start and two READMEs drop gpt-4o --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
paveln83
|
05518975a5
|
test(e2e): move audio, realtime, and Langfuse tests off model ids with shutdown dates (#51816)
* test(e2e): move audio and realtime tests off OpenAI model ids with shutdown dates * test(e2e): move langfuse completion, image, and vertex realtime ids off dated models * test(e2e): register the weather tool in the first realtime session update and read Langfuse v2 flattened attributes * test(e2e): expect the OCR upload's file summary in the Langfuse generation input The OCR upload route has sent the uploaded file, not a data URL, since the native OCR route, and the generation input summarizes that form as the mime type and file name. The broken Langfuse lookup hid the stale data URL expectation * test(e2e): move the image generation test off gpt-image-1-mini gpt-image-1-mini has a 2026-12-01 deprecation_date, so the image generation e2e would start failing that day. gpt-image-2.5-flare has no date and already serves the Langfuse image tests * test(e2e): use gpt-image-2.6-flare and check every n=2 choice reaches Langfuse OpenAI names gpt-image-2.6-flare as the successor of gpt-image-1-mini, so the image tests use it. The n=2 completions test now needs one logged message per choice, since both choices repeat the same text and a single logged message used to pass --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
yujonglee
|
2be96aed13
|
refactor(redis): delete the Python RedisBatch now that every request op is native (#51383)
The request scope keeps only the native batches. RedisBatch, its ops, the cluster fallback, the Python post-call deadline and the DualCache post-call helpers had no production caller left, and the pre-call SET, DEL and miss tracking that UserApiKeyCache never reached go with them. The mixed-families pipeline metadata had no producer left |
||
|
jesus-berri
|
f0021859a7
|
feat(ui): set per-threshold budget alert recipients on virtual keys (#51061)
* feat(ui): set per-threshold budget alert recipients on virtual keys Create Key and Edit Key get the threshold and recipients rows the team member budget alerts already use, writing metadata.max_budget_alert_emails. The helpers and the row editor move to key_team_helpers so the team form and both key forms share them. * test(ui): cover the empty metadata box path and drop a redundant payload case * fix(ui): keep comma-separated string alert recipients stored through the API The backend accepts a string of recipients per threshold, so Edit Key and the team form now load them as rows instead of dropping them and deleting the alert on the next save * fix(ui): keep the stored key alert map on saves that do not edit its rows Edit Key rebuilt max_budget_alert_emails from the rows on every save, so API-written entries the rows cannot show (a null recipient list, a list with a null, thresholds outside 1 to 100) were deleted by unrelated edits. The map is now rebuilt only when the rows were edited and otherwise sent back as stored * fix(ui): load null and mixed alert recipient lists as rows and drop a typed map on Create Key A null recipient list loads as an owner-only row and a mixed list keeps its strings, matching what the backend sends. Create Key now ignores a max_budget_alert_emails typed into the Metadata box, as Edit Key does, and the hint says thresholds replace the default 80% alert * fix(ui): keep Create Key alert rows when the Metadata box holds null * fix(ui): move typed alert maps into rows, keep hidden alert entries, hide alert rows from team admins A max_budget_alert_emails map typed into the Metadata box on Edit Key or Create Key now moves into the threshold rows on blur with a toast, the same way a tags array moves to the Tags field. Entries the rows cannot show stay in the JSON box. When the rows are edited, the saved map keeps the stored entries the rows cannot show (thresholds outside 1 to 100, non-numeric keys, unreadable values). Create Key keeps such typed entries too. Team admins editing a member's key no longer see the alert rows, since that save only accepts budget fields * fix(ui): let the Metadata box own key alert entries the threshold rows cannot show Edit Key now shows max_budget_alert_emails entries outside whole numbers 1 to 100 in the Metadata JSON box and saves them from the box as the admin left it, so they can be seen, edited and removed. Rows still own every entry they can show, and an untouched save sends the stored map unchanged * fix(ui): leave zero-padded alert thresholds to the Metadata box instead of duplicating a row * fix(ui): keep loading zero-padded team member alert thresholds as rows The canonical-key rule applies only to key alert maps, where the Metadata box holds entries the rows cannot show. The team form has no such fallback, so it keeps loading "080" as row 80 as it did before * fix(ui): let the Metadata box own a non-object max_budget_alert_emails value A string, list, number or null stored under max_budget_alert_emails stays in the Edit Key Metadata box, is saved back as the box holds it, and is stored as typed on Create Key. Threshold rows only own an object map, so rows the admin adds replace a non-object value * fix(ui): move typed alert thresholds into the rows when Edit Key saves without a blur Submit now moves a max_budget_alert_emails map typed into the Metadata box into the rows, as it already does for tags, and reads the edited fields at submit time so the moved rows are saved * refactor(ui): name the Edit Key submit handler |
||
|
Yassin Kortam
|
db6dfb6bb6
|
chore: move workspace packages under litellm (#51000)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
f6bee81c73
|
chore: move ui and vscode-extension under litellm and delete docs (#50992)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
6efbaf539c
|
chore: move deployment sources under litellm/deploy (#50981)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
b0e389503f
|
chore: move config and docs off the repo root (#50966)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
a6ce922f07
|
ci: remove CircleCI (#50950) | ||
|
yuneng-jiang
|
587902becf
|
test(e2e): remove the secret manager suite (#51241)
The secret manager lanes ran the proxy against a Vault dev container and a Conjur container that we start ourselves, so nothing in them needed a real third-party service. They have not run a single test since late October, because the co-located proxy cannot boot in the lane pod, and they were soft_fail outside the promotion gate, so they only added noise. The AWS test in the folder was skipped on every lane, since no aws lane ever existed. The four coverage registry rows go with the tests, because the registry is the e2e denominator and this behavior belongs in the integration tier if it gets covered again. The Vault and CyberArk clients keep their unit tests. |
||
|
yuneng-jiang
|
63d2f4fef7
|
refactor(ui): remove the deprecated API Reference page (#51246)
The API Reference page was already behind a deprecation banner. This deletes the /ui/api-reference route and its components, the Developer Tools sidebar entry and page description, the legacy ?page=api_ref redirects, and the i18n strings only that page used. Old ?page=api_ref and ?page=api-reference bookmarks now fall through to the dashboard like any other unknown page id, since there is no replacement page. Tests that listed the page are updated: the sidebar and legacy-route vitest files, the uiHref examples, the e2e sidebar fixtures and Internal Viewer nav check, and the UI path detection unit test. |
||
|
devin-ai-integration[bot]
|
9a4a587e75
|
feat(docker)!: one shipped image and one entrypoint beside the Dockerfile (#43332)
Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
84b0a4297a
|
fix(ui): preserve credential actions across dashboard renders (#50845)
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yuneng-jiang
|
07be67603c
|
test: update the spend tests that still expected the pre-ledger accounting queues (#50814)
* test: update the spend tests that still expected the pre-ledger accounting queues #49872 made the SpendLogs ledger the default accounting path on purpose. A batch with no billable request now writes no cost row, a call the guardrail fails closed during a vendor outage keeps one spend row of either status while its key is still charged once, and charges left at a graceful shutdown are applied by the next start. The daily spend batch shutdown tests exercise the old queue, so they pin use_spend_logs_ledger to false. * test(integration): price the guardrail relay model so the outage test can assert the charge The relay model had no price, so every stream cost $0 and the outage test could not tell a charged call from an uncharged one. Both rigs now price it, and the test expects each served stream to cost 9 input and 4 output tokens and the key to be charged for all sixteen. Also notes the skipped batch cost row case in the e2e rules file, next to the other explicit skips. * test: repair four tests that fail for reasons outside the product The OpenAPI docs gate test rebuilds the Rust extension with uv, and the integration lane blocks outbound traffic, so uv could not refresh its PyPI index for maturin. The test now runs uv offline against the cache the dependency install already filled. The response cache test expected a chat reply blocked with empty content to be cached. Since #50057 a reply with no output is never cached, so that case now expects the repeat to reach the upstream and the real answer to be cached. The Gemini cache hit e2e test gave the model 16 tokens, which its thinking can use up, and a reply with no text is not cached. It now allows 512 and checks the first reply has text. The Together cache read e2e test picks the cheapest model with a cache read price. Since #50491 that is a -latest alias, whose target reports no cached tokens. Aliases are now left out of the pick. |
||
|
yuneng-jiang
|
1d06a1ca69
|
test(e2e): repin the Together reasoning replay backend to MiniMax-M3.1-Flash (#50601)
MiniMax-M3 reached its 2026-11-06 deprecation date in the cost map, which names MiniMax-M3.1-Flash as its successor |
||
|
ryan-crabbe-berri
|
093caa52fa
|
test(e2e): add a Codex row where a sub-agent must read the task its parent hands it (#50494) | ||
|
HootRock
|
8f19607259
|
fix(proxy)!: preserve Responses SSE event framing (#47980)
* fix(proxy): preserve Responses SSE event framing
* style(proxy): format Responses stream terminator condition
* fix(proxy): validate Responses guardrail event names
* fix(proxy): keep Responses first-event errors and keepalive failures in Responses framing
* test(proxy): use controlled clocks for Responses keepalive failures
* fix(proxy): end late Responses failures with an error event the SDK raises on
Errors that arrive after keepalive pings, and guardrail error strings yielded
mid-stream, now end /v1/responses with a named `error` event carrying the
same error object chat sends, so the OpenAI SDK still raises APIError with a
clean message and provider_specific_fields. Background polling passes the
same framing and records an `error` event as a failed response. The first
data line of a frame is found by splitting on newlines only, so a U+2028 in
an error message no longer hides a first-event error.
* fix(proxy): match OpenAI's Responses error event and stream late passthrough blocks as block responses
* fix(proxy): keep the reason when a background Responses request fails
A background poll of a request a post-call guardrail blocked saved a
null code and message, because the guardrail's error event nests them.
A request whose first chunk was an error polled as completed with no
error. Both now poll failed with the same code and message a late
failure gets.
* test(proxy): type the first-chunk poll test parameters
* fix(proxy): keep first-chunk error detection to lone error frames and handle string poll errors
* fix(proxy): keep the HTTP status on late Responses errors and end Responses streams on error events
A late error body without a code now takes the response's status, an upstream error event counts as the stream's terminal event, and a Responses stream that fails before its first chunk ends with an error event instead of data: [DONE].
* test(proxy): type the first-chunk failure test's parameters
* fix(proxy): keep cursor chat framing and JSON-encode Responses hook errors
* fix(proxy): encode non-dict Mapping Responses chunks as JSON objects
* fix(proxy): encode mappings and models nested in Responses stream chunks as JSON objects
* test(proxy): freeze the hook-output fixture
* fix(proxy): keep masked mappings and odd keys safe in Responses stream frames
Only read-only dict views and pydantic models nested in a hook chunk or error event expand into JSON objects. Other mappings, such as httpx.Headers, keep their own string form so masked values stay masked. Keys JSON cannot take become strings, and a chunk JSON still cannot encode goes out as the plain data frame it was before.
* fix(proxy): never fail a Responses stream frame on a value JSON cannot encode
Only dicts and read-only dict views expand into JSON, at the top level too, so a header mapping yielded as a whole chunk keeps its masked string form. Any failure while encoding a chunk falls back to the plain data frame the base sent, and an error event that cannot encode still goes out with its code and message, so a late block never drops the connection.
* fix(proxy): keep what a nested value's repr hides out of Responses stream frames
* fix(proxy): print values in Responses stream frames the way each base path did
* fix(proxy): send string and number subclasses whose repr shows their value as that value in Responses stream frames
* fix(proxy): print container subclasses and SDK models in Responses stream frames the way base did
* fix(proxy): read hook container storage the way repr does and send error-event models as str()
* fix(proxy): frame streamed Responses reads and send unreadable hook values as base did
Track pre-framed and raw SSE frames so a late failure never follows a terminal event, accept CRLF and CR line breaks in a first named error, frame GET /v1/responses/{id}?stream=true as Responses events, print a whole hook proxy chunk once and fall back to its text, ignore a non-integer status_code in a hook error frame, read a late-block dict subclass through items() when that view differs from its storage, and keep the order and inline form base's repr printed for nested mappings
* fix(proxy): match base on Responses stream edge cases
Read a decimal-string or integral-float status_code as its integer again, so
"429" and 429.0 still map to rate_limit_exceeded. Print each hook value's repr
once and reuse it, so a repr that changes between calls cannot reach the
client. Track only pre-framed sequence numbers from 0 to 2**63-1, so a huge
one cannot break the failure event. Build nested JSON with one frame per level
again, which restores the nesting depth sent as JSON. Count a pre-framed frame
as terminal from its data type, and a named error frame only when its data
carries an error, the way the OpenAI SDKs read them.
* fix(proxy): print hook proxies once and keep base's chunk tracking
Read-only mappings in hook chunks now emit the text from their first print, hook
chunks are read without their error field, only a non-empty error object makes a
framed error event terminal, and framed sequence numbers stay below 2**63-1 while
hook chunks keep base's unbounded tracking
* fix(proxy): print each read-only mapping item once and poll framed error events as failed
A MappingProxyType in a hook chunk or error detail now expands from the
mapping it wraps, so each item is printed once, the way base printed it.
A value whose repr changes between calls stays masked, a value whose repr
works once is kept, and deep nesting keeps one stack frame per level
The background reader takes each frame's type from the same terminal rule
the stream uses, so an event: error frame with an error object polls
failed even when its data names another type or none
* fix(proxy): print self-holding hook values once and poll how the stream ended
A container that holds itself now goes out as the marker its repr prints, like {...}, [...] or mappingproxy({...}), and the dict behind a read-only mapping joins the cycle check, so no item is printed a second time. A dict whose item repr adds a key keeps going after the items already shown, the way dict's repr does, instead of printing the whole dict again
A whole read-only mapping chunk that cannot go out as JSON is printed once by the formatter, so a print that fails fails the stream as it did on the base
The background poll reads the event type from the block it parses, as on the base, and keeps the first error frame the stream treats as terminal anywhere in a chunk. That frame fails the poll only when no response.completed, response.failed or response.incomplete follows it
* fix(proxy): name hook chunk events from the sent JSON and keep polls on frames they cannot read
A mapping chunk now takes its event: line from the JSON it sends, so a masked type value never reaches that line, and a chunk the event model cannot read goes out as a plain data frame instead of failing the stream
A dict whose item repr deletes or swaps keys keeps printing every key not yet shown, and an error code past the integer digit limit reads as no status
The background poll skips a frame whose shape it cannot read, as base skipped frames it could not parse, stops reading a chunk at [DONE], and lets only the last terminal frame in a chunk decide its status
A Responses stream's first chunk answers as a JSON error only for codes 400 and up
* fix(proxy): read every frame of a polled chunk and name model chunks from the sent JSON
A background poll parsed only the first data frame of each chunk, so a terminal event behind an error or delta frame in the same chunk was lost, and the poll stored a status its own stream never ended with. It now reads every frame up to [DONE]
A terminal event whose response is a string or null kept no status and polled completed. Its status now comes from the event type, and final output items that are not objects are skipped, so the rest of the final output is still stored
Model chunks now take their event line from the JSON they send, as mapping chunks already do, so a serializer that drops type no longer sends event: response.completed over {}
* fix(proxy): read polled chunks as SSE events and keep draining past a mid-chunk [DONE]
A background poll split a chunk into data lines, so one event with two data lines parsed as two payloads, and a frame with an integer past the digit limit raised a ValueError the parse guard did not catch, which failed the poll and dropped its spend row. The poll now joins each event's data lines the way an SSE reader does and skips any frame it cannot parse
A [DONE] behind another frame in one chunk ended the poll and left the stream unbilled. Only a chunk that opens with [DONE] ends the poll now, as on main, and the poll stops reading a chunk at its first terminal event, so two terminal events in one chunk poll the first one as main did
* fix(proxy): JSON-encode only plain hook chunks on Responses streams
A hook chunk whose whole tree is builtin JSON types streams as its named
event. Any other chunk keeps the base data frame, so nothing a custom
repr hides reaches the wire. Error event details encode the way the chat
error frame does.
* fix(proxy): skip containers already visited when checking hook chunks for JSON
* fix(responses): keep streaming past an event with no type
The Responses stream iterator read event.type directly when it noted a
yielded event, so an event whose JSON had no type, like a custom hook's
dict that the proxy now sends as JSON, raised AttributeError and ended
the stream. Read it with getattr, as the iterator's other checks do.
* fix(proxy): keep hook dicts with no type on the base Responses frame
A hook dict with no type is not a Responses event, so it gets no event
name either way. Sending it as JSON only made litellm SDKs that read
type on every event stop the stream. Keep such dicts on the frame the
base sends, so every SDK version reads them as before.
* fix(responses): keep a background poll's first terminal status and take a first-chunk error code from its HTTP status
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
yuneng-jiang
|
e555a79cee
|
test(e2e): follow the Aria sidebar move and the Skills Hub rename in UI specs (#50453)
#50009 moved Aria out of the dashboard header into the sidebar with its own page, and #50195 renamed the Skills nav link and AI Hub tab to Skills Hub and relabeled the skills search box. The UI e2e specs still looked for the old header button, dialog and labels, so the scheduled litellm-e2e run has been red since build 815. |
||
|
devin-ai-integration[bot]
|
848c267b76
|
test(e2e): wait for saved MCP credentials before deleting the server (#50366)
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
ryan-crabbe-berri
|
6fa0477d84
|
test(e2e): add a Codex row that calls a local MCP tool on a first and a resumed turn (#49817)
* test(e2e): add a Codex row that calls a local MCP tool on a first and a resumed turn * test(e2e): assert the codex mcp row got each value back from the mcp tool call itself * test(e2e): check the codex mcp row exits cleanly on both turns * test(e2e): drop the unused content type from the codex mcp result model |
||
|
tin-berri
|
2ffbdf0d7a
|
feat(ui)!: redesign public AI Hub as a catalog (#50153) | ||
|
Meryem Sakin
|
4a7fb30b0a
|
fix(guardrails)!: check Anthropic and Responses request tools in tool_permission pre_call (#41011)
* fix(guardrails): check Anthropic-format request tools in tool_permission pre_call
The pre_call hook only collected tools shaped as {"type": "function", "function": {...}},
so flat tools from /v1/messages ({"name", "input_schema"}) and /v1/responses
({"type": "function", "name"}) were never checked and deny rules silently passed.
Reuse the shared anthropic_tool_names helper so every request tool name is checked,
and handle Anthropic tool_choice {"type": "tool"} in both block and rewrite modes.
Fixes #40583
* fix(guardrails): detect Responses API named tool_choice in tool_permission
Responses API named choices put the name at the top level ({"type": "function", "name": ...}),
so rewrite mode removed the denied tool but left a tool_choice pointing at it.
* test(guardrails): run tool_permission pre_call tests through real default_on gating
Build the guardrail as an always-on pre_call guardrail instead of patching should_run_guardrail.
* test(guardrails): type the new tool_permission pre_call test parameters
* fix(guardrails): keep the Responses API custom tool type distinct
Anthropic's user-defined tools carry type "custom", so pre_call mapped
that to "function" for tool_type rules. The Responses API has its own
custom tool type, and mapping it let a custom tool slip past a
tool_type deny rule when the default action is allow.
Normalize only for the formats where custom means a plain function, and
read tool names directly in the rewrite path, which never needed a type.
* test(guardrails): cover flat tools through Messages and Responses
Add blocked and permitted flat-tool cases through the running proxy, with
endpoint coverage and metadata for the current e2e harness
Require a guardrail policy verdict when checking denied tools so an
unrelated provider error cannot satisfy the regression test
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
* fix(guardrails): check namespace and nameless tools and relax tool_choice in tool permission rewrite
* fix(guardrails): check forced, grouped, loaded, and hosted tools by their real type
* fix(guardrails): drop references to removed tools and count loaded tools in tool permission rewrite
* fix(guardrails): check tools sent outside top-level tools in tool permission
* fix(guardrails): judge namespace members by their bare name as a fallback and keep rewrites valid for Gemini, Bedrock and hosted tools
A namespace member is checked by namespace__name first, and by its bare name when no rule names the joined one. Rewrite mode now prunes Gemini allowedFunctionNames and turns off forced calling when no function is left, drops a removed hosted tool from allowed_tools, reads Bedrock toolSpec and systemTool blocks by name, skips cachePoint markers, and leaves a tools field that is not a list for the provider to reject instead of failing with a 500
* fix(guardrails): let only name rules decide a namespace member's bare-name fallback
* fix(guardrails): judge malformed tool types, empty Gemini name lists and pass-through Anthropic tools like the endpoints do
A tool or tool_choice whose type is not a string now reads as untyped instead of crashing the guardrail with a 500, and a null type key is never read as a Gemini tool part. An empty allowedFunctionNames list no longer turns off the functions a rewrite kept. An Anthropic custom tool reads as a function by its input_schema on every route, so a pass-through route judges it the same as /v1/messages. A Responses WebSocket frame now forwards the extra_body tools a rewrite stripped
* fix(guardrails): judge a named hosted tool by its type and keep a nested chat custom tool custom
* fix(guardrails): keep judging Anthropic client tools and type-named server tools by their name
* fix(guardrails): judge a named tool by its name alone only when the name is its type's own
* fix(guardrails): judge the function a tool names beside a declaration list or a top-level name
* fix(guardrails): judge Gemini hosted keys whatever the tool type and a namespace's function
* fix(guardrails): judge legacy functions entries like tools and send Gemini a namespace's members
* fix(guardrails): judge a namespace member by every name a provider reads and send Gemini its schema converted
* fix(guardrails): judge a legacy functions entry as a function and relax a tool_choice in extra_body
* fix(gemini): keep a namespace whole when inlining its members would repeat a function name
* fix(guardrails): judge a namespace member by every bare name that shares its qualified name
* fix(vertex_ai): declare a repeated namespace member name once instead of sending namespace names to Gemini
* fix(guardrails): judge a tool_choice sent in extra_body like a top-level one
A forced name only extra_body.tool_choice carried was never checked, so block mode let a denied name through to the provider
* fix(guardrails): judge a Gemini namespace member returned by its bare name and force a named namespace's members
* test: keep formatter churn out of the tool_permission and Gemini test files
* fix(guardrails): judge extra_body.input tools, argument-aware alias rules, and exact forced Gemini functions
Tools loaded in extra_body.input are now checked and stripped like top-level input. A returned
namespace member is judged by the alias whose first name rule its arguments satisfy. A forced
Gemini name that matches a top-level function keeps that function. Alias grouping and Gemini
declaration dedupe now run in one pass instead of quadratic scans.
* fix(guardrails): judge and rewrite every tool field sent inside extra_body
extra_body is merged into the upstream body, so functions, function_call,
mcp_servers and toolConfig nested there reached the model unjudged. Treat
extra_body as a second request layer that is collected and rewritten the
same way as the top-level request.
* fix(guardrails): judge Gemini hosted keys by key and count only the tools a provider receives after extra_body
* fix(guardrails): relax a tool_choice that names a dropped namespace member by its bare name
Gemini and Anthropic chat send namespace members under their bare names, so rewrite mode now treats a dropped member's bare name, and an emptied namespace's own name, as removed when no kept tool still carries it
* fix(gemini,guardrails): keep a top-level function over a same-named namespace member and leave malformed tool values to the provider
Gemini and Vertex chat now declare a top-level function over a namespace member that shares its name, so a forced call gets the top-level schema. post_call judges a returned name a top-level tool carries as that tool, and pre_call no longer raises on an unhashable or non-string tool name, input item type, or function_call name.
* fix(guardrails): judge a returned namespace member named like a hosted MCP server as that member
* fix(gemini,guardrails): send a namespace as main did when another tool has a member's name
Gemini chat inlines a namespace's members only when no other tool, namespace
member, or legacy functions entry declares the same name, and otherwise sends the
namespace by its own name as main did. A tool_choice naming such a namespace
forces the namespace itself. The tool permission post_call reads top-level names
the way Gemini declares them and judges a call the Responses bridge tags with a
namespace as that member.
* fix(gemini): send colliding namespace members as namespace__name functions and judge them in post_call
When a namespace member shares a name with another tool, Gemini chat now gets that namespace's members as namespace__name functions instead of one function named after the namespace, so every function it can call is a name the rules judged. Declared names are counted once, and a forced member maps to each holder's qualified name.
post_call judges a returned namespace__name as that member, by its qualified or bare name the way pre_call does, so a bare-name allowlist or denylist applies to it.
* fix(gemini): keep generated member names unique and let a top-level function own its exact name
A namespace that listed a member twice, or a generated namespace__name that
repeated a top-level function, sent Gemini two declarations with one name, and
Gemini answered 400 where main answered 200. A name a namespace lists twice is
now one member, and a top-level function keeps its exact name.
post_call now judges a returned name a top-level function declares as that
function, never as the namespace member the name could also mean.
* fix(gemini): qualify namespace members by the namespace's top-level name
The guardrail judges a member as `name__member` from the namespace's top-level
name. Gemini built the name from `function.name` when the tool also carried one,
so a request could pass pre-call and reach Gemini under a name no rule judged.
* fix(vertex_ai): force only the namespace members Gemini is sent
A forced namespace or member on Gemini chat could still allow a generated name that a top-level function or an earlier namespace owns, so tool_choice forced an unrelated function. The forced names now come from the same collision checks the declarations use
* fix(vertex_ai): reject a forced namespace Gemini gets no member of, and resolve forced names in one pass
* fix(guardrails): judge a namespace member by a rule on its namespace's name
---------
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
Kerry Lu
|
0d70b9b9a1
|
test(perf): remove the database_upgrade_5rps e2e case and its harness tests (#50076)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
yuneng-jiang
|
b18188dee2
|
test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984)
* test(e2e): retire the quiet_stack lock quiet_stack held an exclusive host-wide lock for six tests, draining every other xdist worker while they ran. On the full e2e lane that left 35-38% of worker time idle. Each of those tests now lives where it can run without pausing the suite - the two memory tests move to perf/ behind the perf marker, so the perf lane runs them on its own fresh stack. Their refusing deployments use a literal key because the perf stack carries no provider keys - the stored-request size check from the memory test becomes an integration test on an owned proxy that stores prompts - the discount and margin spend-row tests are dropped from e2e. Their breakdown assertions join the existing integration pricing test, which already owns the discount and margin repricing flow - the Bedrock guardrail cost test stays in e2e without the pricing reset fixture, since nothing else on the stack writes pricing config now - the MCP semantic tool filter test turns on a proxy-wide setting, so it gets a new proxy_wide_config opt-in marker and runs after the main run * test(e2e): register the claude_code deployments as one batch The claude_code session fixture registered its 24 deployments one at a time on every xdist worker, and each one waited out the 15 s propagation window on its own, about 6 minutes per worker. ProxyClient.register_models posts the whole batch, polls each model until every replica serves it, then waits out propagation once from the last write. A model that fails to post or never becomes servable comes back as ModelNotRegistered, and an unservable one is deleted, so the fixture keeps reporting per-deployment failures the way it did |
||
|
yuneng-jiang
|
cbe361d970
|
feat(ui): rebuild the public AI Hub as a searchable list with a details pane (#49496)
* feat(ui): rebuild the public AI Hub as a searchable list with a details pane The public AI Hub at /ui/model_hub_table (and the older /ui/model_hub) now renders PublicAiHub, built from shadcn parts only: a header with the gateway's base URL and the admin's links, segmented tabs for models, agents, MCP servers and skills, and a list beside a details pane on desktop that becomes a bottom sheet on phones. The open item and tab live in the URL. It keeps every feature the old page had: server-side search, provider, mode and feature filters, sorting, paging, health, rate limits, prices, custom metadata, supported params, usage snippets, the require-auth redirect and the not-enabled state. cURL shows only for chat models, since the old one called chat completions for every mode. The navbar hides the community buttons below the sm breakpoint on every page, and the public navbar uses a smaller logo on phones, so Docs and Blog no longer collide with them. * fix(ui): harden the public AI Hub after adversarial review - Show "This AI Hub isn't available" only when every hub call fails; a failed models call now shows an inline error with Try again and leaves agents, MCP servers and skills usable (a gateway with no models answers the models list with 400) - Row links keep the page's other query params, so open-in-new-tab and copy link no longer 404 - A shared ?open= link resolves models beyond the first page, keeps the opened item when a search filters it out, and says when it is no longer listed instead of showing a different item - Searching, sorting and paging no longer blank the details pane or drop the Operational badge - Key MCP servers by their listed name, which is never null - Keep sub-cent prices and show Free for zero cost, fold evaluation into Decisions, sort agents, MCP servers and skills by name, restore word-wise search, and show a skill's source, author and date added - Drop the invented description for models without a mode, the empty agent modes row, the stray auth dash on MCP rows and NaN dates - Do not retry the UI config call, and close the 1023-1024px gap between the sheet and the details pane - Update the public hub Playwright spec for the new tabs and list, and prune the lint suppressions the old route pages needed * fix(ui): surface hub load errors and keep the open item steady Agents, MCP servers and skills now load through apiClient so an HTTP error shows the inline error instead of an empty "published yet" state, and an all-failing gateway reaches the unavailable page. Try again shows loading instead of an empty hub, the open model stays in the details pane when a search or page change hides its row, a zero-result search keeps the pane, phones get the not-listed sheet for a stale link, an empty open param means nothing opened, free models read Free in the list, and an invalid skill date is hidden |
||
|
yuneng-jiang
|
bb37a11b3b
|
feat(ui): rebuild the command palette on shadcn Command (#49528)
Replace the hand-rolled combobox in the Cmd/Ctrl+K palette with shadcn's base-vega Command and CommandDialog (cmdk) and bind the shortcut with react-hotkeys-hook. cmdk now owns arrow keys, wraparound, Home/End and IME composition, Base UI keeps the dialog mounted for its closing animation and returns focus to the trigger, and mod+k is ignored while focus is inside another dialog Results keep their fixed Keys, Actions, Pages order with shouldFilter disabled. The highlight resets to the first row whenever debounced key results load, so Enter after typing still opens the top key cmdk 1.1.1 leaves aria-activedescendant empty or stale after it picks a row on its own (dip/cmdk#413), so a MutationObserver keeps it on the row that is actually aria-selected until upstream fixes it |
||
|
yuneng-jiang
|
e1c03c5423
|
fix(e2e): restore main qualification across providers and UI (#49763)
* test: check an unsupported budget sort field
* test: wait for dashboard navigation readiness
* test: support clustered provider caches with live fallback
* Revert "test: check an unsupported budget sort field"
This reverts commit
|
||
|
Emerson Gomes
|
ecb8ef5ca8
|
feat(realtime)!: support latest OpenAI audio models (#35600)
* feat(realtime): support latest OpenAI audio models * fix(transcription): type SDK stream duration metadata explicitly * fix(realtime): preserve transport settings and translation duration * refactor(realtime): reuse SDK client initialization * fix(audio): bill finalized streams and authorize transcription models * refactor(audio): keep transcription pricing selection together * fix(realtime): reconcile model metadata and test contracts * fix(realtime): preserve provider duration and model capabilities * fix(realtime): keep transcription validation compatible with cost map guard * fix(realtime): address transcription and billing review findings * fix(realtime): preserve input usage and provider routing * fix(realtime): bill retained output and route Azure aliases * fix(realtime): reject malformed provider durations * fix(realtime): preserve batch cache rates and Azure v1 routes * fix(transcription): reject nonfinite provider durations * refactor(realtime): align annotations with current lint rules * fix(realtime): preserve translation deployment pricing and input usage * test(transcription): verify synchronous stream finalization * fix(realtime): retain missing translation input usage at close * fix(transcription): reject streams that bypass output guardrails * fix(realtime): reconcile audio protocols billing and current CI * fix(realtime): trust confirmed formats and authorize default models * fix(realtime): preserve absent models and reject unsupported guardrails * fix(realtime): reject translation policy pipelines * chore(ci): rerun transient infrastructure timing failure * fix(realtime): reconcile audio protocols with current upstream * fix(audio): validate provider responses at SDK boundaries * fix(realtime): preserve minimum OpenAI SDK compatibility * fix(realtime): preserve deferred Live session authorization * fix(realtime): preserve upstream authentication and transcription behavior * fix(audio): scope provider duration precedence to configured models * fix(typing): avoid Final declarations inside transcription cost loop * refactor(audio): reuse transcription mapping and duration resolution * fix(realtime): enforce model budgets and preserve translation routing * fix(audio): log streamed transcripts whose done event reports detected languages * fix(audio): pass stream, languages and keywords through unchecked for non-OpenAI transcription providers Providers without a streaming transcription handler got UnsupportedParamsError for stream=True, languages and keywords, where the merge base forwarded them as provider-specific params. Only openai and azure validate them now. * fix(audio): bill streamed gpt-transcribe from the provider's reported duration The stream path computed cost after logging had replaced the transcription's duration usage, so it fell back to the measured file length while the unstreamed call billed the provider's seconds. * fix(transcription): log streamed transcription usage from the provider response The stream logging path built the spend payload from the response after its duration usage was transformed away, so whisper-1 with stream=true and streamed gpt-transcribe logged an empty usage object. Bill and log from the provider's TranscriptionResponse in both the sync and async paths. * fix(realtime): keep main's routing, protocol and transport behavior around the new translation routes - Send OpenAI realtime and translation client_secrets and calls through litellm's httpx client again, so ssl_verify and custom CA settings apply and upstream errors keep main's shape - Use the input transcription model as the session model only for transcription sessions - Drop the new /openai/v1 translation aliases and record each lazy feature's own routes, so the /openai pass-through and the MCP routes resolve as on main - Let a configured Azure realtime_protocol win for every model, with GA as the default only when none is set - Return x-litellm headers on streamed transcriptions and mark the request done - Refuse translation sessions before connecting upstream when realtime guardrails apply or the provider is not OpenAI or Azure - Accept json and text for gpt-transcribe, and drop other formats under drop_params instead of failing - Bill translation sessions on output audio seconds only, at the single published rate - Document the 201 SDP answer on the translation calls routes and the transcription 200 response * fix(realtime): restore main behavior for guardrails, Azure transcription, and realtime clients Streamed transcription runs unstreamed when a deployment post_call guardrail applies, so the guardrail still sees the transcript. Azure transcription maps v1, latest and preview to the default dated version on the deployments route for every model. The SDK client secret keeps session.model ahead of model, realtime WebSockets connect to api_base again, realtime metadata keeps only guardrails, and managed agents need an explicit transcription model. Reverts the two unrelated streamed Responses error codes and the duplicate cached_tokens_details field, and the gpt-transcribe format check only reads the request provider's cost map entry. * fix(realtime): keep main's transcription, client secret and route behavior where the PR changed it Transcription reaches every provider as on main again, sends stream only when asked, forwards language and languages together, keeps form key order, bills provider seconds only when above zero, honors base_model for the new flags, keeps main's Azure api_version on the proxy path, and streams carry chat's headers, an error event on a mid-stream failure, and a spend row when the stream ends early. Translation cost reads both per-second rates from one source and spend rows carry the billed seconds. Realtime transcription WebSockets keep model in the upstream URL outside OpenAI's hosts, client secrets forward explicit nulls, accept conversation and delay, and answer 400 on a malformed body, the /openai translation paths are back to main, translation refuses only for guardrails realtime sessions run, nested aliases resolve and later session.update models are authorized, translation client secrets run per-model auth, beta clients bridged to GA get beta-shaped session events, and transcription sessions send the deployment model in both shapes. * fix(realtime): address review findings on transcription billing, streaming and realtime auth Per-second pricing replaces token billing only when the priced entry sets no token rates, translation rates fall back per key to the cost map, transcription cache keys include languages and keywords, a stream request a post_call guardrail forces unstreamed comes back as redacted SSE, Azure beta clients are bridged only for GA-only deployments, nested transcription models on the realtime WebSocket are checked against the key, and translation spend carries key and team metadata * fix(azure): pick the GA realtime protocol for deployments whose model_info.base_model is GA-only * fix(realtime): preserve prior merge resolutions after rebase * fix(realtime): enforce translation policies and normalize Azure transcription URLs * fix(cookbook): stop translation once output idles after the upload ends * fix(realtime): bill translation on input audio and drop pre-read stream cost headers Translation spend now uses the input audio seconds the session sent, priced through input_cost_per_second, instead of the translated output duration. A streamed transcription no longer returns cost headers before the stream is read, since the cost is only known after the last event. Also removes an unused realtime guardrail helper. * fix(realtime): drop top-level session nulls, bill dropped transcription streams, and bridge beta response fields Client secret, translation and transcription session bodies drop top-level nulls again, as main did, while nested nulls such as turn_detection still pass through. Transcription client secrets send the deployment model as session.model. A streamed transcription the client drops still runs its success callbacks, and the stream ends with data: [DONE] like OpenAI's. The beta to GA bridge maps max_response_output_tokens, speed and noise reduction, remaps response.create, and drops temperature under drop_params. The Entra token for realtime HTTP routes is fetched off the event loop * test(e2e): cover realtime translation, live transcription, client secrets, and streamed transcription * fix(realtime): close review gaps in translation billing, transcription streams, and Azure GA Bill translation sessions from the deployment's own per-second rates, add per-second prices for Azure gpt-4o transcribe models, keep stream-scoped guardrails seeing the caller's stream flag, map mid-stream transcription errors to LiteLLM exceptions with a [DONE] frame, map GA responses back to the beta shape for bridged beta clients, honor the minting key's guardrail opt-out on translation calls, restore the Azure transcription session URL, and flag GA-only Azure realtime models * fix(realtime): bill a translation rate the deployment omits from the session model * test(e2e): check translated speech is not English instead of exact French wording * fix(realtime): keep main's behavior for non-bridged backends, stream error frames, gateway URLs and Azure transcribe prices Remap beta events to GA only for bridged beta clients and translation sessions, bridge beta clients on GA-pinned Azure deployments, log error-only upstream sessions as failures, flag dated Azure GA realtime models, drop the unpublished Azure per-second transcribe rates, log a mid-stream transcription failure once, keep main's SSE error frame outside transcription, drop unsupported transcription params under drop_params, keep the caller's Azure auth headers, keep a gateway /v1 path, let key opt-outs release string-callback guardrails on translation, and keep the cookbook waiting while a long upload is still sending * fix(realtime): keep the proxy's realtime session gate off translation sessions A key that opts out of a legacy callback guardrail now gets a working translation session, since the transcription-guardrail gate no longer sends a realtime session.update the translation upstream rejects * fix(realtime): keep websocket model limits, bill measured transcription length, route transcription sessions on the allowed model A WebSocket opened without a query model now authorizes as an empty model, transcription cost prefers the measured upload length over the provider's reported seconds, transcription sessions overwrite nested models with the authorized top-level model again, and a non-boolean transcription stream value answers 400 or is dropped under drop_params. * fix(ci): copy the rust bridge extension into the openapi docs gate's base worktree * fix(model-catalog): accept the realtime translation endpoints in the cost map * fix(cookbook): close the translation session and read until session.closed * fix(realtime): treat an empty transcription stream as unset and pass the cost-map guard An empty `stream` form value runs the transcription unstreamed again, as OpenAI does, instead of answering 400. The four translate cost-map entries drop the translation routes from supported_endpoints until the main branch's catalog parser accepts them, and the nested transcription model lookup uses the router registry entry point. * test(proxy): drop the mid-stream error frame pin that main's error shape change replaced * fix(transcription): bill uploaded seconds when a stream fails mid-transcript * fix(proxy): bill uploaded seconds when a streamed transcription breaks mid-stream * fix(audio): keep transcription's positional arguments and redact streamed error details - Move the new languages, keywords and stream options to the end of transcription's parameters, so a prompt or format passed by position keeps its meaning - Redact internal hostnames, paths and tracebacks from a streamed transcription's mid-stream error event, as chat streams already do - Move the CA-bundle check for the realtime HTTP routes, which serves real TLS on loopback, from the unit suite to the SDK integration suite * fix(audio): stream the guarded transcript to SDK callers that asked for a stream * fix(logging): keep a streamed transcript out of callbacks that turned off message logging A streamed transcription's assembled response reached success callbacks with its text intact under turn_off_message_logging or a callback's message_logging=False, because the shared streaming redactor had no transcription branch. It now redacts the text, as the unstreamed call already does * fix(proxy): stream a transcription the deployment streamed when the client sent no stream field --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
ryan-crabbe-berri
|
50af27eaa6
|
test(e2e): add Codex file edit, long session and auto-compaction rows through LiteLLM (#49386)
* test(e2e): add the Codex file edit and long session rows The file edit row has Codex change a random value into config.txt with the patch tool and checks the file on disk. The long session row has it follow a chain of random file names one cat at a time. new_workspace now returns the resolved path, because Codex reports patched files by their real path (/private/var on macOS). * test(e2e): make the Codex long session prove every earlier command output reached the model * test(e2e): Codex auto-compaction through LiteLLM (#49389) * test(e2e): add the Codex auto-compaction row Codex reads a random value, compacts through the gateway with its remote compaction forced on, and is asked for the value after the file holding it is deleted. The row reads the compaction from the thread's rollout file and marks Claude on Bedrock as a strict expected failure, since the Responses-to-chat bridge ignores the compaction_trigger item * refactor(e2e): validate Codex rollout lines with the models directly |
||
|
ryan-crabbe-berri
|
6254860560
|
test(e2e): drive the Codex CLI through LiteLLM, command round trip (#49374)
* test(e2e): drive the real Codex CLI through the gateway and add the command round trip row Adds tests/e2e/codex: a harness that runs codex exec --json with a throwaway HOME and CODEX_HOME, a custom responses provider pointed at the gateway, the model list read from the gateway's /v1/models, the sandbox on and approvals off. Each test registers its setup's deployment via /model/new and calls it with a key scoped to it, both deleted on teardown. The first row checks that a value only a command can print comes back through the gateway on all five setups * test(e2e): type the Codex self-test event helper as a Mapping * test(e2e): clean up Codex test keys even when setup fails, and fail runs where Codex exits with an error * test(e2e): record a step when the Codex harness sets up its home and workspace The suite rules ask every public harness helper that does something to record a step, so a failed run shows whether it died while setting up Codex or while running it * Update run.py wip |
||
|
simonp9
|
250af8f027
|
fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493)
* fix(responses): log the encrypted response id on streamed /v1/responses spend rows
With response id security on, the stream hook encrypts the resp_ id on every
frame, but logging had already received the SDK's own copy of the terminal
response, so the spend row, the standard logging payload and callbacks got the
managed id (or the chatcmpl id on a bridged stream) the client never saw. The
hook now records the exact id it advertises on the request's logging object,
and the logging layer stamps that id on the logged streamed response
* fix(responses): return the id the client sent on GET and cancel, and keep read rows off the create row's id
Every GET /v1/responses/{id} re-encrypted the response id, so a client that continued a bridged conversation with the
id a GET returned referenced a ciphertext no spend row was filed under. The follow-up lost turn 1 and opened a new
session. GET and cancel now answer with the exact id the client sent when it is an id this proxy issued and it decodes
to the response the provider returned. A create with previous_response_id still gets a fresh id for its own response,
and a raw provider id still gets a managed id
Because a read now carries the create's id, a GET or cancel spend row is filed as <id>_retrieve_<call id> or
<id>_cancel_<call id>. That keeps one row per read, as before, and a read row can never take the create row's
request_id, so the create row survives whichever row is written first and a follow-up replays turn 1 once. The
background settlement row keeps <id>_background_cost, and the cost poll's own status checks keep the response id
* fix(responses): log a fresh id for each read and cancel, echo the held id only to its owner
A GET or cancel of a stored response now logs a new encrypted id of its own,
so callbacks and spend rows never reuse the create's id, while the owner of
the id still gets back the exact id they sent. An admin reading someone
else's response gets an id bound to the admin, as before. The spend row id
suffix for reads and cancels is gone, so with response id security off the
rows and session history match the default branch. A streamed GET follows
the same rules.
* fix(responses): echo the sent id to every reader that passes the response access check
A GET, streamed GET or cancel of a response now answers with the id the caller sent whenever the caller passed
the access check, not only when the caller is the user or team the id was issued to. An admin, the master key
or a teammate who read someone else's streamed response got a fresh id back, and the session lookup could not
reach the first turn from it, so a bridged follow-up through that id lost the conversation history
Every read and cancel still logs a fresh id bound to the reader, never the create's id
* fix(responses): keep a read or cancel row in session history only when it brings new output
A GET, streamed GET or cancel sent with a session header joins the
create's session, and its stored response repeats the create's output.
The replay then sent the same assistant turn once per read.
Session history now keeps an aget_responses or acancel_responses row
only when its stored Responses output has an item id that no earlier
kept row carried. A polled background response keeps the first poll
that returns the finished output, since its queued create row has none,
and later polls of the same output are dropped. Every other row keeps
today's rule
* fix(responses): replay each history item once, echo the sent id on delete, keep background cost files apart
A create row now owns its output items in session history, and a read or cancel row adds only items no
create and no earlier read carried, in one linear pass, so a background response polled while it grows,
or a read that ends before its streamed create, replays the answer once and after its prompt
DELETE /v1/responses/{id} answers with the id the caller sent, on a copy, and logs a fresh encrypted id
like a read does
Azure Data Lake Storage per-event files and unbatched GCS objects are named after the event's spend row
id, so a streamed background create and its cost event, which now share the client's id, keep separate
files
The integration tests that decoded the managed id now match the exact id the client received
* fix(responses): keep main's answer for a raw-id delete and main's storage names outside background cost events
A DELETE by a raw provider id returns the provider's id as main did, and logging still records a fresh
encrypted id. A streamed GET matches the held id by upstream response id, like a plain GET
Only a background response's cost event gets the _background_cost suffix on its Azure Data Lake Storage
file and unbatched GCS object, so batch retrieves and events with no id keep main's names
An item a session read replays takes the newest read's copy at the first read's place, so an item a
poll caught half written replays finished
Tests pin the client's id over the router's continued response id, and the storage test no longer reads
the clock after the upload
* fix(spend): give background cost events their own OpenMeter and DynamoDB ids, replay id-less read items
A streamed background create and its later cost event share the client's
id, so OpenMeter dropped the second event and DynamoDB overwrote the
create's item. Both now key the cost event as <id>_background_cost, like
its spend row and storage file.
Session history again replays a read's output items that carry no id,
as main did.
* fix(integrations): keep OpenMeter and DynamoDB logging working without the proxy extras
* fix(integrations): give background cost events their own ClickHouse request id and DeepEval span id
A streamed background create and its later cost event now share the id the
client received, so ClickHouse and DeepEval saw both events under one id.
Route both through logged_event_key, the way the Azure storage, GCS,
OpenMeter and DynamoDB loggers already do, so the cost event's ClickHouse
request_id matches its spend row's `<id>_background_cost` and its DeepEval
span gets its own uuid. ClickHouse response_id stays the raw provider id.
* test(deepeval): annotate the background cost span test's locals with Final
* test(responses): run the DELETE wire test that pins the echoed client id
DELETE /v1/responses/{id} now answers with the id the caller sent, so the
skip that marked it as a known bug no longer applies. The test passes on
the /v1, /openai/v1 and bare routes.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
ryan-crabbe-berri
|
3af23024bc
|
test(e2e): share the coding-CLI harness between suites (#49370)
* test(e2e): share the CLI version rule, env lockdown and rate limiter in tests/e2e/cli_harness Moves the npm release rule (now taking the package name), the env allowlist plus fresh HOME, and the per-upstream rate limiter out of claude_code so the Codex suite can reuse them. Claude Code behavior is unchanged, its CI, cron and Dockerfile paths now call the shared resolver with @anthropic-ai/claude-code * test(e2e): freeze the packument map and note the rate limiter self-test * test(e2e): race real processes on one rate limiter bucket so a missing lock fails Eight spawned processes start together and drain a bucket that never refills. With the flock they get exactly its burst between them, without it every attempt is granted * test(e2e): keep the old Claude Code version resolver path for the Buildkite runner images The runner Dockerfiles still run tests/e2e/claude_code/pr_gate_version_resolver.py and silently install @latest when it is missing, which drops the three day review window. The old path now forwards to cli_harness/npm_version_resolver.py |
||
|
Mateo Wang
|
b4a80d175c
|
ci: remove workflows whose secrets are dead (#49722)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
tavren27
|
1046716b6d
|
fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409)
* fix(anthropic): sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept * test(e2e): send the azure_ai beta check without the code execution tool * test(anthropic): require a vertex_ai and azure_ai decision for every anthropic beta * fix(anthropic): forward the betas Claude Code sends that anthropic, vertex_ai, and azure_ai accept * test(e2e): send Vertex's header-only betas in the beta acceptance case * fix(anthropic): keep forwarding Vertex computer-use betas and point beta tests at platforms that still lack them A live probe found each Claude 4.5 and 4.6 model on Vertex needs one of the two computer-use betas for the computer tool, so dropping both broke it. The e2e acceptance test leaves those two out of its all-at-once Vertex call, and the integration and unit tests that relied on a beta being dropped now use one the platform still lacks. * test(anthropic): point unsupported-beta tests at betas the platforms still drop The Azure AI handler test used claude-code-20250219 as the beta Azure drops, and the Rust parse test used it as an unknown value. This PR forwards it on Azure AI and maps it, so both now use values that stay dropped or unknown. * fix(anthropic): add computer-use-2025-11-24 for the computer_20251124 tool The computer_20251124 tool fell through to the default computer-use-2024-10-22 beta, which the provider rejects for that tool version. Map it to computer-use-2025-11-24, which the beta map now forwards on anthropic and azure_ai * fix(anthropic): send no computer use beta for an unknown computer tool version An unknown computer tool version fell back to computer-use-2024-10-22, which the anthropic and azure_ai columns now forward. It now gets no beta, as every platform effectively sent before. Every computer tool version also skips defer_loading and allowed_callers, computer_20251124 included * test(azure_ai): read the branch's beta map in the auto-injected beta wire test * fix(anthropic): keep defer_loading and allowed_callers on the computer_20251124 chat tool * test(anthropic): read the branch's beta map in the azure_ai safeguards beta wire test --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
yuneng-jiang
|
60fc121ce8
|
test(a2a): replace the third-party agent e2e test with an integration test (#49541)
Some checks failed
unit / auth-checks (push) Blocked by required conditions
unit / budgets (push) Blocked by required conditions
unit / custom-logging (push) Blocked by required conditions
unit / db-and-spend (push) Blocked by required conditions
unit / endpoints-and-responses (push) Blocked by required conditions
unit / enterprise-routing (push) Blocked by required conditions
unit / guardrails-hooks (push) Blocked by required conditions
unit / jwt-and-keys (push) Blocked by required conditions
unit / key-generation (push) Blocked by required conditions
unit / logging-misc (push) Blocked by required conditions
unit / mcp-oauth (push) Blocked by required conditions
unit / proxy-runtime (push) Blocked by required conditions
unit / proxy-server-core (push) Blocked by required conditions
unit / root (push) Blocked by required conditions
unit / proxy-utils (push) Blocked by required conditions
unit / ui-unit (push) Waiting to run
unit / docs (push) Waiting to run
unit / helm (push) Waiting to run
unit / OpenAI 2.20.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.0.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.25.0 compatibility (push) Blocked by required conditions
unit / redis-py 5.3.1 compatibility (push) Blocked by required conditions
unit / redis-py 6.4.0 compatibility (push) Blocked by required conditions
unit / redis-py 7.4.1 compatibility (push) Blocked by required conditions
unit / redis-py 8.0.1 compatibility (push) Blocked by required conditions
unit / coverage (push) Blocked by required conditions
unit / unit passed (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
LiteLLM Rust DB / db-queries (push) Has been cancelled
VS Code Extension / vscode-extension (push) Has been cancelled
Deletes the A2A e2e test that registered a third-party agent, which depended on a site the project does not control, along with its helpers and its coverage registry row. Adds an integration test where a scripted A2A agent receives a skill call as a data part and returns a data artifact, asserting the agent and the caller each get theirs unchanged |
||
|
yuneng-jiang
|
1b3da63703
|
test: move Sail, Weave and Presidio e2e tests to integration tests (#49533)
* test: move Sail and Weave e2e tests to integration tests The Sail service_tier and completion_window cases now run against the scripted upstream in tests/integration/pricing, asserting the outbound body and the billed rates for chat, responses and messages The key-scoped Weave delivery cases now run against an OTLP collector in tests/integration/observability. The case where only the otel callback is configured is skipped with a BUG reason Removes the e2e copies, the Weave read-back client, the Sail registry rows, the Weave docs paragraph and the unused devin MCP server entry from the stage mirror config * test: move Presidio masking e2e tests to integration tests Moving them to integration tests. The pre_call, post_call and default scope cases now run against a mock Presidio analyzer and anonymizer that follow the real wire contract, and a scripted model upstream, on chat completions, messages and responses, with and without streaming The spend log cases check the stored response and the guardrail record with a proxy that stores prompts in spend logs. The e2e copy, its Presidio guardrail params body, its registry rows and the doc paragraphs naming the two PRESIDIO variables are removed |
||
|
yuneng-jiang
|
68dbf54b09
|
chore(e2e): remove the unused arize_phoenix callback from the stage mirror config (#49538)
Phoenix is unused by the e2e suite and has no account behind it. Drops the callback from the stage mirror gateway config, and updates the OTEL trace test's failure message and the e2e contributing notes that still pointed at the arize_phoenix preset. |
||
|
Mateo Wang
|
a713ed6bab
|
fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692)
* test(e2e): add gemini tool call id length repro for LIT-8912 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them * fix(gemini): pair a tool result with its call when the client re-adds the thought signature to the call id Strands, which the AgentCore harness runs, puts provider_specific_fields.thought_signature back on the assistant tool call id as <id>__thought__<signature> while its tool result keeps the bare id, so the Gemini conversion found no matching call and answered 500. Ids now match with the signature suffix stripped. Also adds thought_signature_in_tool_call_id to the litellm_params inventory test and gives the pass-through tests' mocked logging objects the litellm_params a real Logging object always has. * fix(gemini): keep the thought signature on streamed Messages and Responses tool calls and accept a string setting * fix(gemini): read the thought signature back from Responses input and coerce the per-request setting A Responses function_call input item now keeps its provider_specific_fields, so a client that sends the item back gets its real thought signature replayed to Gemini. A per-request thought_signature_in_tool_call_id of "false" or 0 now keeps ids short, and a value that is not a boolean answers 400, or is dropped under drop_params, instead of a 500. The e2e test now uses the sdk fixture and @meta subjects * fix(caching): key the response cache on thought_signature_in_tool_call_id * fix(responses): send thought signature fields only with short ids and drop malformed routing kwargs whole * fix(main): validate routing kwargs after the locals snapshot so fallbacks and model_list calls keep working The e2e test now uses gemini-3.8-flash with enough max_tokens to finish thinking before the tool call. * fix(gemini): keep non-chat routes tolerant of a malformed thought_signature_in_tool_call_id, match tool results by exact id first, and keep the deployment's value when drop_params drops a malformed override * fix(router): validate strict litellm params through a helper so star imports cannot re-bind a public Final * perf(router): skip strict validation when a request overrides no deployment param * fix(responses): keep a dict tool call's thought signature on the streamed function_call done item * refactor(main): return invalid supplemental params as a value and test the streaming tool call with a real response --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Kerry Lu
|
1ccb970d52
|
refactor(ui): remove the deprecated Old Usage page (#49284)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Mateo Wang
|
4c8b90945f
|
fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864)
* fix(realtime): send Gemini Live setup on the client's first message so session.update modalities apply Gemini and Vertex AI Live accept exactly one setup per session, and the proxy sent its default audio setup at connect, so a client's session.update (GA output_modalities or beta modalities) was dropped and every answer came back as audio. The setup now goes out on the client's first message: a session.update becomes that setup, anything else gets the model default. Bedrock Nova Sonic keeps its connect-time setup through a provider hook. Text setups label deltas response.output_text.delta, and downgrading TEXT to AUDIO on an audio-only model logs a warning * fix(realtime): keep one Gemini Live response modality and skip the "inf" token limit in the setup * fix(realtime): merge every session.update sent before the first content frame into the Gemini Live setup Gemini and Vertex Live accept one setup per connection, so the bridge used to turn only the first session.update into that setup and drop the rest without telling the client. Now every session.update that arrives before the first content frame is held, deep-merged (later values win per field), acknowledged with a session.updated built from the setup it would produce, and the merged session becomes the one setup sent with the first conversation item, audio frame, or tool result. The session events also surface the systemInstruction text as instructions, which the old string check never matched because the setup carries it as a Content object. * fix(realtime): cap the session.update merge depth so the recursive detector accepts it _merge_session_fields recurses once per nested mapping level, so CI's recursive_detector flagged it. Bound the descent at eight levels (the OpenAI session object nests four deep at most), replace anything deeper wholesale with the later session.update's value, and list the helper with the other depth-capped functions the detector allows. * fix(gemini-live): accept a string output modality and drop values Live cannot produce A bare string output_modalities is read as a one-element list and modality values Gemini Live cannot produce are dropped with a warning instead of reaching the backend, so a malformed session.update no longer costs the session. Transcription guardrails now fold their VAD disable into the first held session.update only, so the merged setup carries it exactly once. * fix(gemini realtime): answer an error for session updates Gemini Live cannot apply An output modality the model cannot produce, or a session.update sent after the first content, now gets an invalid_request_error event instead of a silent swap or drop, unless drop_params is set. The synthetic session.created reports the model's default modality, setupComplete no longer adds an extra session.updated, and merging nested session updates is linear in their size * fix(gemini realtime): send the setup before guardrail blocks and idle deadlines, mark push-to-talk turns A pre_call guardrail blocking the first typed message now goes out after the setup, so Gemini and Vertex no longer close the session with 1007. Push-to-talk sessions (turn_detection null or create_response false) wrap each audio turn in empty activityStart and activityEnd messages, which Gemini needs when automatic activity detection is off. A session with no content after 60 seconds sends its setup, since Vertex refuses a setup 300 seconds after connect. Held session.update frames past 10 MB answer session_update_too_large, and deferred mode now errors on a second session.update unless drop_params is set. * fix(realtime): ack unchanged Gemini Live session.update resends and honor null turn_detection An update that leaves the applied setup unchanged now gets session.updated instead of session_update_not_applied, so clients that resend their session with the first response keep working. Deferred setup mode merges held updates like the default mode, turn_detection null turns Gemini's activity detection off, drop_params drops a voice outside the documented Live voices, and the lazy setup deadline moves to 240 seconds * fix(realtime): keep Vertex activity detection off for null turn_detection and set up deferred sessions before text content * fix(realtime): refuse unscreenable Gemini audio, send held setup after buffered audio, reject unknown Vertex voices A realtime_input_transcription guardrail cannot screen Gemini Live audio before the answer starts, so audio input under one is refused with an audio_input_not_screenable error instead of reaching the model. In deferred setup mode a session.update that follows buffered audio is sent as setup at once. Only conversation content triggers the setup, so control events no longer lock in a default one. Vertex rejects an undocumented voice on the session.update, or drops it with drop_params, while Google AI Studio forwards any name. * fix(realtime): hold first content so a following session.update joins the Gemini setup Convert flat realtime tool schemas through Gemini's schema builder, and collapse a text and audio pair to what the model can produce on text-only models too * fix(realtime): check AI Studio Live voices like Vertex * fix(realtime): forward any voice to AI Studio Live models that accept it * fix(realtime): map OpenAI TTS voices before the Live voice check, keep outage spend rows, share the playground session config The Live voice gate now judges the Gemini voice an OpenAI TTS name maps to, so fable and onyx reach Vertex as Umbriel and Orus again. An upstream close before the held setup was sent is logged as a session, not a refusal. The playground sends one server_vad session config for typed and voice turns, and the e2e, integration, and dashboard tests follow lazy setup. * fix(realtime): take the lazy setup clock as a dependency so its tests run on a stepped clock The deadline and grace tests waited on real 50 ms timers, and the merge cost check compared wall-clock runs. The session now reads an injected clock, time.monotonic by default, the two tests step it, and the merge cost check moves to the CodSpeed benchmarks. The Nova Sonic style handler test passes the backend through wrap_backend, which main added after the test was written |
||
|
yuneng-jiang
|
85af2f65c9
|
refactor(helm)!: connect both charts to a separately deployed Lens (#49169)
Neither chart bundles Lens anymore. The Lens subchart, its lock entry, the vendored chart archive, the templates/lens resources and the bundled-Lens install smoke are gone. A connection-only lens block wires the gateway and backend to a Lens deployed from its own chart, and an enabled lensWorker value fails the render instead of being ignored. Supersedes #47869, which made the same change for helm/litellm only. |
||
|
Mateo Wang
|
326f17594e
|
fix(batches): return real batch token counts next to OpenAI's batch usage fields (#47740)
* fix(batches): read OpenAI batch usage into the batch token counts * fix(batches): serve OpenAI's batch usage shape from retrieve and the cost poller * fix(batches): keep the chat usage keys next to OpenAI's batch usage fields Batch usage now carries every chat key it had plus OpenAI's five batch fields with the same counts, so poller-billed batches keep prompt_tokens and cache-write tokens, and older proxies still read the new rows. Usage with missing or null detail counts reads its real input and output counts. * test(batches): check the batch usage counts in the e2e terminal-state test * fix(types): apply nested include and exclude to both batch usage shapes * fix(batches): log no cost for a batch retrieve that is not final yet A retrieve now reads OpenAI's batch usage into the chat counts, so a retrieve that saw the batch completed before its output file arrived priced those counts for the logging callbacks, and the final retrieve priced the whole batch again. A batch that is not final now logs no cost, as it did before. Batch usage dumps also pass exclude_unset and exclude_defaults through to the usage keys. * fix(batches): pass through every usage detail field the provider sends Batch dumps rebuilt input_tokens_details and output_tokens_details from the chat keys, which dropped any detail field past cached_tokens and reasoning_tokens that the merge base passed through. The dump now starts from the provider's stored details and only sets the two counts derived from the chat keys * fix(batches): skip sync loggers on the cost poller's own retrieve and honor exclude_none on provider details * fix(batches): keep sync logger events at $0 on poller-owned retrieves, pass unparsed provider counts through, log batch usage in chat shape * fix(batches): store only chat usage extras on batch spend rows Spend rows for batch retrieves copied OpenAI's batch fields (input_tokens, output_tokens and their details) into metadata.additional_usage_values next to the chat counts they duplicate. Build the row from the batch's chat usage, shared with the logging payload through batch_chat_usage. |
||
|
Mateo Wang
|
6cdb9959df
|
feat(router): opt in to non-chat deployment filtering (#46257)
* feat(proxy): reject chat routes on models declared non-chat via model_info.supported_endpoints A model whose every deployment sets a non-chat model_info.mode and a supported_endpoints list with no chat endpoint now answers /v1/chat/completions, /v1/messages and /v1/responses with a 400 before any provider call. Models without both fields keep today's behavior. * fix(proxy): let fallbacks answer and cover more chat routes in the non-chat model check The check now steps aside when a request, key, team or router fallback would answer, skips such models in the rate-limit fallback loop, ignores mode and supported_endpoints copied from the cost map by old Admin UI saves, treats a blank mode as unset, and also covers /v1/completions, the Responses WebSocket, Gemini generateContent and /v1beta/interactions * fix(proxy): only step aside for fallbacks that can answer chat, gate the Responses WebSocket after alias resolution, and compare declarations to the loaded catalog * fix(proxy): gate the Responses WebSocket with the router after aliases resolve, and ignore client-set internal fallback flags * fix(proxy): walk fallback groups breadth first and release reservations on a rejected WebSocket Each fallback group is looked up once however often the request repeats it, and a WebSocket closed by the endpoint gate runs the failure hook so rate-limit reservations are released * fix(proxy): walk fallback layers without recursion and gate the compact Responses route The fallback walk now folds one layer at a time, so the recursion check passes, and /v1/responses/compact on a model declared non-chat gets the same 400 as the other chat-shaped routes * fix(proxy): keep chat on rows whose cost map entry serves chat A row that carries the cost map key of an entry that serves chat counts as declaring nothing, so a catalog edit that adds a chat path never turns old Admin UI copies into 400s * fix(proxy): check chat sent to a deployment id or backend model name * fix(proxy): check a model the read-through loads from the database * fix(proxy): step aside for fallbacks the check cannot follow and wildcards that serve a backend name A fallback target with no deployments of its own, a client routing stamp, and a wildcard that does not match the requested backend name no longer turn a working request into a 400 * fix(proxy): ignore a client model_group that is not a string when building rate-limit headers A client metadata.model_group sent as an int or a list crashed the error handler with a 500 once the embedding-only check rejected the call before routing. The header helper now treats a non-string stamp as no model group. Drops the pattern clause in the backend-name lookup, since the group lookup already returns wildcard matches. Adds the third audit round's routing-stamp cells. * fix(proxy): keep a non-string client model_group out of spend rows * fix(proxy): validate standard fallbacks before the typed fallback-group lookup * feat(router): opt in to non-chat deployment filtering * fix(router): skip unresolved local rate-limit fallbacks * fix(router): ignore caller-provided endpoint selector values * fix(router): normalize declared chat endpoint prefixes * test(router): cover preflight error-value behavior * test(router): derive replay metadata defaults from model info * fix(router): preserve fallback aliases and check synchronous text calls * test(routing): rewrite the WebSocket model with a pre-call callback, since main no longer runs guardrails at socket admission * test(routing): keep the endpoint gate's owned proxies on the worker's database * fix(router): keep the endpoint gate on model_info patches that omit it and on same-group stream retries A partial PATCH of model_info reset a stored enforce_supported_endpoints true back to false, because the field defaulted to False and the update merges model_dump(exclude_none=True). The field now defaults to None so an omitted flag keeps the stored value The same-group retry of a stream dropped before content called the provider without the endpoint selection context, so it could land on an enforced embedding-only deployment of the same group. It now runs inside endpoint_selection like every other router attempt * fix(router): probe rate-limit fallbacks through get_routable_deployments so the router surface does not grow Main's router fence lets code outside the router use only listed Router members and router_utils modules. The proxy's fallback probe now calls the listed get_routable_deployments with the request's call type, and the gate's 400 moves to litellm.exceptions as EndpointNotSupportedByModelError. It stays separate from main's UnsupportedEndpointError, which the router retries on the group's other deployments, while the gate refuses the whole group * test(responses): cover the WebSocket error frame for a model that refuses the endpoint * fix(proxy): let the rate-limit fallback probe try a fallback this worker has not loaded yet The probe now skips a fallback only when the endpoint check removed every deployment it has, so a model added on another worker still reaches route_request and its database lookup * fix(proxy): keep key router settings off the Responses WebSocket, read unrooted or single-string endpoint lists, and add status to the refusal frame * fix(router): warn when enforce_supported_endpoints has no usable list and match chat endpoints regardless of case, trailing slash, or Gemini path * test(router): type the load-warning test's caplog and router * test(routing): match the endpoint gate's integration tests to base fallback behavior on unloaded fallbacks and the Responses WebSocket --------- Co-authored-by: Mateo <mateo@Mateos-MBP-2.localdomain> |
||
|
Mateo Wang
|
65d912fc2e
|
feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972)
* feat(responses): support Codex client tool_search on Anthropic and other bridged providers * test(router): cover get_search_tool_support provider defaults and model_info overrides * fix(responses): keep tools loaded by earlier Codex tool searches across previous_response_id * perf(responses): read stored turns once on client tool search continuations * fix(router): avoid OAuth refreshes when listing search capabilities * test(cli): expect disabled search on fallback Codex models |
||
|
moe-berri
|
12deb60e27
|
fix(responses): forward top-level moderation param to OpenAI (#48708)
* test: cover Responses moderation forwarding Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): forward moderation parameter Co-authored-by: Poppy Huang <ruihong_huang@foxmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: remove Perplexity drop_params moderation case Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(responses): preserve typed moderation forwarding Co-authored-by: Poppy Huang <ruihong_huang@foxmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Poppy Huang <ruihong_huang@foxmail.com> |
||
|
yuneng-jiang
|
21ab2cb87b
|
test: update stale integration and guardrail e2e tests to intended behavior (#48665)
* test(integration): update stale expectations to intended provider behavior Each expectation moved because a merged PR changed behavior on purpose. - chatgpt string input is wrapped as one user message (#41522) - Gemini 2.5 tool media sits beside function_response (#48428) - sampling knobs move to Gemini 3.5 Flash, which keeps them (#47645) - a body base_model is stripped without allow_client_pricing_override (#47639) - ai_decide answers without usage return usage null (#48261) - azure_ai forwards mapped client anthropic-beta values (#48033) - the OpenCode scripted upstream mints a fresh id per request * test(e2e-ui): type the guardrail name before deleting it GuardrailDeleteModal requires the exact name before Delete enables since #45893. All three guardrail delete flows now go through one helper. * test(observability): accept any sampled traceresponse flags and expect a 400 for a non-object body OpenTelemetry 1.45 (#48055) sets the W3C random trace id flag on new traces, so the gateway's traceresponse now ends in -03. The producer copies the server span's own trace flags, so the tests now accept any two-digit flags with the sampled bit set instead of pinning -01 #48438 made a non-object JSON body a 400 "Invalid JSON payload: expected an object" on purpose, so the [1] body case is renamed non_object_body and expects that 400 * test(observability): stop waiting on OpenTelemetry's export log line in the langtrace outage test The outage test waited for "Exception while exporting Span batch" in the proxy log, which is OpenTelemetry's wording and no longer appears with 1.45. The test keeps asserting what the gateway guarantees, callers unaffected during the outage, a live gateway, and every span after recovery delivered once with no outage span duplicated * test(guardrails): count the Responses disconnect spend rows #47611 added Since #47611 a streamed /v1/responses disconnect writes its own spend row, so the concurrent disconnect test saw 20 rows where it counted 10 by model group. It now reads each row's call type and expects ten acompletion rows with the outage verdict and ten aresponses rows with success. /v1/messages disconnects still write no row on main, so they are left out of the count rather than pinned as absent The responses-httpx case of the single disconnect test no longer needs its LIT-8603 skip and passes * test(observability): size K4's timeout from the harness boot budget K4 boots three two-worker proxies in a row. On CircleCI each boot takes 43 to 58 seconds, so a passing run took 179.97s against its fixed 180s cap and 6 of 30 runs hit pytest-timeout. The cap and the restart gate wait now come from readiness_seconds and graceful_stop_seconds, like other multi-boot tests |
||
|
yuneng-jiang
|
9b2fc9db40
|
test: remove checks that pin implementation details and add a shadcn testing rule (#48663)
* test: remove e2e checks and a CI line count that pin implementation details Deletes the Auto Router template select anchoring spec, the 390px logs pagination variant, and the call id tooltip and clipboard step in the logs spec. Each one tests library or test helper behavior rather than what users see, and each has been red or flaky on main without a product regression Also drops the exact line count on the fan-in requirements export in the integration coverage job. It went from 6 to 10 when uv.lock gained a Python 3.15 fork, while the install was still correct. A package the filter drops still fails the job, either at the hash-checked install or at the next step's import * docs(agents): do not test what shadcn and Base UI already guarantee Adds the boundary to the dashboard AGENTS.md next to the test tiers, and points the Playwright suite's entry in tests/e2e/AGENTS.md at it |
||
|
Mateo Wang
|
0a1dbc0fb4
|
refactor(mcp): drop the experimental label from the MCP server and client paths (#48513)
* refactor(mcp): drop the experimental label from the MCP server and client paths Move litellm/proxy/_experimental/mcp_server to litellm/proxy/mcp_server and litellm/experimental_mcp_client to litellm/mcp_client, with the mirrored tests moved under tests/unit/proxy/mcp_server and tests/unit/mcp_client. Both old packages stay as shims that warn with DeprecationWarning, re-export the new package's public names and alias every submodule in sys.modules so deep imports keep resolving to the same objects. The OpenAPI docs gate now puts the merge base's enterprise package on PYTHONPATH so its base counts do not mix the checked-out enterprise code with the base litellm tree Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(mcp): keep tests/unit/proxy/_experimental as a sharded root for the shim tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): alias namespace subpackages and bind attributes in the deprecated MCP shims pkgutil.walk_packages skipped mcp_server/auth because it has no __init__.py, so old imports of litellm.proxy._experimental.mcp_server.auth.* failed, and sys.modules entries alone left litellm.experimental_mcp_client.tools unreachable as an attribute. Walk the package on disk and setattr each alias on its old parent. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Mateo Wang
|
14193435b8
|
refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510)
* refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages Move the unified /v1/messages code out of a directory called pass_through, since it is not the passthrough feature. The inner messages sub-package becomes native, and the Vertex AI partner model experimental_pass_through package becomes messages. The old packages stay as import shims that warn once and alias every submodule in sys.modules, so old deep imports still resolve Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(anthropic): keep Final variables out of loops in the rename shim Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(anthropic): match the warning text in the rename shim test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(anthropic): resolve old pass_through imports lazily through a meta path finder Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(anthropic): mark the alias loader spec restore for the type discipline gate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(anthropic): format the renamed package shim Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(anthropic): keep the rebind-ok marker on the spec restore line Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(anthropic): carry the new package search path on alias specs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(anthropic): declare the shim test locals Final Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |