The gateway acts on the core decline. A replay layer records each native
request before its handler runs. When the error response carries a Decline
it forwards the recording to Python as the client sent it, or keeps the
501 when the gateway runs standalone. Paths and methods with no native
route go to Python through the merged router's fallbacks, before
authentication. The 501 stubs for embeddings and completions and the
gateway's own Unsupported variant are gone, since a matched stub would
never reach Python.
LITELLM_PYTHON_UPSTREAM picks the Fallback at startup. Forward uses its
own hyper client to the loopback proxy, strips hop-by-hop headers, adds
x-litellm-gateway-declined with the reason and appends the client address
to X-Forwarded-For. An upstream that is not an http origin is rejected
with InvalidUpstream, which keeps the URI parse failure as its source.
Error responses attach the decline in one place, so the messages handler
builds its Anthropic envelope through Error::anthropic_response instead of
a tuple that dropped it.
Forwarding is a gateway concern, so the dispatch module lives in the
gateway crate and replays declines for every mounted router. The replay
layer is installed only when a Python upstream is set, so a standalone
gateway does not buffer request bodies. Tests run the gateway and a
stand-in Python on real listeners through litellm_gateway::serve, which
main.rs uses too. The rust-error skill says where an unserved path and a
forwarded decline go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rust core says it cannot serve a request with one Decline that travels
unchanged through llms and inference up to the host. The Python bridge
raises RustBridgeDeclined for it in route_failure before the public
exception mapping, so dispatch falls back to legacy Python instead of
failing the call, and the gateway answers 501. Every route collapses
errors after the send with Error::after_send, which keeps a provider
answer and a response fault as they are and nests anything else as the
source of an invalid response, so a billed call can never read as a
decline. Response-side cases that used to decline are now
ErrorDetail::Unhandled response faults.
The rust-error skill documents declined() as the first question both
boundaries ask, and the Python suites pin that a native decline reaches
the dispatcher before any provider IO.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
host-python keeps a native error's chain and returns blocking-pool
cancellation instead of panicking, Vault transport failures convert
through litellm_http::Error so no URL reaches Python, the bridge asks
predicates instead of matching three levels into Vault errors, and the
last string-built PyErrs become typed errors mapped once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Auth rejections hide their backend cause on every inference renderer,
path parameters reject in the surface envelope, Textract's unsupported
document and Azure DI's unparseable poll URL get typed variants, the
credential chain exposes its last failure as the source, gateway-mcp
asks predicates over rmcp errors, and the S3 and Azure blob transports
classify through litellm_http::Error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OCR provider error and get_error_class take reqwest's HeaderMap, the
bridge turns it into Python's (name, value) pairs once. Credential
acquisition and provider SDK failures are no longer the caller's fault,
the UI hides auth and backend causes, and chat keeps a provider answer
as is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RustUpstreamError has one constructor that always sets headers, argument
checks raise typed errors mapped once, OCR asks upstream_response(), the
Responses stream error event comes from the error type, gateway-inference
loses its string variants, and auth faults classify through one
litellm_auth predicate on every route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
litellm_inference::Error nests litellm_llms::Error instead of mirroring
its variants, host driver faults get their own variant, every predicate
matches exhaustively, and route tests ask predicates instead of matching
transport variants.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ErrorDetail loses its string variant, OCR provider variants become data,
host faults stop counting as the caller's, a tunnelled transport error
classifies the same on every stream path, and each gateway crate has one
Error with one status() and extractors that reject in its envelope.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Messages describe their own layer and every cause is a typed source,
predicates match exhaustively, and boundaries render with
litellm_error::Report. Media downloads no longer count as dispatched,
Error::upstream bounds the provider body itself, and a forwarded
header SigV4 cannot sign is the caller's fault.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Messages describe their own layer and the cause is the source, boundaries
render the chain once with litellm_error::Report. Corrects skill claims
the audit found stale against the code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Chat, Messages, Responses and transcription each get a route-level test that
fails if a reqwest error reaches the caller with its URL. The Messages stream
decoder also takes body errors through litellm_http::Error like every other
reqwest error
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The former axum page toured gateway-inference: its status match, the envelope
table, an extractor, the unsupported route and the Sse closure. Those details
belong to the gateway crates and go stale with them. The page is now gateway.md
and follows an error from the provider failure through each From into
IntoResponse, where it is classified once and rendered by surface and framing.
The gateway-structure material moves to a new rust-gateway skill: envelopes per
API surface, extractors and path registration, SSE relay, and the router test
harness.
The four WebSocket handshake variants collapse into Handshake(HandshakeFault), and media download failures no longer answer dispatched() since they happen before the provider call
ErrorDetail in llms and auth-types, the inference error and the OCR error now store their cause in a field named cause with no #[source], so to_string() carries it exactly once and source() is None. Tests that walked the chain now destructure the variant and downcast the cause
The gateway asks is_unsupported() and is_missing_credential() on the inference and OCR errors instead of matching through nested enums, and litellm_auth::Error answers is_missing_credential() for both
The rust-error skill gains axum and pyo3 references, boundaries.md describes the real predicate order and the known divergences, the testing example asserts dispatched() correctly, and shape, classification, conversions, messages-and-causes and recipes agree with the code again
transport::Error and media::Error fold into litellm_http::Error, an
Arc<Repr> newtype whose private variants keep the typed reqwest, io,
rustls and tungstenite causes and own every message. Callers can no
longer build an error with a message of their own. They ask through
dispatched, is_request, status, upstream_response and tls_source
One From<reqwest::Error> now classifies connect, network and timeout
and strips the URL. The WebSocket handshake reports each failed check
as its own variant and owns its connect timeout. OCR drops the media
variants it mirrored, and inference::Error maps a header the signer
computes to Unsupported for every route, not only chat
The error rules move out of AGENTS.md into the rust-error skill, which
also records the decisions behind this design
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OutboundRequest was a hand-rolled copy of reqwest::Request. Routes now build
a reqwest::Request directly and RequestSigner reads the body bytes back out
of it, so the signer still covers exactly what is sent. SigV4 signing moves
onto HeaderMap and takes the method, which drops the BTreeMap plumbing and
the header casing table
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Responses WebSocket connected through tokio-tungstenite with its own
native-roots TLS, so it ignored the configured CA bundle, verify flag,
client certificate, proxies and timeouts. The handshake now goes through
a pooled reqwest client (ClientVariant::WebSocket, HTTP/1.1, no
redirects) and upgrades the response, so it shares the settings every
other outbound request uses
The upgrade response is checked for the Sec-WebSocket-Accept key, the
Upgrade and Connection headers and a subprotocol that was offered
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The workspace moves to reqwest 0.13, so litellm-http no longer carries a
second reqwest just for rmcp. The mcp feature, its pool and its proxy
conversion go away, and the gateway hands rmcp a pooled NoRedirect client
that already carries the TLS, proxy and timeout settings
The Client test helpers collapse into for_test, a no-redirect client
without TLS or proxy settings, and wrap_for_test for the few tests that
need a hand-built reqwest client
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Update certifi, Rust trust roots, and the Docker CA bundle to remove emSign ECC Root CA - G3
Require the fixed certifi release in litellm-core and retain libssl3 in Docker runtimes for Prisma compatibility
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Every other crate names its error Error. The format crates already re-exported
it under that name, so only the inference crate and a few importers change
* refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines
* refactor(rust-bridge): split and document route-specific detectors
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Native Responses now hands back the same managed ids as Python: the
resp_ id carries the provider and deployment, encrypted-content items get
encitem_ ids and wrapped content under affinity routing, and container ids
become cntr_ ids. Encoded encrypted-content items in the input are restored
before the provider request. The codec lives in inference-responses with
goldens taken from litellm/responses/utils.py
* fix(databricks): use the native Anthropic route for Claude /v1/messages
Claude models on a Databricks workspace were bridged through chat completions on /v1/messages, which drops the safeguards field and never returns safeguard_results. They now call the workspace's /serving-endpoints/anthropic/v1/messages, the proxy forwards the client's anthropic-beta and anthropic-version headers to Databricks deployments, and the beta header map marks dangerous-tool-use-2026-09-03 as supported there. Any route whose platform does not take that beta drops safeguards together with it instead of answering 400. Non-Claude Databricks models keep the chat completions bridge
* fix(databricks): derive the dangerous-tool-use beta from safeguards instead of forwarding client betas
A Databricks Claude serving endpoint answers 400 to betas Claude Code sends on every turn, prompt-caching-scope-2026-01-05 among them, so the proxy no longer forwards the client's anthropic-beta header to Databricks deployments. The beta the classifier needs is derived from the safeguards body field, the same way the Azure AI route derives it. The Rust beta-header mirror stops rejecting the Databricks dangerous-tool-use cell so its config-cell test matches the JSON again
* fix(databricks): keep the billing block, build the messages route once, and pass the partner user agent
* fix(databricks): route Unity Catalog Claude names to the gateway messages route and keep env credentials
Unity Catalog model-service names now reach the AI Gateway's native Anthropic
route instead of a serving-endpoints path that answers 404 for them, and the
native route resolves the key and base through the same order the chat route
uses, so environment-only deployments keep working.
The native route is limited to Databricks foundation-model Claude names, an
unlisted provider forwards safeguards and the beta as sent, and the ten betas
the workspace rejects are marked unsupported for Databricks in the beta map
and its Rust mirror.
* test(rust): expect Databricks to drop the advanced-tool-use beta
* fix(databricks): keep passthrough safeguards, partner user agent, and classifier fallback
Only the OpenAI-compatible JSON providers drop safeguards now, so a
deployment opted into /v1/messages passthrough forwards the field and
the beta as sent. When a route drops safeguards and the proxy has a
classifier model, the proxy answers the verdict itself, as the chat
completions bridge did. The Databricks Messages route also reads the
databricks_user_agent param like the chat route.
* fix(databricks): drop rejected server tools under drop_params, short-circuit web search, route UC names from a full Messages URL
* fix(databricks): drop container on the native Messages route, as the chat completions bridge did
* fix(databricks): keep every api_base on a route the workspace serves and gate the derived beta on the map
Gate the dangerous-tool-use beta derivation on the beta map the way #43955 does, so GitHub Copilot, DeepSeek, and opt-in passthroughs keep the merge base wire. Recognize /serving-endpoints/anthropic api_bases, fall back to the merge base routing for an api_base path the workspace trim does not know, strip Bedrock invocation metrics from the streamed message_stop, forward client betas to Databricks filtered by the map, and keep tool_choice when a dropped server tool has no name
* test(databricks): cover the native Messages route, derived beta and JSON provider safeguards drop on the wire
* test(anthropic): expect the dangerous-tool-use beta derived from safeguards next to the client's betas
* fix(databricks): send client anthropic headers only on the native Messages route and keep SDK auth
Databricks chat completions, called directly or through the Messages bridge,
no longer receive the client's anthropic-beta and anthropic-version, as at the
merge base. The native Messages route still forwards them, filtered by the
beta map, without counting them as caller credentials, so a deployment with no
api_key keeps falling back to Databricks SDK auth. A pathless api_base goes to
the native route only for Unity Catalog names, matching the chat route.
* test(databricks): cover SDK auth with client anthropic headers and a Unity Catalog workspace root on the native route
* fix(databricks): keep client anthropic headers off an opted-in passthrough api_base
A databricks Claude deployment whose api_base is no workspace route and that opts
into /v1/messages with supported_endpoints falls back to the OpenAI-like
passthrough. That config does not filter betas through the beta map, yet it
received the native-only Databricks header entry, so Claude Code's
prompt-caching-scope beta reached the upstream raw and got a 400 where the merge
base answered 200.
Each Messages config now says whether it is the provider's native route, the
OpenAI-like passthrough says no, and the handler passes that answer when it
reads provider-specific headers, so native-only entries reach only native configs.
* test(databricks): cover client anthropic betas on an opted-in passthrough api_base
A databricks Claude deployment whose api_base is no workspace route and whose
model_info.supported_endpoints opts into /v1/messages takes the OpenAI-like
passthrough. The cell sends Claude Code's prompt-caching-scope and context-1m
betas through the proxy and asserts the caller gets the upstream's message and
the gateway receives no anthropic-beta, as on the merge base. The scripted
gateway answers 400 on the prompt-caching-scope beta, as the real one does
* fix(anthropic): keep beta map loads off shared threads and build the safeguards evaluator only when the proxy classifies
Unfiltered configs such as first-party Anthropic never read the beta map to decide safeguards support. Filtered configs await the map on its dedicated loader before reading it, on both the native and the MCP paths. The native handler decides after the upstream call and builds the evaluator only when the proxy classifies, so malformed messages keep the upstream 400. The Databricks dispatch tests assert the dispatched route instead of the config class, and the sail and Eden AI wire rows expect safeguards dropped, as JSON providers do.
* test(databricks): cover a loaded beta map that marks the safeguards beta unsupported
A remote beta map that nulls dangerous-tool-use for databricks while the bundled map allows it must send neither safeguards nor the beta to the serving endpoint, with or without a client beta, and the caller gets a 200 with no verdict.
* test(anthropic): annotate the MCP beta map recordings as Final
* fix(anthropic): drop safeguards on routes without the classifier only under drop_params
A route whose upstream does not answer safeguards (a JSON provider, or a map
that marks the dangerous-tool-use beta unsupported) now forwards the field and
the beta as the merge base did unless drop_params is set or the proxy runs its
own classifier, matching how litellm treats any unsupported param. Databricks
keeps dropping both, since its merge base bridge never forwarded the field.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(openinfer): add OpenInfer as OpenAI-compatible provider
Register OpenInfer in the JSON provider registry with catalog models,
proxy create fields, and unit coverage so requests route via openinfer/
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openinfer): assert catalog invariants instead of vendor rates
Pinning per-token prices made CI fail when the vendor list changed. Keep
the backup agreement and positivity checks our code owns.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openinfer): add openinfer to the Rust OpenAI-compatible provider mirror
* fix(openinfer): align catalog with OpenInfer's live /v1/models
Replace the four catalog rows with the two models OpenInfer serves and
publishes a price for (@oi/GLM-5.3-Flash, @oi/Qwen3.5-27B), using its
listed per-token rates and context windows, and price-check them in tests
* fix(openinfer): use OpenInfer's native Responses and Messages APIs, allow tools on unpriced models, price Qwen3.5-9B
Declare /v1/responses and /v1/messages for openinfer, since OpenInfer serves both natively. Let JSON providers declare provider-wide capabilities in providers.json, read through get_provider_info, so tool calls reach OpenInfer for models without a cost map row. Send max_completion_tokens as is, set output caps to each model's context window, mark the three rows as reasoning models, add Qwen3.5-9B at its listed rate, and wire openinfer into models_by_provider so the openinfer/* wildcard lists its models.
* fix(openinfer): let a model's own capability row win over provider defaults in get_model_info
Provider defaults now only fill capability fields the cost-map row leaves unset, so a row or deployment that turns function calling off stays off. Also marks openinfer_models as Final.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards
* test(rust): check the dangerous-tool-use beta reaches the wire with safeguards
* fix(rust): derive the dangerous-tool-use beta only for hosts with a beta map column
DeepSeek and the other hosts without a column in anthropic_beta_headers_config.json
keep forwarding safeguards and client betas as sent. Adds the wire-level audit tests.
* fix(rust): add the safeguards beta on Azure AI and Claude Platform hosts
Azure AI and Claude Platform Messages transforms now add the
dangerous-tool-use beta for a non-null safeguards field, through the
Azure AI and Anthropic beta-map columns. The safeguards wire and chaos
tests get unique file names so they do not collide with the Python
fix's tests, the Python-route test asserts the fixed behavior behind a
BUG skip, and the DeepSeek test checks that no beta is added
* fix(rust): keep Claude Platform out of the safeguards beta until a route can prove it
Claude Platform has no beta-map column and no QA account, so it goes back to sending
safeguards with no derived beta, like DeepSeek. Also release held chaos-test requests
before the thread pool waits on them, and mark the no-column test as rstest.
* fix(rust): derive the safeguards beta for Claude Platform and keep the null-cell test off Databricks
Claude Platform on AWS answers 400 to safeguards without dangerous-tool-use-2026-09-03 and 200 with it, so its Messages transform derives the beta through the Anthropic column again. The null-cell case now uses only Bedrock Converse, since Databricks support for the beta is changing in a sibling PR.
* fix(rust): drop a null safeguards key instead of sending it without its beta
Anthropic rejects any safeguards key, null included, when the
dangerous-tool-use beta is missing, so an explicit "safeguards": null on the
Rust gateway still answered 400. The shared Messages transform now drops a
null safeguards key on every host, as the proxy already drops None params
before calling Rust, so it answers 200 with no beta.
The scripted peer in the integration wire tests now rejects the key on
presence, matching the real API.
* test: cover Bedrock Mantle in the null safeguards case and match the vendor's non-list wording
The Rust null-safeguards regression test now also runs for bedrock_mantle, whose beta is keyed on the field being present. The scripted Anthropic peer answers a non-list safeguards value with the real API's "valid array" wording.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci(cost map): remove the generated model prices JSON schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost map): keep schema-independent cost map tests after dropping the schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(cost map): keep the schema file until main's guard stops requiring it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(cost map): let bot sync PRs keep touching the schema file until it is deleted
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): let each route host pick the body params
The native routes projected request body params from a name list, hard-coded in
Rust for Messages and read from `PARAMETERS` for Chat Completions and Responses.
Each route host now answers `body_params(request)` over the keywords the call would
send, so the Python side owns which params reach the provider. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-host): use read-only body parameter types
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): move the route host modules under host/routes
Each native route's Python half lived in its own `<route>/route_host.py`, apart
from the shared host modules Rust also imports by name. They now sit together in
`litellm/rust_bridge/host/routes/<route>.py`, with their unit tests moved to the
mirrored path. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-host): reference messages route alias
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The route host now reports get_configured_request_timeout() in MessagesSettings, and prepare uses the call timeout first, then that setting, then the 600s default. Before this, litellm_settings.request_timeout and REQUEST_TIMEOUT were ignored on the native path, so a long turn that works on Python timed out after 10 minutes
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The Responses route now resolves openrouter alongside openai. The
OpenRouter config reuses the OpenAI request and response transforms and
owns only its bearer credential (OPENROUTER_API_KEY, then OR_API_KEY) and
base URL (OPENROUTER_API_BASE). Wire model ids follow Python's
get_llm_provider, including the openrouter/auto native id with an
explicit provider. The Python route host no longer hands litellm.openai_key
to OpenRouter.
The OpenRouter live test lives in tests/live/openrouter.rs and returns
early when the provider key is not exported, so cargo test stays offline
by default instead of hiding it behind --ignored.
Co-authored-by: nate-berri <nate@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(vertex): report an unreadable credentials file as a read failure
Tell a path apart from inline JSON by shape, not os.path.exists().
Fixes#40166
* test(vertex): narrow pytest.raises to the expected message
* fix(vertex): log the credentials path instead of returning it to the caller
* refactor(vertex): log the credential source decision where it is made
* fix(vertex): handle paths open() rejects with a bare ValueError
* fix(vertex): redact the credentials path before it reaches the log
* test(vertex): cover the brace fallback and non-text credentials file
* test(vertex): exclude the unreachable exhaustiveness guard from coverage
* refactor(vertex): read the failure's fields off the matched object
CodeQL's py/uninitialized-local-variable does not follow keyword capture
patterns in a match statement and reported `path` and `reason` as possibly
unbound. Narrowing on the class and reading the attributes says the same thing
without a suppression.
* fix(vertex): accept a read-only mapping in extract_aws_params
* fix(vertex): keep encoded credentials out of the log and cover the legacy route
* fix(vertex): give one message for every credentials file failure and pick file or JSON by shape in Rust
* fix(vertex): read a `{`-named key file in Rust, name base64 there, and give one message when Google auth rejects a credentials file
* fix(vertex): read an unreadable `{`-named key file as a file, cap key file reads, and drop storage advice from the file message
* test(vertex): integration cells for credentials file and inline value failures
* test(vertex): drive the Rust messages cell through Vertex Claude
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python
Rust's run_call built Python streams itself and imported
litellm.rust_bridge.streams by name, so sync calls were driven in Rust
while async calls came back as a Rust-made coroutine. Every native
inference entrypoint now returns one unstarted Execution and
public_execution.py drives it with lifecycle.run or lifecycle.drive,
wrapping streams and _hidden_params on the Python side. Rust only
imports the lifecycle step classes through StepTypes, and
python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone.
The adapter trusts the _native.pyi contract with a single cast instead
of a per-request runtime_checkable isinstance. Binding tests assert the
adapter starts the execution and returns its completed value rather
than checking callable identity.
* refactor(rust-bridge): fold lifecycle and public_execution into one execution module
The public stream wrappers only forwarded to the lifecycle streams and added the
header dict, so the streams now carry _hidden_params themselves. lifecycle.py no
longer held any call lifecycle, just the Execution driver, so it is renamed to
execution.py and absorbs the synchronous and asynchronous adapters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-bridge): keep execution.py free of third-party imports
The host-python tests load execution.py into a bare interpreter, so the
pydantic header validation broke them. The stream head comes from the
binding's encode_stream_head, which always builds a str-keyed dict
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(host-python): preserve pre-async hook coverage after execution handoff
* style(rust-bridge): sort litellm_host_python imports
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* fix(rust-bridge): mark execution TypeVar as Final
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): cover stream close, read failure, and sync suspension in execution
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): drive executions with generator-style send, throw and close
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): name stream steps StreamOpen and StreamChunk and reject out-of-phase steps
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): drive with ExitStack and require the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(host-python): give the hook_chain lifecycle binding the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(host-python): run the Python driver as one async body
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(host-python): give the driver a PythonCaller with named suspensions
PythonTask becomes PythonCaller with await_python, open_stream and yield_chunk, so the call body no longer builds Suspension values or maps stream replies itself
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(host-python): collect host failures in a thiserror Error enum
missing_state() and the inline PyRuntimeError/PyTypeError messages become Error variants with one From<Error> for PyErr, and InvokeError derives thiserror so From<PyErr> comes from #[from]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(host-python): note the canonical HostError import alias in AGENTS.md
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: lower aiohttp keepalive timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http): restore the 15s keep-alive default and apply it to the Rust client
15s is aiohttp's own default and what every release before v1.78.0 used. The Rust HTTP client reads the same AIOHTTP_KEEPALIVE_TIMEOUT variable, so its default moves with it.
* test(integration): cover upstream connections dropped after the keep-alive idle window
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Chat Completions, Messages and Responses each get a streaming module with
the same three names, event_stream, decode and encode, built on a shared
SSE helper in base_llm/sse.rs that owns the [DONE] marker. Callers use the
module path instead of vendor-prefixed names. Encoding matches what the
Python proxy writes to clients: Messages frames carry an event line, the
OpenAI formats are bare data frames.
Responses gets one event type for SSE and WebSocket. ResponsesEventType
names every event on OpenAI's streaming-events reference, with unknown
names riding in Other, and is_terminal and is_failure follow Python's
iterator. The framer adds a bulk decode_all and an SseBlockCodec that
keeps each event's wire bytes. An event the stream ends inside is dropped,
as the SSE spec and the openai SDK's decoder do.
ChatCompletionChunk no longer requires id, created or object, since
compatible hosts omit them.
* feat(edenai): settle native Messages reported cost
* test(rust): add live EdenAI Messages contract cases
* refactor(rust): settle call cost once as an ExecutionFacts variant
The route now decides Cost (Reported, CacheHit or Deferred) from the
result source and the provider's reported figure. The legacy callback
adapter only projects a Reported amount onto provider_reported_cost and
no longer writes response_cost, which the Python calculator derives
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust): keep EdenAI wire shaping independent of Invoke
* style(rust): format cost settlement imports
* fix(rust): trust provider costs without bypassing configured pricing
* fix(rust): clear untrusted response costs before spend settlement
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* chore: drop Greptile from the PR template, AGENTS.md and CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs: require bot reviews to finish before ticking the review checkbox
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: migrate proxy security, schema migration and MCP legacy tests
* test: run the MCP proxy e2e upstream under the SDK1 peer in integration-mcp
* test: warm the baseline store before timing the locked projection
* test(e2e): hold the stack exclusively while the semantic filter flag is flipped
* test(mcp): scope the in-process proxy e2e fixtures to their module
The session-scoped autouse fixture deleted DATABASE_URL and overrode LITELLM_MASTER_KEY for the rest of the worker, which broke every later test in the integration mcp suite
* test(e2e): assert the semantic filter keeps listed tools instead of a ranking
Which Datadog tool an OpenAI embedding ranks first is not something litellm owns, so the test now checks that the kept tools are a non-empty subset of the key's listing within top_k
* test(mcp): keep the in-process proxy e2e tests on one worker
loadgroup spread the file's tests across workers and interleaved them with other files, so the module-scoped in-process proxy was torn down and restarted between tests and requests timed out
* test(mcp): name the upstream requests that carried the wrong token
The bare all() assertion hid which request broke the forwarded token, which made an intermittent CircleCI failure undiagnosable
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* feat(rust): support Claude Platform and Mantle Messages
* fix(ci): regenerate .greptile/files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): preserve Mantle body betas
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): resolve Mantle default region
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): prefer regional endpoint host regions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(rust): format Bedrock regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
