auth_cache::Error::kind maps into the store's ErrorKind, adding only its encode
kind, so the bridge raises NativeRedisStoreError through kind_error. Its declared
reads and writes are NativePending, so NativeAuthCacheError,
NativeDeclaredAuthReads and NativeDeclaredAuthWrite are gone.
With enable_redis_auth_cache on, everything user_api_key_cache keeps (user-key
objects under bare hashed tokens, team/user/alias/org/end-user/project and the
other management objects, the global proxy spend scalar) now lives in
litellm-redis-store's auth_cache module: both memory partitions plus Redis,
behind litellm/redis/stores/auth_cache.py. redis-py never reads or writes the
keyspace and UserApiKeyCache.redis_cache stays None; features that only borrow
the connection (MCP refresh lock, user provider credential cache) get it from
redis_connection. With the flag off the cache stays Python memory as before.
Values cross pyo3 as plain data and are written as main's json.dumps (str() for
the sync set_cache path), read back with json.loads then literal_eval, with
main's TTL rules, pinned by matching inline cases in the Rust and Python tests.
The identity and owner prefetches read on the request's native pre-call batch
and arm the cooldown read right before, so both leave in one pipeline;
write-backs and pre-call SET/DEL ride the same batch. The invalidation
publisher and subscriber stay in Python and forward evictions to the native
memory tier.
The auth cache sends the commands UserApiKeyCache does. Prefetch reads are one
MGET, set_many writes the key-object partition and then the rest as two
pipelines, delete runs both partitions as concurrent multi-key DELs in chunks
of 1000, and increment_if_present is a plain EVAL with the amount written the
way redis-py encodes it.
ErrorKind names a store failure the way NativeRedisStoreError reports it, with
spend_counters::Error::kind mapping into it, and Reading spells its redis or
memory source with strum, so the bridge only formats the exception.
redis_error_kind maps a Redis client failure to the kind callers branch on, so
NativeSpendCounterError becomes the shared NativeRedisStoreError. Spend methods
go through redis_async and the declared read is a NativePending.
Every spend:* reader and writer (auth budget checks, reservations and lease
renewal, post-call increments, reseed, resets, key/team/user/tag endpoints,
shadow eval) now goes through litellm/redis/stores/spend_counters.py, backed by
litellm-redis-store's spend_counters module. Keys, value bytes (json.dumps for
SET, str() for the set-max script), TTLs and the set-max Lua script match what
main writes, pinned by matching inline cases in the Rust and Python tests.
The auth budget read rides the request's native pre-call batch through a
native flush hook, so it leaves in the same pipeline as the armed cooldown
read, and post-call increments ride the native post-call batch. The
auth-cache invalidation subscriber forwards spend:* broadcasts to the native
memory tier.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spend store sends the commands main does. One MGET for a batched read,
INCRBYFLOAT with EXPIRE on its own pipeline for increment_many, async_set_max
as a plain EVAL, and the reset job's deletes as multi-key DEL in chunks of
1000.
Cooldown converts to and from its CooldownCacheValue dict and router_usage::Kind
derives its tpm and rpm key segment with strum, so the bridge carries those
values through PyValue and FromStr instead of spelling the fields itself.
PyValue moves down from the auth cache store to serve both.
redis_async and redis_sync capture the ServiceLogging labels, scope the
operation and run it, so each store method makes one call. NativePending
replaces NativeDeclaredCooldowns as the single resolve-once handle, and ttl()
is the one seconds-to-Duration conversion.
Router cooldowns and the global TPM/RPM counters move behind two Python stores backed by the new redis-store crate. The read-through memory tier lives in redis-store as MemoryTier, so crates/cache keeps no Redis dependency. Folds in the Python-only encapsulation step that used to be its own PR
native_client takes the NativeRedisSettings a RedisCache captures when it is built, the kwargs, namespace and default TTL, instead of reading those attributes off the cache. A stand-in that skips the RedisCache constructor leaves the native stores on memory, and a test double that wants them on Redis builds the settings itself
The stores send the commands main does: one MGET for the cooldown and usage
reads instead of a GET per key, and a deployment's tpm and rpm increments on
their own async_increment_pipeline round trip instead of the request's
post-call batch.
A credential_provider object reaches the resolver as a ProviderHandle, so it
resolves to Auth::Provider and two RedisCaches share a client only when they
share the provider. The process registry of native clients now holds each
client weakly, so a discarded Router drops its client and its provider
instead of keeping them for the life of the process.
NativeRedisClient resolves a RedisCache's redis_kwargs through the RedisCache
kwargs surface and hands every native store in the process one shared client,
breaker and ServiceLogging event sink. A setting Rust ignores is logged with
its consequence, and a config it cannot use leaves the stores on memory
NativeRequestRedis carries one request's pre-call and post-call batches from
RequestRedisBatches into the stores. The keyspaces that use these land in the
PRs above
RedisSpec carries RedisCache.redis_kwargs into the resolver as plain values
plus the names of keys whose value is a Python object, instead of replacing
each object with its type name. ConfiguredRedis::with_host_objects takes those
names, so the resolver reports them in Resolution::host_objects, warns for the
ones Rust cannot honor and lets each hide its REDIS_* value. A
credential_provider object without a ProviderHandle fails with
ProviderWithoutHandle, so the stores stay on memory instead of connecting
without auth
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): inject database repositories through per-repository dependencies
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): model Prisma client cleanup
* test(rust): include NativeDatabase in bridge API surface
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ocr): support outputContentFormat=markdown for Azure Document Intelligence
* fix(ocr): map only span endpoints when slicing Azure Markdown pages
* fix(ocr): accept any casing of outputContentFormat and report code point offsets
Azure reads outputContentFormat case-insensitively, so Markdown and TEXT
now map to the same values instead of returning 400. When the proxy asks
Azure for code point offsets to split Markdown pages, the response now
carries stringIndexType so callers know the unit of table and key-value
spans. Tests use rstest and the enum derives VariantArray per the Rust
workspace rules.
* fix(ocr): ask Azure for code point offsets only on prebuilt-layout and keep line-joined pages when spans are malformed
* fix(ocr): trim Azure DI outputContentFormat, accept output_content_format, and drop invalid values under drop_params
* fix(ocr): fall back to output_content_format when outputContentFormat is empty
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(rust-bridge): require the native extension
_native ships in every wheel and the sdist will not install without it, yet the bridge loaded it optionally and carried None paths that could fail open for fork safety, exception mapping and OTLP errors. Delete loader.py and import _native directly. Tests inject a fake module through NativeBinding(source=...) or with_source() instead of patching the loader
The install smoke checks now verify the compiled module, and tests fail instead of skipping when _native is missing. The interactions spec canary installs the project so it has the extension
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ci): build Rust bridge in Windows smoke tests
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): one typed decline from core up to each host
Rust core says it cannot serve a request with one Decline that travels
unchanged through llms and inference up to the host. The Python bridge
raises RustBridgeDeclined for it in route_failure before the public
exception mapping, so dispatch falls back to legacy Python instead of
failing the call, and the gateway answers 501. Every route collapses
errors after the send with Error::after_send, which keeps a provider
answer and a response fault as they are and nests anything else as the
source of an invalid response, so a billed call can never read as a
decline. Response-side cases that used to decline are now
ErrorDetail::Unhandled response faults.
The rust-error skill documents declined() as the first question both
boundaries ask, and the Python suites pin that a native decline reaches
the dispatcher before any provider IO.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust): keep post-provider hook failures terminal
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http
The workspace moves to reqwest 0.13, so litellm-http no longer carries a
second reqwest just for rmcp. The mcp feature, its pool and its proxy
conversion go away, and the gateway hands rmcp a pooled NoRedirect client
that already carries the TLS, proxy and timeout settings
The Client test helpers collapse into for_test, a no-redirect client
without TLS or proxy settings, and wrap_for_test for the few tests that
need a hand-built reqwest client
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(http): honor transport settings for Rust WebSockets
The Responses WebSocket connected through tokio-tungstenite with its own
native-roots TLS, so it ignored the configured CA bundle, verify flag,
client certificate, proxies and timeouts. The handshake now goes through
a pooled reqwest client (ClientVariant::WebSocket, HTTP/1.1, no
redirects) and upgrades the response, so it shares the settings every
other outbound request uses
The upgrade response is checked for the Sec-WebSocket-Accept key, the
Upgrade and Connection headers and a subprotocol that was offered
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(http): build outbound requests as reqwest::Request
OutboundRequest was a hand-rolled copy of reqwest::Request. Routes now build
a reqwest::Request directly and RequestSigner reads the body bytes back out
of it, so the signer still covers exactly what is sent. SigV4 signing moves
onto HeaderMap and takes the method, which drops the BTreeMap plumbing and
the header casing table
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(http): make litellm_http::Error the one opaque error
transport::Error and media::Error fold into litellm_http::Error, an
Arc<Repr> newtype whose private variants keep the typed reqwest, io,
rustls and tungstenite causes and own every message. Callers can no
longer build an error with a message of their own. They ask through
dispatched, is_request, status, upstream_response and tls_source
One From<reqwest::Error> now classifies connect, network and timeout
and strips the URL. The WebSocket handshake reports each failed check
as its own variant and owns its connect timeout. OCR drops the media
variants it mirrored, and inference::Error maps a header the signer
computes to Unsupported for every route, not only chat
The error rules move out of AGENTS.md into the rust-error skill, which
also records the decisions behind this design
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): split rust-error skill into thin SKILL.md with references
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): render causes once and ask predicates at the boundaries
The four WebSocket handshake variants collapse into Handshake(HandshakeFault), and media download failures no longer answer dispatched() since they happen before the provider call
ErrorDetail in llms and auth-types, the inference error and the OCR error now store their cause in a field named cause with no #[source], so to_string() carries it exactly once and source() is None. Tests that walked the chain now destructure the variant and downcast the cause
The gateway asks is_unsupported() and is_missing_credential() on the inference and OCR errors instead of matching through nested enums, and litellm_auth::Error answers is_missing_credential() for both
The rust-error skill gains axum and pyo3 references, boundaries.md describes the real predicate order and the known divergences, the testing example asserts dispatched() correctly, and shape, classification, conversions, messages-and-causes and recipes agree with the code again
* docs(rust): focus the rust-error gateway page on the error flow and add a rust-gateway skill
The former axum page toured gateway-inference: its status match, the envelope
table, an extractor, the unsupported route and the Sse closure. Those details
belong to the gateway crates and go stale with them. The page is now gateway.md
and follows an error from the provider failure through each From into
IntoResponse, where it is classified once and rendered by surface and framing.
The gateway-structure material moves to a new rust-gateway skill: envelopes per
API surface, extractors and path registration, SSE relay, and the router test
harness.
* test(inference): pin provider URL redaction on every route's transport error
Chat, Messages, Responses and transcription each get a route-level test that
fails if a reqwest error reaches the caller with its URL. The Messages stream
decoder also takes body errors through litellm_http::Error like every other
reqwest error
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust-error): adopt std source chains and add litellm-error Report
Messages describe their own layer and the cause is the source, boundaries
render the chain once with litellm_error::Report. Corrects skill claims
the audit found stale against the code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): keep causes as sources across http, auth, secrets, cache and utility crates
Messages describe their own layer and every cause is a typed source,
predicates match exhaustively, and boundaries render with
litellm_error::Report. Media downloads no longer count as dispatched,
Error::upstream bounds the provider body itself, and a forwarded
header SigV4 cannot sign is the caller's fault.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): keep causes as sources in llms and the gateway crates
ErrorDetail loses its string variant, OCR provider variants become data,
host faults stop counting as the caller's, a tunnelled transport error
classifies the same on every stream path, and each gateway crate has one
Error with one status() and extractors that reject in its envelope.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): nest llms errors in inference and ask predicates in route tests
litellm_inference::Error nests litellm_llms::Error instead of mirroring
its variants, host driver faults get their own variant, every predicate
matches exhaustively, and route tests ask predicates instead of matching
transport variants.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): map bridge and gateway-inference errors once and render them with Report
RustUpstreamError has one constructor that always sets headers, argument
checks raise typed errors mapped once, OCR asks upstream_response(), the
Responses stream error event comes from the error type, gateway-inference
loses its string variants, and auth faults classify through one
litellm_auth predicate on every route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-error): carry OCR provider headers as a HeaderMap and keep outages off the caller
The OCR provider error and get_error_class take reqwest's HeaderMap, the
bridge turns it into Python's (name, value) pairs once. Credential
acquisition and provider SDK failures are no longer the caller's fault,
the UI hides auth and backend causes, and chat keeps a provider answer
as is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust-error): correct skill claims against the aligned code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): fix the remaining sweep findings in gateway, llms, auth and cache crates
Auth rejections hide their backend cause on every inference renderer,
path parameters reject in the surface envelope, Textract's unsupported
document and Azure DI's unparseable poll URL get typed variants, the
credential chain exposes its last failure as the source, gateway-mcp
asks predicates over rmcp errors, and the S3 and Azure blob transports
classify through litellm_http::Error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-error): fix the remaining sweep findings on the Python side
host-python keeps a native error's chain and returns blocking-pool
cancellation instead of panicking, Vault transport failures convert
through litellm_http::Error so no URL reaches Python, the bridge asks
predicates instead of matching three levels into Vault errors, and the
last string-built PyErrs become typed errors mapped once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* build(deps): refresh compatible dependencies flagged by OSV
* refactor(rust): remove redundant client construction comment
* test(rust): cover configured MCP proxy and CA bundle
* fix(http): preserve native WebSocket trust and optional protocols
* fix(http): allow plain WebSockets without native roots
* fix(http): normalize provider URLs before building outbound requests
* fix(rust): preserve OCR download status and reduce error size
* fix(rust): preserve upstream bodies and bound diagnostics only
* test(rust): preserve bridge serialization boundary in body regression
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http): honor transport settings for Rust WebSockets
The Responses WebSocket connected through tokio-tungstenite with its own
native-roots TLS, so it ignored the configured CA bundle, verify flag,
client certificate, proxies and timeouts. The handshake now goes through
a pooled reqwest client (ClientVariant::WebSocket, HTTP/1.1, no
redirects) and upgrades the response, so it shares the settings every
other outbound request uses
The upgrade response is checked for the Sec-WebSocket-Accept key, the
Upgrade and Connection headers and a subprotocol that was offered
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(http): preserve native WebSocket trust and optional protocols
* fix(http): allow plain WebSockets without native roots
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http
The workspace moves to reqwest 0.13, so litellm-http no longer carries a
second reqwest just for rmcp. The mcp feature, its pool and its proxy
conversion go away, and the gateway hands rmcp a pooled NoRedirect client
that already carries the TLS, proxy and timeout settings
The Client test helpers collapse into for_test, a no-redirect client
without TLS or proxy settings, and wrap_for_test for the few tests that
need a hand-built reqwest client
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* build(deps): refresh compatible dependencies flagged by OSV
* refactor(rust): remove redundant client construction comment
* test(rust): cover configured MCP proxy and CA bundle
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): name the timeout phase and the limit that fired instead of 'after None seconds'
Every HTTP verb on both handlers raised litellm.Timeout with the per-call
timeout argument interpolated into the message, which is None whenever the
caller relied on the client default, and always said "Connection" even for a
read timeout. One helper now reads the limit httpx actually enforced from the
failed request and names the phase that fired
Fixes#14635
* fix(rust_bridge): name the timeout phase in the vault timeout message to match the httpx handler
* fix(http_handler): drop the unreachable timeout fallbacks httpx already rules out
* refactor(rust_bridge): drop a doc comment that restated the timeout helper
Every other crate names its error Error. The format crates already re-exported
it under that name, so only the inference crate and a few importers change
* refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines
* refactor(rust-bridge): split and document route-specific detectors
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Native Responses now hands back the same managed ids as Python: the
resp_ id carries the provider and deployment, encrypted-content items get
encitem_ ids and wrapped content under affinity routing, and container ids
become cntr_ ids. Encoded encrypted-content items in the input are restored
before the provider request. The codec lives in inference-responses with
goldens taken from litellm/responses/utils.py
* refactor(rust-bridge): let each route host pick the body params
The native routes projected request body params from a name list, hard-coded in
Rust for Messages and read from `PARAMETERS` for Chat Completions and Responses.
Each route host now answers `body_params(request)` over the keywords the call would
send, so the Python side owns which params reach the provider. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-host): use read-only body parameter types
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): move the route host modules under host/routes
Each native route's Python half lived in its own `<route>/route_host.py`, apart
from the shared host modules Rust also imports by name. They now sit together in
`litellm/rust_bridge/host/routes/<route>.py`, with their unit tests moved to the
mirrored path. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-host): reference messages route alias
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python
Rust's run_call built Python streams itself and imported
litellm.rust_bridge.streams by name, so sync calls were driven in Rust
while async calls came back as a Rust-made coroutine. Every native
inference entrypoint now returns one unstarted Execution and
public_execution.py drives it with lifecycle.run or lifecycle.drive,
wrapping streams and _hidden_params on the Python side. Rust only
imports the lifecycle step classes through StepTypes, and
python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone.
The adapter trusts the _native.pyi contract with a single cast instead
of a per-request runtime_checkable isinstance. Binding tests assert the
adapter starts the execution and returns its completed value rather
than checking callable identity.
* refactor(rust-bridge): fold lifecycle and public_execution into one execution module
The public stream wrappers only forwarded to the lifecycle streams and added the
header dict, so the streams now carry _hidden_params themselves. lifecycle.py no
longer held any call lifecycle, just the Execution driver, so it is renamed to
execution.py and absorbs the synchronous and asynchronous adapters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-bridge): keep execution.py free of third-party imports
The host-python tests load execution.py into a bare interpreter, so the
pydantic header validation broke them. The stream head comes from the
binding's encode_stream_head, which always builds a str-keyed dict
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(host-python): preserve pre-async hook coverage after execution handoff
* style(rust-bridge): sort litellm_host_python imports
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* fix(rust-bridge): mark execution TypeVar as Final
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): cover stream close, read failure, and sync suspension in execution
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): drive executions with generator-style send, throw and close
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): name stream steps StreamOpen and StreamChunk and reject out-of-phase steps
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): drive with ExitStack and require the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(host-python): give the hook_chain lifecycle binding the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(host-python): collect host failures in a thiserror Error enum
missing_state() and the inline PyRuntimeError/PyTypeError messages become Error variants with one From<Error> for PyErr, and InvokeError derives thiserror so From<PyErr> comes from #[from]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(host-python): note the canonical HostError import alias in AGENTS.md
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): group the Python modules Rust imports by name under rust_bridge/host
Rust reads Python-owned state (legacy callback registries, model capabilities) through
modules it imports by path. Put them in one folder with an AGENTS.md that states the
contract, and update the module path constants on the Rust side.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust_bridge): route Python state reads through host accessors
* docs(rust_bridge): point test_support at the moved host contract test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): move secret manager, transport and timeout accessors under host
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(greptile): register rust_bridge/host AGENTS.md in files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* refactor(rust): give Messages configs typed litellm params
* feat(rust): port _normalize_system_role_messages for Messages hosts
mid_conversation_system mirrors Python's module function for function: a
leading run of system turns is hoisted into the top-level system, a later
turn stays in place when the model map flags supports_mid_conversation_system
and otherwise becomes a user turn where it was, never between a tool_use and
its tool_result, and billing blocks are stripped from the hoisted system. The
capability joins MessagesModelCapabilities and the bridge projection. Azure AI
uses it in place of fold_system_role_messages, which hoisted every turn.
* fix(rust): keep the Bedrock runtime endpoint behind a blank api_base
* refactor(rust): import LitellmParams from its crate instead of a re-export
* refactor(rust): template the Messages request decode errors
Both decode failures go through ErrorDetail::invalid with the subject and the
serde error as source instead of a preformatted sentence.
* refactor(rust_bridge): read the module globals the param specs name
The Messages host no longer keeps its own list of litellm globals; a spec
whose spellings the call leaves out names the global to read, so the next
family that has one needs no host change.
* refactor(rust): give each route lookup Python's implicit None
ProviderConfigManager's per-route methods list only the providers the route
serves and fall through to None; the Rust lookups now do the same, so a new
LlmProviders variant touches only the route that serves it.
VertexParams in auth-types holds the vertex_* fields of CredentialLiteLLMParams
in both spellings, with one ParamSpec per setting: the wire names, the litellm
module global VertexBase.get_vertex_ai_* consults, and the environment names.
A credential deserializes from text or the JSON object a config spells, an
empty object counts as absent, and both credential fields are redacted in
Debug. auth-gcp is split so lib.rs is the entrypoint (config, auth,
constants), its lookups resolve through the specs, secret_names() derives
from them, and get_vertex_ai_project_from_credentials reads a service
account's project_id for hosts that need it before a token exchange. The OCR
host folds the module globals in by spec at projection, so OcrSettings no
longer carries vertex_project and vertex_location.
* refactor(python-bridge): ship a signature base and read the resolved call
NativeCall carries base (positionals by name plus signature defaults) instead
of the fully bound dict. resolved lays kwargs over base, which is what bound
held, so every pre-hook read and every route host keeps seeing the same values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(bedrock): build the transcription NativeCall with an empty base
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(dispatch): read the resolved call instead of bound
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(host-python): rename effective to effective_py_args and note the shallow copy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): derive strum VariantArray and string conversions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): use strum conversions directly with explicit spellings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): use rstest values for key management systems
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(python-bridge): take NativeCall directly and fold routes into per-route folders
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(python-bridge): use rstest for updated tests and keep embedding's stub parameter name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-bridge): share field and response marshaling
* test(rust-bridge): isolate response factory test modules per case
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust-bridge): document shared marshaling helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): separate Messages settings from capability inputs
* refactor(rust-bridge): unify Messages OCR and Responses call inputs
* refactor(rust-bridge): share NativeCall across inference entrypoints
* fix(rust-bridge): preserve Responses URL aliases and public test inputs
* refactor(rust_bridge): remove rule-gated native secret-manager selection
Mirror the cache treatment: SecretManagerRule/SecretManagerContext and the resolve_native_* binding plumbing are gone. The Rust bridge now selects the native backend from the explicitly configured client (capture_secret_manager / _SecretManagerRuntime.from_client), and get_secret_from_manager is the plain Python handler path.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): bump sharp to 0.35.5 for GHSA-wq5f-xc86-pv6w
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(rust): prune unused inference crate dependencies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): rewrite inference layering docs for the split format crates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): note transcription and RouteError alias exceptions in inference AGENTS.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
