* feat(gateway): Azure OpenAI chat completions through the OpenAI passthrough
Adds azure/<deployment> chat completions, streaming and not, on the same
passthrough the openai/ route uses. The request goes to the deployment
path with the api-version query, or to /openai/v1/ for the v1, latest and
preview versions, with api_base, api_key and api_version falling back to
the AZURE_* variables and os.environ references resolved as the proxy
does. A key is sent as api-key, and an azure_ad_token or an Entra ID
token from auth-azure goes as a Bearer token. The body follows the
proxy's gpt, o-series and gpt-5 param rules, picked by base_model or the
deployment name and reading model capabilities from the cost map, so an
unsupported param is refused unless drop_params allows it to be dropped.
Spend is priced as the proxy prices Azure, by azure/<base_model> when one
is set and otherwise by the model the response names, and a content
filter error keeps Azure's inner error for the client
* feat(gateway): Anthropic and Bedrock chat completions streaming
Adds "stream": true to /v1/chat/completions for anthropic/ and bedrock/
deployments, which were refused with 501 before. The request is the
non-streaming one with "stream": true added for Anthropic and sent to
/converse-stream for Bedrock, and the provider's events are decoded the
way the proxy's chunk parsers decode them: text, tool call deltas with
empty arguments sent as "{}", reasoning_content and thinking_blocks with
their index and signature, citations and the mapped finish_reason. A
wrapper in the shape of the proxy's stream wrapper gives every chunk one
id and the client's model name, sends role on the first chunk, adds the
empty finish chunk, strips usage from the chunks and, when
stream_options.include_usage is set, ends with the usage chunk before
[DONE]. The gateway waits for the first chunk before answering, so an
upstream error up to there is retried or falls back to another
deployment, and an error after it ends the stream with the OpenAI error
frame. Usage, cache reads and writes included, is handed to the meter
through the response whether or not the client asked for it, and
dropping the client stream drops the upstream request
* feat(gateway): full openai chat request translation for anthropic and bedrock (#47088)
* feat(gateway): share router cooldowns and follow proxy key invalidations through Redis
Gateway replicas now share state through the coordination Redis the
Python proxy uses (general_settings.coordination_redis, a redis
response cache, or REDIS_*):
- A deployment one replica cools down is written under Python's
deployment:<id>:cooldown key and applied by every other replica's
background poll (one MGET a second), so the fleet stops sending to a
failing deployment together. allowed_fails counts failures across
replicas, as Python's v1 rule does; the v2 failure-rate rule stays
per replica, as in Python. Deployments are named by model_info.id, or
an id derived from the config that is the same on every replica.
- The virtual key cache follows the proxy's
litellm_proxy.auth_cache_invalidation channel, so a key blocked,
deleted or edited through the admin API (or its team or user) is
re-read on every replica at once instead of after up to 60 s.
general_settings.user_api_key_cache_ttl sets the key TTL.
Nothing is added to the path of a successful request. Without Redis,
or while it is down, each replica keeps its own state as before.
Redis Cluster and Sentinel are refused at startup for now.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): hooks and observers around inference requests, with usage and cost
Adds the litellm-gateway-hooks crate: request facts built only when something
reads them, on-path hooks that declare what they read and what they may
change and run in config order, off-path observers on bounded queues, a
usage tap for SSE and JSON responses, and per-deployment prices from the
model cost map overlaid by config prices. The messages and chat handlers run
through it when anything is installed; with nothing installed requests take
the old path.
The key store now also returns the key alias, user, team alias and key and
team metadata, and router attempts carry the deployment id.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): write spend to the proxy's database from a native observer
Adds a spend writer observer that records every gateway request in the
proxy's own Postgres tables with the same statements the Python proxy
uses: spend logs, key/user/team/team member/end user spend, and the
daily user, team and end-user activity tables. Writes are batched on the
proxy's timers and are additive, so replicas and Python proxies can
write side by side. The database pool is now shared between virtual
keys and the writer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): enforce key, team, user, member and end-user budgets on shared spend counters
Adds a built-in budgets hook that runs the proxy's budget checks, in its
order and with its messages, on the proxy's Redis spend:* counters. Each
request reads and seeds the counters, checks them and reserves its
maximum cost in one Redis script, then settles the reservation to its
real cost off the request path, so replicas and Python proxies sharing
the Redis enforce one budget. Without Redis the counters are kept per
replica. The key store now reads user, team member and end-user budgets,
and auth no longer checks budgets against stored spend.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): enforce key, user, team, member and end-user rate limits on shared counters
Adds a built-in rate limits hook with the proxy's v3 limiter semantics:
requests per window, tokens per window and in-flight requests per key,
on the proxy's own {scope:id}:window|requests|tokens|max_parallel_requests
Redis keys and with its scripts, so replicas and Python proxies sharing
the Redis enforce one limit. Tokens are reserved from the proxy's
estimate and settled to usage when the response ends. Refusals carry the
proxy's message, body and headers on each endpoint, and complete
responses carry its x-ratelimit-* headers. Without Redis the counters are
kept per replica. The key store now reads key, user, team and member
rpm/tpm limits and the key's max_parallel_requests, and end users' budget
limits are read with their budgets; keys with plain rate limits are no
longer refused.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): configurable header, spend metadata and provider metadata hooks
Adds four built-in hooks a config lists under litellm_settings.hooks and
runs in that order: forward_headers copies named client headers to the
provider, strip_headers removes headers from the provider request
(including the gateway's default pass-through), spend_metadata adds
header and key-metadata values to the spend log's spend_logs_metadata,
and provider_metadata sets Messages body fields and headers from
templates over request facts. Hooks' upstream header edits now reach the
provider on both endpoints, and body edits can set nested fields.
Credentials and signer headers are never forwarded or set, and a
misconfigured hook stops the gateway at startup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): model capacity hook, the proxy's dynamic rate limiter
Adds the model_capacity hook, turned on by listing dynamic_rate_limiter_v3
(or model_capacity) in litellm_settings.callbacks. It caps each model
group at the sum of its deployments' rpm and tpm across every replica,
and with priority_reservation holds each priority's pool to its share
once the group's saturation reaches the threshold, counting it below.
It uses the proxy's model_saturation_check and priority_model keys and
scripts, its refusal messages and headers, and adds a successful
response's tokens to the group and the pool admission chose. The rate
limit counters gain a general check-and-add over request and token
pairs, and the throttling refusal shape is shared.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* test(gateway-auth): expect the narrowed unsupported rate limit message
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): prometheus observer with the proxy's metric and label names
Adds the prometheus observer, turned on by listing prometheus in
litellm_settings.callbacks, success_callback or failure_callback. Each
finished request updates the proxy's request, spend, token, latency and
deployment metrics under its names and labels, and /metrics serves them
after the gateway's own. prometheus_metrics_config, the exclude lists
and prometheus_latency_buckets are read as the proxy reads them. The
metrics the gateway already served under the same names come from the
observer alone when it is on. Outcomes now carry when the attempt was
sent upstream, for the LLM API latency and time to first token.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): s3_v2 observer writing the proxy's logging payload to S3
Adds the s3_v2 observer, turned on by listing s3_v2 (or s3_log) in
litellm_settings.callbacks, success_callback or failure_callback. Each
finished request becomes the proxy's StandardLoggingPayload, with the
messages from the request body and the response as the client got it,
streams put back together. It is queued and written every flush interval
under the proxy's key layout, with its Content-MD5, Content-Disposition
and encryption headers, one PUT per request or one NDJSON object per
folder with s3_batch_file_upload. s3_callback_params is read as the
proxy reads it, os.environ references included. Uploads are retried,
dropped and kept for the next flush by the proxy's rules, and what is
queued is written at shutdown. auth-aws gains an S3 signer that signs
the path as sent and every header, as botocore does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138knkqKpkoSr6PApenZan5
* feat(gateway): Azure OpenAI chat completions through the OpenAI passthrough
Adds azure/<deployment> chat completions, streaming and not, on the same
passthrough the openai/ route uses. The request goes to the deployment
path with the api-version query, or to /openai/v1/ for the v1, latest and
preview versions, with api_base, api_key and api_version falling back to
the AZURE_* variables and os.environ references resolved as the proxy
does. A key is sent as api-key, and an azure_ad_token or an Entra ID
token from auth-azure goes as a Bearer token. The body follows the
proxy's gpt, o-series and gpt-5 param rules, picked by base_model or the
deployment name and reading model capabilities from the cost map, so an
unsupported param is refused unless drop_params allows it to be dropped.
Spend is priced as the proxy prices Azure, by azure/<base_model> when one
is set and otherwise by the model the response names, and a content
filter error keeps Azure's inner error for the client
* feat(gateway): Anthropic and Bedrock chat completions streaming
Adds "stream": true to /v1/chat/completions for anthropic/ and bedrock/
deployments, which were refused with 501 before. The request is the
non-streaming one with "stream": true added for Anthropic and sent to
/converse-stream for Bedrock, and the provider's events are decoded the
way the proxy's chunk parsers decode them: text, tool call deltas with
empty arguments sent as "{}", reasoning_content and thinking_blocks with
their index and signature, citations and the mapped finish_reason. A
wrapper in the shape of the proxy's stream wrapper gives every chunk one
id and the client's model name, sends role on the first chunk, adds the
empty finish chunk, strips usage from the chunks and, when
stream_options.include_usage is set, ends with the usage chunk before
[DONE]. The gateway waits for the first chunk before answering, so an
upstream error up to there is retried or falls back to another
deployment, and an error after it ends the stream with the OpenAI error
frame. Usage, cache reads and writes included, is handed to the meter
through the response whether or not the client asked for it, and
dropping the client stream drops the upstream request
* feat(gateway): full OpenAI chat request translation for Anthropic and Bedrock
Anthropic and Bedrock Converse deployments now take the whole OpenAI chat request the
Python proxy accepts instead of a narrow allowlist. Messages translate as Python builds
them: system as a string, parts or several messages, text, image_url (URL or data URL),
file and document parts, assistant tool_calls, the tool and function roles, consecutive
role merging, empty content, cache_control on Anthropic and cachePoint on Bedrock.
Params map per provider: max_tokens and max_completion_tokens with Anthropic's model
default, temperature, top_p, stop, tools and tool_choice, parallel_tool_calls,
response_format as a json tool or native output format, reasoning_effort and thinking,
user as metadata, with n and any other unsupported param refused with a 400 unless
drop_params is set globally, on the deployment or on the request, and
allowed_openai_params and additional_drop_params honoured. Responses carry tool calls,
thinking blocks and response_format answers back in OpenAI shape, streamed and not. The
route downloads media URLs a provider cannot take by reference, merges the beta headers
a request needs with forwarded ones, and the gateway hands the cost map to both
providers for the capability rules that read it.
* fix(rust-bridge): read Python's model_cost for chat translation capabilities
Chat translation for Anthropic and Bedrock reads capability flags from the
cost map. The Python bridge passed none, so every SDK chat call through the
Rust path failed. It now reads litellm.model_cost live, so lookups match
Python's, runtime-registered models included. ModelInfo::entry returns a Cow
so a map read from Python can hand over a copy.
Two tests sent temperature != 1 to a model that removed sampling params,
which Python itself refuses; they now send temperature=1.
* fix(model-catalog): read token counts the cost map writes as whole floats
The published cost map now has prompt_cache_min_tokens: 4096.0, which the
typed u64 fields rejected, so the gateway dropped the whole map at startup
and priced every request at zero. Counts accept any whole non-negative
number, as Python reads any JSON number, and a test keeps the checked-in
map parsing.
* feat(gateway): Azure OpenAI chat completions through the OpenAI passthrough (#47074)
Adds azure/<deployment> chat completions, streaming and not, on the same
passthrough the openai/ route uses. The request goes to the deployment
path with the api-version query, or to /openai/v1/ for the v1, latest and
preview versions, with api_base, api_key and api_version falling back to
the AZURE_* variables and os.environ references resolved as the proxy
does. A key is sent as api-key, and an azure_ad_token or an Entra ID
token from auth-azure goes as a Bearer token. The body follows the
proxy's gpt, o-series and gpt-5 param rules, picked by base_model or the
deployment name and reading model capabilities from the cost map, so an
unsupported param is refused unless drop_params allows it to be dropped.
Spend is priced as the proxy prices Azure, by azure/<base_model> when one
is set and otherwise by the model the response names, and a content
filter error keeps Azure's inner error for the client
* feat(gateway): Anthropic and Bedrock chat completions streaming (#47077)
Adds "stream": true to /v1/chat/completions for anthropic/ and bedrock/
deployments, which were refused with 501 before. The request is the
non-streaming one with "stream": true added for Anthropic and sent to
/converse-stream for Bedrock, and the provider's events are decoded the
way the proxy's chunk parsers decode them: text, tool call deltas with
empty arguments sent as "{}", reasoning_content and thinking_blocks with
their index and signature, citations and the mapped finish_reason. A
wrapper in the shape of the proxy's stream wrapper gives every chunk one
id and the client's model name, sends role on the first chunk, adds the
empty finish chunk, strips usage from the chunks and, when
stream_options.include_usage is set, ends with the usage chunk before
[DONE]. The gateway waits for the first chunk before answering, so an
upstream error up to there is retried or falls back to another
deployment, and an error after it ends the stream with the OpenAI error
frame. Usage, cache reads and writes included, is handed to the meter
through the response whether or not the client asked for it, and
dropping the client stream drops the upstream request
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(gateway): gemini chat completions on google ai studio and vertex ai (#47098)
* feat(gateway): Gemini chat completions on Google AI Studio and Vertex AI
gemini/ deployments and vertex_ai/ deployments of Gemini models now serve
/v1/chat/completions, streamed and not, instead of being refused. Both surfaces share
one translation ported from the Python VertexGeminiConfig. Params map as Python maps
them: temperature, top_p, max_tokens and max_completion_tokens, n, stop, penalties
where the model takes them, seed, logprobs, tools with function declarations and the
built in search, code execution, url context, maps and computer use tools,
tool_choice, response_format as a response schema, reasoning_effort and thinking as a
thinking config, modalities, audio and labels. Messages become contents with a system
instruction when the cost map allows it, images, files, audio, function calls and
responses, and thought signatures carried back on tool call ids
AI Studio sends the key in x-goog-api-key from the deployment or GOOGLE_API_KEY,
GEMINI_API_KEY or PALM_API_KEY, and calls v1alpha for Gemini 3 and v1beta otherwise.
Vertex AI calls the project and location URL with a forwarded bearer or a Google
access token from vertex_credentials, the environment or ADC, and grafts the path onto
a custom api_base. Responses carry text, reasoning, tool calls, grounding annotations,
logprobs and the vertex_ai metadata, a blocked prompt or flagged answer comes back as
content_filter, and usage counts thoughts and cached tokens as Python does so cost is
recorded. The gateway resolves os.environ references in the deployment key, base and
Vertex params, passes the deployment Vertex params to the call, and loads the cost map
for gemini and vertex_ai deployments. Claude and partner models on Vertex AI stay
refused
* test(gateway): gemini cost map test double returns a Cow entry
* feat(gateway): native chat for openai-compatible providers (#47099)
Serve chat completions for hosted_vllm, nvidia_nim, groq, together_ai,
fireworks_ai, xai, mistral, deepseek, openrouter, cerebras and perplexity
through the OpenAI passthrough, with each provider's Python config ported
as a request transform in llms and its response and stream fixups applied
on the way back.
Streams ask for stream_options.include_usage on the client's behalf where
the provider takes it, record the usage through ReportedUsage for the spend
hooks, and keep it from the client unless the client asked for it.
* feat(gateway): claude chat completions on vertex ai (#47101)
vertex_ai/ deployments of Claude models now serve /v1/chat/completions,
streamed and not, instead of being refused. The request is the Anthropic
chat translation built for the vertex_ai host, as Python's
VertexAIAnthropicConfig builds it on top of AnthropicConfig: capabilities
read the vertex_ai/ cost map entry first and native structured output
reads only that entry, service_tier and speed are refused or dropped with
drop_params, and the Anthropic API betas are not added. The body drops
model, carries anthropic_version vertex-2023-10-16, folds output_format
into output_config and keeps effort only for models that take it, and
names its context management and caller betas in anthropic_beta and the
anthropic-beta header, filtered to what Vertex accepts, with the web
search beta alone when there are none.
The call goes to the publisher rawPredict URL for the project and
location, regional or global, or a custom api_base or VERTEXAI_API_BASE,
with a Google access token from vertex_credentials, the environment or
ADC. Streams call streamRawPredict with stream true, alt=sse on a custom
base, decode the Anthropic events as the anthropic/ path does and hand
usage to the hooks through ReportedUsage. Every remote image is
downloaded first, since Vertex takes none by URL, and the response names
the requested model.
* fix(gateway): find anthropic stream usage by event name (#47206)
The stream meter kept a data line only when "message_start" or
"message_delta" appeared in its first 96 bytes. An upstream that puts
"type" after the message object (JSON key order is not fixed) had every
message_start skipped, so input tokens were recorded as 0 and spend,
budgets, rate limits, metrics and S3 logs undercounted.
Anthropic streams name every event, so the event: line now decides; the
byte-window check remains for streams without event lines.
* fix(gateway): stop hooks pinning connection read buffers for a stream's life (#47220)
The s3 logger asks for the response text, and the tap kept every
response chunk as a Bytes slice of the upstream connection's read
buffer, so each open stream held a buffer per chunk until it ended
(about 460 KB per open 10k-token stream locally). Chunks are now copied
into one owned buffer.
The same applies on the request side: the facts kept the raw request
slice and a clone of the client headers, both slices of the client
connection's read buffer. The body is now released once hooks have run
(an owned copy is kept only when an observer reads it) and the headers
are copied.
* feat(loadtest): full-feature mode with recorded-data verification (#47207)
* feat(loadtest): full-feature mode with recorded-data verification
genconfig -features full turns on what both gateways do per request:
owners with rpm/tpm/budgets/max_parallel, deployment capacity, the
dynamic_rate_limiter_v3, prometheus and s3_v2 callbacks, and the four
request hooks with Python stand-ins (custom_callbacks.py). -key-sets
gives each gateway its own keys, users and teams in a shared database.
verify checks spend logs, owner spend, daily tables, Prometheus counters,
S3 logs and the headers the provider received against what loadgen sent,
and compares per-deployment prices across sides. s3stub serves S3 for
the logs. mockllm takes in= for exact input tokens, returns unique
response ids, and counts chosen request headers; loadgen takes -keys
and -header.
* feat(loadtest): s3stub -max-string keeps long runs' logs in memory
* feat(loadtest): bifrost as a third side of the full-feature run (#47236)
genconfig writes a "bifrost" key set as a Bifrost config: a custom provider
per deployment, a routing rule per model group, virtual keys, and with
-features full key/team/customer budgets and rate limits, model configs for
deployment limits, the logs store with S3 payloads, the telemetry plugin and
header forwarding/stripping/logging. A pricing file copies the cost map's
prices per deployment.
mockllm answers Bedrock Converse (streamed and not), serves a self-signed
TLS listener for Bifrost's HTTPS-only Bedrock provider, and echoes the
request's model, which Bifrost prices by. s3stub answers HeadBucket.
loadgen takes -chat-path and -messages-path for Bifrost's
/anthropic/v1/messages.
verify reads a side named bifrost in Bifrost's schema and reports what it
has no counterpart for (users, members, daily tables, provider_metadata,
cross-replica state) as n/a; with three sides it compares every pair.
Claude-Session: https://claude.ai/code/session_01XF7rGf49TWYH5Hqkb14eSL
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* build(deps): bump aiohttp to 3.14.5 to clear osv-scan (#47369)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): let each route host pick the body params
The native routes projected request body params from a name list, hard-coded in
Rust for Messages and read from `PARAMETERS` for Chat Completions and Responses.
Each route host now answers `body_params(request)` over the keywords the call would
send, so the Python side owns which params reach the provider. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-host): use read-only body parameter types
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): move the route host modules under host/routes
Each native route's Python half lived in its own `<route>/route_host.py`, apart
from the shared host modules Rust also imports by name. They now sit together in
`litellm/rust_bridge/host/routes/<route>.py`, with their unit tests moved to the
mirrored path. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-host): reference messages route alias
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python
Rust's run_call built Python streams itself and imported
litellm.rust_bridge.streams by name, so sync calls were driven in Rust
while async calls came back as a Rust-made coroutine. Every native
inference entrypoint now returns one unstarted Execution and
public_execution.py drives it with lifecycle.run or lifecycle.drive,
wrapping streams and _hidden_params on the Python side. Rust only
imports the lifecycle step classes through StepTypes, and
python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone.
The adapter trusts the _native.pyi contract with a single cast instead
of a per-request runtime_checkable isinstance. Binding tests assert the
adapter starts the execution and returns its completed value rather
than checking callable identity.
* refactor(rust-bridge): fold lifecycle and public_execution into one execution module
The public stream wrappers only forwarded to the lifecycle streams and added the
header dict, so the streams now carry _hidden_params themselves. lifecycle.py no
longer held any call lifecycle, just the Execution driver, so it is renamed to
execution.py and absorbs the synchronous and asynchronous adapters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-bridge): keep execution.py free of third-party imports
The host-python tests load execution.py into a bare interpreter, so the
pydantic header validation broke them. The stream head comes from the
binding's encode_stream_head, which always builds a str-keyed dict
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(host-python): preserve pre-async hook coverage after execution handoff
* style(rust-bridge): sort litellm_host_python imports
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* fix(rust-bridge): mark execution TypeVar as Final
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): cover stream close, read failure, and sync suspension in execution
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): drive executions with generator-style send, throw and close
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): name stream steps StreamOpen and StreamChunk and reject out-of-phase steps
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): drive with ExitStack and require the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(host-python): give the hook_chain lifecycle binding the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(host-python): collect host failures in a thiserror Error enum
missing_state() and the inline PyRuntimeError/PyTypeError messages become Error variants with one From<Error> for PyErr, and InvokeError derives thiserror so From<PyErr> comes from #[from]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(host-python): note the canonical HostError import alias in AGENTS.md
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): group the Python modules Rust imports by name under rust_bridge/host
Rust reads Python-owned state (legacy callback registries, model capabilities) through
modules it imports by path. Put them in one folder with an AGENTS.md that states the
contract, and update the module path constants on the Rust side.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust_bridge): route Python state reads through host accessors
* docs(rust_bridge): point test_support at the moved host contract test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): move secret manager, transport and timeout accessors under host
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(greptile): register rust_bridge/host AGENTS.md in files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* refactor(rust): give Messages configs typed litellm params
* feat(rust): port _normalize_system_role_messages for Messages hosts
mid_conversation_system mirrors Python's module function for function: a
leading run of system turns is hoisted into the top-level system, a later
turn stays in place when the model map flags supports_mid_conversation_system
and otherwise becomes a user turn where it was, never between a tool_use and
its tool_result, and billing blocks are stripped from the hoisted system. The
capability joins MessagesModelCapabilities and the bridge projection. Azure AI
uses it in place of fold_system_role_messages, which hoisted every turn.
* fix(rust): keep the Bedrock runtime endpoint behind a blank api_base
* refactor(rust): import LitellmParams from its crate instead of a re-export
* refactor(rust): template the Messages request decode errors
Both decode failures go through ErrorDetail::invalid with the subject and the
serde error as source instead of a preformatted sentence.
* refactor(rust_bridge): read the module globals the param specs name
The Messages host no longer keeps its own list of litellm globals; a spec
whose spellings the call leaves out names the global to read, so the next
family that has one needs no host change.
* refactor(rust): give each route lookup Python's implicit None
ProviderConfigManager's per-route methods list only the providers the route
serves and fall through to None; the Rust lookups now do the same, so a new
LlmProviders variant touches only the route that serves it.
VertexParams in auth-types holds the vertex_* fields of CredentialLiteLLMParams
in both spellings, with one ParamSpec per setting: the wire names, the litellm
module global VertexBase.get_vertex_ai_* consults, and the environment names.
A credential deserializes from text or the JSON object a config spells, an
empty object counts as absent, and both credential fields are redacted in
Debug. auth-gcp is split so lib.rs is the entrypoint (config, auth,
constants), its lookups resolve through the specs, secret_names() derives
from them, and get_vertex_ai_project_from_credentials reads a service
account's project_id for hosts that need it before a token exchange. The OCR
host folds the module globals in by spec at projection, so OcrSettings no
longer carries vertex_project and vertex_location.
* refactor(python-bridge): ship a signature base and read the resolved call
NativeCall carries base (positionals by name plus signature defaults) instead
of the fully bound dict. resolved lays kwargs over base, which is what bound
held, so every pre-hook read and every route host keeps seeing the same values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(bedrock): build the transcription NativeCall with an empty base
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(dispatch): read the resolved call instead of bound
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(host-python): rename effective to effective_py_args and note the shallow copy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): derive strum VariantArray and string conversions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): use strum conversions directly with explicit spellings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): use rstest values for key management systems
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(python-bridge): take NativeCall directly and fold routes into per-route folders
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(python-bridge): use rstest for updated tests and keep embedding's stub parameter name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust-bridge): share field and response marshaling
* test(rust-bridge): isolate response factory test modules per case
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust-bridge): document shared marshaling helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): separate Messages settings from capability inputs
* refactor(rust-bridge): unify Messages OCR and Responses call inputs
* refactor(rust-bridge): share NativeCall across inference entrypoints
* fix(rust-bridge): preserve Responses URL aliases and public test inputs
* refactor(rust_bridge): remove rule-gated native secret-manager selection
Mirror the cache treatment: SecretManagerRule/SecretManagerContext and the resolve_native_* binding plumbing are gone. The Rust bridge now selects the native backend from the explicitly configured client (capture_secret_manager / _SecretManagerRuntime.from_client), and get_secret_from_manager is the plain Python handler path.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): bump sharp to 0.35.5 for GHSA-wq5f-xc86-pv6w
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(rust): prune unused inference crate dependencies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): rewrite inference layering docs for the split format crates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): note transcription and RouteError alias exceptions in inference AGENTS.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): rename litellm-core to litellm-inference
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): expose inference base API for format crates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): move shared inference test helpers behind test-support
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The native response-cache runtime attached through Cache._native_cache is
unreachable since the V2 cache replaced it. Drop the Python branches and
wrapper, the _ResponseCacheRuntime pyclass and its backend/activation/
semantic modules, the python-bridge deps only they used, and the tests and
fixtures dedicated to that path.
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): track ClickHouse migrations in a checksummed ledger
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(clickhouse): harden migration execution and reuse retention SQL
* refactor(rust): track ClickHouse migrations in a checksummed ledger
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): keep ClickHouse retention TTLs in the current policy list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(lens): own trace reads behind a cached TraceStore port
Move storage-independent trace reads into litellm-traces-cache behind a
TraceStore port that ClickHouse implements. One keyset pager drives the
span, list span and spend reads, and a run list batch reads spend once.
Trace opens, pages and list summaries share one resolved read per trace
in an in-process cache with single-flight loading. Live traces and reads
with unknown spend expire after 5s, quiet traces after 10 minutes, failed
reads are never cached, and the accepted list page size is remembered per
scope.
Trace read failures map to their own status and code (400, 409, 413, 503
with Retry-After), and the trace drawer retries temporary failures while
offering only a refresh for changed or oversized traces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* perf(lens): seed large profiles with server-side copies and long sessions
Replay one copy through the proxy, then copy it inside ClickHouse and
PostgreSQL with INSERT ... SELECT, rewriting trace, span and call IDs so
every copy keeps its own spend. Add three long single-trace sessions for
drawer paging and the oversized read path
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chores
* style(lens): float the investigation setup badge on the tab edge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(lens): restructure the trace drawer and polish its layout
Split the 547-line TraceDrawer into run/, tree/, span/, content/ and
conversation/ modules. Step rows now sit on one line with colored span
family tiles, and the per-row timing bar moved into an optional Waterfall
layout with a time axis. The steps and details panes are separated by the
shadcn Resizable handle, with the split remembered per orientation.
Span payloads go through one pure classifier (payloadView) that picks
messages, a tool result, a nested field tree or text. JSON-encoded field
values unfold into a tree, prose renders as markdown, repr and tracebacks
stay monospace, and every section offers a Raw view. LangChain's
serialized messages now render as conversation cards.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* typesafety
* wip
* fmt
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: seed Lens dev with configurable load profiles
* feat: run Lens UI live through the dev launcher
* fix: verify Lens UI startup before seeding
* fix(ui): render run timestamps on one compact line
The agent runs table printed the long locale form with timezone, which
wrapped to two lines per row. Use a fixed-width 24h form with
milliseconds that matches the timeline axis, keep the long form in the
hover title, and show the timezone once in the column header
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(ui): use the root query client for the Lens demo
Drop the demo's nested QueryClient. Cache keys are already partitioned by
scope, and the root client now skips retries on 4xx ApiErrors, which covers
the demo's not-in-demo and read-only rejections and live 4xx alike.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: drop unused synthetic_spend reference from query help
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(traces): drop the deeplite fixtures and map every SDK to a logo
The deeplite captures predate the example repository and carried
synthetic spend rows, which leaked a fixture-only column into the
query help SQL and pinned tests to its shape. Replace them with the
SDK captures in the ClickHouse round trip and query API tests, and
derive the seed tenant lookup from the capture metadata
The runs table only knew the two Anthropic framework slugs. Register
the slugs the normalizer emits for LangChain, LangGraph, Deep Agents,
CrewAI, Google ADK, LlamaIndex, OpenAI Agents, Pydantic AI, Strands,
Vercel AI SDK, Codex, Cursor and Copilot, and fall back to the generic
agent glyph when a run has no known framework
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(ui): inject the Lens sample through data sources instead of demo checks
Components no longer ask whether they are in the demo. The traces source
carries live and handoff, navigation state comes from a URL or memory
route, and the preview action comes from context instead of onDemo props
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(ui): infinite scroll for the agent runs list
Replace the Load more button with a sentinel that fetches the next
cursor page as the list nears its end. Placeholder rows hold the tail
while more runs exist, and a failed page stops auto-loading until Retry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(ui): keep the whole Lens view in the URL
Lens navigation now lives entirely in query params through nuqs: the
sample session (demo=true, with a Demo data switch in the header), the
open run (trace, trace_ref), the selected step, view and detail section
(span, view, span_tab) and the list filters and range (q, agent, status,
hours). Any Lens view is a shareable link and the back button walks runs
RunView takes its selection injected: the drawer feeds it URL state and
the investigations evidence sheet keeps a local one, so a finding's
original run never writes step ids into the URL. Leaving the sample
session clears every Lens key except the tab so sample ids never point
at live data
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* style(traces): cargo fmt captures tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(dev): seed linked tracing and spend fixtures
* chore(dev): use OpenAI model in tracing config
* chore(dev): align tracing credentials with UI E2E
* fix(dev): update fixture seeder query scope
* feat(dev): seed linked tracing and spend fixtures
* chore(dev): use OpenAI model in tracing config
* chore(dev): align tracing credentials with UI E2E
* fix(dev): update fixture seeder query scope
* wip
* wip
* wip
* chore(trace): checkpoint ongoing Rust migration
* refactor(trace): group Python bridge under trace package
* refactor(traces): read span conventions through a Convention trait
Each span format (Claude Code, LangSmith, OpenInference, gen_ai) now lives under
normalize/convention/ as a unit struct implementing Convention, owning both its
detection and its extraction. Precedence is one ordered registry instead of an
if-chain in mod.rs that reached into each module differently.
The modules now share one way to read attributes: present() for the first
non-empty key and Payload for a text that also reports the key it consumed,
replacing three different idioms and the &mut Vec threaded through payload
readers. Instrumentation::adjust returns a new Extraction instead of mutating
one, with each SDK rule as its own function, and the LangChain middleware
suffix list exists once.
* feat(trace): export Rust-owned wire schemas and enforce contract bounds
* fix(trace): bound quoted counts in ClickHouse wire schemas
* feat(trace): generate Python wire contracts with datamodel-code-generator
* test(trace): validate migrated callers and generated contracts at the native boundary
* refactor(traces): rename normalization convention to format
* fix(traces): reconcile spend evidence and preserve unknown costs
* feat(traces): normalize additional telemetry formats
* test(traces): cover captured normalization fixtures
* refactor(traces): isolate SDK normalization rules
* feat(tracing): seed all trace exports for local dashboard
* fix(clickhouse): preserve custom LiteLLM request metadata
* docs(traces): define normalization module boundaries
* docs(traces): define resolution and OTLP boundaries
* fix(ui): normalize nullable trace message names
* refactor(traces): split resolver modules and cover resolution behavior
* test(traces): replace normalization snapshots with behavior assertions
* fix(ui): align dashboard API contracts with generated types
* refactor(traces): type normalization and storage boundaries
* fix(traces): seed captured SDK spend and preserve provider identities
* wip
* test(traces): verify guide discovery and content ordering
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* wip
* wip
* test(traces): separate root status from diagnostic error counts
* test(traces): cover normalization precedence and fallbacks
* chore(cache): remove stray comments from trace PR
* test(traces): name lens test for shared query path
* fix(traces): place query implementation before test module
* test(traces): use unified read scope in migration tests
* ci(rust): allow feature checks to finish
* ci(mcp): allow dependency resolution to finish
* fix(traces): preserve key visibility and safe spend attribution
* fix(tracing): use ClickHouse URL for reads by default
* fix(tracing): unify ClickHouse storage configuration
* fix(tracing): update dashboard setup copy for one URL
* test(tracing): make tests/unit/tracing a package
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(config): drop legacy string tracing store variant
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tracing): own ClickHouse defaults in constants and reject unset env references
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(tracing): use raw regex patterns in config tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tracing): read ClickHouse env defaults when tracing config resolves
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): split audit log query guard to fit condition-chain budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(tracing): add SQL queries and schema-aware query help
* test(tracing): verify help requests and sync API types
* refactor(tracing): render query help with Askama
* refactor(tracing): use jinja extension for query guide
* fix(tracing): preserve query help when discovery fails
* feat(tracing): enforce team SQL scope with managed ClickHouse readers
* test(tracing): verify reads with one ClickHouse URL
* fix(tracing): revoke rotated trace reader credentials
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(tracing): streamline query help catalog assembly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tracing): run query help discovery sequentially
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(tracing): update reader setup request expectations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(tracing): bring current ingestion prerequisite onto main
Port the prerequisite implementation from BerriAI/litellm#43915 at 5aacd57455 so Lens does not depend on the retired tracing stack.
* feat(lens): add trace analysis and standalone worker
* fix(lens): clarify review limits and finalize main integration
* fix(lens): simplify worker setup and show the next check
* fix(lens): simplify analyzer setup and resolve integration failures
* fix(lens): preserve durations and evidence from later trace reads
* fix(lens): trust server context for internal analysis exclusion
* fix(lens): pin reviewed analyzer image and verify request inclusion
* test(lens): select time units before entering custom duration
* test(lens): allow the standalone analyzer lifetime HTTP client
* test(lens): run analyzer tests in active proxy coverage shard
* wip
* feat(traces): establish shared Rust storage foundation
* fix(traces): escape ClickHouse text parameters
* test(traces): exercise response cap with bounded strings
* fix(traces): remove unnecessary lint expectation
* fix(traces): encode ClickHouse timestamp units in Rust
* test(traces): mark exception match as a regex
* refactor(traces): execute schema setup in Rust
* refactor(traces): use shared logging execution wrapper
* docs(traces): replace foundation README with boundary rules
* fix(traces): use current bridge execution facade
* fix(traces): account for protocol cast in lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): extract litellm-host-native as the shared Rust host driver
Move service and hook dispatch out of host-http into a Driver that owns the
machine and Rust handlers, returning at completion or a stream boundary and
holding the demand reply until the consumer advances. Move the in-process
runner onto the same driver. host-http now layers encoding, SSE, body polling
and lifecycle observation over it. host-python keeps driving litellm-host
directly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): interrupt the machine when the in-process stream consumer fails
Restores the pre-refactor interruption path for StreamConsumer errors via
Driver::fail and ports the generic run lifecycle tests into host-native.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): separate the machine contract from coroutine execution
* auth update
* refactor(rust): use standard flow control for host requests
* style(rust): keep host driver imports formatted
* chores
* mostly relocation
* refactor(rust): separate interceptors from queued observers
* refactor(rust): centralize legacy callback mappings and lifecycle
* docs: define Python host boundaries and migration plan
* refactor: enforce Python host and bridge boundaries
* refactor(rust): separate operations from callback composition
* refactor(rust): compose SDK policy through call hooks
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
