Native Responses now hands back the same managed ids as Python: the
resp_ id carries the provider and deployment, encrypted-content items get
encitem_ ids and wrapped content under affinity routing, and container ids
become cntr_ ids. Encoded encrypted-content items in the input are restored
before the provider request. The codec lives in inference-responses with
goldens taken from litellm/responses/utils.py
* fix(databricks): use the native Anthropic route for Claude /v1/messages
Claude models on a Databricks workspace were bridged through chat completions on /v1/messages, which drops the safeguards field and never returns safeguard_results. They now call the workspace's /serving-endpoints/anthropic/v1/messages, the proxy forwards the client's anthropic-beta and anthropic-version headers to Databricks deployments, and the beta header map marks dangerous-tool-use-2026-09-03 as supported there. Any route whose platform does not take that beta drops safeguards together with it instead of answering 400. Non-Claude Databricks models keep the chat completions bridge
* fix(databricks): derive the dangerous-tool-use beta from safeguards instead of forwarding client betas
A Databricks Claude serving endpoint answers 400 to betas Claude Code sends on every turn, prompt-caching-scope-2026-01-05 among them, so the proxy no longer forwards the client's anthropic-beta header to Databricks deployments. The beta the classifier needs is derived from the safeguards body field, the same way the Azure AI route derives it. The Rust beta-header mirror stops rejecting the Databricks dangerous-tool-use cell so its config-cell test matches the JSON again
* fix(databricks): keep the billing block, build the messages route once, and pass the partner user agent
* fix(databricks): route Unity Catalog Claude names to the gateway messages route and keep env credentials
Unity Catalog model-service names now reach the AI Gateway's native Anthropic
route instead of a serving-endpoints path that answers 404 for them, and the
native route resolves the key and base through the same order the chat route
uses, so environment-only deployments keep working.
The native route is limited to Databricks foundation-model Claude names, an
unlisted provider forwards safeguards and the beta as sent, and the ten betas
the workspace rejects are marked unsupported for Databricks in the beta map
and its Rust mirror.
* test(rust): expect Databricks to drop the advanced-tool-use beta
* fix(databricks): keep passthrough safeguards, partner user agent, and classifier fallback
Only the OpenAI-compatible JSON providers drop safeguards now, so a
deployment opted into /v1/messages passthrough forwards the field and
the beta as sent. When a route drops safeguards and the proxy has a
classifier model, the proxy answers the verdict itself, as the chat
completions bridge did. The Databricks Messages route also reads the
databricks_user_agent param like the chat route.
* fix(databricks): drop rejected server tools under drop_params, short-circuit web search, route UC names from a full Messages URL
* fix(databricks): drop container on the native Messages route, as the chat completions bridge did
* fix(databricks): keep every api_base on a route the workspace serves and gate the derived beta on the map
Gate the dangerous-tool-use beta derivation on the beta map the way #43955 does, so GitHub Copilot, DeepSeek, and opt-in passthroughs keep the merge base wire. Recognize /serving-endpoints/anthropic api_bases, fall back to the merge base routing for an api_base path the workspace trim does not know, strip Bedrock invocation metrics from the streamed message_stop, forward client betas to Databricks filtered by the map, and keep tool_choice when a dropped server tool has no name
* test(databricks): cover the native Messages route, derived beta and JSON provider safeguards drop on the wire
* test(anthropic): expect the dangerous-tool-use beta derived from safeguards next to the client's betas
* fix(databricks): send client anthropic headers only on the native Messages route and keep SDK auth
Databricks chat completions, called directly or through the Messages bridge,
no longer receive the client's anthropic-beta and anthropic-version, as at the
merge base. The native Messages route still forwards them, filtered by the
beta map, without counting them as caller credentials, so a deployment with no
api_key keeps falling back to Databricks SDK auth. A pathless api_base goes to
the native route only for Unity Catalog names, matching the chat route.
* test(databricks): cover SDK auth with client anthropic headers and a Unity Catalog workspace root on the native route
* fix(databricks): keep client anthropic headers off an opted-in passthrough api_base
A databricks Claude deployment whose api_base is no workspace route and that opts
into /v1/messages with supported_endpoints falls back to the OpenAI-like
passthrough. That config does not filter betas through the beta map, yet it
received the native-only Databricks header entry, so Claude Code's
prompt-caching-scope beta reached the upstream raw and got a 400 where the merge
base answered 200.
Each Messages config now says whether it is the provider's native route, the
OpenAI-like passthrough says no, and the handler passes that answer when it
reads provider-specific headers, so native-only entries reach only native configs.
* test(databricks): cover client anthropic betas on an opted-in passthrough api_base
A databricks Claude deployment whose api_base is no workspace route and whose
model_info.supported_endpoints opts into /v1/messages takes the OpenAI-like
passthrough. The cell sends Claude Code's prompt-caching-scope and context-1m
betas through the proxy and asserts the caller gets the upstream's message and
the gateway receives no anthropic-beta, as on the merge base. The scripted
gateway answers 400 on the prompt-caching-scope beta, as the real one does
* fix(anthropic): keep beta map loads off shared threads and build the safeguards evaluator only when the proxy classifies
Unfiltered configs such as first-party Anthropic never read the beta map to decide safeguards support. Filtered configs await the map on its dedicated loader before reading it, on both the native and the MCP paths. The native handler decides after the upstream call and builds the evaluator only when the proxy classifies, so malformed messages keep the upstream 400. The Databricks dispatch tests assert the dispatched route instead of the config class, and the sail and Eden AI wire rows expect safeguards dropped, as JSON providers do.
* test(databricks): cover a loaded beta map that marks the safeguards beta unsupported
A remote beta map that nulls dangerous-tool-use for databricks while the bundled map allows it must send neither safeguards nor the beta to the serving endpoint, with or without a client beta, and the caller gets a 200 with no verdict.
* test(anthropic): annotate the MCP beta map recordings as Final
* fix(anthropic): drop safeguards on routes without the classifier only under drop_params
A route whose upstream does not answer safeguards (a JSON provider, or a map
that marks the dangerous-tool-use beta unsupported) now forwards the field and
the beta as the merge base did unless drop_params is set or the proxy runs its
own classifier, matching how litellm treats any unsupported param. Databricks
keeps dropping both, since its merge base bridge never forwarded the field.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(openinfer): add OpenInfer as OpenAI-compatible provider
Register OpenInfer in the JSON provider registry with catalog models,
proxy create fields, and unit coverage so requests route via openinfer/
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openinfer): assert catalog invariants instead of vendor rates
Pinning per-token prices made CI fail when the vendor list changed. Keep
the backup agreement and positivity checks our code owns.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(openinfer): add openinfer to the Rust OpenAI-compatible provider mirror
* fix(openinfer): align catalog with OpenInfer's live /v1/models
Replace the four catalog rows with the two models OpenInfer serves and
publishes a price for (@oi/GLM-5.3-Flash, @oi/Qwen3.5-27B), using its
listed per-token rates and context windows, and price-check them in tests
* fix(openinfer): use OpenInfer's native Responses and Messages APIs, allow tools on unpriced models, price Qwen3.5-9B
Declare /v1/responses and /v1/messages for openinfer, since OpenInfer serves both natively. Let JSON providers declare provider-wide capabilities in providers.json, read through get_provider_info, so tool calls reach OpenInfer for models without a cost map row. Send max_completion_tokens as is, set output caps to each model's context window, mark the three rows as reasoning models, add Qwen3.5-9B at its listed rate, and wire openinfer into models_by_provider so the openinfer/* wildcard lists its models.
* fix(openinfer): let a model's own capability row win over provider defaults in get_model_info
Provider defaults now only fill capability fields the cost-map row leaves unset, so a row or deployment that turns function calling off stays off. Also marks openinfer_models as Final.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards
* test(rust): check the dangerous-tool-use beta reaches the wire with safeguards
* fix(rust): derive the dangerous-tool-use beta only for hosts with a beta map column
DeepSeek and the other hosts without a column in anthropic_beta_headers_config.json
keep forwarding safeguards and client betas as sent. Adds the wire-level audit tests.
* fix(rust): add the safeguards beta on Azure AI and Claude Platform hosts
Azure AI and Claude Platform Messages transforms now add the
dangerous-tool-use beta for a non-null safeguards field, through the
Azure AI and Anthropic beta-map columns. The safeguards wire and chaos
tests get unique file names so they do not collide with the Python
fix's tests, the Python-route test asserts the fixed behavior behind a
BUG skip, and the DeepSeek test checks that no beta is added
* fix(rust): keep Claude Platform out of the safeguards beta until a route can prove it
Claude Platform has no beta-map column and no QA account, so it goes back to sending
safeguards with no derived beta, like DeepSeek. Also release held chaos-test requests
before the thread pool waits on them, and mark the no-column test as rstest.
* fix(rust): derive the safeguards beta for Claude Platform and keep the null-cell test off Databricks
Claude Platform on AWS answers 400 to safeguards without dangerous-tool-use-2026-09-03 and 200 with it, so its Messages transform derives the beta through the Anthropic column again. The null-cell case now uses only Bedrock Converse, since Databricks support for the beta is changing in a sibling PR.
* fix(rust): drop a null safeguards key instead of sending it without its beta
Anthropic rejects any safeguards key, null included, when the
dangerous-tool-use beta is missing, so an explicit "safeguards": null on the
Rust gateway still answered 400. The shared Messages transform now drops a
null safeguards key on every host, as the proxy already drops None params
before calling Rust, so it answers 200 with no beta.
The scripted peer in the integration wire tests now rejects the key on
presence, matching the real API.
* test: cover Bedrock Mantle in the null safeguards case and match the vendor's non-list wording
The Rust null-safeguards regression test now also runs for bedrock_mantle, whose beta is keyed on the field being present. The scripted Anthropic peer answers a non-list safeguards value with the real API's "valid array" wording.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci(cost map): remove the generated model prices JSON schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost map): keep schema-independent cost map tests after dropping the schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(cost map): keep the schema file until main's guard stops requiring it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(cost map): let bot sync PRs keep touching the schema file until it is deleted
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): let each route host pick the body params
The native routes projected request body params from a name list, hard-coded in
Rust for Messages and read from `PARAMETERS` for Chat Completions and Responses.
Each route host now answers `body_params(request)` over the keywords the call would
send, so the Python side owns which params reach the provider. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-host): use read-only body parameter types
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): move the route host modules under host/routes
Each native route's Python half lived in its own `<route>/route_host.py`, apart
from the shared host modules Rust also imports by name. They now sit together in
`litellm/rust_bridge/host/routes/<route>.py`, with their unit tests moved to the
mirrored path. No behavior change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust-host): reference messages route alias
Co-Authored-By: yujonglee.dev@gmail.com <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The route host now reports get_configured_request_timeout() in MessagesSettings, and prepare uses the call timeout first, then that setting, then the 600s default. Before this, litellm_settings.request_timeout and REQUEST_TIMEOUT were ignored on the native path, so a long turn that works on Python timed out after 10 minutes
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The Responses route now resolves openrouter alongside openai. The
OpenRouter config reuses the OpenAI request and response transforms and
owns only its bearer credential (OPENROUTER_API_KEY, then OR_API_KEY) and
base URL (OPENROUTER_API_BASE). Wire model ids follow Python's
get_llm_provider, including the openrouter/auto native id with an
explicit provider. The Python route host no longer hands litellm.openai_key
to OpenRouter.
The OpenRouter live test lives in tests/live/openrouter.rs and returns
early when the provider key is not exported, so cargo test stays offline
by default instead of hiding it behind --ignored.
Co-authored-by: nate-berri <nate@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(vertex): report an unreadable credentials file as a read failure
Tell a path apart from inline JSON by shape, not os.path.exists().
Fixes#40166
* test(vertex): narrow pytest.raises to the expected message
* fix(vertex): log the credentials path instead of returning it to the caller
* refactor(vertex): log the credential source decision where it is made
* fix(vertex): handle paths open() rejects with a bare ValueError
* fix(vertex): redact the credentials path before it reaches the log
* test(vertex): cover the brace fallback and non-text credentials file
* test(vertex): exclude the unreachable exhaustiveness guard from coverage
* refactor(vertex): read the failure's fields off the matched object
CodeQL's py/uninitialized-local-variable does not follow keyword capture
patterns in a match statement and reported `path` and `reason` as possibly
unbound. Narrowing on the class and reading the attributes says the same thing
without a suppression.
* fix(vertex): accept a read-only mapping in extract_aws_params
* fix(vertex): keep encoded credentials out of the log and cover the legacy route
* fix(vertex): give one message for every credentials file failure and pick file or JSON by shape in Rust
* fix(vertex): read a `{`-named key file in Rust, name base64 there, and give one message when Google auth rejects a credentials file
* fix(vertex): read an unreadable `{`-named key file as a file, cap key file reads, and drop storage advice from the file message
* test(vertex): integration cells for credentials file and inline value failures
* test(vertex): drive the Rust messages cell through Vertex Claude
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python
Rust's run_call built Python streams itself and imported
litellm.rust_bridge.streams by name, so sync calls were driven in Rust
while async calls came back as a Rust-made coroutine. Every native
inference entrypoint now returns one unstarted Execution and
public_execution.py drives it with lifecycle.run or lifecycle.drive,
wrapping streams and _hidden_params on the Python side. Rust only
imports the lifecycle step classes through StepTypes, and
python-bridge/src/lifecycle.rs, PythonLifecycle and streams.py are gone.
The adapter trusts the _native.pyi contract with a single cast instead
of a per-request runtime_checkable isinstance. Binding tests assert the
adapter starts the execution and returns its completed value rather
than checking callable identity.
* refactor(rust-bridge): fold lifecycle and public_execution into one execution module
The public stream wrappers only forwarded to the lifecycle streams and added the
header dict, so the streams now carry _hidden_params themselves. lifecycle.py no
longer held any call lifecycle, just the Execution driver, so it is renamed to
execution.py and absorbs the synchronous and asynchronous adapters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust-bridge): keep execution.py free of third-party imports
The host-python tests load execution.py into a bare interpreter, so the
pydantic header validation broke them. The stream head comes from the
binding's encode_stream_head, which always builds a str-keyed dict
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(host-python): preserve pre-async hook coverage after execution handoff
* style(rust-bridge): sort litellm_host_python imports
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): drive unstarted native calls through execution in the split lifecycle tests
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* fix(rust-bridge): mark execution TypeVar as Final
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(rust-bridge): cover stream close, read failure, and sync suspension in execution
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust-bridge): drive executions with generator-style send, throw and close
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): name stream steps StreamOpen and StreamChunk and reject out-of-phase steps
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(rust-bridge): drive with ExitStack and require the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* test(host-python): give the hook_chain lifecycle binding the stream factory
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(host-python): run the Python driver as one async body
Co-Authored-By: Yujong Lee <yujong@berri.ai>
* refactor(host-python): give the driver a PythonCaller with named suspensions
PythonTask becomes PythonCaller with await_python, open_stream and yield_chunk, so the call body no longer builds Suspension values or maps stream replies itself
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(host-python): collect host failures in a thiserror Error enum
missing_state() and the inline PyRuntimeError/PyTypeError messages become Error variants with one From<Error> for PyErr, and InvokeError derives thiserror so From<PyErr> comes from #[from]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(host-python): note the canonical HostError import alias in AGENTS.md
Co-Authored-By: Yujong Lee <yujong@berri.ai>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: lower aiohttp keepalive timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http): restore the 15s keep-alive default and apply it to the Rust client
15s is aiohttp's own default and what every release before v1.78.0 used. The Rust HTTP client reads the same AIOHTTP_KEEPALIVE_TIMEOUT variable, so its default moves with it.
* test(integration): cover upstream connections dropped after the keep-alive idle window
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Chat Completions, Messages and Responses each get a streaming module with
the same three names, event_stream, decode and encode, built on a shared
SSE helper in base_llm/sse.rs that owns the [DONE] marker. Callers use the
module path instead of vendor-prefixed names. Encoding matches what the
Python proxy writes to clients: Messages frames carry an event line, the
OpenAI formats are bare data frames.
Responses gets one event type for SSE and WebSocket. ResponsesEventType
names every event on OpenAI's streaming-events reference, with unknown
names riding in Other, and is_terminal and is_failure follow Python's
iterator. The framer adds a bulk decode_all and an SseBlockCodec that
keeps each event's wire bytes. An event the stream ends inside is dropped,
as the SSE spec and the openai SDK's decoder do.
ChatCompletionChunk no longer requires id, created or object, since
compatible hosts omit them.
* feat(edenai): settle native Messages reported cost
* test(rust): add live EdenAI Messages contract cases
* refactor(rust): settle call cost once as an ExecutionFacts variant
The route now decides Cost (Reported, CacheHit or Deferred) from the
result source and the provider's reported figure. The legacy callback
adapter only projects a Reported amount onto provider_reported_cost and
no longer writes response_cost, which the Python calculator derives
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust): keep EdenAI wire shaping independent of Invoke
* style(rust): format cost settlement imports
* fix(rust): trust provider costs without bypassing configured pricing
* fix(rust): clear untrusted response costs before spend settlement
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* chore: drop Greptile from the PR template, AGENTS.md and CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs: require bot reviews to finish before ticking the review checkbox
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: migrate proxy security, schema migration and MCP legacy tests
* test: run the MCP proxy e2e upstream under the SDK1 peer in integration-mcp
* test: warm the baseline store before timing the locked projection
* test(e2e): hold the stack exclusively while the semantic filter flag is flipped
* test(mcp): scope the in-process proxy e2e fixtures to their module
The session-scoped autouse fixture deleted DATABASE_URL and overrode LITELLM_MASTER_KEY for the rest of the worker, which broke every later test in the integration mcp suite
* test(e2e): assert the semantic filter keeps listed tools instead of a ranking
Which Datadog tool an OpenAI embedding ranks first is not something litellm owns, so the test now checks that the kept tools are a non-empty subset of the key's listing within top_k
* test(mcp): keep the in-process proxy e2e tests on one worker
loadgroup spread the file's tests across workers and interleaved them with other files, so the module-scoped in-process proxy was torn down and restarted between tests and requests timed out
* test(mcp): name the upstream requests that carried the wrong token
The bare all() assertion hid which request broke the forwarded token, which made an intermittent CircleCI failure undiagnosable
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* feat(rust): support Claude Platform and Mantle Messages
* fix(ci): regenerate .greptile/files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): preserve Mantle body betas
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): resolve Mantle default region
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): prefer regional endpoint host regions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(rust): format Bedrock regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): group the Python modules Rust imports by name under rust_bridge/host
Rust reads Python-owned state (legacy callback registries, model capabilities) through
modules it imports by path. Put them in one folder with an AGENTS.md that states the
contract, and update the module path constants on the Rust side.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust_bridge): route Python state reads through host accessors
* docs(rust_bridge): point test_support at the moved host contract test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): move secret manager, transport and timeout accessors under host
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(greptile): register rust_bridge/host AGENTS.md in files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): rename the evaluation model mode to decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): canonicalize the health check mode without rebinding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): replace canonical_model_mode with is_decisions_model_mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): preserve evaluation mode compatibility
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: drop the rebuilt dashboard bundle from this PR
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(dashboard): format decision mode helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* wip
* refactor(rust): share Bedrock paths and Converse/Invoke wire types via llms-types
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): share Vertex AI locations, base URL and rawPredict paths via llms-types
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): give every llms-types format and provider its own folder
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): keep single-adapter projections in llms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): restructure llms and llms-types AGENTS.md into rules and references
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): keep provider mod.rs files as re-export entrypoints
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): split llms and llms-types AGENTS.md rules into topic sections
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): move llms-types tests inline and default the workspace to inline tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): move llms tests inline beside the code they cover
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): cover every provider format with AGENTS.md and keep each upstream URL in one file
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: drop the check_rust_agents_md step
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: regenerate greptile files.json for the new AGENTS.md files
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): make every llms-types mod.rs a re-export entrypoint and drop check_rust_agents_md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(rust): register Messages providers in one match and drop the nested AGENTS.md list
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* feat(rust): add the DeepSeek Anthropic Messages config
* refactor(rust): drop the tool discriminator without mutation and use rstest in DeepSeek tests
* feat(rust): add the Vertex AI Anthropic Messages config
* fix(rust): normalize Vertex system turns the way Python does
Vertex uses the shared normalize_system_role_messages: a leading run is
hoisted, a later turn stays or is converted in place by the model's
supports_mid_conversation_system, and billing blocks are filtered after the
hoist so one inside a system-role message no longer reaches Vertex.
* fix(rust_bridge): keep non-Claude Vertex Messages calls on Python
The catalog admits every vertex_ai Messages call, but Rust serves only Claude
there and reports anything else as a terminal InvalidProvider instead of a
decline. The dispatcher now bypasses Rust for those models, as
get_provider_anthropic_messages_config does.
* refactor(rust): template the Vertex location and project errors
The location failure is the shared InvalidType detail and the missing project
is MissingSetting with the provider, setting, param and variable name, so no
sentence is spelled at the return site.
* test(rust): use rstest in the Vertex AI Messages tests
* refactor(rust): derive the missing Vertex project error from its spec
* refactor(rust): give Messages configs typed litellm params
* feat(rust): port _normalize_system_role_messages for Messages hosts
mid_conversation_system mirrors Python's module function for function: a
leading run of system turns is hoisted into the top-level system, a later
turn stays in place when the model map flags supports_mid_conversation_system
and otherwise becomes a user turn where it was, never between a tool_use and
its tool_result, and billing blocks are stripped from the hoisted system. The
capability joins MessagesModelCapabilities and the bridge projection. Azure AI
uses it in place of fold_system_role_messages, which hoisted every turn.
* fix(rust): keep the Bedrock runtime endpoint behind a blank api_base
* refactor(rust): import LitellmParams from its crate instead of a re-export
* refactor(rust): template the Messages request decode errors
Both decode failures go through ErrorDetail::invalid with the subject and the
serde error as source instead of a preformatted sentence.
* refactor(rust_bridge): read the module globals the param specs name
The Messages host no longer keeps its own list of litellm globals; a spec
whose spellings the call leaves out names the global to read, so the next
family that has one needs no host change.
* refactor(rust): give each route lookup Python's implicit None
ProviderConfigManager's per-route methods list only the providers the route
serves and fall through to None; the Rust lookups now do the same, so a new
LlmProviders variant touches only the route that serves it.
litellm-router-types mirrors litellm/types/router.py: LitellmParams holds the
model, the credentials, one flattened group per credential family from
auth-types (aws, vertex), the deployment settings a config spells, and every
other key in extra, as Python's extra="allow" keeps it. Config parses it
directly, so config::LiteLlmParams and its untyped additional_fields are gone
and a mistyped provider param is a parse error. fields() and specs() derive
from the families' param specs for hosts that project kwargs and fold module
globals. Spelled<T> is the one untagged shape for a typed value or the text a
config spells, organization takes the list Python's router expands, and the
callback shorthand keeps a value-level one-or-many. No credentials wrapper
group: serde's flatten only consumes keys for struct-shaped children.
VertexParams in auth-types holds the vertex_* fields of CredentialLiteLLMParams
in both spellings, with one ParamSpec per setting: the wire names, the litellm
module global VertexBase.get_vertex_ai_* consults, and the environment names.
A credential deserializes from text or the JSON object a config spells, an
empty object counts as absent, and both credential fields are redacted in
Debug. auth-gcp is split so lib.rs is the entrypoint (config, auth,
constants), its lookups resolve through the specs, secret_names() derives
from them, and get_vertex_ai_project_from_credentials reads a service
account's project_id for hosts that need it before a token exchange. The OCR
host folds the module globals in by spec at projection, so OcrSettings no
longer carries vertex_project and vertex_location.
AwsParams in auth-types holds the aws_* fields of CredentialLiteLLMParams
with one ParamSpec per field: its wire name and the environment names it
falls back to. fields() and secret_names() derive from the specs, the AWS
helpers take the struct instead of a map, aws_auth_config and the region
resolution read through the specs, and a missing region is
AwsParams::REGION.missing("AWS"), whose message is rendered from the spec.
Bedrock Converse, Bedrock transcription, Textract OCR and the Bedrock
Messages config build the struct at their one untyped boundary.
Moves 58 legacy nodes (cyberark, logging callback manager, langfuse handler and helpers, SQS, app crypto, bedrock nova and invoke, cloudflare, litellm_proxy, nvidia nim, replicate, triton, convert_dict_to_response, get_model_info, vcr live-call probe) into the tests/unit files that mirror the modules they exercise. Calls that reached huggingface.co, langfuse or 0.0.0.0 are refused at the HTTP boundary with respx. Tests that need credentials, a license, a spawned server, sleeps, a subprocess, or assert nothing stay in their legacy files
* refactor(python-bridge): ship a signature base and read the resolved call
NativeCall carries base (positionals by name plus signature defaults) instead
of the fully bound dict. resolved lays kwargs over base, which is what bound
held, so every pre-hook read and every route host keeps seeing the same values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(bedrock): build the transcription NativeCall with an empty base
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(dispatch): read the resolved call instead of bound
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(host-python): rename effective to effective_py_args and note the shallow copy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): derive strum VariantArray and string conversions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): use strum conversions directly with explicit spellings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): use rstest values for key management systems
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(python-bridge): take NativeCall directly and fold routes into per-route folders
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(python-bridge): use rstest for updated tests and keep embedding's stub parameter name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(rust): add the Anthropic beta header policy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): resolve known betas held as Other in AnthropicBeta::on
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): use named rstest cases for Other beta resolution
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(rust): make the rstest named-case rule explicit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): move BetaPolicy tests to the public API test crate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
rust-analyzer cannot resolve the bare alias names emitted by
macro_rules_attribute::apply, so every aliased type was invisible to it
(no go-to-definition or find-references). Referencing the aliases through
crate:: resolves them via the re-export that attribute_alias! generates.
Co-authored-by: Nate Armstrong <narmstrong@Nates-MBP.localdomain>
* docs(rust): add ADR folder with ADR 000 and ADR 001 scaffold
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): add ADR 000 and ADR 001 on typing requests inside Rust
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): list four ADR sections in ADR 000
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): move ADR process doc from adr_000 to AGENTS.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Nate Armstrong <narmstrong@Nates-MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): add typed LLM wire contracts to llms-types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(rust): drop blank lines left in new wire types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): complete additive wire type contracts
* feat(rust): complete additive nested wire contracts
* feat(rust): give Messages content blocks concrete per-block contracts
Replace the shared all-optional ContentBlockPayload with one struct per
documented block and nested tagged unions for server tool results, so
required fields from the official Messages reference are enforced.
Make CustomTool a plain struct with required name and input_schema, add
MessagesToolParam plus the toolset and undated tool-search tags, model
MiniMax media sources as a tagged union, require documented fields on
chat audio, image URLs, and logprobs, and rename the new block enum to
MessagesContentPart so it no longer shadows the streaming struct.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(rust): tighten new wire contracts against the official references
Give each Messages builtin tool a concrete contract with its documented
required fields, fixed names, and typed allowed callers in place of the
shared all-optional ToolDefinition bag. Make MCP tool result content and
fallback triggers optional as the request params document, and narrow
tool search references, web fetch content, and bash results to their
documented block types.
Keep unmodeled Responses output items and new usage iteration and stop
detail types as Recognized values instead of rejecting the whole payload,
accept find_in_page and optional open_page URLs, preserve JSON Schema
property order, and move prompt_cache_breakpoint to Chat Completions
content parts where OpenAI documents it.
Drop OCR table and key/value types that only matched Azure Document
Intelligence and would reject other providers' normalized passthrough,
along with unsourced bounding box and applied edit fields.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(rust): type every documented Responses output item and close wire gaps
Add the 18 remaining documented Responses output item variants with
concrete nested actions, outputs, and safety checks, and add the
documented namespace, async, caller, and image generation fields to the
existing items.
Restrict MCP tool result content to a string or text blocks, model the
Messages diagnostics cache-miss union and its request param with null
distinct from absent, and accept draft-07 tuple items and prefixItems in
JSON Schema.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(rust): keep Messages contract assertions in test bodies
Compare built-in tool decodes against fully built expected values and
split the required-field rejections per type, so no case carries its
assertions in a callback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(rust): cite the Mistral OCR reference beside the OCR wire types
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
