* feat(agents): support Databricks Agent Platform agents in the agent gateway
Add a databricks_agent provider that speaks the MLflow ResponsesAgent wire
(Responses-shaped input, output_text items, output_text.delta SSE) to Model
Serving endpoints (/invocations or the unified /serving-endpoints/responses)
and Databricks Apps (/responses), with a PAT or OAuth M2M client credentials
Register it through the agents config block or the Admin UI agent catalog and
call it through /a2a/<agent> (message/send, message/stream) or model:
a2a/<agent> chat completions. The chat route now runs every completion-bridge
agent through its own provider, and attaches the minted Databricks OAuth token
for both bridge and URL agents. Consumed OAuth settings never reach the
provider call
* fix(agents): harden databricks_agent input, output, and OAuth handling
Non-text content parts answer 400 unless drop_params is set, a reply without
an output list answers 502, flat OAuth fields resolve an os.environ api_base,
the bridge debug log no longer echoes the request model, routing tests stub
only the HTTP boundary, and the catalog description fits the dropdown
* fix(agents): merge agent and client extra_headers on the a2a chat route, and reject an empty databricks_oauth block
* fix(agents): bridge only databricks_agent agents on the a2a chat route, keep the card url over a client api_base, and require workspace_url for a Databricks App
* refactor(agents): read the a2a chat bridge providers from a constants set instead of branching on a provider name
* fix(databricks_agent): let the configured credential win, reject malformed client extra_headers, and keep a missing endpoint name empty
* test(agents): drive the flat OAuth a2a send through the real bridge
* fix(agents): keep an empty databricks_oauth block ignored on url agents, as before this change
* fix(agents): accept bare App hosts, env service principals, and os.environ refs on Databricks agents
* fix(databricks_agent): address review caveats on chat bridging, params, and auth
Map max_tokens and max_completion_tokens to max_output_tokens and pass
temperature, top_p, and user, so /v1/messages works without drop_params.
Send A2A message metadata only with string values, answering 400 or
dropping the rest under drop_params. Join message items with a blank line
in both stream and non-stream replies. Cache the env service principal
token until shortly before it expires. Keep databricks_agent replies out
of the response cache.
On the chat bridge, let request fields win over the agent's stored params
while pinning the agent's api_base and api_key, and apply the agent's
static_headers between client and agent extra_headers, as /a2a does.
Resolve os.environ/ only for databricks_agent api_base and api_key instead
of every bridge provider. Ignore an unusable databricks_oauth block on URL
agents on chat, as at the merge base, instead of answering 400.
* test(internal_context): list the databricks agent token cache as process-local
* fix(databricks_agent): keep the agent's endpoint, input, and target out of client control
extra_body model and input answer 400, or are dropped under drop_params.
A client api_base or base_url never replaces the agent's card URL on the
bridge. Chat to a URL agent keeps its merge-base call, and an incomplete
databricks_oauth block on a URL agent keeps its merge-base ValueError.
* test(databricks): expect the merge-base 400 for a client api_base on agent chat
* fix(databricks): keep agent-owned inputs and identity headers out of a chat caller's reach
* fix(databricks): bound the agent-over-caller merge to two levels so it needs no recursion
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
/public/providers/fields re-read and re-parsed the bundled 85K
provider_create_fields.json on every request, and /public/agents/fields did the
same for that file plus agent_create_fields.json. Both are unauthenticated
public endpoints, and both did the work inside an `async def`, so the read and
the parse ran on the event loop and serialized against every other in-flight
request. Measured against the bundled files, that is roughly 0.40 ms of blocking
work per request for the provider endpoint and 0.45 ms for the agent endpoint
Both now memoize with `lru_cache`, which is what `/public/endpoints` in the same
module already does through its `_cached_endpoints` global, so each file is read
once per process rather than once per request
The agent endpoint mutated the dicts it had just loaded, appending inherited
provider credential fields and popping `inherit_credentials_from_provider`. That
is only safe while the data is re-read every time, so caching it meant making the
merge pure: `_agent_with_inherited_credentials` builds a new mapping per agent,
and the cached value is the merged result rather than the raw file contents.
Responses are byte identical before and after, checked on both endpoints against
a live proxy
The bundled records are typed as Mapping[str, object] and narrowed with
isinstance at the point of use, rather than typing.Any, which ruff-strict bans
Side effects: 3 of the repo's ASYNC230 violations go away, and basedpyright
errors in the file drop from 63 to 52
Rebased onto main after /public/providers/fields started validating the bundled
JSON through a pydantic adapter on every request. That validation now runs once
and the validated records are cached; the agent endpoint keeps merging the raw
provider records, so both responses stay byte identical to main.
Co-authored-by: Alok Sharma <alok44170@gmail.com>
* feat(ui): add decisions and classifier providers to the Add Model form
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): use literal upstream decisions paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: drop duplicate TYPESAFE provider fields entry superseded by main's TypeSafe
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): drop stray blank line in provider_info_helpers test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): type the OSS decisions provider fields test
* fix(decisions): keep Laya and Bespoke env keys on their configured server
Route laya and bespoke Decisions deployments through the same oss_connection
and validate_oss_model rules the pass-through and the classifier router use,
so LAYA_API_KEY and BESPOKE_API_KEY only go to LAYA_API_BASE and BESPOKE_API_BASE,
an api_base with credentials, a query, or a non-HTTP scheme answers 400, and
an unknown checkpoint name answers 400 before any upstream call
* fix(decisions): send the Laya and Bespoke env key to a form-added model on the env server
A deployment added through Add Model always carries an API base, so the
env key only reached deployments with no base. The env key now goes to a
deployment whose base matches LAYA_API_BASE or BESPOKE_API_BASE, an empty
deployment key counts as no key, and the Bespoke placeholder is the name
Ollama serves.
* test(decisions): cover the env-base classifier case in the complexity router test
* fix(decisions): match the env server with or without /v1 and point Ollama users at bespoke/nimble
* fix(decisions): call /v1/systemone once from classifiers and pass-through on a /v1 base
* fix(decisions): strip /v1 from the API base path only, so a host named v1 still works
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(claude-code): emit installationPreference in marketplace.json entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): satisfy marketplace CI checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): qualify marketplace entry as read only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): suppress typed marketplace assembly warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): restrict installation_preference to the documented values
Accept only available, auto_install, or required so a typo returns 422 instead of being silently dropped, and cover the PUT full-replace clearing behavior with a test
* fix(claude-code): echo installation_preference on the public skill hub
* refactor(claude-code): read installation_preference with one membership check and cover hand-edited rows
* test(integration): audit installation_preference on the Claude Code marketplace
* test(integration): kill a serving worker in the marketplace chaos cell
* test(integration): give the marketplace worker-kill cell the two-boot timeout
* fix(proxy): keep installationPreference on the MarketplacePluginEntry model
* fix(proxy): refuse self-installing preferences off sha256 archives, accept the camelCase key, and echo the preference on POST and PUT
* fix(claude-code): serve installation_preference only on archive plugins and refuse both key spellings
Claude Code 2.1.75 and older reject a marketplace.json once any plugin carries installationPreference,
and every Claude Code release before archive support rejects a catalog with an archive plugin. Requiring
an archive source for every value, available included, keeps the field off catalogs those clients can read.
A request that sends installation_preference and installationPreference together now gets a 422.
* fix(claude-code): read the preference spelling that carries a value when the other is null
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): source Home What's new from day 0 support blog posts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): Home cards read card_title and card_summary from blog tags with provider logo
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(home): serve the bundled whats_new.json by default so a release shows only its own launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4dd96800e8)
* style: annotate whats_new_url with str | None
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit d1ae7ea085)
* test(home): pin whats_new_url to the env var and regenerate the dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4bfec53865)
* test(public_endpoints): prove the /public/whats_new route serves the bundled list when no URL is configured
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(public_endpoints): drop the unused os import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(home): pin whats_new_url to the env var so a baked-in default URL fails the suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): sort the import blocks ruff flagged in test_public_endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(home): let the route test read the real whats_new_url default so a baked-in URL fails it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): load the Home page What's new launches from /public/whats_new at runtime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the three newest What's new launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build: ship whats_new.json in the wheel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add a Home page as the dashboard root with usage summary, recent launches, blog updates and a model leaderboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back to spend-log request counts on Home when the gateway has recorded none
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover the Home page totals, launches, leaderboard and updates feed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): render Home Updates as rows and return up to 6 blog posts from /public/litellm_blog_posts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): filter Home Updates by the new-model and api-change blog tags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(blog): import NotRequired from typing_extensions for py3.10 and test the endpoint passes every fetched post through
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(ui): drop the Updates filter chips, keep Updates as a plain row list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): restore generated schema.d.ts to main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the Home key button Create New Virtual Key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the What's new sublabel on Home
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show an error with retry when Home usage fails to load and move the Home test to the integration tier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Add Model selector read the merged runtime cost map, so deployment
ids and provider-prefixed backend keys registered for proxy deployments
showed up as duplicate model choices. The public cost map endpoint now
accepts catalog_only=true to return the catalog as loaded, and the Add
Model panel requests that view. The default response is unchanged.
Resolves LIT-9263
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add TypeSafe and Strands Decider to the Add Model provider list
* test(public_endpoints): type the provider fields helper against the route's model
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(proxy): expose public names for private proxy helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep original class names behind public aliases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): preserve internal callback filtering
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep _PROXY_ class names for managed files hooks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep old private names in package exports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): preserve recursive auth helper name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): align MCP limiter tests with server enforcement
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): match main's MCP limiter tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep old private names bound in importing modules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): preserve compatibility imports through strict lint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): use exact pyright suppression in password helper test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): add reasons to compatibility import noqa comments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): update IN-list baseline for renamed helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): register typesafe as a provider so Jev deployments load
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): move provider endpoints under llms and validate proxy bodies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): add Cloudflare Clef and Strands Decider backends
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): register decisions routes for managed agents and gateway
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): use raw regex for cloudflare missing account match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): avoid cast in Cloudflare response unwrapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): default model, evaluation health probe, short Cloudflare names
The proxy validates only state and questions, so a request without a
model falls through to the configured default model like every other
route. Health checks probe evaluation-mode deployments through the
Decisions API instead of failing with an unsupported mode, and
cloudflare/clef and cloudflare/clef-flash get cost-map rows so the short
names resolve a mode and a price. The registry no longer claims typed
decisions for a provider with no backend.
* fix(decisions): let health_check_params override the evaluation probe
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit the decisions endpoint across providers, limits, health and chaos
Adds the /v1/decisions audit cells: one wire contract per provider (path, key, body and cost-map billing), the gateway-only fields and tags, the sad paths (invalid bodies, unknown model, key checks, api_base in the body, upstream 401/429/500, a 200 without answers, an unreachable upstream), the two evaluation-mode health probes, and three chaos cells (a mixed-failure burst over both routes, a worker SIGKILL mid-burst, an upstream outage and restart on the same port).
The PR's cost case read the upstream observations through the gateway, which answers 404 for that path; it now reads them from the upstream URL. The owned proxy harness takes extra CLI arguments, and its graceful stop waits as long as a worker boot may take, since a worker still starting honors SIGTERM only once it is up and the 30 second wait forced a cleanup under load.
* fix(decisions): send env API keys to a configured api_base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): add zero-cost evaluation cost-map entry for Strands Decider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): register the routes through the lazy feature registry
The Decisions router was included at import, ahead of the config and DB
pass-through endpoints, so a pass-through configured at /v1/decisions
was skipped and answered 400 as an unknown Decisions provider. The
routes now register through LAZY_FEATURES, which splices them in after
every eager route, so a pass-through at /v1/decisions keeps its route
while /decisions still serves natively. The lazy OpenAPI snapshot carries
the two paths so the schema shows them before the first call.
The audit cells add the env-key egress to a configured api_base, the
client api_base opt-in shared with chat, the pass-through precedence on
an owned proxy, and the Strands evaluation health check resolved from
the cost map. The integration config exports the Perplexity env key the
first cell needs.
* fix(decisions): keep the Cloudflare api_base message in its transformation and read the audit upstream once per cell
* fix(proxy): let a config pass-through beat a lazily registered route in eager mode
With LITELLM_DISABLE_LAZY_ROUTES set the decisions routes are registered at
startup, so SafeRouteAdder treated a config pass-through at exactly
/v1/decisions as already registered and dropped it. In lazy mode a pass-through
created through the API after the first native call was skipped the same way.
Routes a lazy feature owns no longer count as registered, and a route added at
one of their paths is placed ahead of them, the precedence lazy mode gives a
config pass-through when the feature has not loaded yet.
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): register tencent in the Add Model provider dropdown
The Add Model provider dropdown is driven by the proxy's
/public/providers/fields endpoint, which serves
provider_create_fields.json. Tencent was frozen in the test's
ADD_MODEL_UNLISTED_PROVIDERS set, so it never appeared in the dropdown.
Add a Tencent entry (optional api_base + required api_key, matching
TENCENT_API_BASE/TENCENT_API_KEY) and unfreeze it in the backend test.
Register Tencent in the UI Providers enum, provider_map, and placeholder
map so the dropdown resolves the display name and model placeholder.
* fix(tencent): drop test docstring to satisfy comment policy
* fix(ui): bundle the Tencent Cloud logo for the provider dropdown
* test(proxy): move utils, agent_endpoints and endpoint tests into tests/unit/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): package moved unit test directories
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): exclude proxy-db-owned files from the misc target
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drop the redundant fixture docstrings in the proxy conftest
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure
* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage
* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests
* ci: fail the unit shard when circleci tests split errors
* test: drop restating comments from the gemini pass-through split
* build: point the local proxy unit targets at the nested tests/unit/proxy tree
* ci: exit the unit shard cleanly when circleci tests split assigns it no files
---------
Co-authored-by: yuneng <yuneng@berri.ai>
