* feat(azure_ai): support Microsoft-Decision-1 on the decisions API
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(azure_ai): strip project path and API suffix together in decisions base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 2fcc780498)
* feat(decisions): add hosted_vllm provider
Adds HostedVLLMDecisionsConfig so hosted_vllm/<served model> works on
POST /v1/systemone and POST /v1/decisions against a vLLM server that
serves the System One body (vllm-project/vllm#59299, after v0.31.0).
vLLM only supports choice questions and rejects others, so the base
decisions config gains a per-provider health_check_questions attribute
and the evaluation-mode health probe sends a choice question for
hosted_vllm while every other provider keeps the noul probe.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): move the hosted_vllm placeholder key to constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 61e5f2dd3c)
* refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): map OpenRouter connection failures to APIConnectionError and drop explanatory docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): keep APIConnectionError for unreachable deployments on every provider
`_map_upstream_exception` now maps a status `_handle_error` synthesized for a
non-HTTP failure to `APIConnectionError` before `exception_type` runs, so a
Perplexity (openai-compatible) deployment that cannot be reached no longer
answers `InternalServerError` where main answers `APIConnectionError`.
Audit cells: the unreachable-deployment integration cell is parametrized over
the five providers, OpenRouter connection failures are pinned on chat, stream,
embeddings, decisions and the SDK, a deployment whose provider has no decisions
config gets the 400 naming every supported provider, and a unit cell pins the
mapper alone. The first docstring sentence of the five translation bases is
dropped.
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 0e09e99146)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236)
Backport of #44236 to rc/1.105.0.
Cherry-picked from 8b1990b4bc (main).
* feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions
Backport of #45184 to rc/1.105.0.
Cherry-picked from 3339469e73 (litellm_systemone_endpoint). The playground System One tab is not on this line, so its UI hunks are dropped.
* fix(decisions): import assert_never from typing_extensions for Python 3.10
Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from 29e715473c (litellm_systemone_endpoint).
* fix(decisions): cap OpenAI-format questions at the System One limit
Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from 7ef132947a (litellm_systemone_endpoint).
* fix(decisions): return 400 for bodies that are not JSON
Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from b6bb885324 (litellm_systemone_endpoint).
* test(decisions): send System One bodies to /v1/systemone in integration tests
Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from 77872f6cda (litellm_systemone_endpoint).
* feat(decisions): add OpenAI as a Decisions provider
System One requests to openai/ models are translated to OpenAI's Decisions shape on the way out and OpenAI's answers are translated back, so both /v1/systemone and /v1/decisions can route to gpt-6-luna.
(cherry picked from commit b94bbbc318)
* refactor(decisions): move the OpenAI wire translation into llms/openai
(cherry picked from commit d209a85076)
* feat(decisions): route both request formats through a shared decisions IR
System One and OpenAI-format bodies now convert to one internal representation, and each provider translates it to its own wire format. OpenAI deployments receive the caller's messages, images, names, typed choice values, level descriptions and safety_identifier unchanged. System One providers return a 400 for image input. Cached and cache-write tokens are priced on both response shapes, and provider response extras survive the round trip.
(cherry picked from commit 0dd207fde1)
* fix(decisions): keep provider fields inside System One answers
(cherry picked from commit 1348de864f)
* fix(decisions): load on Python 3.10 and 3.11 and bill cached tokens once under custom pricing
Decisions IR dataclasses used a mappingproxy default, which is only hashable from Python 3.12, so import litellm failed on 3.10 and 3.11. Decisions usage now reaches cost_per_token as a Usage with prompt_tokens_details, so custom pricing no longer adds cached and cache-write tokens on top of input_tokens that already include them
(cherry picked from commit 39012f1c22)
* fix(decisions): honor litellm OpenAI key and base settings for the openai provider
OpenAI decisions now resolve the key from litellm.api_key, litellm.openai_key, then OPENAI_API_KEY, and the base from litellm.api_base, OPENAI_BASE_URL, then OPENAI_API_BASE, matching other OpenAI calls. Providers own their configured key and base lookup through the endpoint config.
(cherry picked from commit e465919004)
* test(decisions): clear every OpenAI base setting in the proxy OpenAI deployment test
(cherry picked from commit a2951b318d)
* fix(decisions): send OpenAI instructions for System One questions without them and reject single-option questions
(cherry picked from commit e4081f77c6)
* fix(decisions): return guardrail_information on OpenAI-format decisions when requested
Backport of the #45184 and #45214 guardrail fix.
Cherry-picked from 855f0bb765 (litellm_systemone_endpoint) as merged into #45214 at c19ada66ed (litellm_openai_decisions_provider).
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): saving echoed model_info never persists cost map pricing as a deployment override (Pylon #6870)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): budget_duration change on /budget/update recomputes budget_reset_at (Pylon #6913)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): count_tokens on a budgeted key reserves no budget and a later completion still succeeds (Pylon #6966)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): /cost/estimate reports configured prices for a deployment absent from the cost map (Pylon #7014)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cache the team member default budget in Redis as JSON (Pylon #7180)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): aggregated team daily activity reports whole-range team spend in one page (Pylon #7224)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed daily user rollup commits are retried so spend report and daily activity agree (Pylon #7268)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): CLI session token without org_id is charged to and capped by the team organization budget (Pylon #7291)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gemini passthrough success releases its budget reservation from the spend counter (Pylon #7295)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): batch retrieval spend row sums reasoning tokens and counts output and error file failures (Pylon #7341)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): uncostable batches retire from the cost poll page so newer batches are costed (Pylon #7342)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): charge a team member added without any budget on its membership row (Pylon #7363)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed dispatched requests keep estimated input tokens in spend logs (Pylon #7519)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock passthrough converse guardrail ignores tool definitions (Pylon #7524)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): explicit null budget_duration on /team/new is not replaced by default_team_params (Pylon #7536)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): PATCH /organization/update with a null limit clears it (Pylon #7577)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): ultrafast service_tier bills ultrafast rates without leaking pricing fields upstream (Pylon #7587)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep the selected model in the response and spend log for an Azure Model Router alias (Pylon #7636)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): disconnected Bedrock /v1/messages stream still bills terminal usage (Pylon #7685)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks cached prompt tokens bill at cache rates (Pylon #7738)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): completed batch spend row records reasoning tokens and error file failures (Pylon #7928)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bill OCR annotation pages at annotation_cost_per_page (Pylon #7958)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): in-flight count tokens request reserves no key budget so a completion still reaches the provider (Pylon #7307)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed /v1/responses success callbacks keep provider response headers (Pylon #7775)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)"
This reverts commit 910348be7a.
* test(integration): reconcile contracts manifest for bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): isolate proxy config writes in bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): address review feedback on budget reset bounds and callback batch accumulation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): avoid rebinding the cache identity accumulator
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert forwarded messages per cache identity call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): make budget reset and team default tests deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vector_stores): keep config-defined vector stores listed and read-only
Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show vector store source and read-only state for config-defined stores
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show a visible read-only hint in the config vector store actions menu
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The gpt-5.6-responses_previous_response_id case sent a literal id the proxy never issued, which the Responses id security hook refuses with a 403 at production defaults. The case now primes a response through the proxy and chains the id it hands back, so the harness drops allow_unmanaged_response_ids and the security hook stays exercised
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.
The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.
The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
