feat(decisions): backport /v1/systemone, OpenAI-format /v1/decisions and the openai Decisions provider to rc/1.105.0 (#44236, #45184, #45214) (#45189)

* feat(decisions): add unified /v1/decisions endpoint for Jev-compatible providers (#44236)

Backport of #44236 to rc/1.105.0.
Cherry-picked from 8b1990b4bc (main).

* feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions

Backport of #45184 to rc/1.105.0.
Cherry-picked from 3339469e73 (litellm_systemone_endpoint). The playground System One tab is not on this line, so its UI hunks are dropped.

* fix(decisions): import assert_never from typing_extensions for Python 3.10

Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from 29e715473c (litellm_systemone_endpoint).

* fix(decisions): cap OpenAI-format questions at the System One limit

Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from 7ef132947a (litellm_systemone_endpoint).

* fix(decisions): return 400 for bodies that are not JSON

Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from b6bb885324 (litellm_systemone_endpoint).

* test(decisions): send System One bodies to /v1/systemone in integration tests

Backport of the #45184 follow-up to rc/1.105.0.
Cherry-picked from 77872f6cda (litellm_systemone_endpoint).

* feat(decisions): add OpenAI as a Decisions provider

System One requests to openai/ models are translated to OpenAI's Decisions shape on the way out and OpenAI's answers are translated back, so both /v1/systemone and /v1/decisions can route to gpt-6-luna.

(cherry picked from commit b94bbbc318)

* refactor(decisions): move the OpenAI wire translation into llms/openai

(cherry picked from commit d209a85076)

* feat(decisions): route both request formats through a shared decisions IR

System One and OpenAI-format bodies now convert to one internal representation, and each provider translates it to its own wire format. OpenAI deployments receive the caller's messages, images, names, typed choice values, level descriptions and safety_identifier unchanged. System One providers return a 400 for image input. Cached and cache-write tokens are priced on both response shapes, and provider response extras survive the round trip.

(cherry picked from commit 0dd207fde1)

* fix(decisions): keep provider fields inside System One answers

(cherry picked from commit 1348de864f)

* fix(decisions): load on Python 3.10 and 3.11 and bill cached tokens once under custom pricing

Decisions IR dataclasses used a mappingproxy default, which is only hashable from Python 3.12, so import litellm failed on 3.10 and 3.11. Decisions usage now reaches cost_per_token as a Usage with prompt_tokens_details, so custom pricing no longer adds cached and cache-write tokens on top of input_tokens that already include them

(cherry picked from commit 39012f1c22)

* fix(decisions): honor litellm OpenAI key and base settings for the openai provider

OpenAI decisions now resolve the key from litellm.api_key, litellm.openai_key, then OPENAI_API_KEY, and the base from litellm.api_base, OPENAI_BASE_URL, then OPENAI_API_BASE, matching other OpenAI calls. Providers own their configured key and base lookup through the endpoint config.

(cherry picked from commit e465919004)

* test(decisions): clear every OpenAI base setting in the proxy OpenAI deployment test

(cherry picked from commit a2951b318d)

* fix(decisions): send OpenAI instructions for System One questions without them and reject single-option questions

(cherry picked from commit e4081f77c6)

* fix(decisions): return guardrail_information on OpenAI-format decisions when requested

Backport of the #45184 and #45214 guardrail fix.
Cherry-picked from 855f0bb765 (litellm_systemone_endpoint) as merged into #45214 at c19ada66ed (litellm_openai_decisions_provider).

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

.github/workflows/test-unit.yml

@@ -61,7 +61,7 @@ jobs:
 
           - shard: core-utils
             artifact-name: core-utils
-            test-path: ""
+            test-path: tests/unit/decisions
             unit-flag: core-utils
             workers: 2
             reruns: 1
@@ -141,6 +141,7 @@ jobs:
             artifact-name: proxy-endpoints
             test-path: >-
               tests/unit/proxy/analytics_endpoints
+              tests/unit/proxy/decisions_endpoints
               tests/unit/proxy/management_endpoints
               tests/unit/proxy/list_api
               tests/unit/proxy/memory

README.md

@@ -390,11 +390,13 @@ Set `LITELLM_PROXY_API_BASE` and `LITELLM_PROXY_API_KEY` and every model call th
 | [Sail (`sail`)](https://docs.litellm.ai/docs/providers/sail) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Sambanova (`sambanova`)](https://docs.litellm.ai/docs/providers/sambanova) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Snowflake (`snowflake`)](https://docs.litellm.ai/docs/providers/snowflake) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
+| [Strands Decider (`strands_decider`)](https://docs.litellm.ai/docs/providers) |  |  |  |  |  |  |  |  |  |  |
 | [Text Completion Codestral (`text-completion-codestral`)](https://docs.litellm.ai/docs/providers/codestral) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Text Completion OpenAI (`text-completion-openai`)](https://docs.litellm.ai/docs/providers/text_completion_openai) | ✅ | ✅ | ✅ |  |  | ✅ | ✅ | ✅ | ✅ |  |
 | [Together AI (`together_ai`)](https://docs.litellm.ai/docs/providers/togetherai) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Topaz (`topaz`)](https://docs.litellm.ai/docs/providers/topaz) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Triton (`triton`)](https://docs.litellm.ai/docs/providers/triton-inference-server) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
+| [Typesafe Decisions API (`typesafe`)](https://docs.litellm.ai/docs/providers) |  |  |  |  |  |  |  |  |  |  |
 | [V0 (`v0`)](https://docs.litellm.ai/docs/providers/v0) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [Vercel AI Gateway (`vercel_ai_gateway`)](https://docs.litellm.ai/docs/providers/vercel_ai_gateway) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |
 | [VLLM (`vllm`)](https://docs.litellm.ai/docs/providers/vllm) | ✅ | ✅ | ✅ |  |  |  |  |  |  |  |

gateway/routes/allowlist.py

@@ -1,7 +1,7 @@
 """Path allowlist for the gateway component.
 
 The gateway exposes the LLM data-plane surface: chat/completions, embeddings,
-audio, batches, files, fine-tuning, rerank, ocr, rag, video, search, image,
+audio, batches, files, fine-tuning, rerank, decisions, ocr, rag, video, search, image,
 responses, vector stores, passthrough providers, realtime websockets, MCP
 tool-call endpoints, and operational endpoints (/health, /metrics, and the
 /debug/memory/summary read of the serving worker's RSS).
@@ -60,6 +60,10 @@ GATEWAY_PATH_PREFIXES: tuple[str, ...] = (
     "/v1/rerank",
     "/v2/rerank",
     "/rerank",
+    "/v1/decisions",
+    "/decisions",
+    "/v1/systemone",
+    "/systemone",
     "/v1/ocr",
     "/ocr",
     "/v1/rag/",

litellm/__init__.py

@@ -1473,6 +1473,7 @@ from .embeddings.dispatch import *
 from .rust_bridge import rust
 from .rag.main import *
 from .sandbox.main import *
+from .decisions.main import *
 from .search.main import *
 from .realtime_api.main import (
     _arealtime,

litellm/cost_calculator.py

@@ -9,7 +9,7 @@ from typing import TYPE_CHECKING, Any, Final, Literal, cast
 
 from httpx import Response
 from pydantic import BaseModel
-from typing_extensions import ReadOnly, TypedDict
+from typing_extensions import ReadOnly, TypedDict, assert_never
 
 import litellm
 import litellm._logging
@@ -99,6 +99,7 @@ from litellm.llms.vertex_ai.cost_calculator import cost_router as google_cost_ro
 from litellm.llms.xai.cost_calculator import cost_per_token as xai_cost_per_token
 from litellm.responses.utils import ResponseAPILoggingUtils
 from litellm.types.agents import LiteLLMSendMessageResponse
+from litellm.types.decisions import DecisionsResponse, DecisionsUsage, OpenAIDecisionResponse, OpenAIDecisionUsage
 from litellm.types.llms.base import CachedTokensDetails
 from litellm.types.llms.openai import (
     HttpxBinaryResponseContent,
@@ -1023,6 +1024,8 @@ def get_usage_object(
         ),
     )
 
+    if isinstance(completion_response, (DecisionsResponse, OpenAIDecisionResponse)):
+        return None if completion_response.usage is None else _decisions_usage(completion_response.usage)
     if usage_obj is None:
         return None
     if isinstance(usage_obj, Usage):
@@ -1053,11 +1056,37 @@ def get_usage_object(
         return None
 
 
+def _decisions_prompt_tokens_details(usage: DecisionsUsage | OpenAIDecisionUsage) -> PromptTokensDetailsWrapper:
+    match usage:
+        case DecisionsUsage():
+            return PromptTokensDetailsWrapper(
+                cached_tokens=usage.cached_tokens, cache_write_tokens=usage.cache_write_tokens
+            )
+        case OpenAIDecisionUsage():
+            return PromptTokensDetailsWrapper(
+                cached_tokens=usage.input_tokens_details.cached_tokens,
+                cache_write_tokens=usage.input_tokens_details.cache_write_tokens,
+            )
+        case _:
+            assert_never(usage)
+
+
+def _decisions_usage(usage: DecisionsUsage | OpenAIDecisionUsage) -> Usage:
+    return Usage(
+        prompt_tokens=usage.input_tokens,
+        completion_tokens=usage.output_tokens,
+        total_tokens=usage.input_tokens + usage.output_tokens,
+        prompt_tokens_details=_decisions_prompt_tokens_details(usage),
+    )
+
+
 def _is_known_usage_objects(usage_obj):
     """Returns True if the usage obj is a known Usage type"""
     return (
         isinstance(usage_obj, litellm.Usage)
         or isinstance(usage_obj, ResponseAPIUsage)
+        or isinstance(usage_obj, DecisionsUsage)
+        or isinstance(usage_obj, OpenAIDecisionUsage)
         or TranscriptionUsageObjectTransformation.is_transcription_usage_object(usage_obj)
     )
 
@@ -1466,7 +1495,9 @@ def completion_cost(
                             "usage",
                             litellm.Usage(**_usage_for_dump.model_dump()),
                         )
-                    if usage_obj is None:
+                    if isinstance(usage_obj, (DecisionsUsage, OpenAIDecisionUsage)):
+                        _usage = _decisions_usage(usage_obj).model_dump()
+                    elif usage_obj is None:
                         _usage = {}
                     elif isinstance(usage_obj, BaseModel):
                         _usage = cast(BaseModel, usage_obj).model_dump()
@@ -1957,7 +1988,9 @@ def response_cost_calculator(
     | LiteLLMRealtimeStreamLoggingObject
     | OpenAIModerationResponse
     | Response
-    | SearchResponse,
+    | SearchResponse
+    | DecisionsResponse
+    | OpenAIDecisionResponse,
     model: str,
     custom_llm_provider: str | None,
     call_type: Literal[
@@ -1979,6 +2012,8 @@ def response_cost_calculator(
         "arerank",
         "search",
         "asearch",
+        "decisions",
+        "adecisions",
     ],
     optional_params: dict,
     cache_hit: bool | None = None,

litellm/decisions/__init__.py

@@ -0,0 +1,3 @@
+from litellm.decisions.main import adecisions, decisions
+
+__all__ = ["adecisions", "decisions"]

litellm/decisions/main.py

@@ -0,0 +1,369 @@
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass, field
+from types import MappingProxyType
+from typing import Final, TypeAlias
+
+import httpx
+from pydantic import TypeAdapter, ValidationError
+from typing_extensions import assert_never
+
+import litellm
+from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
+from litellm.llms.base_llm.decisions.transformation import (
+    DecisionsProviderConfig,
+    ir_to_systemone_response,
+    systemone_request_to_ir,
+)
+from litellm.llms.cloudflare.decisions.transformation import CLOUDFLARE_DECISIONS_ENDPOINT
+from litellm.llms.custom_httpx.http_handler import _get_httpx_client, get_async_httpx_client
+from litellm.llms.openai.decisions.transformation import (
+    OPENAI_DECISIONS_ENDPOINT,
+    ir_to_openai_response,
+    openai_request_to_ir,
+)
+from litellm.llms.openrouter.decisions.transformation import OPENROUTER_DECISIONS_ENDPOINT
+from litellm.llms.perplexity.decisions.transformation import PERPLEXITY_DECISIONS_ENDPOINT
+from litellm.llms.strands_decider.decisions.transformation import STRANDS_DECIDER_DECISIONS_ENDPOINT
+from litellm.llms.typesafe.decisions.transformation import TYPESAFE_DECISIONS_ENDPOINT
+from litellm.types.decisions import (
+    DecisionQuestion,
+    DecisionsIRRequest,
+    DecisionsIRResponse,
+    DecisionsJSON,
+    DecisionsRequestBody,
+    DecisionsResponse,
+    OpenAIDecisionInput,
+    OpenAIDecisionQuestion,
+    OpenAIDecisionRequestBody,
+    OpenAIDecisionResponse,
+    UnsupportedDecisionsRequest,
+)
+from litellm.utils import client
+
+DECISIONS_ENDPOINTS: Final[Mapping[str, DecisionsProviderConfig]] = MappingProxyType(
+    {
+        "perplexity": PERPLEXITY_DECISIONS_ENDPOINT,
+        "typesafe": TYPESAFE_DECISIONS_ENDPOINT,
+        "openrouter": OPENROUTER_DECISIONS_ENDPOINT,
+        "cloudflare": CLOUDFLARE_DECISIONS_ENDPOINT,
+        "strands_decider": STRANDS_DECIDER_DECISIONS_ENDPOINT,
+        "openai": OPENAI_DECISIONS_ENDPOINT,
+    }
+)
+
+DecisionsQuestions: TypeAlias = (
+    Mapping[str, DecisionQuestion | Mapping[str, object]] | Sequence[OpenAIDecisionQuestion | Mapping[str, object]]
+)
+DecisionsRequestFormat: TypeAlias = DecisionsRequestBody | OpenAIDecisionRequestBody
+
+_SYSTEMONE_REQUEST_ADAPTER: Final[TypeAdapter[DecisionsRequestBody]] = TypeAdapter(DecisionsRequestBody)
+_OPENAI_REQUEST_ADAPTER: Final[TypeAdapter[OpenAIDecisionRequestBody]] = TypeAdapter(OpenAIDecisionRequestBody)
+_DECISIONS_PAYLOAD_ADAPTER: Final[TypeAdapter[object]] = TypeAdapter(object)
+
+
+@dataclass(frozen=True, slots=True, repr=False)
+class _PreparedDecisionsRequest:
+    config: DecisionsProviderConfig
+    provider: str
+    upstream_model: str
+    url: str
+    api_key: str | None = field(repr=False)
+    headers: Mapping[str, str] = field(repr=False)
+    body: Mapping[str, object] = field(repr=False)
+    request: DecisionsRequestFormat = field(repr=False)
+    ir_request: DecisionsIRRequest = field(repr=False)
+
+
+def _resolve_provider_model(model: str, custom_llm_provider: str | None) -> tuple[str, str]:
+    provider: Final = model.partition("/")[0] if custom_llm_provider is None else custom_llm_provider
+    if provider not in DECISIONS_ENDPOINTS:
+        supported: Final = ", ".join(DECISIONS_ENDPOINTS)
+        raise litellm.BadRequestError(
+            message=f"Unknown Decisions provider '{provider}'. Supported providers: {supported}",
+            model=model,
+            llm_provider=provider,
+        )
+    upstream_model: Final = model.removeprefix(f"{provider}/") if model.startswith(f"{provider}/") else model
+    if not upstream_model:
+        raise litellm.BadRequestError(
+            message="A model name is required for the Decisions API",
+            model=model,
+            llm_provider=provider,
+        )
+    return provider, upstream_model
+
+
+def _resolve_api_key(
+    *,
+    provider: str,
+    model: str,
+    endpoint: DecisionsProviderConfig,
+    api_key: str | None,
+) -> str | None:
+    if api_key is not None:
+        return api_key
+    configured_api_key: Final = endpoint.configured_api_key()
+    if configured_api_key:
+        return configured_api_key
+    if not endpoint.api_key_required:
+        return None
+    raise litellm.AuthenticationError(
+        message=f"Missing API key for Decisions provider '{provider}'",
+        model=model,
+        llm_provider=provider,
+    )
+
+
+def _validate_request(
+    *,
+    state: DecisionsJSON | None,
+    questions: DecisionsQuestions | None,
+    decision_input: OpenAIDecisionInput | None,
+    safety_identifier: str | None,
+) -> DecisionsRequestFormat:
+    if decision_input is None:
+        return _SYSTEMONE_REQUEST_ADAPTER.validate_python({"state": state, "questions": questions})
+    return _OPENAI_REQUEST_ADAPTER.validate_python(
+        {"input": decision_input, "questions": questions, "safety_identifier": safety_identifier}
+    )
+
+
+def _ir_request(request: DecisionsRequestFormat) -> DecisionsIRRequest:
+    match request:
+        case DecisionsRequestBody():
+            return systemone_request_to_ir(request)
+        case OpenAIDecisionRequestBody():
+            return openai_request_to_ir(request)
+        case _:
+            assert_never(request)
+
+
+def _prepare_request(
+    *,
+    model: str,
+    state: DecisionsJSON | None,
+    questions: DecisionsQuestions | None,
+    decision_input: OpenAIDecisionInput | None,
+    safety_identifier: str | None,
+    api_key: str | None,
+    api_base: str | None,
+    custom_llm_provider: str | None,
+    extra_headers: Mapping[str, str] | None,
+) -> _PreparedDecisionsRequest:
+    provider, upstream_model = _resolve_provider_model(model, custom_llm_provider)
+    if state is not None and decision_input is not None:
+        raise litellm.BadRequestError(
+            message="Pass either state (System One format) or input (OpenAI format) to the Decisions API, not both",
+            model=model,
+            llm_provider=provider,
+        )
+    try:
+        validated_request: Final = _validate_request(
+            state=state, questions=questions, decision_input=decision_input, safety_identifier=safety_identifier
+        )
+    except ValidationError as error:
+        raise litellm.BadRequestError(
+            message=f"Invalid Decisions request: {error}",
+            model=model,
+            llm_provider=provider,
+        ) from error
+
+    endpoint: Final = DECISIONS_ENDPOINTS[provider]
+    resolved_api_base: Final = api_base or endpoint.configured_api_base()
+    if resolved_api_base is None:
+        raise litellm.BadRequestError(
+            message=endpoint.missing_api_base_message(provider),
+            model=model,
+            llm_provider=provider,
+        )
+
+    resolved_api_key: Final = _resolve_api_key(
+        provider=provider,
+        model=model,
+        endpoint=endpoint,
+        api_key=api_key,
+    )
+
+    canonical_model: Final = endpoint.canonical_model(upstream_model)
+    outbound_headers: Final = MappingProxyType(
+        {
+            **{
+                name: value
+                for name, value in (extra_headers or {}).items()
+                if name.lower() not in {"authorization", "content-type"}
+            },
+            **({"Authorization": f"Bearer {resolved_api_key}"} if resolved_api_key is not None else {}),
+            "Content-Type": "application/json",
+        }
+    )
+    ir_request: Final = _ir_request(validated_request)
+    body: Final = endpoint.request_body(endpoint.request_model(canonical_model), ir_request)
+    if isinstance(body, UnsupportedDecisionsRequest):
+        raise litellm.BadRequestError(
+            message=f"Decisions provider '{provider}' cannot serve this request: {body.reason}",
+            model=model,
+            llm_provider=provider,
+        )
+    return _PreparedDecisionsRequest(
+        config=endpoint,
+        provider=provider,
+        upstream_model=canonical_model,
+        url=endpoint.endpoint_url(resolved_api_base, canonical_model),
+        api_key=resolved_api_key,
+        headers=outbound_headers,
+        body=MappingProxyType(body),
+        request=validated_request,
+        ir_request=ir_request,
+    )
+
+
+def _log_request(
+    prepared: _PreparedDecisionsRequest,
+    kwargs: Mapping[str, object],
+) -> LiteLLMLoggingObj | None:
+    logging_obj: Final = kwargs.get("litellm_logging_obj")
+    if not isinstance(logging_obj, LiteLLMLoggingObj):
+        return None
+    logging_obj.update_from_kwargs(
+        kwargs=dict(kwargs),
+        model=prepared.upstream_model,
+        litellm_params={
+            "litellm_call_id": kwargs.get("litellm_call_id"),
+            "api_base": prepared.url,
+        },
+        custom_llm_provider=prepared.provider,
+    )
+    request_body: Final = dict(prepared.body)
+    request_headers: Final = dict(prepared.headers)
+    logging_obj.pre_call(
+        input=request_body,
+        api_key=prepared.api_key,
+        model=prepared.upstream_model,
+        additional_args={
+            "api_base": prepared.url,
+            "complete_input_dict": request_body,
+            "headers": request_headers,
+        },
+    )
+    return logging_obj
+
+
+def _format_response(
+    response: DecisionsIRResponse, prepared: _PreparedDecisionsRequest, model: str
+) -> DecisionsResponse | OpenAIDecisionResponse:
+    match prepared.request:
+        case DecisionsRequestBody():
+            return ir_to_systemone_response(response, prepared.ir_request)
+        case OpenAIDecisionRequestBody():
+            return ir_to_openai_response(response, prepared.ir_request, model)
+        case _:
+            assert_never(prepared.request)
+
+
+def _parse_response(
+    response: httpx.Response,
+    prepared: _PreparedDecisionsRequest,
+    model: str,
+) -> DecisionsResponse | OpenAIDecisionResponse:
+    response.raise_for_status()
+    payload: Final[object] = _DECISIONS_PAYLOAD_ADAPTER.validate_json(response.content)
+    result: Final = _format_response(prepared.config.parse_response(payload, prepared.ir_request), prepared, model)
+    result._hidden_params.update(
+        {
+            "model": f"{prepared.provider}/{prepared.upstream_model}",
+            "custom_llm_provider": prepared.provider,
+            "provider_response_model": f"{prepared.provider}/{prepared.upstream_model}",
+        }
+    )
+    return result
+
+
+def _map_upstream_exception(error: Exception, prepared: _PreparedDecisionsRequest) -> Exception:
+    return litellm.exception_type(
+        model=f"{prepared.provider}/{prepared.upstream_model}",
+        custom_llm_provider=prepared.provider,
+        original_exception=error,
+    )
+
+
+@client
+async def adecisions(
+    model: str,
+    state: DecisionsJSON | None = None,
+    questions: DecisionsQuestions | None = None,
+    api_key: str | None = None,
+    api_base: str | None = None,
+    timeout: float | httpx.Timeout | None = None,
+    custom_llm_provider: str | None = None,
+    extra_headers: Mapping[str, str] | None = None,
+    input: OpenAIDecisionInput | None = None,
+    safety_identifier: str | None = None,
+    **kwargs: object,
+) -> DecisionsResponse | OpenAIDecisionResponse:
+    prepared: Final = _prepare_request(
+        model=model,
+        state=state,
+        questions=questions,
+        decision_input=input,
+        safety_identifier=safety_identifier,
+        api_key=api_key,
+        api_base=api_base,
+        custom_llm_provider=custom_llm_provider,
+        extra_headers=extra_headers,
+    )
+    logging_obj: Final = _log_request(prepared, kwargs)
+    try:
+        handler: Final = get_async_httpx_client(llm_provider=prepared.provider)
+        response: Final = await handler.post(
+            prepared.url,
+            json=dict(prepared.body),
+            headers=dict(prepared.headers),
+            timeout=timeout,
+            logging_obj=logging_obj,
+        )
+        return _parse_response(response=response, prepared=prepared, model=model)
+    except Exception as error:
+        raise _map_upstream_exception(error, prepared) from error
+
+
+@client
+def decisions(
+    model: str,
+    state: DecisionsJSON | None = None,
+    questions: DecisionsQuestions | None = None,
+    api_key: str | None = None,
+    api_base: str | None = None,
+    timeout: float | httpx.Timeout | None = None,
+    custom_llm_provider: str | None = None,
+    extra_headers: Mapping[str, str] | None = None,
+    input: OpenAIDecisionInput | None = None,
+    safety_identifier: str | None = None,
+    **kwargs: object,
+) -> DecisionsResponse | OpenAIDecisionResponse:
+    prepared: Final = _prepare_request(
+        model=model,
+        state=state,
+        questions=questions,
+        decision_input=input,
+        safety_identifier=safety_identifier,
+        api_key=api_key,
+        api_base=api_base,
+        custom_llm_provider=custom_llm_provider,
+        extra_headers=extra_headers,
+    )
+    logging_obj: Final = _log_request(prepared, kwargs)
+    try:
+        handler: Final = _get_httpx_client()
+        response: Final = handler.post(
+            prepared.url,
+            json=dict(prepared.body),
+            headers=dict(prepared.headers),
+            timeout=timeout,
+            logging_obj=logging_obj,
+        )
+        return _parse_response(response=response, prepared=prepared, model=model)
+    except Exception as error:
+        raise _map_upstream_exception(error, prepared) from error
+
+
+__all__ = ["DECISIONS_ENDPOINTS", "adecisions", "decisions"]

litellm/litellm_core_utils/health_check_helpers.py

@@ -168,6 +168,7 @@ class HealthCheckHelpers:
             "batch",
             "responses",
             "ocr",
+            "evaluation",
         ],
         Callable,
     ]:
@@ -190,7 +191,7 @@ class HealthCheckHelpers:
         from litellm.litellm_core_utils.audio_utils.utils import (
             get_audio_file_for_health_check,
         )
-        from litellm.litellm_core_utils.health_check_utils import _filter_model_params
+        from litellm.litellm_core_utils.health_check_utils import DECISIONS_CALL_PARAMS, _filter_model_params
         from litellm.realtime_api.main import _realtime_health_check
 
         return {
@@ -257,4 +258,13 @@ class HealthCheckHelpers:
                 **_filter_model_params(model_params=model_params),
                 document=_ocr_health_check_document(model=model, custom_llm_provider=custom_llm_provider),
             ),
+            "evaluation": lambda: litellm.adecisions(
+                **DECISIONS_CALL_PARAMS.validate_python(
+                    {
+                        "state": prompt or "health check",
+                        "questions": {"reachable": {"type": "noul", "instructions": "Is the service reachable?"}},
+                        **_filter_model_params(model_params=model_params),
+                    }
+                )
+            ),
         }

litellm/litellm_core_utils/health_check_utils.py

@@ -4,6 +4,12 @@ Utils used for litellm.ahealth_check()
 
 from typing import Final
 
+from pydantic import TypeAdapter
+
+from litellm.types.decisions import DecisionsCallParams
+
+DECISIONS_CALL_PARAMS: Final[TypeAdapter[DecisionsCallParams]] = TypeAdapter(DecisionsCallParams)
+
 
 def _filter_model_params(model_params: dict) -> dict:
     """Remove 'messages' param from model params."""

litellm/litellm_core_utils/litellm_logging.py

@@ -118,6 +118,7 @@ from litellm.llms.base_llm.search.transformation import SearchResponse
 from litellm.responses.utils import ResponseAPILoggingUtils
 from litellm.types.agents import LiteLLMSendMessageResponse
 from litellm.types.containers.main import ContainerObject
+from litellm.types.decisions import DecisionsResponse, OpenAIDecisionResponse
 from litellm.types.integrations.s3_v2 import S3PartitionGranularity
 from litellm.types.interactions import (
     InteractionsAPIResponse,
@@ -1812,6 +1813,7 @@ class Logging(LiteLLMLoggingBaseClass):
             LiteLLMRealtimeStreamLoggingObject,
             OpenAIModerationResponse,
             "SearchResponse",
+            DecisionsResponse,
             dict,
             list,
         ],
@@ -2588,6 +2590,8 @@ class Logging(LiteLLMLoggingBaseClass):
             or isinstance(logging_result, OpenAIModerationResponse)
             or isinstance(logging_result, OCRResponse)  # OCR
             or isinstance(logging_result, SearchResponse)  # Search API
+            or isinstance(logging_result, DecisionsResponse)
+            or isinstance(logging_result, OpenAIDecisionResponse)
             or (
                 isinstance(logging_result, InteractionsAPIResponse)
                 and logging_result.usage is not None

litellm/llms/base_llm/decisions/__init__.py

@@ -0,0 +1,3 @@
+from .transformation import DecisionsProviderConfig, JevCompatibleDecisionsEndpoint
+
+__all__ = ["DecisionsProviderConfig", "JevCompatibleDecisionsEndpoint"]

litellm/llms/base_llm/decisions/transformation.py

@@ -0,0 +1,350 @@
+import json
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from types import MappingProxyType
+from typing import Final, Protocol
+
+from pydantic import TypeAdapter
+from typing_extensions import assert_never
+
+from litellm.secret_managers.main import get_secret_str
+from litellm.types.decisions import (
+    ChoiceAnswer,
+    ChoiceQuestion,
+    DecisionAnswer,
+    DecisionQuestion,
+    DecisionsIRAnswer,
+    DecisionsIRChoiceAnswer,
+    DecisionsIRChoiceOption,
+    DecisionsIRChoiceProbability,
+    DecisionsIRChoiceQuestion,
+    DecisionsIRMessages,
+    DecisionsIRPredicateAnswer,
+    DecisionsIRPredicateQuestion,
+    DecisionsIRQuestion,
+    DecisionsIRRefusal,
+    DecisionsIRRequest,
+    DecisionsIRResponse,
+    DecisionsIRScoreAnswer,
+    DecisionsIRScoreLevel,
+    DecisionsIRScoreProbability,
+    DecisionsIRScoreQuestion,
+    DecisionsIRState,
+    DecisionsIRUsage,
+    DecisionsJSON,
+    DecisionsRequestBody,
+    DecisionsResponse,
+    DecisionsUsage,
+    NoulAnswer,
+    NoulQuestion,
+    OpenAIDecisionInputImage,
+    OpenAIDecisionInputMessage,
+    OpenAIDecisionInputText,
+    ScoreAnswer,
+    ScoreQuestion,
+    UnsupportedDecisionsRequest,
+    systemone_choice_key,
+)
+
+_SYSTEMONE_RESPONSE_ADAPTER: Final[TypeAdapter[DecisionsResponse]] = TypeAdapter(DecisionsResponse)
+_TEXT_ONLY: Final = UnsupportedDecisionsRequest(
+    reason="input_image content parts are not supported because System One providers accept text input only"
+)
+
+
+def decisions_text(value: DecisionsJSON) -> str:
+    return value if isinstance(value, str) else json.dumps(value)
+
+
+def systemone_keys(questions: Sequence[DecisionsIRQuestion]) -> tuple[str, ...]:
+    names: Final = tuple(question.name for question in questions if question.name is not None)
+    if len(frozenset(names)) == len(questions):
+        return names
+    return tuple(str(index) for index in range(len(questions)))
+
+
+def _ir_question(name: str, question: DecisionQuestion) -> DecisionsIRQuestion:
+    extra: Final = MappingProxyType(question.model_extra or {})
+    match question:
+        case NoulQuestion():
+            return DecisionsIRPredicateQuestion(
+                name=name, instructions=question.instructions, criteria=question.criteria, extra=extra
+            )
+        case ChoiceQuestion():
+            return DecisionsIRChoiceQuestion(
+                name=name,
+                instructions=question.instructions,
+                choices=tuple(
+                    DecisionsIRChoiceOption(value=value, description=description)
+                    for value, description in question.criteria.items()
+                ),
+                extra=extra,
+            )
+        case ScoreQuestion():
+            return DecisionsIRScoreQuestion(
+                name=name,
+                instructions=question.instructions,
+                levels=tuple(
+                    DecisionsIRScoreLevel(label=criterion, description=None) for criterion in question.criteria
+                ),
+                extra=extra,
+            )
+        case _:
+            assert_never(question)
+
+
+def systemone_request_to_ir(request: DecisionsRequestBody) -> DecisionsIRRequest:
+    return DecisionsIRRequest(
+        input=DecisionsIRState(state=request.state),
+        questions=tuple(_ir_question(name, question) for name, question in request.questions.items()),
+    )
+
+
+def _has_image(message: OpenAIDecisionInputMessage) -> bool:
+    return not isinstance(message.content, str) and any(
+        isinstance(part, OpenAIDecisionInputImage) for part in message.content
+    )
+
+
+def _message_text(message: OpenAIDecisionInputMessage) -> str:
+    if isinstance(message.content, str):
+        return message.content
+    return "\n\n".join(part.text for part in message.content if isinstance(part, OpenAIDecisionInputText))
+
+
+def _systemone_state(
+    decision_input: DecisionsIRState | DecisionsIRMessages,
+) -> DecisionsJSON | UnsupportedDecisionsRequest:
+    match decision_input:
+        case DecisionsIRState():
+            return decision_input.state
+        case DecisionsIRMessages():
+            if any(_has_image(message) for message in decision_input.messages):
+                return _TEXT_ONLY
+            return "\n\n".join(_message_text(message) for message in decision_input.messages)
+        case _:
+            assert_never(decision_input)
+
+
+def _optional(key: str, value: object) -> Mapping[str, object]:
+    return {} if value is None else {key: value}
+
+
+def _level_criterion(level: DecisionsIRScoreLevel) -> DecisionsJSON:
+    return level.label if level.description is None else f"{decisions_text(level.label)}: {level.description}"
+
+
+def _systemone_question(question: DecisionsIRQuestion) -> Mapping[str, object]:
+    match question:
+        case DecisionsIRPredicateQuestion():
+            return {
+                "type": "noul",
+                **_optional("instructions", question.instructions),
+                **_optional("criteria", question.criteria),
+                **question.extra,
+            }
+        case DecisionsIRChoiceQuestion():
+            return {
+                "type": "choice",
+                **_optional("instructions", question.instructions),
+                "criteria": {systemone_choice_key(option.value): option.description for option in question.choices},
+                **question.extra,
+            }
+        case DecisionsIRScoreQuestion():
+            return {
+                "type": "score",
+                **_optional("instructions", question.instructions),
+                "criteria": [_level_criterion(level) for level in question.levels],
+                **question.extra,
+            }
+        case _:
+            assert_never(question)
+
+
+def ir_to_systemone_request(
+    model: str, request: DecisionsIRRequest
+) -> Mapping[str, object] | UnsupportedDecisionsRequest:
+    state: Final = _systemone_state(request.input)
+    if isinstance(state, UnsupportedDecisionsRequest):
+        return state
+    keyed_questions: Final = zip(systemone_keys(request.questions), request.questions, strict=True)
+    return {
+        "model": model,
+        "state": state,
+        "questions": {key: _systemone_question(question) for key, question in keyed_questions},
+    }
+
+
+def _ir_choice_answer(question: DecisionsIRChoiceQuestion, answer: ChoiceAnswer) -> DecisionsIRChoiceAnswer:
+    typed_values: Final = {systemone_choice_key(option.value): option.value for option in question.choices}
+    return DecisionsIRChoiceAnswer(
+        choice=typed_values.get(answer.choice, answer.choice),
+        confidence=answer.confidence,
+        probabilities=tuple(
+            DecisionsIRChoiceProbability(value=typed_values.get(key, key), probability=probability)
+            for key, probability in answer.probabilities.items()
+        ),
+        extra=MappingProxyType(answer.model_extra or {}),
+    )
+
+
+def _ir_score_answer(question: DecisionsIRScoreQuestion, answer: ScoreAnswer) -> DecisionsIRScoreAnswer:
+    return DecisionsIRScoreAnswer(
+        score=answer.score,
+        confidence=answer.confidence,
+        probabilities=tuple(
+            DecisionsIRScoreProbability(
+                value=index,
+                label=answer.legend.get(str(index), level.label),
+                probability=answer.probabilities.get(str(index), 0.0),
+            )
+            for index, level in enumerate(question.levels)
+        ),
+        extra=MappingProxyType(answer.model_extra or {}),
+    )
+
+
+def _ir_answer(question: DecisionsIRQuestion, answer: DecisionAnswer | None) -> DecisionsIRAnswer:
+    match question, answer:
+        case DecisionsIRPredicateQuestion(), NoulAnswer():
+            return DecisionsIRPredicateAnswer(probability=answer.noul, extra=MappingProxyType(answer.model_extra or {}))
+        case DecisionsIRChoiceQuestion(), ChoiceAnswer():
+            return _ir_choice_answer(question, answer)
+        case DecisionsIRScoreQuestion(), ScoreAnswer():
+            return _ir_score_answer(question, answer)
+        case _:
+            return DecisionsIRRefusal()
+
+
+def systemone_response_to_ir(response: DecisionsResponse, request: DecisionsIRRequest) -> DecisionsIRResponse:
+    usage: Final = response.usage or DecisionsUsage()
+    keyed_questions: Final = zip(systemone_keys(request.questions), request.questions, strict=True)
+    return DecisionsIRResponse(
+        model=response.model,
+        answers=tuple(_ir_answer(question, response.answers.get(key)) for key, question in keyed_questions),
+        usage=DecisionsIRUsage(
+            input_tokens=usage.input_tokens,
+            output_tokens=usage.output_tokens,
+            cached_tokens=usage.cached_tokens,
+            cache_write_tokens=usage.cache_write_tokens,
+            extra=MappingProxyType(usage.model_extra or {}),
+        ),
+        extra=MappingProxyType(response.model_extra or {}),
+    )
+
+
+def parse_systemone_response(payload: object, request: DecisionsIRRequest) -> DecisionsIRResponse:
+    return systemone_response_to_ir(_SYSTEMONE_RESPONSE_ADAPTER.validate_python(payload), request)
+
+
+def _systemone_answer(answer: DecisionsIRAnswer) -> DecisionAnswer | None:
+    match answer:
+        case DecisionsIRPredicateAnswer():
+            return NoulAnswer.model_validate({**answer.extra, "type": "noul", "noul": answer.probability})
+        case DecisionsIRChoiceAnswer():
+            return ChoiceAnswer.model_validate(
+                {
+                    **answer.extra,
+                    "type": "choice",
+                    "choice": systemone_choice_key(answer.choice),
+                    "confidence": answer.confidence,
+                    "probabilities": {
+                        systemone_choice_key(item.value): item.probability for item in answer.probabilities
+                    },
+                }
+            )
+        case DecisionsIRScoreAnswer():
+            return ScoreAnswer.model_validate(
+                {
+                    **answer.extra,
+                    "type": "score",
+                    "score": answer.score,
+                    "confidence": answer.confidence,
+                    "legend": {str(item.value): item.label for item in answer.probabilities},
+                    "probabilities": {str(item.value): item.probability for item in answer.probabilities},
+                }
+            )
+        case DecisionsIRRefusal():
+            return None
+        case _:
+            assert_never(answer)
+
+
+def ir_to_systemone_response(response: DecisionsIRResponse, request: DecisionsIRRequest) -> DecisionsResponse:
+    keyed_answers: Final = zip(
+        systemone_keys(request.questions), (_systemone_answer(answer) for answer in response.answers), strict=True
+    )
+    return DecisionsResponse.model_validate(
+        {
+            **response.extra,
+            "model": response.model,
+            "answers": {key: answer for key, answer in keyed_answers if answer is not None},
+            "usage": DecisionsUsage.model_validate(
+                {
+                    **response.usage.extra,
+                    "input_tokens": response.usage.input_tokens,
+                    "output_tokens": response.usage.output_tokens,
+                    "cached_tokens": response.usage.cached_tokens,
+                    "cache_write_tokens": response.usage.cache_write_tokens,
+                }
+            ),
+        }
+    )
+
+
+@dataclass(frozen=True, slots=True)
+class JevCompatibleDecisionsEndpoint:
+    default_api_base_value: str | None
+    path: str
+    api_key_env: tuple[str, ...]
+    api_base_env: str
+    api_key_required: bool = True
+
+    def configured_api_key(self) -> str | None:
+        return next((key for key in (get_secret_str(name) for name in self.api_key_env) if key), None)
+
+    def configured_api_base(self) -> str | None:
+        return get_secret_str(self.api_base_env) or self.default_api_base_value
+
+    def missing_api_base_message(self, provider: str) -> str:
+        return f"api_base is required for Decisions provider '{provider}'"
+
+    def canonical_model(self, model: str) -> str:
+        return model
+
+    def request_model(self, model: str) -> str:
+        return model
+
+    def endpoint_url(self, api_base: str, model: str) -> str:
+        return f"{api_base.rstrip('/').removesuffix('/v1')}{self.path}"
+
+    def request_body(
+        self, model: str, request: DecisionsIRRequest
+    ) -> Mapping[str, object] | UnsupportedDecisionsRequest:
+        return ir_to_systemone_request(model, request)
+
+    def parse_response(self, payload: object, request: DecisionsIRRequest) -> DecisionsIRResponse:
+        return parse_systemone_response(payload, request)
+
+
+class DecisionsProviderConfig(Protocol):
+    @property
+    def api_key_required(self) -> bool: ...
+
+    def configured_api_key(self) -> str | None: ...
+
+    def configured_api_base(self) -> str | None: ...
+
+    def missing_api_base_message(self, provider: str) -> str: ...
+
+    def canonical_model(self, model: str) -> str: ...
+
+    def request_model(self, model: str) -> str: ...
+
+    def endpoint_url(self, api_base: str, model: str) -> str: ...
+
+    def request_body(
+        self, model: str, request: DecisionsIRRequest
+    ) -> Mapping[str, object] | UnsupportedDecisionsRequest: ...
+
+    def parse_response(self, payload: object, request: DecisionsIRRequest) -> DecisionsIRResponse: ...

litellm/llms/cloudflare/decisions/transformation.py

@@ -0,0 +1,73 @@
+from collections.abc import Mapping
+from dataclasses import dataclass
+from typing import Final
+
+from pydantic import TypeAdapter
+
+from litellm.llms.base_llm.decisions.transformation import ir_to_systemone_request, parse_systemone_response
+from litellm.secret_managers.main import (
+    get_secret_str,
+    normalize_nonempty_secret_str,
+)
+from litellm.types.decisions import DecisionsIRRequest, DecisionsIRResponse, UnsupportedDecisionsRequest
+
+_RESPONSE_MAPPING_ADAPTER: Final[TypeAdapter[Mapping[str, object]]] = TypeAdapter(Mapping[str, object])
+
+
+@dataclass(frozen=True, slots=True)
+class CloudflareDecisionsEndpoint:
+    api_key_required: bool = True
+
+    def configured_api_key(self) -> str | None:
+        return get_secret_str("CLOUDFLARE_API_KEY") or None
+
+    def configured_api_base(self) -> str | None:
+        env_api_base: Final = get_secret_str("CLOUDFLARE_API_BASE")
+        if env_api_base:
+            return env_api_base
+        account_id: Final = normalize_nonempty_secret_str(get_secret_str("CLOUDFLARE_ACCOUNT_ID"))
+        if account_id is None:
+            return None
+        return f"https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run"
+
+    def missing_api_base_message(self, provider: str) -> str:
+        return "Missing CLOUDFLARE_ACCOUNT_ID - set CLOUDFLARE_ACCOUNT_ID or pass api_base explicitly"
+
+    def canonical_model(self, model: str) -> str:
+        if model.startswith("@cf/"):
+            return model
+        return f"@cf/cloudflare/{model}"
+
+    def request_model(self, model: str) -> str:
+        return model.rsplit("/", maxsplit=1)[-1]
+
+    def endpoint_url(self, api_base: str, model: str) -> str:
+        normalized_api_base: Final = api_base.rstrip("/")
+        if normalized_api_base.endswith("/ai/v1"):
+            return f"{normalized_api_base.removesuffix('/ai/v1')}/ai/run/{model}"
+        if normalized_api_base.endswith("/ai/run"):
+            return f"{normalized_api_base}/{model}"
+        return f"{normalized_api_base}/ai/run/{model}"
+
+    def request_body(
+        self, model: str, request: DecisionsIRRequest
+    ) -> Mapping[str, object] | UnsupportedDecisionsRequest:
+        return ir_to_systemone_request(model, request)
+
+    def parse_response(self, payload: object, request: DecisionsIRRequest) -> DecisionsIRResponse:
+        return parse_systemone_response(_unwrap_result(payload), request)
+
+
+def _unwrap_result(payload: object) -> object:
+    if not isinstance(payload, Mapping):
+        return payload
+    response_mapping: Final = _RESPONSE_MAPPING_ADAPTER.validate_python(payload)
+    if "answers" in response_mapping:
+        return payload
+    result: Final = response_mapping.get("result")
+    if isinstance(result, Mapping):
+        return result
+    return payload
+
+
+CLOUDFLARE_DECISIONS_ENDPOINT: Final[CloudflareDecisionsEndpoint] = CloudflareDecisionsEndpoint()

litellm/llms/openai/decisions/transformation.py

@@ -0,0 +1,330 @@
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from typing import Final
+
+from pydantic import TypeAdapter
+from typing_extensions import assert_never
+
+import litellm
+from litellm.llms.base_llm.decisions.transformation import decisions_text
+from litellm.secret_managers.main import get_secret_str
+from litellm.types.decisions import (
+    DecisionsIRAnswer,
+    DecisionsIRChoiceAnswer,
+    DecisionsIRChoiceOption,
+    DecisionsIRChoiceProbability,
+    DecisionsIRChoiceQuestion,
+    DecisionsIRMessages,
+    DecisionsIRPredicateAnswer,
+    DecisionsIRPredicateQuestion,
+    DecisionsIRQuestion,
+    DecisionsIRRefusal,
+    DecisionsIRRequest,
+    DecisionsIRResponse,
+    DecisionsIRScoreAnswer,
+    DecisionsIRScoreLevel,
+    DecisionsIRScoreProbability,
+    DecisionsIRScoreQuestion,
+    DecisionsIRState,
+    DecisionsIRUsage,
+    DecisionsJSON,
+    OpenAIChoiceAnswer,
+    OpenAIChoiceProbability,
+    OpenAIChoiceQuestion,
+    OpenAIDecisionAnswer,
+    OpenAIDecisionInput,
+    OpenAIDecisionInputTokensDetails,
+    OpenAIDecisionOutputTokensDetails,
+    OpenAIDecisionQuestion,
+    OpenAIDecisionRequestBody,
+    OpenAIDecisionResponse,
+    OpenAIDecisionUsage,
+    OpenAIPredicateAnswer,
+    OpenAIPredicateQuestion,
+    OpenAIRefusalAnswer,
+    OpenAIScoreAnswer,
+    OpenAIScoreProbability,
+    OpenAIScoreQuestion,
+    UnsupportedDecisionsRequest,
+    systemone_choice_key,
+)
+
+_OPENAI_RESPONSE_ADAPTER: Final[TypeAdapter[OpenAIDecisionResponse]] = TypeAdapter(OpenAIDecisionResponse)
+_SINGLE_OPTION: Final = UnsupportedDecisionsRequest(
+    reason="OpenAI needs at least 2 choices or levels on every choice or score question"
+)
+
+
+def _ir_input(decision_input: OpenAIDecisionInput) -> DecisionsIRState | DecisionsIRMessages:
+    if isinstance(decision_input, str):
+        return DecisionsIRState(state=decision_input)
+    return DecisionsIRMessages(messages=tuple(decision_input))
+
+
+def _ir_question(question: OpenAIDecisionQuestion) -> DecisionsIRQuestion:
+    match question:
+        case OpenAIPredicateQuestion():
+            return DecisionsIRPredicateQuestion(name=question.name, instructions=question.instructions)
+        case OpenAIChoiceQuestion():
+            return DecisionsIRChoiceQuestion(
+                name=question.name,
+                instructions=question.instructions,
+                choices=tuple(
+                    DecisionsIRChoiceOption(value=option.value, description=option.description)
+                    for option in question.choices
+                ),
+            )
+        case OpenAIScoreQuestion():
+            return DecisionsIRScoreQuestion(
+                name=question.name,
+                instructions=question.instructions,
+                levels=tuple(
+                    DecisionsIRScoreLevel(label=level.label, description=level.description) for level in question.levels
+                ),
+            )
+        case _:
+            assert_never(question)
+
+
+def openai_request_to_ir(request: OpenAIDecisionRequestBody) -> DecisionsIRRequest:
+    return DecisionsIRRequest(
+        input=_ir_input(request.input),
+        questions=tuple(_ir_question(question) for question in request.questions),
+        safety_identifier=request.safety_identifier,
+    )
+
+
+def _text_field(key: str, value: DecisionsJSON | None) -> Mapping[str, str]:
+    return {} if value is None else {key: decisions_text(value)}
+
+
+def _instructions(value: DecisionsJSON | None, default: str) -> str:
+    return default if value is None else decisions_text(value)
+
+
+def _predicate_instructions(question: DecisionsIRPredicateQuestion) -> str:
+    instructions: Final = () if question.instructions is None else (decisions_text(question.instructions),)
+    criteria: Final = tuple(
+        f"Answer {answer} when: {decisions_text(rule)}"
+        for answer, rule in (question.criteria or {}).items()
+        if rule is not None
+    )
+    return "\n\n".join((*instructions, *criteria))
+
+
+def _openai_question(question: DecisionsIRQuestion) -> Mapping[str, object]:
+    match question:
+        case DecisionsIRPredicateQuestion():
+            return {
+                "type": "predicate",
+                **_text_field("name", question.name),
+                "instructions": _predicate_instructions(question) or "Is this true of the input?",
+            }
+        case DecisionsIRChoiceQuestion():
+            return {
+                "type": "choice",
+                **_text_field("name", question.name),
+                "instructions": _instructions(question.instructions, "Which choice best fits the input?"),
+                "choices": [
+                    {"value": option.value, **_text_field("description", option.description)}
+                    for option in question.choices
+                ],
+            }
+        case DecisionsIRScoreQuestion():
+            return {
+                "type": "score",
+                **_text_field("name", question.name),
+                "instructions": _instructions(question.instructions, "Which level best fits the input?"),
+                "levels": [
+                    {"label": decisions_text(level.label), **_text_field("description", level.description)}
+                    for level in question.levels
+                ],
+            }
+        case _:
+            assert_never(question)
+
+
+def _openai_input(decision_input: DecisionsIRState | DecisionsIRMessages) -> str | Sequence[Mapping[str, object]]:
+    match decision_input:
+        case DecisionsIRState():
+            return decisions_text(decision_input.state)
+        case DecisionsIRMessages():
+            return [message.model_dump(mode="json", exclude_none=True) for message in decision_input.messages]
+        case _:
+            assert_never(decision_input)
+
+
+def _has_one_option(question: DecisionsIRQuestion) -> bool:
+    match question:
+        case DecisionsIRChoiceQuestion():
+            return len(question.choices) < 2
+        case DecisionsIRScoreQuestion():
+            return len(question.levels) < 2
+        case _:
+            return False
+
+
+def ir_to_openai_request(model: str, request: DecisionsIRRequest) -> Mapping[str, object] | UnsupportedDecisionsRequest:
+    if any(_has_one_option(question) for question in request.questions):
+        return _SINGLE_OPTION
+    return {
+        "model": model,
+        "input": _openai_input(request.input),
+        "questions": [_openai_question(question) for question in request.questions],
+        **_text_field("safety_identifier", request.safety_identifier),
+    }
+
+
+def _level_label(question: DecisionsIRScoreQuestion, probability: OpenAIScoreProbability) -> DecisionsJSON:
+    if 0 <= probability.value < len(question.levels):
+        return question.levels[probability.value].label
+    return probability.label
+
+
+def _ir_answer(question: DecisionsIRQuestion, answer: OpenAIDecisionAnswer | None) -> DecisionsIRAnswer:
+    match question, answer:
+        case DecisionsIRPredicateQuestion(), OpenAIPredicateAnswer():
+            return DecisionsIRPredicateAnswer(probability=answer.probability)
+        case DecisionsIRChoiceQuestion(), OpenAIChoiceAnswer():
+            return DecisionsIRChoiceAnswer(
+                choice=answer.choice,
+                confidence=answer.confidence,
+                probabilities=tuple(
+                    DecisionsIRChoiceProbability(value=item.value, probability=item.probability)
+                    for item in answer.probabilities
+                ),
+            )
+        case DecisionsIRScoreQuestion(), OpenAIScoreAnswer():
+            return DecisionsIRScoreAnswer(
+                score=answer.score,
+                confidence=answer.confidence,
+                probabilities=tuple(
+                    DecisionsIRScoreProbability(
+                        value=item.value, label=_level_label(question, item), probability=item.probability
+                    )
+                    for item in answer.probabilities
+                ),
+            )
+        case _:
+            return DecisionsIRRefusal()
+
+
+def openai_response_to_ir(response: OpenAIDecisionResponse, request: DecisionsIRRequest) -> DecisionsIRResponse:
+    answers: Final = response.answers
+    return DecisionsIRResponse(
+        model=response.model,
+        answers=tuple(
+            _ir_answer(question, answers[index] if index < len(answers) else None)
+            for index, question in enumerate(request.questions)
+        ),
+        usage=DecisionsIRUsage(
+            input_tokens=response.usage.input_tokens,
+            output_tokens=response.usage.output_tokens,
+            cached_tokens=response.usage.input_tokens_details.cached_tokens,
+            cache_write_tokens=response.usage.input_tokens_details.cache_write_tokens,
+            reasoning_tokens=response.usage.output_tokens_details.reasoning_tokens,
+        ),
+    )
+
+
+def _openai_choice_answer(question: DecisionsIRChoiceQuestion, answer: DecisionsIRChoiceAnswer) -> OpenAIChoiceAnswer:
+    probabilities: Final = {systemone_choice_key(item.value): item.probability for item in answer.probabilities}
+    return OpenAIChoiceAnswer(
+        name=question.name,
+        choice=answer.choice,
+        probabilities=tuple(
+            OpenAIChoiceProbability(
+                value=option.value, probability=probabilities.get(systemone_choice_key(option.value), 0.0)
+            )
+            for option in question.choices
+        ),
+        confidence=answer.confidence,
+    )
+
+
+def _openai_score_answer(question: DecisionsIRScoreQuestion, answer: DecisionsIRScoreAnswer) -> OpenAIScoreAnswer:
+    probabilities: Final = {item.value: item.probability for item in answer.probabilities}
+    return OpenAIScoreAnswer(
+        name=question.name,
+        score=answer.score,
+        probabilities=tuple(
+            OpenAIScoreProbability(
+                value=index, label=decisions_text(level.label), probability=probabilities.get(index, 0.0)
+            )
+            for index, level in enumerate(question.levels)
+        ),
+        confidence=answer.confidence,
+    )
+
+
+def _openai_answer(question: DecisionsIRQuestion, answer: DecisionsIRAnswer) -> OpenAIDecisionAnswer:
+    match question, answer:
+        case DecisionsIRPredicateQuestion(), DecisionsIRPredicateAnswer():
+            return OpenAIPredicateAnswer(name=question.name, probability=answer.probability)
+        case DecisionsIRChoiceQuestion(), DecisionsIRChoiceAnswer():
+            return _openai_choice_answer(question, answer)
+        case DecisionsIRScoreQuestion(), DecisionsIRScoreAnswer():
+            return _openai_score_answer(question, answer)
+        case _:
+            return OpenAIRefusalAnswer(name=question.name)
+
+
+def ir_to_openai_response(
+    response: DecisionsIRResponse, request: DecisionsIRRequest, requested_model: str
+) -> OpenAIDecisionResponse:
+    return OpenAIDecisionResponse(
+        model=response.model or requested_model,
+        answers=tuple(
+            _openai_answer(question, answer)
+            for question, answer in zip(request.questions, response.answers, strict=True)
+        ),
+        usage=OpenAIDecisionUsage(
+            input_tokens=response.usage.input_tokens,
+            input_tokens_details=OpenAIDecisionInputTokensDetails(
+                cached_tokens=response.usage.cached_tokens,
+                cache_write_tokens=response.usage.cache_write_tokens,
+            ),
+            output_tokens=response.usage.output_tokens,
+            output_tokens_details=OpenAIDecisionOutputTokensDetails(reasoning_tokens=response.usage.reasoning_tokens),
+            total_tokens=response.usage.input_tokens + response.usage.output_tokens,
+        ),
+    )
+
+
+@dataclass(frozen=True, slots=True)
+class OpenAIDecisionsEndpoint:
+    api_key_required: bool = True
+
+    def configured_api_key(self) -> str | None:
+        return litellm.api_key or litellm.openai_key or get_secret_str("OPENAI_API_KEY")
+
+    def configured_api_base(self) -> str | None:
+        return (
+            litellm.api_base
+            or get_secret_str("OPENAI_BASE_URL")
+            or get_secret_str("OPENAI_API_BASE")
+            or "https://api.openai.com"
+        )
+
+    def missing_api_base_message(self, provider: str) -> str:
+        return f"api_base is required for Decisions provider '{provider}'"
+
+    def canonical_model(self, model: str) -> str:
+        return model
+
+    def request_model(self, model: str) -> str:
+        return model
+
+    def endpoint_url(self, api_base: str, model: str) -> str:
+        return f"{api_base.rstrip('/').removesuffix('/v1')}/v1/decisions"
+
+    def request_body(
+        self, model: str, request: DecisionsIRRequest
+    ) -> Mapping[str, object] | UnsupportedDecisionsRequest:
+        return ir_to_openai_request(model, request)
+
+    def parse_response(self, payload: object, request: DecisionsIRRequest) -> DecisionsIRResponse:
+        return openai_response_to_ir(_OPENAI_RESPONSE_ADAPTER.validate_python(payload), request)
+
+
+OPENAI_DECISIONS_ENDPOINT: Final[OpenAIDecisionsEndpoint] = OpenAIDecisionsEndpoint()

litellm/llms/openrouter/decisions/transformation.py

@@ -0,0 +1,10 @@
+from typing import Final
+
+from litellm.llms.base_llm.decisions.transformation import JevCompatibleDecisionsEndpoint
+
+OPENROUTER_DECISIONS_ENDPOINT: Final[JevCompatibleDecisionsEndpoint] = JevCompatibleDecisionsEndpoint(
+    default_api_base_value="https://openrouter.ai/api",
+    path="/alpha/decisions",
+    api_key_env=("OPENROUTER_API_KEY",),
+    api_base_env="OPENROUTER_API_BASE",
+)

litellm/llms/perplexity/decisions/transformation.py

@@ -0,0 +1,10 @@
+from typing import Final
+
+from litellm.llms.base_llm.decisions.transformation import JevCompatibleDecisionsEndpoint
+
+PERPLEXITY_DECISIONS_ENDPOINT: Final[JevCompatibleDecisionsEndpoint] = JevCompatibleDecisionsEndpoint(
+    default_api_base_value="https://api.perplexity.ai",
+    path="/v1/decisions",
+    api_key_env=("PERPLEXITYAI_API_KEY", "PERPLEXITY_API_KEY"),
+    api_base_env="PERPLEXITY_API_BASE",
+)

litellm/llms/strands_decider/decisions/transformation.py

@@ -0,0 +1,11 @@
+from typing import Final
+
+from litellm.llms.base_llm.decisions.transformation import JevCompatibleDecisionsEndpoint
+
+STRANDS_DECIDER_DECISIONS_ENDPOINT: Final[JevCompatibleDecisionsEndpoint] = JevCompatibleDecisionsEndpoint(
+    default_api_base_value=None,
+    path="/v1/systemone",
+    api_key_env=("STRANDS_DECIDER_API_KEY",),
+    api_base_env="STRANDS_DECIDER_API_BASE",
+    api_key_required=False,
+)

litellm/llms/typesafe/decisions/transformation.py

@@ -0,0 +1,10 @@
+from typing import Final
+
+from litellm.llms.base_llm.decisions.transformation import JevCompatibleDecisionsEndpoint
+
+TYPESAFE_DECISIONS_ENDPOINT: Final[JevCompatibleDecisionsEndpoint] = JevCompatibleDecisionsEndpoint(
+    default_api_base_value="https://api.typesafe.ai",
+    path="/v1/systemone",
+    api_key_env=("TYPESAFE_API_KEY",),
+    api_base_env="TYPESAFE_API_BASE",
+)

litellm/model_prices_and_context_window_backup.json

@@ -34158,7 +34158,8 @@
         "supported_endpoints": [
             "/v1/chat/completions",
             "/v1/batch",
-            "/v1/responses"
+            "/v1/responses",
+            "/v1/decisions"
         ],
         "supported_modalities": [
             "text",
@@ -79714,5 +79715,55 @@
         "supports_response_schema": true,
         "supports_tool_choice": true,
         "supports_vision": true
+    },
+    "cloudflare/clef": {
+        "input_cost_per_token": 2.4e-07,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "cloudflare/clef-flash": {
+        "input_cost_per_token": 9e-08,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "cloudflare/@cf/cloudflare/clef": {
+        "input_cost_per_token": 2.4e-07,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "cloudflare/@cf/cloudflare/clef-flash": {
+        "input_cost_per_token": 9e-08,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "perplexity/pplx-decider-v1-27b": {
+        "input_cost_per_token": 4e-08,
+        "litellm_provider": "perplexity",
+        "max_input_tokens": 262144,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://docs.perplexity.ai/api-reference/decisions-post"
+    },
+    "strands_decider/strands-decider-2B-hobson-v19": {
+        "input_cost_per_token": 0.0,
+        "litellm_provider": "strands_decider",
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://huggingface.co/StrandsAgents/strands-decider-2B-hobson-v19",
+        "metadata": {
+            "notes": "Self-hosted decision model; infrastructure costs are paid separately"
+        }
     }
 }

litellm/proxy/_lazy_features.py

@@ -16,6 +16,7 @@ from collections.abc import Set as AbstractSet
 from contextlib import asynccontextmanager
 from dataclasses import dataclass, field
 from functools import partial
+from itertools import chain
 from types import MappingProxyType
 from typing import TYPE_CHECKING, Final
 
@@ -266,6 +267,11 @@ LAZY_FEATURES: Final[tuple[LazyFeature, ...]] = (
         module_path="litellm.proxy.openai_evals_endpoints.endpoints",
         path_prefixes=("/v1/evals", "/evals"),
     ),
+    LazyFeature(
+        name="decisions",
+        module_path="litellm.proxy.decisions_endpoints.endpoints",
+        path_prefixes=("/v1/decisions", "/decisions", "/v1/systemone", "/systemone"),
+    ),
     LazyFeature(
         name="claude_code_marketplace",
         module_path="litellm.proxy.anthropic_endpoints.claude_code_endpoints",
@@ -376,6 +382,10 @@ def _lazy_slots(app: "FastAPI") -> Mapping[str, BaseRoute | None]:
     return app.state.lazy_slots if hasattr(app.state, "lazy_slots") else MappingProxyType({})
 
 
+def _lazy_routes(app: "FastAPI") -> Mapping[str, tuple[BaseRoute, ...]]:
+    return app.state.lazy_routes if hasattr(app.state, "lazy_routes") else MappingProxyType({})
+
+
 def reserve_lazy_slot(app: "FastAPI", name: str, features: tuple[LazyFeature, ...] = LAZY_FEATURES) -> None:
     """Record the route the feature's router used to be included after, so its routes
     are spliced back in there once it loads and keep the same precedence. Anchoring on
@@ -472,11 +482,8 @@ def _lazy_lock(app: "FastAPI", module_path: str) -> asyncio.Lock:
 def _register_feature(app: "FastAPI", feat: LazyFeature, module: object, features: tuple[LazyFeature, ...]) -> None:
     before: Final = len(app.router.routes)
     feat.register_fn(app, module)
-    previous: Final[Mapping[str, tuple[BaseRoute, ...]]] = (
-        app.state.lazy_routes if hasattr(app.state, "lazy_routes") else MappingProxyType({})
-    )
     lazy_routes: Final[Mapping[str, tuple[BaseRoute, ...]]] = MappingProxyType(
-        {**previous, feat.module_path: tuple(app.router.routes[before:])}
+        {**_lazy_routes(app), feat.module_path: tuple(app.router.routes[before:])}
     )
     app.state.lazy_routes = lazy_routes  # rebind-ok: the app owns the record of which routes each feature added
     app.router.routes[:] = hot_routes_first(  # rebind-ok: the app owns its route table
@@ -541,11 +548,8 @@ def _register_all_on_startup(inner: "Lifespan[FastAPI]", features: tuple[LazyFea
 
 def _restore_registry_order(app: "FastAPI", features: tuple[LazyFeature, ...]) -> None:
     present: Final = frozenset(id(route) for route in app.router.routes)
-    registered: Final[Mapping[str, tuple[BaseRoute, ...]]] = (
-        app.state.lazy_routes if hasattr(app.state, "lazy_routes") else MappingProxyType({})
-    )
     still_routed: Final = MappingProxyType(
-        {module_path: tuple(r for r in routes if id(r) in present) for module_path, routes in registered.items()}
+        {module_path: tuple(r for r in routes if id(r) in present) for module_path, routes in _lazy_routes(app).items()}
     )
     app.router.routes[:] = hot_routes_first(  # rebind-ok: the app owns its route table
         _in_registry_order(app.router.routes, still_routed, features, _lazy_slots(app))
@@ -597,6 +601,13 @@ def _make_warmup_router(app: "FastAPI", features: tuple[LazyFeature, ...] = LAZY
     return router
 
 
+def lazy_owned_routes(app: "FastAPI") -> frozenset[int]:
+    """ids of the routes lazy features have registered on this app. A route added later at
+    one of their paths (a config pass-through at /v1/decisions) goes ahead of them, the
+    precedence lazy mode gives it when the feature has not loaded by the time the config is read."""
+    return frozenset(id(route) for route in chain.from_iterable(_lazy_routes(app).values()))
+
+
 def loaded_lazy_modules(app: "FastAPI") -> frozenset[str]:
     """The set of lazy feature modules whose routers are actually registered
     on this app (tracked by _install), empty until a feature loads or eager startup runs.

litellm/proxy/_lazy_openapi_snapshot.json

@@ -9492,6 +9492,109 @@
       }
     }
   },
+  "decisions": {
+    "components": {
+      "schemas": {}
+    },
+    "paths": {
+      "/decisions": {
+        "post": {
+          "operationId": "decisions_decisions_post",
+          "responses": {
+            "200": {
+              "content": {
+                "application/json": {
+                  "schema": {}
+                }
+              },
+              "description": "Successful Response"
+            }
+          },
+          "security": [
+            {
+              "APIKeyHeader": []
+            }
+          ],
+          "summary": "Decisions",
+          "tags": [
+            "decisions"
+          ]
+        }
+      },
+      "/systemone": {
+        "post": {
+          "operationId": "systemone_systemone_post",
+          "responses": {
+            "200": {
+              "content": {
+                "application/json": {
+                  "schema": {}
+                }
+              },
+              "description": "Successful Response"
+            }
+          },
+          "security": [
+            {
+              "APIKeyHeader": []
+            }
+          ],
+          "summary": "Systemone",
+          "tags": [
+            "decisions"
+          ]
+        }
+      },
+      "/v1/decisions": {
+        "post": {
+          "operationId": "decisions_v1_decisions_post",
+          "responses": {
+            "200": {
+              "content": {
+                "application/json": {
+                  "schema": {}
+                }
+              },
+              "description": "Successful Response"
+            }
+          },
+          "security": [
+            {
+              "APIKeyHeader": []
+            }
+          ],
+          "summary": "Decisions",
+          "tags": [
+            "decisions"
+          ]
+        }
+      },
+      "/v1/systemone": {
+        "post": {
+          "operationId": "systemone_v1_systemone_post",
+          "responses": {
+            "200": {
+              "content": {
+                "application/json": {
+                  "schema": {}
+                }
+              },
+              "description": "Successful Response"
+            }
+          },
+          "security": [
+            {
+              "APIKeyHeader": []
+            }
+          ],
+          "summary": "Systemone",
+          "tags": [
+            "decisions"
+          ]
+        }
+      }
+    }
+  },
   "evals": {
     "components": {
       "schemas": {

litellm/proxy/_types.py

@@ -475,6 +475,10 @@ class LiteLLMRoutes(enum.Enum):
         "/v1/search",
         "/search/{search_tool_name}",
         "/v1/search/{search_tool_name}",
+        "/decisions",
+        "/v1/decisions",
+        "/systemone",
+        "/v1/systemone",
         # OCR
         "/ocr",
         "/v1/ocr",

litellm/proxy/agent_endpoints/auth/managed_authorization.py

@@ -29,6 +29,8 @@ _MANAGED_MODEL_ROUTES: Final = frozenset(
             "audio/speech",
             "moderations",
             "rerank",
+            "decisions",
+            "systemone",
             "ocr",
         ),
     )
@@ -72,6 +74,8 @@ _MODEL_ROUTE_KINDS: Final[
         "/audio/transcriptions": "moderation",
         "/audio/speech": "speech",
         "/rerank": "body",
+        "/decisions": "body",
+        "/systemone": "body",
         "/messages/count_tokens": "body",
         ":countTokens": "path",
     }

litellm/proxy/common_request_processing.py

@@ -173,6 +173,7 @@ ProxyRouteType: TypeAlias = Literal[
     "avector_store_file_delete",
     "aocr",
     "asearch",
+    "adecisions",
     "avideo_generation",
     "avideo_list",
     "avideo_status",
@@ -1958,6 +1959,7 @@ class ProxyBaseLLMRequestProcessing:
             "avector_store_file_delete",
             "aocr",
             "asearch",
+            "adecisions",
             "avideo_generation",
             "avideo_list",
             "avideo_status",

litellm/proxy/decisions_endpoints/__init__.py

@@ -0,0 +1 @@
+__all__ = ()

litellm/proxy/decisions_endpoints/endpoints.py

@@ -0,0 +1,161 @@
+from collections.abc import Mapping
+from typing import Annotated, Final
+
+from fastapi import APIRouter, Depends, Request, Response
+from fastapi.responses import ORJSONResponse  # pyright: ignore[reportDeprecated]  # required endpoint contract
+from pydantic import TypeAdapter, ValidationError
+
+from litellm.exceptions import BadRequestError
+from litellm.proxy.auth.user_api_key_auth import UserAPIKeyAuth, user_api_key_auth
+from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
+from litellm.types.decisions import DecisionsRequestBody, OpenAIDecisionRequestBody
+
+router: Final = APIRouter()
+_REQUEST_DATA_ADAPTER: Final[TypeAdapter[dict[str, object]]] = TypeAdapter(dict[str, object])
+_DECISIONS_REQUEST_BODY_ADAPTER: Final[TypeAdapter[DecisionsRequestBody]] = TypeAdapter(DecisionsRequestBody)
+_OPENAI_DECISION_REQUEST_BODY_ADAPTER: Final[TypeAdapter[OpenAIDecisionRequestBody]] = TypeAdapter(
+    OpenAIDecisionRequestBody
+)
+_GENERAL_SETTINGS_ADAPTER: Final[TypeAdapter[dict[str, object]]] = TypeAdapter(dict[str, object])
+_OPTIONAL_STRING_ADAPTER: Final[TypeAdapter[str | None]] = TypeAdapter(str | None)
+_OPTIONAL_FLOAT_ADAPTER: Final[TypeAdapter[float | None]] = TypeAdapter(float | None)
+
+
+async def _invalid_request(
+    raw_data: Mapping[str, object], error: ValidationError, user_api_key_dict: UserAPIKeyAuth
+) -> Exception:
+    from litellm.proxy.proxy_server import proxy_logging_obj, version
+
+    return await ProxyBaseLLMRequestProcessing(data=dict(raw_data))._handle_llm_api_exception(
+        e=BadRequestError(
+            message=f"Invalid Decisions request: {error}",
+            model=str(raw_data.get("model", "")),
+            llm_provider="",
+        ),
+        user_api_key_dict=user_api_key_dict,
+        proxy_logging_obj=proxy_logging_obj,
+        version=version,
+    )
+
+
+async def _request_data(request: Request, user_api_key_dict: UserAPIKeyAuth) -> dict[str, object]:
+    body: Final = await request.body()
+    try:
+        return _REQUEST_DATA_ADAPTER.validate_json(body)
+    except ValidationError as error:
+        raise await _invalid_request(raw_data={}, error=error, user_api_key_dict=user_api_key_dict)
+
+
+async def _process_decisions(
+    request: Request,
+    fastapi_response: Response,
+    user_api_key_dict: UserAPIKeyAuth,
+    body_adapter: TypeAdapter[DecisionsRequestBody] | TypeAdapter[OpenAIDecisionRequestBody],
+) -> object:
+    from litellm.proxy.proxy_server import (
+        general_settings as proxy_general_settings,
+    )
+    from litellm.proxy.proxy_server import (
+        llm_router,
+        proxy_config,
+        proxy_logging_obj,
+        user_max_tokens,
+        user_request_timeout,
+        version,
+    )
+    from litellm.proxy.proxy_server import (
+        user_api_base as proxy_user_api_base,
+    )
+    from litellm.proxy.proxy_server import (
+        user_model as proxy_user_model,
+    )
+    from litellm.proxy.proxy_server import (
+        user_temperature as proxy_user_temperature,
+    )
+
+    data: Final = await _request_data(request, user_api_key_dict)
+    try:
+        body_adapter.validate_python(data)
+    except ValidationError as error:
+        raise await _invalid_request(raw_data=data, error=error, user_api_key_dict=user_api_key_dict)
+    general_settings: Final = _GENERAL_SETTINGS_ADAPTER.validate_python(proxy_general_settings)
+    user_api_base: Final = _OPTIONAL_STRING_ADAPTER.validate_python(proxy_user_api_base)
+    user_model: Final = _OPTIONAL_STRING_ADAPTER.validate_python(proxy_user_model)
+    user_temperature: Final = _OPTIONAL_FLOAT_ADAPTER.validate_python(proxy_user_temperature)
+    processor: Final = ProxyBaseLLMRequestProcessing(data=data)
+    try:
+        return await processor.base_process_llm_request(
+            request=request,
+            fastapi_response=fastapi_response,
+            user_api_key_dict=user_api_key_dict,
+            route_type="adecisions",
+            proxy_logging_obj=proxy_logging_obj,
+            llm_router=llm_router,
+            general_settings=general_settings,
+            proxy_config=proxy_config,
+            select_data_generator=None,
+            model=None,
+            user_model=user_model,
+            user_temperature=user_temperature,
+            user_request_timeout=user_request_timeout,
+            user_max_tokens=user_max_tokens,
+            user_api_base=user_api_base,
+            version=version,
+        )
+    except Exception as error:
+        raise await processor._handle_llm_api_exception(
+            e=error,
+            user_api_key_dict=user_api_key_dict,
+            proxy_logging_obj=proxy_logging_obj,
+            version=version,
+        )
+
+
+@router.post(
+    "/v1/systemone",
+    dependencies=[Depends(user_api_key_auth)],
+    response_class=ORJSONResponse,  # pyright: ignore[reportDeprecated]  # required endpoint contract
+    tags=["decisions"],
+)
+@router.post(
+    "/systemone",
+    dependencies=[Depends(user_api_key_auth)],
+    response_class=ORJSONResponse,  # pyright: ignore[reportDeprecated]  # required endpoint contract
+    tags=["decisions"],
+)
+async def systemone(
+    request: Request,
+    fastapi_response: Response,
+    user_api_key_dict: Annotated[UserAPIKeyAuth, Depends(user_api_key_auth)],
+):
+    return await _process_decisions(
+        request=request,
+        fastapi_response=fastapi_response,
+        user_api_key_dict=user_api_key_dict,
+        body_adapter=_DECISIONS_REQUEST_BODY_ADAPTER,
+    )
+
+
+@router.post(
+    "/v1/decisions",
+    dependencies=[Depends(user_api_key_auth)],
+    response_class=ORJSONResponse,  # pyright: ignore[reportDeprecated]  # required endpoint contract
+    tags=["decisions"],
+)
+@router.post(
+    "/decisions",
+    dependencies=[Depends(user_api_key_auth)],
+    response_class=ORJSONResponse,  # pyright: ignore[reportDeprecated]  # required endpoint contract
+    tags=["decisions"],
+)
+async def decisions(
+    request: Request,
+    fastapi_response: Response,
+    user_api_key_dict: Annotated[UserAPIKeyAuth, Depends(user_api_key_auth)],
+):
+    return await _process_decisions(
+        request=request,
+        fastapi_response=fastapi_response,
+        user_api_key_dict=user_api_key_dict,
+        body_adapter=_OPENAI_DECISION_REQUEST_BODY_ADAPTER,
+    )

litellm/proxy/pass_through_endpoints/pass_through_endpoints.py

@@ -27,6 +27,7 @@ from fastapi import (
 )
 from fastapi.responses import StreamingResponse
 from starlette.datastructures import UploadFile as StarletteUploadFile
+from starlette.routing import BaseRoute, Route
 from starlette.websockets import WebSocketState
 from websockets.asyncio.client import connect
 from websockets.exceptions import (
@@ -65,6 +66,7 @@ from litellm.llms.base_llm.managed_resources.utils import (
 )
 from litellm.llms.custom_httpx.http_handler import get_async_httpx_client
 from litellm.passthrough import BasePassthroughUtils
+from litellm.proxy._lazy_features import lazy_owned_routes
 from litellm.proxy._types import (
     ConfigFieldInfo,
     ConfigFieldUpdate,
@@ -2873,35 +2875,34 @@ def _extract_model_from_vertex_ai_setup(setup_response: Mapping[str, object]) ->
     return None
 
 
+def _placed_ahead(routes: Sequence[BaseRoute], moving: BaseRoute, before: BaseRoute) -> tuple[BaseRoute, ...]:
+    kept: Final = tuple(route for route in routes if route is not moving)
+    at: Final = next(index for index, route in enumerate(kept) if route is before)
+    return (*kept[:at], moving, *kept[at:])
+
+
 class SafeRouteAdder:
     """
     Wrapper class for adding routes to FastAPI app.
-    Only adds routes if they don't already exist on the app.
+    Only adds routes if they don't already exist on the app. A route a lazy feature registered
+    does not count: a route added at its path goes ahead of it, the precedence a config
+    pass-through at /v1/decisions gets in lazy mode, where the feature has not loaded yet.
     """
 
+    @staticmethod
+    def _colliding_routes(app: FastAPI, path: str, methods: Sequence[str]) -> tuple[Route, ...]:
+        wanted: Final = frozenset(methods)
+        return tuple(
+            route
+            for route in app.routes
+            if isinstance(route, Route) and route.path == path and not wanted.isdisjoint(route.methods or ())
+        )
+
     @staticmethod
     def _is_path_registered(app: FastAPI, path: str, methods: list[str]) -> bool:
-        """
-        Check if a path with any of the specified methods is already registered on the app.
-
-        Args:
-            app: The FastAPI application instance
-            path: The path to check (e.g., "/v1/chat/completions")
-            methods: List of HTTP methods to check (e.g., ["GET", "POST"])
-
-        Returns:
-            True if the path is already registered with any of the methods, False otherwise
-        """
-        for route in app.routes:
-            # Use getattr to safely access route attributes
-            route_path = getattr(route, "path", None)
-            route_methods = getattr(route, "methods", None)
-
-            if route_path == path and route_methods is not None:
-                # Check if any of the methods overlap
-                if any(method in route_methods for method in methods):
-                    return True
-        return False
+        """True when a route the app itself defines already serves the path with one of the methods."""
+        lazy_owned: Final = lazy_owned_routes(app)
+        return any(id(route) not in lazy_owned for route in SafeRouteAdder._colliding_routes(app, path, methods))
 
     @staticmethod
     def add_api_route_if_not_exists(
@@ -2932,12 +2933,17 @@ class SafeRouteAdder:
             )
             return False
 
+        shadowed: Final = SafeRouteAdder._colliding_routes(app, path, methods)
         app.add_api_route(
             path=path,
             endpoint=endpoint,
             methods=methods,
             dependencies=dependencies,
         )
+        if shadowed:
+            app.router.routes[:] = _placed_ahead(  # rebind-ok: the app owns its route table
+                app.router.routes, app.router.routes[-1], shadowed[0]
+            )
         verbose_proxy_logger.debug(
             "Successfully added route: %s with methods %s",
             path,

litellm/proxy/route_llm_request.py

@@ -90,6 +90,7 @@ ROUTE_ENDPOINT_MAPPING: Final = {
     "acompact_responses": "/responses/compact",
     "aocr": "/ocr",
     "asearch": "/search",
+    "adecisions": "/decisions",
     "avideo_generation": "/videos",
     "avideo_list": "/videos",
     "avideo_status": "/videos/{video_id}",
@@ -373,6 +374,7 @@ RouteType = Literal[
     "avector_store_file_delete",
     "aocr",
     "asearch",
+    "adecisions",
     "avideo_generation",
     "avideo_list",
     "avideo_status",

litellm/router.py

@@ -2086,6 +2086,11 @@ class Router:
         self.asearch = self.factory_function(asearch, call_type="asearch")
         self.search = self.factory_function(search, call_type="search")
 
+        from litellm.decisions import adecisions, decisions
+
+        self.adecisions = self.factory_function(adecisions, call_type="adecisions")
+        self.decisions = self.factory_function(decisions, call_type="decisions")
+
     def _initialize_video_endpoints(self):
         """Initialize video endpoints."""
         from litellm.videos import (
@@ -6639,6 +6644,8 @@ class Router:
             "ocr",
             "asearch",
             "search",
+            "adecisions",
+            "decisions",
             "aadapter_generate_content",
             "avideo_generation",
             "video_generation",
@@ -6712,6 +6719,7 @@ class Router:
             "generate_content_stream",
             "ocr",
             "search",
+            "decisions",
             "video_generation",
             "video_list",
             "video_status",
@@ -6879,6 +6887,7 @@ class Router:
                 "agenerate_content_stream",
                 "aocr",
                 "ocr",
+                "adecisions",
                 "avideo_generation",
                 "avideo_list",
                 "avideo_status",

litellm/types/decisions.py

@@ -0,0 +1,443 @@
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass, field
+from types import MappingProxyType
+from typing import Annotated, Final, Literal, TypeAlias
+
+from pydantic import ConfigDict, Field, PrivateAttr, model_validator, with_config
+from typing_extensions import ReadOnly, Required, TypedDict
+
+from litellm.types.llms.base import LiteLLMPydanticObjectBase
+
+DecisionsJSON: TypeAlias = str | Mapping[str, object] | Sequence[object]
+NoulCriteria: TypeAlias = Mapping[Literal["true", "false"], DecisionsJSON | None]
+MAX_DECISION_QUESTIONS: Final = 128
+
+
+class NoulQuestion(LiteLLMPydanticObjectBase):
+    type: Literal["noul"]
+    instructions: DecisionsJSON | None = None
+    criteria: NoulCriteria | None = None
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+    @model_validator(mode="after")
+    def require_instructions_or_criteria(self) -> "NoulQuestion":
+        if self.instructions is None and self.criteria is None:
+            raise ValueError("A noul question requires instructions or criteria")
+        return self
+
+
+class ChoiceQuestion(LiteLLMPydanticObjectBase):
+    type: Literal["choice"]
+    instructions: DecisionsJSON | None = None
+    criteria: Annotated[Mapping[str, DecisionsJSON | None], Field(min_length=1, max_length=255)]
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class ScoreQuestion(LiteLLMPydanticObjectBase):
+    type: Literal["score"]
+    instructions: DecisionsJSON | None = None
+    criteria: Annotated[Sequence[DecisionsJSON], Field(min_length=1, max_length=10)]
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+DecisionQuestion: TypeAlias = Annotated[
+    NoulQuestion | ChoiceQuestion | ScoreQuestion,
+    Field(discriminator="type"),
+]
+
+DecisionQuestionMap: TypeAlias = Annotated[
+    Mapping[Annotated[str, Field(min_length=1)], DecisionQuestion],
+    Field(min_length=1, max_length=MAX_DECISION_QUESTIONS),
+]
+
+
+class DecisionsRequestBody(LiteLLMPydanticObjectBase):
+    state: DecisionsJSON
+    questions: DecisionQuestionMap
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class DecisionsRequest(DecisionsRequestBody):
+    model: str
+
+
+@with_config(ConfigDict(extra="allow"))
+class DecisionsCallParams(TypedDict, total=False):
+    model: Required[ReadOnly[str]]
+    state: Required[ReadOnly[DecisionsJSON]]
+    questions: Required[ReadOnly[DecisionQuestionMap]]
+    api_key: ReadOnly[str | None]
+    api_base: ReadOnly[str | None]
+    timeout: ReadOnly[float | None]
+    custom_llm_provider: ReadOnly[str | None]
+    extra_headers: ReadOnly[Mapping[str, str] | None]
+
+
+class NoulAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["noul"]
+    noul: float
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class ChoiceAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["choice"]
+    choice: str
+    confidence: float
+    probabilities: Mapping[str, float]
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class ScoreAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["score"]
+    score: float
+    confidence: float
+    legend: Mapping[str, DecisionsJSON]
+    probabilities: Mapping[str, float]
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+DecisionAnswer: TypeAlias = Annotated[
+    NoulAnswer | ChoiceAnswer | ScoreAnswer,
+    Field(discriminator="type"),
+]
+
+
+class DecisionsUsage(LiteLLMPydanticObjectBase):
+    input_tokens: int = 0
+    output_tokens: int = 0
+    cached_tokens: Annotated[int, Field(exclude=True)] = 0
+    cache_write_tokens: Annotated[int, Field(exclude=True)] = 0
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class _HiddenParamsResponse(LiteLLMPydanticObjectBase):
+    _hidden_params: dict[str, object] = PrivateAttr(default_factory=dict)
+
+
+class DecisionsResponse(_HiddenParamsResponse):
+    model: str | None = None
+    answers: Mapping[str, DecisionAnswer]
+    usage: DecisionsUsage | None = None
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class OpenAIDecisionInputText(LiteLLMPydanticObjectBase):
+    type: Literal["input_text"]
+    text: str
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+class OpenAIDecisionInputImage(LiteLLMPydanticObjectBase):
+    type: Literal["input_image"]
+    image_url: str
+    detail: str | None = None
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+OpenAIDecisionContentPart: TypeAlias = Annotated[
+    OpenAIDecisionInputText | OpenAIDecisionInputImage,
+    Field(discriminator="type"),
+]
+
+
+class OpenAIDecisionInputMessage(LiteLLMPydanticObjectBase):
+    role: Literal["user"] = "user"
+    type: Literal["message"] = "message"
+    content: str | Sequence[OpenAIDecisionContentPart]
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+OpenAIDecisionInput: TypeAlias = str | Sequence[OpenAIDecisionInputMessage]
+
+
+class OpenAIPredicateQuestion(LiteLLMPydanticObjectBase):
+    type: Literal["predicate"]
+    name: str | None = None
+    instructions: str
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+class OpenAIChoiceOption(LiteLLMPydanticObjectBase):
+    value: str | bool
+    description: str | None = None
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+def systemone_choice_key(value: str | bool) -> str:
+    if isinstance(value, bool):
+        return "true" if value else "false"
+    return value
+
+
+class OpenAIChoiceQuestion(LiteLLMPydanticObjectBase):
+    type: Literal["choice"]
+    name: str | None = None
+    instructions: str
+    choices: Annotated[Sequence[OpenAIChoiceOption], Field(min_length=2, max_length=255)]
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+    @model_validator(mode="after")
+    def require_unique_systemone_keys(self) -> "OpenAIChoiceQuestion":
+        keys: Final = frozenset(systemone_choice_key(option.value) for option in self.choices)
+        if len(keys) != len(self.choices):
+            raise ValueError("Choice values must be unique, and a boolean cannot share its text with a string choice")
+        return self
+
+
+class OpenAIScoreLevel(LiteLLMPydanticObjectBase):
+    label: str
+    description: str | None = None
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+class OpenAIScoreQuestion(LiteLLMPydanticObjectBase):
+    type: Literal["score"]
+    name: str | None = None
+    instructions: str
+    levels: Annotated[Sequence[OpenAIScoreLevel], Field(min_length=2, max_length=10)]
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+OpenAIDecisionQuestion: TypeAlias = Annotated[
+    OpenAIPredicateQuestion | OpenAIChoiceQuestion | OpenAIScoreQuestion,
+    Field(discriminator="type"),
+]
+
+
+class OpenAIDecisionRequestBody(LiteLLMPydanticObjectBase):
+    input: OpenAIDecisionInput
+    questions: Annotated[Sequence[OpenAIDecisionQuestion], Field(min_length=1, max_length=MAX_DECISION_QUESTIONS)]
+    safety_identifier: str | None = None
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+class OpenAIPredicateAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["predicate"] = "predicate"
+    name: str | None
+    probability: float
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIChoiceProbability(LiteLLMPydanticObjectBase):
+    value: str | bool
+    probability: float
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIChoiceAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["choice"] = "choice"
+    name: str | None
+    choice: str | bool
+    probabilities: tuple[OpenAIChoiceProbability, ...]
+    confidence: float
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIScoreProbability(LiteLLMPydanticObjectBase):
+    value: int
+    label: str
+    probability: float
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIScoreAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["score"] = "score"
+    name: str | None
+    score: float
+    probabilities: tuple[OpenAIScoreProbability, ...]
+    confidence: float
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIRefusalAnswer(LiteLLMPydanticObjectBase):
+    type: Literal["refusal"] = "refusal"
+    name: str | None
+
+    model_config = ConfigDict(frozen=True)
+
+
+OpenAIDecisionAnswer: TypeAlias = Annotated[
+    OpenAIPredicateAnswer | OpenAIChoiceAnswer | OpenAIScoreAnswer | OpenAIRefusalAnswer,
+    Field(discriminator="type"),
+]
+
+
+class OpenAIDecisionInputTokensDetails(LiteLLMPydanticObjectBase):
+    cached_tokens: int = 0
+    cache_write_tokens: int = 0
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIDecisionOutputTokensDetails(LiteLLMPydanticObjectBase):
+    reasoning_tokens: int = 0
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIDecisionUsage(LiteLLMPydanticObjectBase):
+    input_tokens: int
+    input_tokens_details: OpenAIDecisionInputTokensDetails = OpenAIDecisionInputTokensDetails()
+    output_tokens: int
+    output_tokens_details: OpenAIDecisionOutputTokensDetails = OpenAIDecisionOutputTokensDetails()
+    total_tokens: int
+
+    model_config = ConfigDict(frozen=True)
+
+
+class OpenAIDecisionResponse(_HiddenParamsResponse):
+    model: str
+    answers: tuple[OpenAIDecisionAnswer, ...]
+    usage: OpenAIDecisionUsage
+
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+
+_NO_EXTRA: Final[Mapping[str, object]] = MappingProxyType({})
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRState:
+    state: DecisionsJSON
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRMessages:
+    messages: tuple[OpenAIDecisionInputMessage, ...]
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRPredicateQuestion:
+    name: str | None
+    instructions: DecisionsJSON | None
+    criteria: NoulCriteria | None = None
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRChoiceOption:
+    value: str | bool
+    description: DecisionsJSON | None
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRChoiceQuestion:
+    name: str | None
+    instructions: DecisionsJSON | None
+    choices: tuple[DecisionsIRChoiceOption, ...]
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRScoreLevel:
+    label: DecisionsJSON
+    description: str | None
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRScoreQuestion:
+    name: str | None
+    instructions: DecisionsJSON | None
+    levels: tuple[DecisionsIRScoreLevel, ...]
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+DecisionsIRQuestion: TypeAlias = DecisionsIRPredicateQuestion | DecisionsIRChoiceQuestion | DecisionsIRScoreQuestion
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRRequest:
+    input: DecisionsIRState | DecisionsIRMessages
+    questions: tuple[DecisionsIRQuestion, ...]
+    safety_identifier: str | None = None
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRPredicateAnswer:
+    probability: float
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRChoiceProbability:
+    value: str | bool
+    probability: float
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRChoiceAnswer:
+    choice: str | bool
+    confidence: float
+    probabilities: tuple[DecisionsIRChoiceProbability, ...]
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRScoreProbability:
+    value: int
+    label: DecisionsJSON
+    probability: float
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRScoreAnswer:
+    score: float
+    confidence: float
+    probabilities: tuple[DecisionsIRScoreProbability, ...]
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRRefusal:
+    pass
+
+
+DecisionsIRAnswer: TypeAlias = (
+    DecisionsIRPredicateAnswer | DecisionsIRChoiceAnswer | DecisionsIRScoreAnswer | DecisionsIRRefusal
+)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRUsage:
+    input_tokens: int = 0
+    output_tokens: int = 0
+    cached_tokens: int = 0
+    cache_write_tokens: int = 0
+    reasoning_tokens: int = 0
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class DecisionsIRResponse:
+    model: str | None
+    answers: tuple[DecisionsIRAnswer, ...]
+    usage: DecisionsIRUsage
+    extra: Mapping[str, object] = field(default_factory=lambda: _NO_EXTRA)
+
+
+@dataclass(frozen=True, slots=True)
+class UnsupportedDecisionsRequest:
+    reason: str

litellm/types/utils.py

@@ -467,6 +467,8 @@ class CallTypes(str, Enum):
     arerank = "arerank"
     search = "search"
     asearch = "asearch"
+    decisions = "decisions"
+    adecisions = "adecisions"
     arealtime = "_arealtime"
     aresponses_websocket = "_aresponses_websocket"
     create_batch = "create_batch"
@@ -653,6 +655,8 @@ CallTypesLiteral = Literal[
     "arerank",
     "search",
     "asearch",
+    "decisions",
+    "adecisions",
     "_arealtime",
     "_aresponses_websocket",
     "create_batch",
@@ -762,6 +766,10 @@ API_ROUTE_TO_CALL_TYPES: Final[Mapping[str, Sequence[CallTypes]]] = {
     # Search
     "/search": [CallTypes.asearch, CallTypes.search],
     "/v1/search": [CallTypes.asearch, CallTypes.search],
+    "/decisions": [CallTypes.adecisions, CallTypes.decisions],
+    "/v1/decisions": [CallTypes.adecisions, CallTypes.decisions],
+    "/systemone": [CallTypes.adecisions, CallTypes.decisions],
+    "/v1/systemone": [CallTypes.adecisions, CallTypes.decisions],
     # Batches
     "/batches": [CallTypes.acreate_batch, CallTypes.create_batch],
     "/v1/batches": [CallTypes.acreate_batch, CallTypes.create_batch],
@@ -4044,6 +4052,8 @@ class LlmProviders(str, Enum):
     OLLAMA_CHAT = "ollama_chat"
     DEEPINFRA = "deepinfra"
     PERPLEXITY = "perplexity"
+    TYPESAFE = "typesafe"
+    STRANDS_DECIDER = "strands_decider"
     MISTRAL = "mistral"
     MILVUS = "milvus"
     GROQ = "groq"

litellm/utils.py

@@ -1217,6 +1217,9 @@ def function_setup(
                 if isinstance(search_query, list)
                 else search_query
             )
+        elif call_type in (CallTypes.decisions.value, CallTypes.adecisions.value):
+            decisions_state: Final = args[1] if len(args) > 1 else kwargs.get("state") or kwargs.get("input") or ""
+            messages = decisions_state if isinstance(decisions_state, str) else json.dumps(decisions_state, default=str)
         elif call_type in (CallTypes.image_edit.value, CallTypes.aimage_edit.value):
             messages = args[1] if len(args) > 1 else kwargs.get("prompt")
         elif call_type in (CallTypes.ocr.value, CallTypes.aocr.value):

model_prices_and_context_window.json

@@ -34158,7 +34158,8 @@
         "supported_endpoints": [
             "/v1/chat/completions",
             "/v1/batch",
-            "/v1/responses"
+            "/v1/responses",
+            "/v1/decisions"
         ],
         "supported_modalities": [
             "text",
@@ -79714,5 +79715,55 @@
         "supports_response_schema": true,
         "supports_tool_choice": true,
         "supports_vision": true
+    },
+    "cloudflare/clef": {
+        "input_cost_per_token": 2.4e-07,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "cloudflare/clef-flash": {
+        "input_cost_per_token": 9e-08,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "cloudflare/@cf/cloudflare/clef": {
+        "input_cost_per_token": 2.4e-07,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "cloudflare/@cf/cloudflare/clef-flash": {
+        "input_cost_per_token": 9e-08,
+        "litellm_provider": "cloudflare",
+        "max_input_tokens": 65536,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://developers.cloudflare.com/workers-ai/platform/pricing/"
+    },
+    "perplexity/pplx-decider-v1-27b": {
+        "input_cost_per_token": 4e-08,
+        "litellm_provider": "perplexity",
+        "max_input_tokens": 262144,
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://docs.perplexity.ai/api-reference/decisions-post"
+    },
+    "strands_decider/strands-decider-2B-hobson-v19": {
+        "input_cost_per_token": 0.0,
+        "litellm_provider": "strands_decider",
+        "mode": "evaluation",
+        "output_cost_per_token": 0.0,
+        "source": "https://huggingface.co/StrandsAgents/strands-decider-2B-hobson-v19",
+        "metadata": {
+            "notes": "Self-hosted decision model; infrastructure costs are paid separately"
+        }
     }
 }

provider_endpoints_support.json

@@ -2551,6 +2551,20 @@
         "image_variations": true
       }
     },
+    "typesafe": {
+      "display_name": "TypeSafe (`typesafe`)",
+      "url": "https://docs.typesafe.ai/models",
+      "endpoints": {
+        "systemone": true
+      }
+    },
+    "strands_decider": {
+      "display_name": "Strands Decider (`strands_decider`)",
+      "url": "https://docs.litellm.ai/docs/providers",
+      "endpoints": {
+        "systemone": true
+      }
+    },
     "tavily": {
       "display_name": "Tavily (`tavily`)",
       "url": "https://docs.litellm.ai/docs/search/tavily",

tests/integration/_support/process.py

@@ -50,12 +50,16 @@ def signal_group(group: int, action: int) -> None:
         pass
 
 
+def graceful_stop_seconds() -> float:
+    return max(30.0, float(os.environ.get("INTEGRATION_PROXY_READY_SECONDS", "70")))
+
+
 def stop_root_process(process: subprocess.Popen[bytes]) -> bool:
     if process.poll() is not None:
         return True
     process.terminate()
     try:
-        process.wait(timeout=30)
+        process.wait(timeout=graceful_stop_seconds())
     except subprocess.TimeoutExpired:
         return False
     return True
@@ -183,6 +187,7 @@ def owned_proxy_process(
     remove_environment: tuple[str, ...] = (),
     workers: int = 1,
     database_setup: tuple[str, ...] = DB_PUSH,
+    extra_arguments: tuple[str, ...] = (),
 ) -> Iterator[OwnedProxy]:
     root: Final = Path(os.environ.get("INTEGRATION_PROXY_ROOT") or Path(__file__).resolve().parents[3])
     environment: Final = {
@@ -209,6 +214,7 @@ def owned_proxy_process(
         "--num_workers",
         str(workers),
         *database_setup,
+        *extra_arguments,
     )
     launch: Final = _launch_until_bound(command, root, environment, output, _PORT_ATTEMPTS)
     process: Final = launch.process

tests/integration/cost_calculation/cost_tracking_case.py

@@ -246,6 +246,7 @@ class CostTrackingTestCase(BaseModel):
             "/v1/audio/speech",
             "/v1/images/generations",
             "/v1/images/edits",
+            "/v1/systemone",
         ]
         | Annotated[str, Field(pattern=r"^/(gemini|anthropic|bedrock)/")]
     ) = "/v1/chat/completions"

tests/integration/cost_calculation/cost_tracking_cases.json

@@ -57,6 +57,13 @@
         "search_context_size_high": 0.012
       }
     },
+    "perplexity/pplx-decider-v1-27b": {
+      "input_cost_per_token": 4e-08,
+      "litellm_provider": "perplexity",
+      "max_input_tokens": 262144,
+      "mode": "evaluation",
+      "output_cost_per_token": 0.0
+    },
     "deepseek/deepseek-v4-chat": {
       "litellm_provider": "deepseek",
       "mode": "chat",
@@ -30925,6 +30932,47 @@
         "completion_tokens": 412
       }
     },
+    {
+      "name": "perplexity/pplx-decider-v1-27b-decisions",
+      "covers": "quota_management.spend_tracking.decisions_costs",
+      "model": "perplexity/pplx-decider-v1-27b",
+      "endpoint": "/v1/systemone",
+      "request": {
+        "model": "$MODEL",
+        "state": {
+          "source": "cost-tracking"
+        },
+        "questions": {
+          "is_defect": {
+            "type": "noul",
+            "instructions": "Is this a defect?"
+          }
+        }
+      },
+      "response": {
+        "content_type": "application/json",
+        "body": {
+          "model": "pplx-decider-v1-27b",
+          "answers": {
+            "is_defect": {
+              "type": "noul",
+              "noul": 0.9
+            }
+          },
+          "usage": {
+            "input_tokens": 367,
+            "output_tokens": 3
+          }
+        }
+      },
+      "expected": {
+        "spend": 1.468e-05,
+        "input_cost": 1.468e-05,
+        "output_cost": 0.0,
+        "prompt_tokens": 367,
+        "completion_tokens": 3
+      }
+    },
     {
       "name": "gpt-5.6-client_disconnect_mid_stream",
       "covers": "quota_management.spend_tracking.scripted_wire.client_disconnect",

tests/integration/cost_calculation/test_cost_tracking.py

@@ -268,6 +268,31 @@ def test_case_bills_expected_cost(gateway: Gateway, case: CostTrackingTestCase)
             assert row.spend == 0, f"{case.name}: failure spend was {row.spend}"
             return
         assert response.is_success, f"{case.name}: proxy returned {response.status_code}: {response.text[:400]}"
+        if case.endpoint == "/v1/systemone":
+            observed: Final = JSON_OBJECT.validate_json(
+                httpx.get(f"{gateway.upstream_url}/__observations", timeout=5, trust_env=False).content
+            )
+            decision_observations: Final = tuple(
+                value
+                for value in observed["requests"]
+                if isinstance(value, dict) and value.get("path") == f"/{scenario_id}/v1/decisions"
+            )
+            assert decision_observations == (
+                {
+                    "path": f"/{scenario_id}/v1/decisions",
+                    "authorization": "Bearer sk-scripted-provider",
+                    "body": {
+                        "model": "pplx-decider-v1-27b",
+                        "state": {"source": "cost-tracking"},
+                        "questions": {
+                            "is_defect": {
+                                "type": "noul",
+                                "instructions": "Is this a defect?",
+                            }
+                        },
+                    },
+                },
+            )
         if case.response.content_type == "text/event-stream":
             _assert_stream_has_no_error(response.text)
         rows: Final = poll_rows(key, len(responses) + (prior_response_id is not None))

tests/integration/management/test_model_health_check.py

@@ -1,8 +1,41 @@
+import os
 import uuid
 from typing import Final
 
 import httpx
-from integration._support.client import Gateway, object_value
+from integration._support.client import Gateway, object_value, string_value
+from integration._support.upstream import ScenarioHandle, delete_scenario, register_scenario
+from integration.cost_calculation.cost_tracking_case import JsonResponse
+from pydantic import JsonValue
+
+_DECISIONS_PROBE_REPLY: Final = JsonResponse(
+    content_type="application/json",
+    body={
+        "model": "pplx-decider-v1-27b",
+        "answers": {"reachable": {"type": "noul", "noul": 1.0}},
+        "usage": {"input_tokens": 10, "output_tokens": 1},
+    },
+)
+_CONFIGURED_PROBE_REPLY: Final = JsonResponse(
+    content_type="application/json",
+    body={
+        "model": "jev-custom",
+        "answers": {"alive": {"type": "choice", "choice": "yes", "confidence": 0.9, "probabilities": {"yes": 0.9}}},
+        "usage": {"input_tokens": 10, "output_tokens": 1},
+    },
+)
+_STRANDS_PROBE_REPLY: Final = JsonResponse(
+    content_type="application/json",
+    body={
+        "model": "strands-decider-2B-hobson-v19",
+        "answers": {"reachable": {"type": "noul", "noul": 1.0}},
+        "usage": {"input_tokens": 10, "output_tokens": 1},
+    },
+)
+_CONFIGURED_STATE: Final[dict[str, JsonValue]] = {"ticket": "health probe"}
+_CONFIGURED_QUESTIONS: Final[dict[str, JsonValue]] = {
+    "alive": {"type": "choice", "criteria": {"yes": "the service answers", "no": "the service is down"}}
+}
 
 
 def test_health_check_of_a_model_added_through_the_api_calls_its_upstream_and_reports_it_healthy(
@@ -32,3 +65,85 @@ def test_health_check_of_a_model_added_through_the_api_calls_its_upstream_and_re
         assert [request["body"]["model"] for request in upstream.get("/__observations").json()["requests"]] == [
             provider_model
         ]
+
+
+def _health_report(gateway: Gateway, model: str) -> dict[str, JsonValue]:
+    health: Final = gateway.request("GET", "/health", params={"model": model})
+    assert health.status_code == 200, health.text
+    return health.json()
+
+
+def _probes_sent_to(gateway: Gateway, handle: ScenarioHandle) -> list[tuple[str, JsonValue]]:
+    with httpx.Client(base_url=gateway.upstream_url, timeout=5, trust_env=False) as upstream:
+        requests: Final = upstream.get("/__observations").json()["requests"]
+    return [
+        (string_value(request["path"]), request["body"])
+        for request in map(object_value, requests)
+        if string_value(request["path"]).startswith(f"/{handle.scenario_id}/")
+    ]
+
+
+def test_evaluation_mode_health_check_resolves_the_mode_from_the_cost_map_and_sends_the_default_probe(
+    gateway: Gateway,
+) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = register_scenario(f"health-decisions-{uuid.uuid4().hex[:12]}", _DECISIONS_PROBE_REPLY)
+        scenario.cleanups.callback(delete_scenario, handle)
+        model: Final = scenario.model(model="perplexity/pplx-decider-v1-27b", api_base=handle.api_base())
+        report: Final = _health_report(gateway, model)
+        assert (report["healthy_count"], report["unhealthy_count"]) == (1, 0), report
+        assert _probes_sent_to(gateway, handle) == [
+            (
+                f"/{handle.scenario_id}/v1/decisions",
+                {
+                    "model": "pplx-decider-v1-27b",
+                    "state": os.environ.get("DEFAULT_HEALTH_CHECK_PROMPT", "test from litellm"),
+                    "questions": {"reachable": {"type": "noul", "instructions": "Is the service reachable?"}},
+                },
+            )
+        ]
+
+
+def test_evaluation_mode_health_check_sends_the_configured_state_and_questions(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = register_scenario(f"health-decisions-{uuid.uuid4().hex[:12]}", _CONFIGURED_PROBE_REPLY)
+        scenario.cleanups.callback(delete_scenario, handle)
+        model: Final = scenario.model(
+            model="typesafe/jev-custom",
+            api_base=handle.api_base(),
+            model_info={
+                "mode": "evaluation",
+                "health_check_params": {"state": _CONFIGURED_STATE, "questions": _CONFIGURED_QUESTIONS},
+            },
+        )
+        report: Final = _health_report(gateway, model)
+        assert (report["healthy_count"], report["unhealthy_count"]) == (1, 0), report
+        assert _probes_sent_to(gateway, handle) == [
+            (
+                f"/{handle.scenario_id}/v1/systemone",
+                {"model": "jev-custom", "state": _CONFIGURED_STATE, "questions": _CONFIGURED_QUESTIONS},
+            )
+        ]
+
+
+def test_evaluation_mode_health_check_of_the_self_hosted_strands_model_resolves_the_mode_from_the_cost_map(
+    gateway: Gateway,
+) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = register_scenario(f"health-decisions-{uuid.uuid4().hex[:12]}", _STRANDS_PROBE_REPLY)
+        scenario.cleanups.callback(delete_scenario, handle)
+        model: Final = scenario.model(
+            model="strands_decider/strands-decider-2B-hobson-v19", api_base=handle.api_base(), api_key=None
+        )
+        report: Final = _health_report(gateway, model)
+        assert (report["healthy_count"], report["unhealthy_count"]) == (1, 0), report
+        assert _probes_sent_to(gateway, handle) == [
+            (
+                f"/{handle.scenario_id}/v1/systemone",
+                {
+                    "model": "strands-decider-2B-hobson-v19",
+                    "state": os.environ.get("DEFAULT_HEALTH_CHECK_PROMPT", "test from litellm"),
+                    "questions": {"reachable": {"type": "noul", "instructions": "Is the service reachable?"}},
+                },
+            )
+        ]

tests/integration/providers/test_decisions_chaos.py

@@ -0,0 +1,262 @@
+import asyncio
+import json
+import re
+import signal
+import threading
+import uuid
+from dataclasses import dataclass
+from pathlib import Path
+from queue import SimpleQueue
+from types import MappingProxyType
+from typing import Final
+from urllib.parse import urlsplit
+
+import httpx
+import psutil
+import pytest
+import yaml
+from integration._support.client import Gateway, eventually
+from integration._support.database import read_rows
+from integration._support.process import owned_proxy_process
+from integration._support.wire import Reply, Request, Wire, wire_server
+from pydantic import JsonValue, TypeAdapter
+
+_BACKEND: Final = "pplx-decider-v1-27b"
+_CONFIG_MODEL: Final = "decisions-chaos"
+_API_KEY: Final = "synthetic-decisions-key"
+_JSON_OBJECT: Final = TypeAdapter(dict[str, JsonValue])
+_STARTED_WORKER: Final = re.compile(r"Started server process \[(\d+)\]")
+_QUESTIONS: Final[dict[str, JsonValue]] = {"fine": {"type": "noul", "instructions": "Is the state fine?"}}
+_ROUTES: Final = ("/v1/systemone", "/systemone")
+
+
+@dataclass(frozen=True, slots=True)
+class _Call:
+    route: str
+    marker: str
+    fail: bool
+
+
+@dataclass(frozen=True, slots=True)
+class _Served:
+    call: _Call
+    status: int
+    call_id: str
+    model_group: str
+    text: str
+
+
+def _calls(count: int, *, fail: bool) -> tuple[_Call, ...]:
+    return tuple(
+        _Call(
+            route=_ROUTES[index % len(_ROUTES)],
+            marker=f"{'fail' if fail and index % 2 else 'ok'}-{uuid.uuid4().hex}",
+            fail=fail and index % 2 == 1,
+        )
+        for index in range(count)
+    )
+
+
+def _marker_of(request: Request) -> str:
+    state: Final = _JSON_OBJECT.validate_json(request.body)["state"]
+    assert isinstance(state, str), request.body
+    return state
+
+
+def _reply(request: Request) -> Reply:
+    marker: Final = _marker_of(request)
+    if marker.startswith("fail-"):
+        return Reply(status=500, body=json.dumps({"error": {"message": f"scripted outage {marker}"}}).encode())
+    answer: Final = {
+        "model": f"model-{marker}",
+        "answers": {"fine": {"type": "noul", "noul": 0.5}},
+        "usage": {"input_tokens": 12, "output_tokens": 1},
+    }
+    return Reply(body=json.dumps(answer).encode())
+
+
+async def _send(client: httpx.AsyncClient, key: str, model: str | None, call: _Call) -> _Served:
+    body: Final[dict[str, JsonValue]] = {
+        **({"model": model} if model is not None else {}),
+        "state": call.marker,
+        "questions": _QUESTIONS,
+        "num_retries": 0,
+    }
+    response: Final = await client.post(call.route, json=body, headers={"Authorization": f"Bearer {key}"})
+    return _Served(
+        call=call,
+        status=response.status_code,
+        call_id=response.headers.get("x-litellm-call-id", ""),
+        model_group=response.headers.get("x-litellm-model-group", ""),
+        text=response.text,
+    )
+
+
+async def _burst(
+    base_url: str, key: str, model: str | None, calls: tuple[_Call, ...], *, tolerate_transport_errors: bool = False
+) -> tuple[_Served, ...]:
+    async with httpx.AsyncClient(base_url=base_url, timeout=60, trust_env=False) as client:
+        results: Final = await asyncio.gather(
+            *(_send(client, key, model, call) for call in calls), return_exceptions=tolerate_transport_errors
+        )
+    for result in results:
+        assert not isinstance(result, BaseException) or isinstance(result, httpx.TransportError), repr(result)
+    return tuple(result for result in results if isinstance(result, _Served))
+
+
+def _assert_served_its_own(served: _Served) -> None:
+    assert served.call_id, served.text
+    if served.call.fail:
+        assert served.status == 500, (served.status, served.text)
+        assert served.call.marker in served.text, served.text
+        return
+    assert served.status == 200, (served.status, served.text)
+    assert _JSON_OBJECT.validate_json(served.text)["model"] == f"model-{served.call.marker}", served.text
+
+
+def _statuses_by_call_id(call_ids: tuple[str, ...]) -> dict[str, JsonValue]:
+    placeholders: Final = ", ".join("%s" for _ in call_ids)
+    rows: Final = eventually(
+        lambda: read_rows(
+            f'SELECT request_id, status FROM "LiteLLM_SpendLogs" WHERE request_id IN ({placeholders})', call_ids
+        ),
+        lambda found: len(found) >= len(call_ids),
+        seconds=70,
+    )
+    assert len(rows) == len(call_ids), rows
+    return {str(row["request_id"]): row["status"] for row in rows}
+
+
+def _expected_statuses(served: tuple[_Served, ...]) -> dict[str, JsonValue]:
+    return {item.call_id: "failure" if item.call.fail else "success" for item in served}
+
+
+def _health(gateway: Gateway, model: str) -> tuple[int, int]:
+    health: Final = gateway.request("GET", "/health", params={"model": model})
+    assert health.status_code in (200, 503), health.text
+    report: Final = health.json()
+    return (report["healthy_count"], report["unhealthy_count"])
+
+
+def _chaos_config(wire: Wire, tmp_path: Path) -> Path:
+    config: Final = {
+        **_JSON_OBJECT.validate_python(yaml.safe_load(Path("tests/integration/proxy_config.yaml").read_text())),
+        "model_list": [
+            {
+                "model_name": _CONFIG_MODEL,
+                "litellm_params": {"model": f"perplexity/{_BACKEND}", "api_base": wire.url, "api_key": _API_KEY},
+            }
+        ],
+    }
+    path: Final = tmp_path / "decisions-chaos.yaml"
+    path.write_text(yaml.safe_dump(config))
+    return path
+
+
+def _worker_startups(log: Path) -> tuple[tuple[int, ...], int]:
+    text: Final = log.read_text()
+    return tuple(int(pid) for pid in _STARTED_WORKER.findall(text)), text.count("Application startup complete.")
+
+
+def _open_upstream_connections(pid: int, upstream: str) -> int:
+    port: Final = urlsplit(upstream).port
+    return sum(
+        1
+        for connection in psutil.Process(pid).net_connections(kind="tcp")
+        if connection.status == psutil.CONN_ESTABLISHED and connection.raddr and connection.raddr.port == port
+    )
+
+
+@pytest.mark.timeout(180)
+async def test_burst_over_both_routes_bills_each_call_once_with_its_own_status(gateway: Gateway) -> None:
+    calls: Final = _calls(30, fail=True)
+    with wire_server(_reply) as wire, gateway.scenario() as scenario:
+        model: Final = scenario.model(model=f"perplexity/{_BACKEND}", api_base=wire.url, api_key=_API_KEY)
+        served: Final = await _burst(str(gateway.client.base_url), gateway.key, model, calls)
+        assert len(served) == 30
+        for item in served:
+            _assert_served_its_own(item)
+        assert len({item.call_id for item in served}) == 30
+        assert _statuses_by_call_id(tuple(item.call_id for item in served)) == _expected_statuses(served)
+        received: Final = wire.drain()
+        assert sorted(_marker_of(request) for request in received) == sorted(call.marker for call in calls)
+        assert {request.target for request in received} == {"/v1/decisions"}, received
+
+
+@pytest.mark.timeout(420)
+async def test_worker_sigkill_mid_burst_leaves_the_sibling_serving_the_default_model(
+    gateway: Gateway, tmp_path: Path
+) -> None:
+    calls: Final = _calls(20, fail=False)
+    release: Final = threading.Event()
+    held_markers: Final[SimpleQueue[str]] = SimpleQueue()
+
+    def held(request: Request) -> Reply:
+        held_markers.put(_marker_of(request))
+        assert release.wait(timeout=60), "The burst was never released"
+        return _reply(request)
+
+    with wire_server(held) as wire:
+        path: Final = _chaos_config(wire, tmp_path)
+        with owned_proxy_process(
+            gateway, tmp_path, {}, config=path, workers=2, extra_arguments=("--model", _CONFIG_MODEL)
+        ) as owned:
+            candidate: Final = owned.gateway
+            base_url: Final = str(candidate.client.base_url)
+            workers, _ = eventually(
+                lambda: _worker_startups(owned.log),
+                lambda found: len(found[0]) == 2 and found[1] == 2,
+                seconds=120,
+            )
+            burst: Final = asyncio.create_task(
+                _burst(base_url, candidate.key, None, calls, tolerate_transport_errors=True)
+            )
+            await asyncio.to_thread(eventually, held_markers.qsize, lambda size: size == 20, 60)
+            held_by: Final = MappingProxyType({pid: _open_upstream_connections(pid, wire.url) for pid in workers})
+            assert sum(held_by.values()) == 20, held_by
+            victim_pid, survivor_pid = sorted(workers, key=held_by.__getitem__)
+            victim: Final = psutil.Process(victim_pid)
+            victim.suspend()
+            victim.send_signal(signal.SIGKILL)
+            release.set()
+            served: Final = await burst
+            assert len(served) == held_by[survivor_pid], (held_by, len(served))
+            follow_up: Final = _Call(route="/systemone", marker=f"ok-{uuid.uuid4().hex}", fail=False)
+            (answered,) = await _burst(base_url, candidate.key, None, (follow_up,))
+            await asyncio.to_thread(
+                eventually,
+                lambda: _worker_startups(owned.log),
+                lambda found: len(found[0]) == 3 and found[1] == 3,
+                180,
+            )
+            for item in (*served, answered):
+                _assert_served_its_own(item)
+                assert item.model_group == _CONFIG_MODEL, item.model_group
+            call_ids: Final = tuple(item.call_id for item in (*served, answered))
+            assert set(_statuses_by_call_id(call_ids).values()) == {"success"}
+            assert len({_marker_of(request) for request in wire.drain()}) == 21
+
+
+@pytest.mark.timeout(180)
+async def test_upstream_outage_fails_its_calls_and_recovery_on_the_same_port_restores_them(gateway: Gateway) -> None:
+    base_url: Final = str(gateway.client.base_url)
+    with gateway.scenario() as scenario:
+        with wire_server(_reply) as wire:
+            port: Final = urlsplit(wire.url).port
+            assert port is not None
+            model: Final = scenario.model(model=f"perplexity/{_BACKEND}", api_base=wire.url, api_key=_API_KEY)
+            before: Final = await _burst(base_url, gateway.key, model, _calls(5, fail=False))
+            assert _health(gateway, model) == (1, 0)
+        during: Final = await _burst(base_url, gateway.key, model, _calls(5, fail=False))
+        assert [item.status for item in during] == [500] * 5, [item.text for item in during]
+        assert _health(gateway, model) == (0, 1)
+        with wire_server(_reply, port=port):
+            after: Final = await _burst(base_url, gateway.key, model, _calls(5, fail=False))
+            assert _health(gateway, model) == (1, 0)
+        for item in (*before, *after):
+            _assert_served_its_own(item)
+        statuses: Final = _statuses_by_call_id(tuple(item.call_id for item in (*before, *during, *after)))
+        assert statuses == {
+            **{item.call_id: "success" for item in (*before, *after)},
+            **{item.call_id: "failure" for item in during},
+        }

tests/integration/providers/test_decisions_wire.py

@@ -0,0 +1,458 @@
+import json
+import math
+import socket
+import uuid
+from collections.abc import Sequence
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Final
+
+import httpx
+import pytest
+import yaml
+from integration._support.client import Gateway, Scenario, eventually, object_value, string_value
+from integration._support.database import read_rows
+from integration._support.process import owned_proxy_process
+from integration._support.upstream import ScenarioHandle, delete_scenario, register_scenario
+from integration.cost_calculation.cost_tracking_case import JsonResponse
+from pydantic import JsonValue, TypeAdapter
+
+import litellm
+
+_API_KEY: Final = "synthetic-decisions-key"
+_ENV_KEY: Final = "synthetic-decisions-env-key"
+_PASS_THROUGH_MODEL: Final = "gpt-6-luna"
+_PASS_THROUGH_AUTHORIZATION: Final = "Bearer customer-held-upstream-key"
+_PASS_THROUGH_NEIGHBOUR: Final = "decisions-beside-a-pass-through"
+_JSON_OBJECT: Final = TypeAdapter(dict[str, JsonValue])
+_USAGE: Final[dict[str, JsonValue]] = {"input_tokens": 367, "output_tokens": 3}
+_STATE: Final[dict[str, JsonValue]] = {"ticket": "The export job hangs at 99%", "component": "billing"}
+_QUESTIONS: Final[dict[str, JsonValue]] = {
+    "defect": {"type": "noul", "instructions": "Is this a defect?"},
+    "severity": {"type": "choice", "criteria": {"low": "cosmetic", "high": "blocks users"}, "weight": 2},
+    "confidence": {"type": "score", "instructions": "How sure are you?", "criteria": ["unsure", "sure"]},
+}
+_ANSWERS: Final[dict[str, JsonValue]] = {
+    "defect": {"type": "noul", "noul": 0.93},
+    "severity": {"type": "choice", "choice": "high", "confidence": 0.8, "probabilities": {"low": 0.2, "high": 0.8}},
+    "confidence": {
+        "type": "score",
+        "score": 1.0,
+        "confidence": 0.7,
+        "legend": {"0": "unsure", "1": "sure"},
+        "probabilities": {"0": 0.3, "1": 0.7},
+    },
+}
+_CHAT_BODY: Final[dict[str, JsonValue]] = {"messages": [{"role": "user", "content": "hi"}]}
+_CHAT_REPLY: Final[dict[str, JsonValue]] = {
+    "id": "chatcmpl-decisions-parity",
+    "object": "chat.completion",
+    "created": 1700000000,
+    "model": "pplx-decider-v1-27b",
+    "choices": [{"index": 0, "message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}],
+    "usage": {"prompt_tokens": 3, "completion_tokens": 1, "total_tokens": 4},
+}
+_SPEND_QUERY: Final = (
+    "SELECT spend, status, call_type, model_group, custom_llm_provider, api_base, prompt_tokens, completion_tokens, "
+    'request_tags FROM "LiteLLM_SpendLogs" WHERE request_id = %s'
+)
+
+
+@dataclass(frozen=True, slots=True)
+class _Provider:
+    name: str
+    model: str
+    path: str
+    body_model: str
+    api_key: str | None
+    wraps_result: bool
+    cost_map_key: str | None
+
+
+_PROVIDERS: Final = (
+    _Provider(
+        "perplexity",
+        "perplexity/pplx-decider-v1-27b",
+        "/v1/decisions",
+        "pplx-decider-v1-27b",
+        _API_KEY,
+        False,
+        "perplexity/pplx-decider-v1-27b",
+    ),
+    _Provider("typesafe", "typesafe/jev-1.13.0", "/v1/systemone", "jev-1.13.0", _API_KEY, False, "typesafe/jev-1.13.0"),
+    _Provider(
+        "openrouter",
+        "openrouter/typesafe/jev-1.13",
+        "/alpha/decisions",
+        "typesafe/jev-1.13",
+        _API_KEY,
+        False,
+        "openrouter/typesafe/jev-1.13",
+    ),
+    _Provider(
+        "strands_decider", "strands_decider/systemone-decider", "/v1/systemone", "systemone-decider", None, False, None
+    ),
+    _Provider(
+        "cloudflare",
+        "cloudflare/clef",
+        "/ai/run/@cf/cloudflare/clef",
+        "clef",
+        _API_KEY,
+        True,
+        "cloudflare/@cf/cloudflare/clef",
+    ),
+)
+_PERPLEXITY: Final = _PROVIDERS[0]
+_INVALID_BODIES: Final[tuple[tuple[str, dict[str, JsonValue]], ...]] = (
+    ("missing questions", {"state": _STATE}),
+    ("missing state", {"questions": _QUESTIONS}),
+    ("numeric state", {"state": 5, "questions": _QUESTIONS}),
+    ("empty questions", {"state": _STATE, "questions": {}}),
+    ("noul without instructions or criteria", {"state": _STATE, "questions": {"q": {"type": "noul"}}}),
+    ("choice without criteria", {"state": _STATE, "questions": {"q": {"type": "choice", "criteria": {}}}}),
+    (
+        "score with eleven criteria",
+        {"state": _STATE, "questions": {"q": {"type": "score", "criteria": [f"level-{index}" for index in range(11)]}}},
+    ),
+    ("unknown question type", {"state": _STATE, "questions": {"q": {"type": "ranking", "criteria": ["a"]}}}),
+)
+
+
+def _number(value: JsonValue) -> float:
+    assert isinstance(value, (int, float)) and not isinstance(value, bool), value
+    return float(value)
+
+
+def _expected_spend(cost_map_key: str | None) -> float:
+    if cost_map_key is None:
+        return 0.0
+    prices: Final = object_value(json.loads(Path("model_prices_and_context_window.json").read_text())[cost_map_key])
+    return _number(_USAGE["input_tokens"]) * _number(prices["input_cost_per_token"]) + _number(
+        _USAGE["output_tokens"]
+    ) * _number(prices["output_cost_per_token"])
+
+
+def _answer_body(provider: _Provider) -> dict[str, JsonValue]:
+    answer: Final[dict[str, JsonValue]] = {"model": provider.body_model, "answers": _ANSWERS, "usage": _USAGE}
+    return {"result": answer, "success": True} if provider.wraps_result else answer
+
+
+def _register(scenario: Scenario, body: dict[str, JsonValue], *, status: int = 200) -> ScenarioHandle:
+    handle: Final = register_scenario(
+        f"decisions-{uuid.uuid4().hex[:12]}", JsonResponse(content_type="application/json", body=body, status=status)
+    )
+    scenario.cleanups.callback(delete_scenario, handle)
+    return handle
+
+
+def _deployment(scenario: Scenario, handle: ScenarioHandle, provider: _Provider) -> str:
+    return scenario.model(model=provider.model, api_base=handle.api_base(), api_key=provider.api_key)
+
+
+def _decide(gateway: Gateway, model: str, *, key: str | None = None, **extra: JsonValue) -> httpx.Response:
+    return gateway.request(
+        "POST", "/v1/systemone", {"model": model, "state": _STATE, "questions": _QUESTIONS, **extra}, key=key
+    )
+
+
+def _chat(gateway: Gateway, model: str, *, key: str | None = None, **extra: JsonValue) -> httpx.Response:
+    return gateway.request("POST", "/v1/chat/completions", {"model": model, **_CHAT_BODY, **extra}, key=key)
+
+
+def _observed_requests(gateway: Gateway) -> tuple[dict[str, JsonValue], ...]:
+    with httpx.Client(base_url=gateway.upstream_url, timeout=5, trust_env=False) as upstream:
+        return tuple(map(object_value, upstream.get("/__observations").json()["requests"]))
+
+
+def _calls_to(requests: Sequence[dict[str, JsonValue]], handle: ScenarioHandle) -> list[dict[str, JsonValue]]:
+    return [request for request in requests if string_value(request["path"]).startswith(f"/{handle.scenario_id}/")]
+
+
+def _upstream_calls(gateway: Gateway, handle: ScenarioHandle) -> list[dict[str, JsonValue]]:
+    return _calls_to(_observed_requests(gateway), handle)
+
+
+def _spend_row(call_id: str) -> dict[str, JsonValue]:
+    rows: Final = eventually(lambda: read_rows(_SPEND_QUERY, (call_id,)), lambda found: len(found) == 1, seconds=70)
+    return rows[0]
+
+
+def _free_closed_port() -> int:
+    with socket.socket() as probe:
+        probe.bind(("127.0.0.1", 0))
+        return probe.getsockname()[1]
+
+
+def _pass_through_config(directory: Path, pass_through_target: str, native_api_base: str) -> Path:
+    base: Final = _JSON_OBJECT.validate_python(yaml.safe_load(Path("tests/integration/proxy_config.yaml").read_text()))
+    config: Final = {
+        **base,
+        "general_settings": {
+            **object_value(base["general_settings"]),
+            "pass_through_endpoints": [
+                {
+                    "path": "/v1/decisions",
+                    "target": pass_through_target,
+                    "headers": {"Authorization": _PASS_THROUGH_AUTHORIZATION},
+                }
+            ],
+        },
+        "model_list": [
+            {
+                "model_name": _PASS_THROUGH_NEIGHBOUR,
+                "litellm_params": {"model": _PERPLEXITY.model, "api_base": native_api_base, "api_key": _API_KEY},
+            }
+        ],
+    }
+    path: Final = directory / "decisions-pass-through.yaml"
+    path.write_text(yaml.safe_dump(config))
+    return path
+
+
+@pytest.mark.parametrize("provider", _PROVIDERS, ids=lambda provider: provider.name)
+def test_each_provider_gets_its_own_path_key_and_body_and_is_billed_from_the_cost_map(
+    gateway: Gateway, provider: _Provider
+) -> None:
+    expected_spend: Final = _expected_spend(provider.cost_map_key)
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(provider))
+        model: Final = _deployment(scenario, handle, provider)
+        response: Final = _decide(gateway, model)
+        assert response.status_code == 200, response.text
+        assert response.json() == {"model": provider.body_model, "answers": _ANSWERS, "usage": _USAGE}
+        assert response.headers["x-litellm-model-group"] == model
+        assert math.isclose(float(response.headers.get("x-litellm-response-cost", "0")), expected_spend, rel_tol=1e-9)
+        (call,) = _upstream_calls(gateway, handle)
+        assert call["path"] == f"/{handle.scenario_id}{provider.path}"
+        assert call["authorization"] == (f"Bearer {provider.api_key}" if provider.api_key else "")
+        assert call["body"] == {"model": provider.body_model, "state": _STATE, "questions": _QUESTIONS}
+        row: Final = _spend_row(response.headers["x-litellm-call-id"])
+        assert (
+            row["status"],
+            row["call_type"],
+            row["custom_llm_provider"],
+            row["model_group"],
+            row["api_base"],
+            row["prompt_tokens"],
+            row["completion_tokens"],
+        ) == ("success", "adecisions", provider.name, model, f"{handle.api_base()}{provider.path}", 367, 3)
+        assert math.isclose(_number(row["spend"]), expected_spend, rel_tol=1e-9), row
+
+
+def test_repeated_identical_requests_each_reach_the_upstream_and_are_each_billed(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        responses: Final = tuple(_decide(gateway, model) for _ in range(2))
+        assert [response.status_code for response in responses] == [200, 200], [r.text for r in responses]
+        call_ids: Final = tuple(response.headers["x-litellm-call-id"] for response in responses)
+        assert len(set(call_ids)) == 2, call_ids
+        assert len(_upstream_calls(gateway, handle)) == 2
+        for call_id in call_ids:
+            assert _spend_row(call_id)["status"] == "success"
+
+
+async def test_sdk_sync_and_async_clients_send_the_same_request(gateway: Gateway) -> None:
+    provider: Final = _PROVIDERS[1]
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(provider))
+        synchronous: Final = litellm.decisions(
+            model=provider.model, state=_STATE, questions=_QUESTIONS, api_base=handle.api_base(), api_key=_API_KEY
+        )
+        asynchronous: Final = await litellm.adecisions(
+            model=provider.model, state=_STATE, questions=_QUESTIONS, api_base=handle.api_base(), api_key=_API_KEY
+        )
+        for response in (synchronous, asynchronous):
+            assert response.model_dump(mode="json") == {
+                "model": provider.body_model,
+                "answers": _ANSWERS,
+                "usage": _USAGE,
+            }
+        calls: Final = _upstream_calls(gateway, handle)
+        assert len(calls) == 2, calls
+        for call in calls:
+            assert call["path"] == f"/{handle.scenario_id}{provider.path}"
+            assert call["authorization"] == f"Bearer {_API_KEY}"
+            assert call["body"] == {"model": provider.body_model, "state": _STATE, "questions": _QUESTIONS}
+
+
+def test_gateway_only_fields_stay_at_the_gateway_and_tags_reach_the_spend_log(gateway: Gateway) -> None:
+    tag: Final = f"decisions-audit-{uuid.uuid4().hex[:8]}"
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        response: Final = _decide(
+            gateway, model, user="auditor", num_retries=0, temperature=0.2, metadata={"tags": [tag]}
+        )
+        assert response.status_code == 200, response.text
+        (call,) = _upstream_calls(gateway, handle)
+        assert call["body"] == {"model": _PERPLEXITY.body_model, "state": _STATE, "questions": _QUESTIONS}
+        row: Final = _spend_row(response.headers["x-litellm-call-id"])
+        tags: Final = row["request_tags"]
+        assert isinstance(tags, list) and tag in tags, row
+
+
+def test_invalid_bodies_are_refused_at_the_gateway_without_an_upstream_call(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        for label, body in _INVALID_BODIES:
+            response: Final = gateway.request("POST", "/v1/systemone", {"model": model, **body})
+            assert response.status_code == 400, (label, response.text)
+            assert "Invalid Decisions request" in response.text, (label, response.text)
+        assert _upstream_calls(gateway, handle) == []
+
+
+def test_unknown_model_is_refused_like_chat(gateway: Gateway) -> None:
+    model: Final = f"missing-{uuid.uuid4().hex}"
+    decisions: Final = _decide(gateway, model)
+    chat: Final = _chat(gateway, model)
+    assert 400 <= decisions.status_code < 500, decisions.text
+    assert decisions.status_code == chat.status_code, (decisions.text, chat.text)
+
+
+def test_key_checks_match_chat(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        anonymous: Final = gateway.client.post(
+            "/v1/systemone", json={"model": model, "state": _STATE, "questions": _QUESTIONS}
+        )
+        assert anonymous.status_code == 401, anonymous.text
+        restricted: Final = scenario.key(models=[f"other-{uuid.uuid4().hex}"])
+        refused: Final = _decide(gateway, model, key=restricted)
+        assert 400 <= refused.status_code < 500, refused.text
+        assert refused.status_code == _chat(gateway, model, key=restricted).status_code, refused.text
+        assert _upstream_calls(gateway, handle) == []
+        spender: Final = scenario.key(max_budget=1e-06)
+        first: Final = _decide(gateway, model, key=spender)
+        assert first.status_code == 200, first.text
+        blocked: Final = eventually(
+            lambda: _decide(gateway, model, key=spender), lambda response: response.status_code != 200, seconds=70
+        )
+        assert 400 <= blocked.status_code < 500, blocked.text
+        assert blocked.status_code == _chat(gateway, model, key=spender).status_code, blocked.text
+
+
+def test_request_body_api_base_is_refused_like_chat_without_an_upstream_call(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        decisions: Final = _decide(gateway, model, api_base=f"http://127.0.0.1:{_free_closed_port()}")
+        chat: Final = _chat(gateway, model, api_base=f"http://127.0.0.1:{_free_closed_port()}")
+        assert 400 <= decisions.status_code < 500, decisions.text
+        assert decisions.status_code == chat.status_code, (decisions.text, chat.text)
+        assert _upstream_calls(gateway, handle) == []
+
+
+def test_a_deployment_without_a_key_sends_the_provider_env_key_to_its_configured_api_base(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        model: Final = scenario.model(model=_PERPLEXITY.model, api_base=handle.api_base(), api_key=None)
+        response: Final = _decide(gateway, model)
+        assert response.status_code == 200, response.text
+        (call,) = _upstream_calls(gateway, handle)
+        assert (call["path"], call["authorization"]) == (f"/{handle.scenario_id}/v1/decisions", f"Bearer {_ENV_KEY}")
+
+
+def test_a_deployment_opted_into_client_api_base_sends_decisions_and_chat_to_the_body_api_base(
+    gateway: Gateway,
+) -> None:
+    with gateway.scenario() as scenario:
+        configured: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        decisions_target: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        chat_target: Final = _register(scenario, _CHAT_REPLY)
+        model: Final = scenario.model(
+            model=_PERPLEXITY.model,
+            api_base=configured.api_base(),
+            api_key=_API_KEY,
+            configurable_clientside_auth_params=["api_base"],
+        )
+        decisions: Final = _decide(gateway, model, api_base=decisions_target.api_base())
+        chat: Final = _chat(gateway, model, api_base=chat_target.api_base())
+        assert decisions.status_code == 200, decisions.text
+        assert chat.status_code == 200, chat.text
+        observed: Final = _observed_requests(gateway)
+        assert [call["path"] for call in _calls_to(observed, decisions_target)] == [
+            f"/{decisions_target.scenario_id}/v1/decisions"
+        ]
+        assert [call["path"] for call in _calls_to(observed, chat_target)] == [
+            f"/{chat_target.scenario_id}/chat/completions"
+        ]
+        assert _calls_to(observed, configured) == []
+
+
+def test_a_config_pass_through_at_v1_decisions_keeps_answering_and_the_native_api_serves_system_one(
+    gateway: Gateway, tmp_path: Path
+) -> None:
+    with gateway.scenario() as scenario:
+        pass_through_target: Final = _register(
+            scenario, {"model": _PASS_THROUGH_MODEL, "answers": _ANSWERS, "usage": _USAGE}
+        )
+        native_target: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        config: Final = _pass_through_config(
+            tmp_path, f"{pass_through_target.api_base()}/v1/decisions", native_target.api_base()
+        )
+        with owned_proxy_process(gateway, tmp_path, {}, config=config) as owned:
+            through: Final = owned.gateway.request(
+                "POST", "/v1/decisions", {"model": _PASS_THROUGH_MODEL, "state": _STATE, "questions": _QUESTIONS}
+            )
+            native: Final = owned.gateway.request(
+                "POST", "/systemone", {"model": _PASS_THROUGH_NEIGHBOUR, "state": _STATE, "questions": _QUESTIONS}
+            )
+        assert through.status_code == 200, through.text
+        assert through.json() == {"model": _PASS_THROUGH_MODEL, "answers": _ANSWERS, "usage": _USAGE}
+        observed: Final = _observed_requests(gateway)
+        (forwarded,) = _calls_to(observed, pass_through_target)
+        assert (forwarded["path"], forwarded["authorization"], object_value(forwarded["body"])["model"]) == (
+            f"/{pass_through_target.scenario_id}/v1/decisions",
+            _PASS_THROUGH_AUTHORIZATION,
+            _PASS_THROUGH_MODEL,
+        )
+        assert native.status_code == 200, native.text
+        assert [call["path"] for call in _calls_to(observed, native_target)] == [
+            f"/{native_target.scenario_id}/v1/decisions"
+        ]
+
+
+@pytest.mark.parametrize("status", (401, 429, 500))
+def test_upstream_errors_keep_their_status_and_log_an_unbilled_failure(gateway: Gateway, status: int) -> None:
+    marker: Final = f"scripted-{status}-{uuid.uuid4().hex[:8]}"
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, {"error": {"message": marker}}, status=status)
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        response: Final = _decide(gateway, model, num_retries=0)
+        assert response.status_code == status, response.text
+        assert marker in response.text
+        row: Final = _spend_row(response.headers["x-litellm-call-id"])
+        assert (row["status"], row["call_type"], row["model_group"], _number(row["spend"])) == (
+            "failure",
+            "adecisions",
+            model,
+            0.0,
+        )
+
+
+def test_upstream_success_without_answers_is_a_gateway_side_server_error(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, {"model": _PERPLEXITY.body_model, "usage": _USAGE})
+        model: Final = _deployment(scenario, handle, _PERPLEXITY)
+        response: Final = _decide(gateway, model, num_retries=0)
+        assert 500 <= response.status_code < 600, response.text
+        assert "answers" in response.text
+        assert _spend_row(response.headers["x-litellm-call-id"])["status"] == "failure"
+
+
+def test_unreachable_upstream_fails_only_its_own_deployment(gateway: Gateway) -> None:
+    with gateway.scenario() as scenario:
+        handle: Final = _register(scenario, _answer_body(_PERPLEXITY))
+        healthy: Final = _deployment(scenario, handle, _PERPLEXITY)
+        dead: Final = scenario.model(
+            model=_PERPLEXITY.model, api_base=f"http://127.0.0.1:{_free_closed_port()}", api_key=_API_KEY
+        )
+        failed: Final = _decide(gateway, dead, num_retries=0)
+        assert 500 <= failed.status_code < 600, failed.text
+        assert _spend_row(failed.headers["x-litellm-call-id"])["status"] == "failure"
+        served: Final = _decide(gateway, healthy)
+        assert served.status_code == 200, served.text
+        assert len(_upstream_calls(gateway, handle)) == 1

tests/integration/proxy_config.yaml

@@ -23,3 +23,5 @@ vector_store_registry:
       api_base: os.environ/INTEGRATION_UPSTREAM_URL
       api_key: integration-provider-key
       vector_store_description: declared in tests/integration/proxy_config.yaml
+environment_variables:
+  PERPLEXITYAI_API_KEY: synthetic-decisions-env-key

tests/unit/decisions/test_main.py

@@ -0,0 +1,841 @@
+from __future__ import annotations
+
+import asyncio
+import json
+from collections.abc import Mapping
+from types import MappingProxyType
+from typing import Final
+
+import pytest
+import respx
+
+import litellm
+from litellm.integrations.custom_logger import CustomLogger
+from litellm.litellm_core_utils.logging_worker import GLOBAL_LOGGING_WORKER
+from litellm.types.decisions import (
+    ChoiceAnswer,
+    DecisionsResponse,
+    DecisionsUsage,
+    NoulAnswer,
+    OpenAIDecisionResponse,
+    ScoreAnswer,
+)
+
+_QUESTIONS: Final[Mapping[str, object]] = MappingProxyType(
+    {
+        "is_defect": {"type": "noul", "instructions": "Is this a defect?", "provider_field": "kept"},
+        "sentiment": {"type": "choice", "criteria": {"positive": None, "negative": "unhappy"}},
+        "severity": {"type": "score", "criteria": ["none", "low", "high"]},
+    }
+)
+_INPUT_TOKENS: Final[int] = 367
+_OUTPUT_TOKENS: Final[int] = 3
+_CACHED_TOKENS: Final[int] = 256
+_CACHE_WRITE_TOKENS: Final[int] = 64
+_RESPONSE: Final[Mapping[str, object]] = {
+    "model": "jev-1.13",
+    "answers": {
+        "is_defect": {"type": "noul", "noul": 0.9},
+        "sentiment": {
+            "type": "choice",
+            "choice": "positive",
+            "confidence": 0.8,
+            "probabilities": {"positive": 0.8, "negative": 0.2},
+        },
+        "severity": {
+            "type": "score",
+            "score": 1,
+            "confidence": 0.7,
+            "legend": {"0": "none", "1": "low", "2": "high"},
+            "probabilities": {"0": 0.1, "1": 0.8, "2": 0.1},
+        },
+    },
+    "usage": {"input_tokens": _INPUT_TOKENS, "output_tokens": _OUTPUT_TOKENS},
+}
+_STRANDS_RESPONSE: Final[Mapping[str, object]] = {
+    "model": "strands-decider-2B-hobson-v19",
+    "answers": {
+        "severity": {
+            "type": "score",
+            "score": 1,
+            "confidence": 0.7,
+            "legend": {"0": "none", "1": "low", "2": "high"},
+            "probabilities": {"0": 0.1, "1": 0.8, "2": 0.1},
+        }
+    },
+    "usage": {"input_tokens": 216, "output_tokens": 3},
+    "latency_ms": 3722.17,
+}
+_PROVIDERS: Final[tuple[tuple[str, str, str, str], ...]] = (
+    (
+        "perplexity",
+        "perplexity/pplx-decider-v1-27b",
+        "https://api.perplexity.ai/v1/decisions",
+        "pplx-decider-v1-27b",
+    ),
+    ("typesafe", "typesafe/jev-1.13", "https://api.typesafe.ai/v1/systemone", "jev-1.13"),
+    (
+        "openrouter",
+        "openrouter/typesafe/jev-1.13",
+        "https://openrouter.ai/api/alpha/decisions",
+        "typesafe/jev-1.13",
+    ),
+)
+
+_OPENAI_RESPONSE: Final[Mapping[str, object]] = {
+    "model": "gpt-6-luna",
+    "answers": [
+        {"type": "predicate", "name": "is_defect", "probability": 0.9},
+        {"type": "refusal", "name": "sentiment"},
+    ],
+    "usage": {
+        "input_tokens": _INPUT_TOKENS,
+        "input_tokens_details": {"cached_tokens": _CACHED_TOKENS, "cache_write_tokens": _CACHE_WRITE_TOKENS},
+        "output_tokens": _OUTPUT_TOKENS,
+        "output_tokens_details": {"reasoning_tokens": 0},
+        "total_tokens": _INPUT_TOKENS + _OUTPUT_TOKENS,
+    },
+}
+
+
+class _RecordingLogger(CustomLogger):
+    def __init__(self) -> None:
+        super().__init__()
+        self.standard_logging_object: Mapping[str, object] | None = None
+
+    async def async_log_success_event(
+        self,
+        kwargs: Mapping[str, object],
+        response_obj: object,
+        start_time: object,
+        end_time: object,
+    ) -> None:
+        standard_logging_object: Final = kwargs.get("standard_logging_object")
+        if isinstance(standard_logging_object, dict):
+            self.standard_logging_object = standard_logging_object
+
+
+async def _drain_logging_worker() -> None:
+    await asyncio.sleep(0)
+    GLOBAL_LOGGING_WORKER.start()
+    await asyncio.wait_for(GLOBAL_LOGGING_WORKER.flush(), timeout=10.0)
+
+
+@pytest.fixture(autouse=True)
+def _httpx_transport(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
+    litellm.in_memory_llm_clients_cache.flush_cache()
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(("provider", "model", "url", "upstream_model"), _PROVIDERS)
+async def test_adecisions_sends_the_provider_wire_contract(
+    provider: str,
+    model: str,
+    url: str,
+    upstream_model: str,
+    respx_mock: respx.MockRouter,
+) -> None:
+    route: Final = respx_mock.post(url).respond(json=_RESPONSE)
+
+    response: Final = await litellm.adecisions(
+        model=model,
+        state={"source": "unit-test"},
+        questions=_QUESTIONS,
+        api_key="caller-key",
+        extra_headers={
+            "x-request-tag": "decisions-test",
+            "AUTHORIZATION": "attacker-key",
+            "Content-Type": "text/plain",
+        },
+        internal_kwarg="must-not-leak",
+    )
+
+    assert route.called
+    assert len(respx_mock.calls) == 1
+    request: Final = respx_mock.calls[0].request
+    assert request.headers["authorization"] == "Bearer caller-key"
+    assert request.headers["content-type"] == "application/json"
+    assert request.headers["x-request-tag"] == "decisions-test"
+    assert json.loads(request.content) == {
+        "model": upstream_model,
+        "state": {"source": "unit-test"},
+        "questions": {
+            "is_defect": {
+                "type": "noul",
+                "instructions": "Is this a defect?",
+                "provider_field": "kept",
+            },
+            "sentiment": {"type": "choice", "criteria": {"positive": None, "negative": "unhappy"}},
+            "severity": {"type": "score", "criteria": ["none", "low", "high"]},
+        },
+    }
+    assert isinstance(response.answers["is_defect"], NoulAnswer)
+    assert isinstance(response.answers["sentiment"], ChoiceAnswer)
+    assert isinstance(response.answers["severity"], ScoreAnswer)
+    assert response._hidden_params["custom_llm_provider"] == provider
+
+
+@pytest.mark.asyncio
+async def test_router_dispatches_typesafe_decisions_without_api_base(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("TYPESAFE_API_KEY", raising=False)
+    monkeypatch.delenv("TYPESAFE_API_BASE", raising=False)
+    provider_resolution: Final = litellm.get_llm_provider("typesafe/jev-latest")
+
+    assert provider_resolution[:2] == ("jev-latest", "typesafe")
+
+    router: Final = litellm.Router(
+        model_list=[
+            {
+                "model_name": "jev",
+                "litellm_params": {
+                    "model": "typesafe/jev-latest",
+                    "api_key": "k",
+                },
+            }
+        ]
+    )
+    upstream: Final = respx_mock.post("https://api.typesafe.ai/v1/systemone").respond(json=_RESPONSE)
+
+    response: Final = await router.adecisions(
+        model="jev",
+        state="router-test",
+        questions={
+            "sentiment": {
+                "type": "choice",
+                "criteria": {"positive": None, "negative": "unhappy"},
+            }
+        },
+    )
+
+    assert upstream.called
+    assert len(respx_mock.calls) == 1
+    assert json.loads(respx_mock.calls[0].request.content) == {
+        "model": "jev-latest",
+        "state": "router-test",
+        "questions": {
+            "sentiment": {
+                "type": "choice",
+                "criteria": {"positive": None, "negative": "unhappy"},
+            }
+        },
+    }
+    assert respx_mock.calls[0].request.headers["authorization"] == "Bearer k"
+    assert isinstance(response.answers["sentiment"], ChoiceAnswer)
+    assert response.answers["sentiment"].choice == "positive"
+
+
+def test_decisions_uses_the_same_wire_contract_for_sync_calls(respx_mock: respx.MockRouter) -> None:
+    route: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = litellm.decisions(
+        model="perplexity/pplx-decider-v1-27b",
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        api_key="caller-key",
+    )
+
+    assert route.called
+    assert response.model == "jev-1.13"
+
+
+def test_openrouter_response_keeps_provider_fields(respx_mock: respx.MockRouter) -> None:
+    payload: Final = {
+        **_RESPONSE,
+        "id": "decision-1",
+        "provider": "typesafe",
+        "usage": {**_RESPONSE["usage"], "cost": 0.25},
+    }
+    respx_mock.post("https://openrouter.ai/api/alpha/decisions").respond(json=payload)
+
+    response: Final = litellm.decisions(
+        model="openrouter/typesafe/jev-1.13",
+        state="review",
+        questions=_QUESTIONS,
+        api_key="caller-key",
+    )
+
+    assert response.model_extra["id"] == "decision-1"
+    assert response.model_extra["provider"] == "typesafe"
+    assert response.usage is not None
+    assert response.usage.model_extra["cost"] == 0.25
+
+
+def test_decisions_cost_uses_litellm_token_pricing() -> None:
+    response: Final = DecisionsResponse(
+        model="pplx-decider-v1-27b",
+        answers={},
+        usage=DecisionsUsage(input_tokens=_INPUT_TOKENS, output_tokens=_OUTPUT_TOKENS),
+    )
+    response._hidden_params = {
+        "model": "perplexity/pplx-decider-v1-27b",
+        "custom_llm_provider": "perplexity",
+    }
+
+    cost: Final = litellm.completion_cost(completion_response=response)
+    perplexity_cost: Final = litellm.model_cost["perplexity/pplx-decider-v1-27b"]
+    expected_cost: Final = _INPUT_TOKENS * float(perplexity_cost["input_cost_per_token"]) + _OUTPUT_TOKENS * float(
+        perplexity_cost["output_cost_per_token"]
+    )
+
+    assert expected_cost > 0
+    assert cost == pytest.approx(expected_cost)
+
+
+@pytest.mark.parametrize(
+    "response",
+    (
+        DecisionsResponse(
+            model="jev-latest",
+            answers={},
+            usage=DecisionsUsage(
+                input_tokens=_INPUT_TOKENS,
+                output_tokens=_OUTPUT_TOKENS,
+                cached_tokens=_CACHED_TOKENS,
+                cache_write_tokens=_CACHE_WRITE_TOKENS,
+            ),
+        ),
+        OpenAIDecisionResponse.model_validate(_OPENAI_RESPONSE),
+    ),
+    ids=("systemone", "openai"),
+)
+def test_custom_token_pricing_bills_cached_decisions_input_tokens_once(
+    response: DecisionsResponse | OpenAIDecisionResponse,
+) -> None:
+    cost: Final = litellm.completion_cost(
+        completion_response=response,
+        model="gpt-6-luna",
+        custom_llm_provider="openai",
+        custom_cost_per_token={
+            "input_cost_per_token": 1.0,
+            "output_cost_per_token": 2.0,
+            "cache_read_input_token_cost": 0.1,
+            "cache_creation_input_token_cost": 1.25,
+        },
+    )
+
+    assert cost == pytest.approx(
+        (_INPUT_TOKENS - _CACHED_TOKENS - _CACHE_WRITE_TOKENS) * 1.0
+        + _CACHED_TOKENS * 0.1
+        + _CACHE_WRITE_TOKENS * 1.25
+        + _OUTPUT_TOKENS * 2.0
+    )
+
+
+@pytest.mark.asyncio
+async def test_decisions_cost_is_in_standard_logging_object(respx_mock: respx.MockRouter) -> None:
+    respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+    recording_logger: Final = _RecordingLogger()
+    original_callbacks: Final = litellm.callbacks
+    litellm.callbacks = [recording_logger]
+
+    try:
+        await litellm.adecisions(
+            model="perplexity/pplx-decider-v1-27b",
+            state="review",
+            questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+            api_key="caller-key",
+        )
+        await _drain_logging_worker()
+    finally:
+        litellm.callbacks = original_callbacks
+
+    assert recording_logger.standard_logging_object is not None
+    perplexity_cost: Final = litellm.model_cost["perplexity/pplx-decider-v1-27b"]
+    expected_cost: Final = _INPUT_TOKENS * float(perplexity_cost["input_cost_per_token"]) + _OUTPUT_TOKENS * float(
+        perplexity_cost["output_cost_per_token"]
+    )
+
+    assert expected_cost > 0
+    assert recording_logger.standard_logging_object["response_cost"] == pytest.approx(expected_cost)
+    assert recording_logger.standard_logging_object["prompt_tokens"] == _INPUT_TOKENS
+    assert recording_logger.standard_logging_object["completion_tokens"] == _OUTPUT_TOKENS
+
+
+@pytest.mark.asyncio
+async def test_unknown_provider_is_rejected_before_http(respx_mock: respx.MockRouter) -> None:
+    with pytest.raises(litellm.BadRequestError, match="Supported providers"):
+        await litellm.adecisions(
+            model="unknown/jev-1.13",
+            state="review",
+            questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+            api_key="caller-key",
+        )
+
+    assert len(respx_mock.calls) == 0
+
+
+@pytest.mark.asyncio
+async def test_empty_custom_provider_is_rejected_before_http(respx_mock: respx.MockRouter) -> None:
+    with pytest.raises(litellm.BadRequestError, match="Supported providers"):
+        await litellm.adecisions(
+            model="perplexity/pplx-decider-v1-27b",
+            state="review",
+            questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+            api_key="caller-key",
+            custom_llm_provider="",
+        )
+
+    assert len(respx_mock.calls) == 0
+
+
+@pytest.mark.asyncio
+async def test_invalid_question_is_rejected_before_http(respx_mock: respx.MockRouter) -> None:
+    with pytest.raises(litellm.BadRequestError, match="Invalid Decisions request"):
+        await litellm.adecisions(
+            model="perplexity/pplx-decider-v1-27b",
+            state="review",
+            questions={"sentiment": {"type": "choice"}},
+            api_key="caller-key",
+        )
+
+    assert len(respx_mock.calls) == 0
+
+
+def test_upstream_bad_request_maps_to_litellm_error(respx_mock: respx.MockRouter) -> None:
+    respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(
+        status_code=400,
+        json={"error": {"message": "invalid decision"}},
+    )
+
+    with pytest.raises(litellm.BadRequestError):
+        litellm.decisions(
+            model="perplexity/pplx-decider-v1-27b",
+            state="review",
+            questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+            api_key="caller-key",
+        )
+
+
+def test_server_key_is_sent_to_an_explicit_api_base(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("PERPLEXITYAI_API_KEY", "server-key")
+    monkeypatch.delenv("PERPLEXITY_API_KEY", raising=False)
+    route: Final = respx_mock.post("https://egress.example/perplexity/v1/decisions").respond(json=_RESPONSE)
+
+    litellm.decisions(
+        model="perplexity/pplx-decider-v1-27b",
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        api_base="https://egress.example/perplexity",
+    )
+
+    assert route.call_count == 1
+    assert route.calls[0].request.headers["authorization"] == "Bearer server-key"
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("model", ("cloudflare/clef", "cloudflare/@cf/cloudflare/clef"))
+@pytest.mark.parametrize("wrapped", (False, True))
+async def test_cloudflare_clef_resolves_model_and_response_envelope(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+    model: str,
+    wrapped: bool,
+) -> None:
+    monkeypatch.setenv("CLOUDFLARE_ACCOUNT_ID", "acct")
+    monkeypatch.setenv("CLOUDFLARE_API_KEY", "cloudflare-key")
+    monkeypatch.delenv("CLOUDFLARE_API_BASE", raising=False)
+    response_body: Final[Mapping[str, object]] = (
+        {"result": _RESPONSE, "success": True, "errors": [], "messages": []} if wrapped else _RESPONSE
+    )
+    route: Final = respx_mock.post(
+        "https://api.cloudflare.com/client/v4/accounts/acct/ai/run/@cf/cloudflare/clef"
+    ).respond(json=response_body)
+
+    response: Final = await litellm.adecisions(
+        model=model,
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    )
+
+    assert route.called
+    request: Final = respx_mock.calls[0].request
+    assert request.headers["authorization"] == "Bearer cloudflare-key"
+    assert json.loads(request.content) == {
+        "model": "clef",
+        "state": "review",
+        "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    }
+    assert response.answers == {"is_defect": NoulAnswer(type="noul", noul=0.9)}
+    assert response._hidden_params["model"] == "cloudflare/@cf/cloudflare/clef"
+
+
+@pytest.mark.asyncio
+async def test_cloudflare_clef_flash_uses_flash_endpoint_and_request_model(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("CLOUDFLARE_ACCOUNT_ID", "acct")
+    monkeypatch.setenv("CLOUDFLARE_API_KEY", "cloudflare-key")
+    monkeypatch.delenv("CLOUDFLARE_API_BASE", raising=False)
+    route: Final = respx_mock.post(
+        "https://api.cloudflare.com/client/v4/accounts/acct/ai/run/@cf/cloudflare/clef-flash"
+    ).respond(json=_RESPONSE)
+
+    await litellm.adecisions(
+        model="cloudflare/clef-flash",
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    )
+
+    assert route.called
+    assert json.loads(respx_mock.calls[0].request.content)["model"] == "clef-flash"
+
+
+@pytest.mark.asyncio
+async def test_cloudflare_api_base_from_env_uses_workers_ai_run_path(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("CLOUDFLARE_API_BASE", "https://api.cloudflare.com/client/v4/accounts/acct/ai/v1")
+    monkeypatch.setenv("CLOUDFLARE_API_KEY", "cloudflare-key")
+    monkeypatch.delenv("CLOUDFLARE_ACCOUNT_ID", raising=False)
+    route: Final = respx_mock.post(
+        "https://api.cloudflare.com/client/v4/accounts/acct/ai/run/@cf/cloudflare/clef"
+    ).respond(json=_RESPONSE)
+
+    await litellm.adecisions(
+        model="cloudflare/clef",
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    )
+
+    assert route.called
+
+
+@pytest.mark.asyncio
+async def test_cloudflare_requires_account_id_or_api_base_before_http(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("CLOUDFLARE_ACCOUNT_ID", raising=False)
+    monkeypatch.delenv("CLOUDFLARE_API_BASE", raising=False)
+    monkeypatch.setenv("CLOUDFLARE_API_KEY", "cloudflare-key")
+
+    with pytest.raises(litellm.BadRequestError, match="Missing CLOUDFLARE_ACCOUNT_ID - set CLOUDFLARE_ACCOUNT_ID"):
+        await litellm.adecisions(
+            model="cloudflare/clef",
+            state="review",
+            questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        )
+
+    assert len(respx_mock.calls) == 0
+
+
+@pytest.mark.asyncio
+async def test_cloudflare_clef_cost_uses_the_model_cost_map(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("CLOUDFLARE_ACCOUNT_ID", "acct")
+    monkeypatch.setenv("CLOUDFLARE_API_KEY", "cloudflare-key")
+    monkeypatch.delenv("CLOUDFLARE_API_BASE", raising=False)
+    respx_mock.post("https://api.cloudflare.com/client/v4/accounts/acct/ai/run/@cf/cloudflare/clef").respond(
+        json=_RESPONSE
+    )
+
+    response: Final = await litellm.adecisions(
+        model="cloudflare/clef",
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    )
+
+    cost: Final = litellm.completion_cost(completion_response=response)
+    clef_cost: Final = litellm.model_cost["cloudflare/@cf/cloudflare/clef"]
+    expected_cost: Final = _INPUT_TOKENS * float(clef_cost["input_cost_per_token"]) + _OUTPUT_TOKENS * float(
+        clef_cost["output_cost_per_token"]
+    )
+
+    assert expected_cost > 0
+    assert cost == pytest.approx(expected_cost)
+
+
+@pytest.mark.asyncio
+async def test_strands_decider_requires_api_base_before_http(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("STRANDS_DECIDER_API_BASE", raising=False)
+    monkeypatch.delenv("STRANDS_DECIDER_API_KEY", raising=False)
+
+    with pytest.raises(litellm.BadRequestError, match="api_base is required"):
+        await litellm.adecisions(
+            model="strands_decider/strands-decider-2B-hobson-v19",
+            state="review",
+            questions={"severity": {"type": "score", "criteria": ["none", "low", "high"]}},
+        )
+
+    assert len(respx_mock.calls) == 0
+
+
+@pytest.mark.asyncio
+async def test_strands_decider_without_key_preserves_response_extras(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("STRANDS_DECIDER_API_BASE", raising=False)
+    monkeypatch.delenv("STRANDS_DECIDER_API_KEY", raising=False)
+    route: Final = respx_mock.post("https://strands.example/v1/systemone").respond(json=_STRANDS_RESPONSE)
+
+    response: Final = await litellm.adecisions(
+        model="strands_decider/strands-decider-2B-hobson-v19",
+        state="review",
+        questions={"severity": {"type": "score", "criteria": ["none", "low", "high"]}},
+        api_base="https://strands.example",
+    )
+
+    assert route.called
+    assert "authorization" not in respx_mock.calls[0].request.headers
+    assert response.model_extra["latency_ms"] == _STRANDS_RESPONSE["latency_ms"]
+    severity: Final = response.answers["severity"]
+    assert isinstance(severity, ScoreAnswer)
+    assert severity.legend == {"0": "none", "1": "low", "2": "high"}
+
+
+@pytest.mark.asyncio
+async def test_strands_decider_uses_key_from_matching_environment_base(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("STRANDS_DECIDER_API_BASE", "https://strands.example")
+    monkeypatch.setenv("STRANDS_DECIDER_API_KEY", "strands-key")
+    route: Final = respx_mock.post("https://strands.example/v1/systemone").respond(json=_STRANDS_RESPONSE)
+
+    await litellm.adecisions(
+        model="strands_decider/strands-decider-2B-hobson-v19",
+        state="review",
+        questions={"severity": {"type": "score", "criteria": ["none", "low", "high"]}},
+        api_base="https://strands.example",
+    )
+
+    assert route.called
+    assert respx_mock.calls[0].request.headers["authorization"] == "Bearer strands-key"
+
+
+@pytest.mark.asyncio
+async def test_strands_decider_provider_resolution_and_router_dispatch(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("STRANDS_DECIDER_API_BASE", raising=False)
+    monkeypatch.delenv("STRANDS_DECIDER_API_KEY", raising=False)
+    provider_resolution: Final = litellm.get_llm_provider("strands_decider/strands-decider-2B-hobson-v19")
+    router: Final = litellm.Router(
+        model_list=[
+            {
+                "model_name": "strands",
+                "litellm_params": {
+                    "model": "strands_decider/strands-decider-2B-hobson-v19",
+                    "api_base": "https://strands.example",
+                },
+            }
+        ]
+    )
+    route: Final = respx_mock.post("https://strands.example/v1/systemone").respond(json=_STRANDS_RESPONSE)
+
+    response: Final = await router.adecisions(
+        model="strands",
+        state="review",
+        questions={"severity": {"type": "score", "criteria": ["none", "low", "high"]}},
+    )
+
+    assert provider_resolution[:2] == ("strands-decider-2B-hobson-v19", "strands_decider")
+    assert route.called
+    assert response.model == _STRANDS_RESPONSE["model"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("api_base", "url"),
+    (
+        (None, "https://api.openai.com/v1/decisions"),
+        ("https://gateway.example/v1", "https://gateway.example/v1/decisions"),
+    ),
+)
+async def test_openai_decisions_translate_systemone_to_the_openai_wire_contract_and_back(
+    api_base: str | None,
+    url: str,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setattr(litellm, "api_base", None)
+    monkeypatch.delenv("OPENAI_BASE_URL", raising=False)
+    monkeypatch.delenv("OPENAI_API_BASE", raising=False)
+    route: Final = respx_mock.post(url).respond(json=_OPENAI_RESPONSE)
+
+    response: Final = await litellm.adecisions(
+        model="openai/gpt-6-luna",
+        state="The package arrived broken.",
+        questions={
+            "is_defect": {"type": "noul", "instructions": "Is this a defect?"},
+            "sentiment": {
+                "type": "choice",
+                "instructions": "How does the customer feel?",
+                "criteria": {"positive": None, "negative": "unhappy"},
+            },
+        },
+        api_key="caller-key",
+        api_base=api_base,
+    )
+
+    assert route.called
+    request: Final = respx_mock.calls[0].request
+    assert request.headers["authorization"] == "Bearer caller-key"
+    assert json.loads(request.content) == {
+        "model": "gpt-6-luna",
+        "input": "The package arrived broken.",
+        "questions": [
+            {"type": "predicate", "name": "is_defect", "instructions": "Is this a defect?"},
+            {
+                "type": "choice",
+                "name": "sentiment",
+                "instructions": "How does the customer feel?",
+                "choices": [{"value": "positive"}, {"value": "negative", "description": "unhappy"}],
+            },
+        ],
+    }
+    assert response.answers == {"is_defect": NoulAnswer(type="noul", noul=0.9)}
+    assert response._hidden_params["custom_llm_provider"] == "openai"
+    luna_cost: Final = litellm.model_cost["gpt-6-luna"]
+    expected_cost: Final = (
+        (_INPUT_TOKENS - _CACHED_TOKENS - _CACHE_WRITE_TOKENS) * float(luna_cost["input_cost_per_token"])
+        + _CACHED_TOKENS * float(luna_cost["cache_read_input_token_cost"])
+        + _CACHE_WRITE_TOKENS * float(luna_cost["cache_creation_input_token_cost"])
+        + _OUTPUT_TOKENS * float(luna_cost["output_cost_per_token"])
+    )
+    assert expected_cost > 0
+    assert litellm.completion_cost(completion_response=response) == pytest.approx(expected_cost)
+
+
+@pytest.mark.parametrize(
+    ("settings", "env", "url", "authorization"),
+    (
+        ({"openai_key": "sdk-key"}, {}, "https://api.openai.com/v1/decisions", "Bearer sdk-key"),
+        (
+            {"api_key": "global-key", "openai_key": "sdk-key"},
+            {"OPENAI_API_KEY": "env-key"},
+            "https://api.openai.com/v1/decisions",
+            "Bearer global-key",
+        ),
+        (
+            {},
+            {"OPENAI_API_KEY": "env-key", "OPENAI_API_BASE": "https://legacy.example/v1"},
+            "https://legacy.example/v1/decisions",
+            "Bearer env-key",
+        ),
+        (
+            {"api_base": "https://sdk.example"},
+            {"OPENAI_API_KEY": "env-key", "OPENAI_BASE_URL": "https://env.example"},
+            "https://sdk.example/v1/decisions",
+            "Bearer env-key",
+        ),
+    ),
+    ids=("openai_key", "api_key_before_env", "openai_api_base_env", "api_base_before_env"),
+)
+def test_openai_decisions_use_the_same_settings_as_other_openai_calls(
+    settings: Mapping[str, str],
+    env: Mapping[str, str],
+    url: str,
+    authorization: str,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    for name in ("api_key", "openai_key", "api_base"):
+        monkeypatch.setattr(litellm, name, settings.get(name))
+    for name in ("OPENAI_API_KEY", "OPENAI_BASE_URL", "OPENAI_API_BASE"):
+        monkeypatch.delenv(name, raising=False)
+    for name, value in env.items():
+        monkeypatch.setenv(name, value)
+    route: Final = respx_mock.post(url).respond(json=_OPENAI_RESPONSE)
+
+    litellm.decisions(
+        model="openai/gpt-6-luna",
+        state="review",
+        questions={"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    )
+
+    assert route.call_count == 1
+    assert route.calls[0].request.headers["authorization"] == authorization
+
+
+@pytest.mark.asyncio
+async def test_openai_format_calls_to_a_systemone_provider_get_openai_format_answers(
+    respx_mock: respx.MockRouter,
+) -> None:
+    route: Final = respx_mock.post("https://api.typesafe.ai/v1/systemone").respond(json=_RESPONSE)
+
+    response: Final = await litellm.adecisions(
+        model="typesafe/jev-1.13",
+        input="review",
+        questions=[
+            {"type": "predicate", "name": "is_defect", "instructions": "Is this a defect?"},
+            {
+                "type": "choice",
+                "name": "sentiment",
+                "instructions": "Tone?",
+                "choices": [{"value": "positive"}, {"value": "negative"}],
+            },
+        ],
+        api_key="caller-key",
+    )
+
+    assert route.called
+    assert tuple(json.loads(respx_mock.calls[0].request.content)["questions"]) == ("is_defect", "sentiment")
+    assert isinstance(response, OpenAIDecisionResponse)
+    assert [answer.model_dump(mode="json") for answer in response.answers] == [
+        {"type": "predicate", "name": "is_defect", "probability": 0.9},
+        {
+            "type": "choice",
+            "name": "sentiment",
+            "choice": "positive",
+            "probabilities": [{"value": "positive", "probability": 0.8}, {"value": "negative", "probability": 0.2}],
+            "confidence": 0.8,
+        },
+    ]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("model", "request_kwargs", "message"),
+    (
+        (
+            "typesafe/jev-1.13",
+            {
+                "input": [
+                    {
+                        "role": "user",
+                        "content": [{"type": "input_image", "image_url": "data:image/png;base64,AA=="}],
+                    }
+                ],
+                "questions": [{"type": "predicate", "instructions": "Is this a defect?"}],
+            },
+            "cannot serve this request",
+        ),
+        (
+            "openai/gpt-6-luna",
+            {
+                "state": "review",
+                "input": "review",
+                "questions": [{"type": "predicate", "instructions": "Is this a defect?"}],
+            },
+            "not both",
+        ),
+    ),
+    ids=("image_to_systemone_provider", "state_and_input"),
+)
+async def test_requests_a_provider_cannot_serve_are_rejected_before_http(
+    respx_mock: respx.MockRouter,
+    model: str,
+    request_kwargs: Mapping[str, object],
+    message: str,
+) -> None:
+    with pytest.raises(litellm.BadRequestError, match=message):
+        await litellm.adecisions(model=model, api_key="caller-key", **request_kwargs)
+
+    assert len(respx_mock.calls) == 0

tests/unit/litellm_core_utils/test_health_check_helpers.py

@@ -1,5 +1,6 @@
 """Test health check helper functions"""
 
+import json
 import socket
 import struct
 import zlib
@@ -8,6 +9,7 @@ from typing import Final
 from unittest.mock import AsyncMock, MagicMock, patch
 
 import pytest
+import respx
 
 import litellm
 from litellm.constants import LITTELM_INTERNAL_HEALTH_SERVICE_ACCOUNT_NAME
@@ -548,3 +550,100 @@ def test_ocr_health_check_document_raises_without_the_extension():
             _ocr_health_check_document(model="mistral/mistral-ocr-latest", custom_llm_provider="mistral")
     finally:
         NATIVE_OCR_HEALTH_CHECK_DOCUMENT.reset()
+
+
+@pytest.mark.parametrize(
+    ("model", "upstream_url"),
+    (
+        ("perplexity/pplx-decider-v1-27b", "https://api.perplexity.ai/v1/decisions"),
+        ("cloudflare/clef", "https://api.cloudflare.com/client/v4/accounts/acct-1/ai/run/@cf/cloudflare/clef"),
+    ),
+)
+async def test_ahealth_check_probes_evaluation_models_through_the_decisions_api(
+    model: str,
+    upstream_url: str,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("CLOUDFLARE_ACCOUNT_ID", "acct-1")
+    monkeypatch.delenv("CLOUDFLARE_API_BASE", raising=False)
+    monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
+    litellm.in_memory_llm_clients_cache.flush_cache()
+    upstream: Final = respx_mock.post(upstream_url).respond(
+        json={
+            "model": model,
+            "answers": {"reachable": {"type": "noul", "noul": 1.0}},
+            "usage": {"input_tokens": 12, "output_tokens": 1},
+        }
+    )
+
+    result: Final = await ahealth_check({"model": model, "api_key": "sk-test"}, mode=None)
+
+    assert "error" not in result, result
+    assert upstream.called
+    sent: Final = json.loads(upstream.calls[0].request.content)
+    assert sent["state"] == "health check"
+    assert sent["questions"]["reachable"]["type"] == "noul"
+
+
+@pytest.mark.asyncio
+async def test_ahealth_check_evaluation_uses_configured_probe_state_and_questions(
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
+    litellm.in_memory_llm_clients_cache.flush_cache()
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(
+        json={
+            "model": "perplexity/pplx-decider-v1-27b",
+            "answers": {"ok": {"type": "noul", "noul": 1.0}},
+            "usage": {"input_tokens": 12, "output_tokens": 1},
+        }
+    )
+
+    result: Final = await ahealth_check(
+        {
+            "model": "perplexity/pplx-decider-v1-27b",
+            "api_key": "sk-test",
+            "state": "custom probe",
+            "questions": {"ok": {"type": "noul", "instructions": "Is it ok?"}},
+        },
+        mode=None,
+    )
+
+    assert "error" not in result, result
+    assert upstream.called
+    sent: Final = json.loads(upstream.calls[0].request.content)
+    assert sent["state"] == "custom probe"
+    assert set(sent["questions"]) == {"ok"}
+
+
+@pytest.mark.asyncio
+async def test_ahealth_check_probes_strands_through_decisions_without_mode(
+    local_model_cost_map: None,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("STRANDS_DECIDER_API_KEY", raising=False)
+    monkeypatch.delenv("STRANDS_DECIDER_API_BASE", raising=False)
+    monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
+    litellm.in_memory_llm_clients_cache.flush_cache()
+    upstream: Final = respx_mock.post("http://strands.local:8080/v1/systemone").respond(
+        json={
+            "model": "strands-decider-2B-hobson-v19",
+            "answers": {"reachable": {"type": "noul", "noul": 1.0}},
+            "usage": {"input_tokens": 12, "output_tokens": 1},
+        }
+    )
+
+    result: Final = await ahealth_check(
+        {
+            "model": "strands_decider/strands-decider-2B-hobson-v19",
+            "api_base": "http://strands.local:8080",
+        },
+        mode=None,
+    )
+
+    assert "error" not in result, result
+    assert upstream.called
+    assert "authorization" not in upstream.calls[0].request.headers

tests/unit/llms/azure/test_azure_common_utils.py

@@ -470,6 +470,7 @@ def test_default_max_retries_env_var_reaches_azure_sdk_client():
             "allm_passthrough_route",
             "llm_passthrough_route",
             "asearch",
+            "adecisions",
             "avector_store_create",
             "avector_store_search",
             "acreate_skill",

tests/unit/llms/base_llm/decisions/test_base_decisions_transformation.py

@@ -0,0 +1,180 @@
+from collections.abc import Mapping
+from typing import Final
+
+import pytest
+from pydantic import TypeAdapter
+
+from litellm.llms.base_llm.decisions.transformation import (
+    ir_to_systemone_request,
+    ir_to_systemone_response,
+    parse_systemone_response,
+    systemone_request_to_ir,
+)
+from litellm.llms.openai.decisions.transformation import openai_request_to_ir
+from litellm.types.decisions import (
+    MAX_DECISION_QUESTIONS,
+    DecisionsIRRefusal,
+    DecisionsIRRequest,
+    DecisionsRequestBody,
+    OpenAIDecisionRequestBody,
+    UnsupportedDecisionsRequest,
+)
+
+_SYSTEMONE_BODY: Final[TypeAdapter[DecisionsRequestBody]] = TypeAdapter(DecisionsRequestBody)
+_OPENAI_BODY: Final[TypeAdapter[OpenAIDecisionRequestBody]] = TypeAdapter(OpenAIDecisionRequestBody)
+
+_SYSTEMONE_REQUEST: Final[Mapping[str, object]] = {
+    "state": {"ticket": 1234, "text": "Screen cracked"},
+    "questions": {
+        "damaged": {
+            "type": "noul",
+            "instructions": "Is the item damaged?",
+            "criteria": {"true": "Visible damage", "false": None},
+            "provider_field": "dropped",
+        },
+        "rubric_only": {"type": "noul", "criteria": {"true": {"signal": "refund"}}},
+        "action": {
+            "type": "choice",
+            "instructions": {"policy": "refund-v2"},
+            "criteria": {"refund": "Within 30 days", "escalate": None},
+        },
+        "severity": {"type": "score", "criteria": ["minor", {"label": "major"}]},
+    },
+}
+
+
+def _openai_ir(raw: Mapping[str, object]) -> DecisionsIRRequest:
+    return openai_request_to_ir(_OPENAI_BODY.validate_python(raw))
+
+
+def test_a_systemone_request_reaches_a_systemone_provider_unchanged() -> None:
+    ir: Final = systemone_request_to_ir(_SYSTEMONE_BODY.validate_python(_SYSTEMONE_REQUEST))
+
+    assert ir_to_systemone_request("jev-latest", ir) == {"model": "jev-latest", **_SYSTEMONE_REQUEST}
+
+
+@pytest.mark.parametrize(
+    ("names", "keys"),
+    (
+        (("damaged", "action"), ("damaged", "action")),
+        (("damaged", None), ("0", "1")),
+        (("damaged", "damaged"), ("0", "1")),
+    ),
+    ids=("all_named", "one_unnamed", "duplicate_names"),
+)
+def test_openai_questions_are_keyed_by_name_only_when_every_name_is_unique(
+    names: tuple[str | None, str | None], keys: tuple[str, str]
+) -> None:
+    questions: Final = [
+        {"type": "predicate", "instructions": f"Question {index}?", **({} if name is None else {"name": name})}
+        for index, name in enumerate(names)
+    ]
+
+    body: Final = ir_to_systemone_request("jev-latest", _openai_ir({"input": "review", "questions": questions}))
+
+    assert tuple(_SYSTEMONE_BODY.validate_python(body).questions) == keys
+
+
+def test_openai_messages_become_systemone_state_text() -> None:
+    ir: Final = _openai_ir(
+        {
+            "input": [
+                {
+                    "role": "user",
+                    "content": [
+                        {"type": "input_text", "text": "The package arrived broken."},
+                        {"type": "input_text", "text": "I want a refund."},
+                    ],
+                },
+                {"role": "user", "content": "Order 1234."},
+            ],
+            "questions": [{"type": "predicate", "instructions": "Is this a defect?"}],
+        }
+    )
+
+    body: Final = ir_to_systemone_request("jev-latest", ir)
+
+    assert isinstance(body, Mapping)
+    assert body["state"] == "The package arrived broken.\n\nI want a refund.\n\nOrder 1234."
+
+
+def test_image_input_is_unsupported_by_systemone_providers() -> None:
+    ir: Final = _openai_ir(
+        {
+            "input": [
+                {
+                    "role": "user",
+                    "content": [
+                        {"type": "input_text", "text": "Is the screen cracked?"},
+                        {"type": "input_image", "image_url": "data:image/png;base64,AA=="},
+                    ],
+                }
+            ],
+            "questions": [{"type": "predicate", "instructions": "Is this a defect?"}],
+        }
+    )
+
+    assert isinstance(ir_to_systemone_request("jev-latest", ir), UnsupportedDecisionsRequest)
+
+
+def test_the_largest_openai_request_accepted_translates_to_a_valid_systemone_request() -> None:
+    ir: Final = _openai_ir(
+        {
+            "input": "review",
+            "questions": [
+                {"type": "predicate", "instructions": f"Question {index}?"} for index in range(MAX_DECISION_QUESTIONS)
+            ],
+        }
+    )
+
+    translated: Final = _SYSTEMONE_BODY.validate_python(ir_to_systemone_request("jev-latest", ir))
+
+    assert len(translated.questions) == MAX_DECISION_QUESTIONS
+
+
+def test_a_systemone_response_reaches_the_caller_unchanged_with_provider_extras() -> None:
+    payload: Final = {
+        "model": "jev-latest",
+        "answers": {
+            "damaged": {"type": "noul", "noul": 0.95, "rationale": "crack visible"},
+            "action": {
+                "type": "choice",
+                "choice": "refund",
+                "confidence": 0.8,
+                "probabilities": {"refund": 0.9, "escalate": 0.1},
+                "calibrated": True,
+            },
+            "severity": {
+                "type": "score",
+                "score": 0.4,
+                "confidence": 0.6,
+                "legend": {"0": "minor", "1": {"label": "major"}},
+                "probabilities": {"0": 0.6, "1": 0.4},
+                "raw_logits": [0.1, 0.2],
+            },
+        },
+        "usage": {"input_tokens": 383, "output_tokens": 2, "cost": 0.25},
+        "latency_ms": 3722.17,
+    }
+    ir: Final = systemone_request_to_ir(_SYSTEMONE_BODY.validate_python(_SYSTEMONE_REQUEST))
+
+    response: Final = ir_to_systemone_response(parse_systemone_response(payload, ir), ir)
+
+    assert response.model_dump(mode="json") == payload
+
+
+def test_missing_or_mismatched_systemone_answers_are_refusals_left_out_of_the_response() -> None:
+    ir: Final = systemone_request_to_ir(_SYSTEMONE_BODY.validate_python(_SYSTEMONE_REQUEST))
+    payload: Final = {
+        "model": "jev-latest",
+        "answers": {
+            "damaged": {"type": "noul", "noul": 0.95},
+            "action": {"type": "noul", "noul": 0.5},
+        },
+        "usage": {"input_tokens": 10, "output_tokens": 1},
+    }
+
+    parsed: Final = parse_systemone_response(payload, ir)
+
+    assert parsed.answers[1:] == (DecisionsIRRefusal(), DecisionsIRRefusal(), DecisionsIRRefusal())
+    assert tuple(ir_to_systemone_response(parsed, ir).answers) == ("damaged",)

tests/unit/llms/openai/decisions/test_openai_decisions_transformation.py

@@ -0,0 +1,256 @@
+from collections.abc import Mapping
+from typing import Final
+
+import pytest
+from pydantic import TypeAdapter
+
+from litellm.llms.base_llm.decisions.transformation import ir_to_systemone_response, systemone_request_to_ir
+from litellm.llms.openai.decisions.transformation import (
+    OPENAI_DECISIONS_ENDPOINT,
+    ir_to_openai_request,
+    ir_to_openai_response,
+    openai_request_to_ir,
+)
+from litellm.types.decisions import (
+    DecisionsIRRequest,
+    DecisionsRequestBody,
+    OpenAIDecisionRequestBody,
+    UnsupportedDecisionsRequest,
+)
+
+_SYSTEMONE_BODY: Final[TypeAdapter[DecisionsRequestBody]] = TypeAdapter(DecisionsRequestBody)
+_OPENAI_BODY: Final[TypeAdapter[OpenAIDecisionRequestBody]] = TypeAdapter(OpenAIDecisionRequestBody)
+
+_OPENAI_REQUEST: Final[Mapping[str, object]] = {
+    "input": [
+        {
+            "type": "message",
+            "role": "user",
+            "content": [
+                {"type": "input_text", "text": "The screen is cracked."},
+                {"type": "input_image", "image_url": "data:image/png;base64,AA==", "detail": "high"},
+            ],
+        },
+        {"type": "message", "role": "user", "content": "Order 1234."},
+    ],
+    "questions": [
+        {"type": "predicate", "name": "damaged", "instructions": "Is the item damaged?"},
+        {
+            "type": "choice",
+            "instructions": "Should we refund?",
+            "choices": [{"value": True, "description": "Refund now"}, {"value": "escalate"}],
+        },
+        {
+            "type": "score",
+            "name": "severity",
+            "instructions": "How severe is it?",
+            "levels": [{"label": "minor"}, {"label": "major", "description": "Product unusable"}],
+        },
+        {"type": "predicate", "name": "fraud", "instructions": "Is this fraud?"},
+    ],
+    "safety_identifier": "end-user-1",
+}
+_PREDICATE_ANSWER: Final[Mapping[str, object]] = {"type": "predicate", "name": "damaged", "probability": 0.95}
+_CHOICE_ANSWER: Final[Mapping[str, object]] = {
+    "type": "choice",
+    "name": None,
+    "choice": True,
+    "probabilities": [{"value": True, "probability": 0.9}, {"value": "escalate", "probability": 0.1}],
+    "confidence": 0.8,
+}
+_REFUSAL_ANSWER: Final[Mapping[str, object]] = {"type": "refusal", "name": "fraud"}
+_USAGE: Final[Mapping[str, object]] = {
+    "input_tokens": 383,
+    "input_tokens_details": {"cached_tokens": 256, "cache_write_tokens": 64},
+    "output_tokens": 2,
+    "output_tokens_details": {"reasoning_tokens": 1},
+    "total_tokens": 385,
+}
+_OPENAI_RESPONSE: Final[Mapping[str, object]] = {
+    "model": "gpt-6-luna",
+    "answers": [
+        _PREDICATE_ANSWER,
+        _CHOICE_ANSWER,
+        {
+            "type": "score",
+            "name": "severity",
+            "score": 0.7,
+            "probabilities": [
+                {"value": 0, "label": "minor", "probability": 0.3},
+                {"value": 1, "label": "major", "probability": 0.7},
+            ],
+            "confidence": 0.6,
+        },
+        _REFUSAL_ANSWER,
+    ],
+    "usage": _USAGE,
+}
+
+
+def _openai_ir(raw: Mapping[str, object]) -> DecisionsIRRequest:
+    return openai_request_to_ir(_OPENAI_BODY.validate_python(raw))
+
+
+def test_an_openai_request_reaches_openai_unchanged() -> None:
+    assert ir_to_openai_request("gpt-6-luna", _openai_ir(_OPENAI_REQUEST)) == {
+        "model": "gpt-6-luna",
+        **_OPENAI_REQUEST,
+    }
+
+
+def test_an_openai_response_reaches_the_caller_unchanged() -> None:
+    ir: Final = _openai_ir(_OPENAI_REQUEST)
+
+    parsed: Final = OPENAI_DECISIONS_ENDPOINT.parse_response(_OPENAI_RESPONSE, ir)
+
+    assert ir_to_openai_response(parsed, ir, "requested").model_dump(mode="json") == _OPENAI_RESPONSE
+
+
+def test_answers_openai_did_not_return_are_refusals() -> None:
+    ir: Final = _openai_ir(_OPENAI_REQUEST)
+    payload: Final = {**_OPENAI_RESPONSE, "answers": [_PREDICATE_ANSWER]}
+
+    response: Final = ir_to_openai_response(OPENAI_DECISIONS_ENDPOINT.parse_response(payload, ir), ir, "requested")
+
+    assert [answer.type for answer in response.answers] == ["predicate", "refusal", "refusal", "refusal"]
+    assert [answer.name for answer in response.answers] == ["damaged", None, "severity", "fraud"]
+
+
+def test_a_systemone_request_becomes_an_openai_request_with_questions_named_by_their_keys() -> None:
+    request: Final = _SYSTEMONE_BODY.validate_python(
+        {
+            "state": {"ticket": 1234, "text": "Screen cracked"},
+            "questions": {
+                "damaged": {
+                    "type": "noul",
+                    "instructions": "Is the item damaged?",
+                    "criteria": {"true": "Visible damage", "false": None},
+                    "provider_field": "dropped",
+                },
+                "rubric_only": {"type": "noul", "criteria": {"true": {"signal": "refund"}}},
+                "action": {
+                    "type": "choice",
+                    "instructions": {"policy": "refund-v2"},
+                    "criteria": {"refund": "Within 30 days", "escalate": None},
+                },
+                "severity": {"type": "score", "criteria": ["minor", {"label": "major"}]},
+            },
+        }
+    )
+
+    assert ir_to_openai_request("gpt-6-luna", systemone_request_to_ir(request)) == {
+        "model": "gpt-6-luna",
+        "input": '{"ticket": 1234, "text": "Screen cracked"}',
+        "questions": [
+            {
+                "type": "predicate",
+                "name": "damaged",
+                "instructions": "Is the item damaged?\n\nAnswer true when: Visible damage",
+            },
+            {"type": "predicate", "name": "rubric_only", "instructions": 'Answer true when: {"signal": "refund"}'},
+            {
+                "type": "choice",
+                "name": "action",
+                "instructions": '{"policy": "refund-v2"}',
+                "choices": [{"value": "refund", "description": "Within 30 days"}, {"value": "escalate"}],
+            },
+            {
+                "type": "score",
+                "name": "severity",
+                "instructions": "Which level best fits the input?",
+                "levels": [{"label": "minor"}, {"label": '{"label": "major"}'}],
+            },
+        ],
+    }
+
+
+def test_systemone_questions_without_instructions_become_valid_openai_questions() -> None:
+    request: Final = _SYSTEMONE_BODY.validate_python(
+        {
+            "state": "Screen cracked",
+            "questions": {
+                "damaged": {"type": "noul", "criteria": {"false": None}},
+                "action": {"type": "choice", "criteria": {"refund": None, "escalate": None}},
+                "severity": {"type": "score", "criteria": ["minor", "major"]},
+            },
+        }
+    )
+
+    body: Final = ir_to_openai_request("gpt-6-luna", systemone_request_to_ir(request))
+
+    assert all(question.instructions for question in _OPENAI_BODY.validate_python(body).questions)
+
+
+@pytest.mark.parametrize(
+    "question",
+    ({"type": "choice", "criteria": {"refund": None}}, {"type": "score", "criteria": ["minor"]}),
+    ids=("choice", "score"),
+)
+def test_a_systemone_question_with_one_option_is_unsupported_by_openai(question: Mapping[str, object]) -> None:
+    request: Final = _SYSTEMONE_BODY.validate_python(
+        {
+            "state": "Screen cracked",
+            "questions": {"damaged": {"type": "noul", "instructions": "Damaged?"}, "q": question},
+        }
+    )
+
+    assert isinstance(ir_to_openai_request("gpt-6-luna", systemone_request_to_ir(request)), UnsupportedDecisionsRequest)
+
+
+def test_an_openai_response_becomes_systemone_answers_with_the_callers_score_labels() -> None:
+    ir: Final = systemone_request_to_ir(
+        _SYSTEMONE_BODY.validate_python(
+            {
+                "state": "Screen cracked",
+                "questions": {
+                    "damaged": {"type": "noul", "instructions": "Damaged?"},
+                    "action": {"type": "choice", "criteria": {"true": None, "escalate": None}},
+                    "severity": {"type": "score", "criteria": ["minor", {"label": "major"}]},
+                    "fraud": {"type": "noul", "instructions": "Fraud?"},
+                },
+            }
+        )
+    )
+    payload: Final = {
+        **_OPENAI_RESPONSE,
+        "answers": [
+            _PREDICATE_ANSWER,
+            _CHOICE_ANSWER,
+            {
+                "type": "score",
+                "name": "severity",
+                "score": 0.7,
+                "probabilities": [
+                    {"value": 0, "label": "minor", "probability": 0.3},
+                    {"value": 1, "label": '{"label": "major"}', "probability": 0.7},
+                ],
+                "confidence": 0.6,
+            },
+            _REFUSAL_ANSWER,
+        ],
+    }
+
+    response: Final = ir_to_systemone_response(OPENAI_DECISIONS_ENDPOINT.parse_response(payload, ir), ir)
+
+    assert response.model_dump(mode="json") == {
+        "model": "gpt-6-luna",
+        "answers": {
+            "damaged": {"type": "noul", "noul": 0.95},
+            "action": {
+                "type": "choice",
+                "choice": "true",
+                "confidence": 0.8,
+                "probabilities": {"true": 0.9, "escalate": 0.1},
+            },
+            "severity": {
+                "type": "score",
+                "score": 0.7,
+                "confidence": 0.6,
+                "legend": {"0": "minor", "1": {"label": "major"}},
+                "probabilities": {"0": 0.3, "1": 0.7},
+            },
+        },
+        "usage": {"input_tokens": 383, "output_tokens": 2},
+    }
+    assert response.usage is not None
+    assert (response.usage.cached_tokens, response.usage.cache_write_tokens) == (256, 64)

tests/unit/proxy/decisions_endpoints/test_endpoints.py

@@ -0,0 +1,583 @@
+from __future__ import annotations
+
+import asyncio
+import json
+from collections.abc import AsyncGenerator, Iterator, Mapping
+from contextlib import asynccontextmanager
+from pathlib import Path
+from typing import Final
+
+import pytest
+import respx
+from fastapi import FastAPI
+from fastapi.routing import APIRoute
+from fastapi.testclient import TestClient
+from starlette.routing import Match
+
+import litellm
+from litellm.proxy._lazy_features import LAZY_FEATURES, LazyFeature, attach_lazy_features
+from litellm.proxy.decisions_endpoints.endpoints import decisions, systemone
+from litellm.proxy.pass_through_endpoints.pass_through_endpoints import SafeRouteAdder
+from litellm.proxy.proxy_server import (
+    app,
+    cleanup_router_config_variables,
+    initialize,
+)
+
+_INPUT_TOKENS: Final[int] = 367
+_OUTPUT_TOKENS: Final[int] = 3
+_RESPONSE: Final[Mapping[str, object]] = {
+    "model": "pplx-decider-v1-27b",
+    "answers": {
+        "is_defect": {"type": "noul", "noul": 0.9},
+    },
+    "usage": {"input_tokens": _INPUT_TOKENS, "output_tokens": _OUTPUT_TOKENS},
+}
+_STRANDS_RESPONSE: Final[Mapping[str, object]] = {
+    "model": "strands-decider-2B-hobson-v19",
+    "answers": {
+        "is_defect": {"type": "noul", "noul": 0.9},
+    },
+    "usage": {"input_tokens": 216, "output_tokens": 3},
+    "latency_ms": 3722.17,
+}
+_REQUEST: Final[Mapping[str, object]] = {
+    "model": "decider",
+    "state": {"source": "proxy-test"},
+    "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+}
+
+
+@pytest.fixture
+def client(monkeypatch: pytest.MonkeyPatch) -> Iterator[TestClient]:
+    monkeypatch.setenv("OPENAI_API_KEY", "fake-openai-key")
+    monkeypatch.setenv("OPENAI_API_BASE", "https://fake-openai.example")
+    monkeypatch.setenv("REDIS_HOST", "localhost")
+    cleanup_router_config_variables()
+    config_path: Final = Path(__file__).parents[1] / "test_configs" / "test_config_no_auth.yaml"
+    asyncio.run(initialize(config=str(config_path), debug=True))
+    monkeypatch.setattr(
+        litellm.proxy.proxy_server,
+        "llm_router",
+        litellm.Router(
+            model_list=[
+                {
+                    "model_name": "decider",
+                    "litellm_params": {
+                        "model": "perplexity/pplx-decider-v1-27b",
+                        "api_key": "test-key",
+                    },
+                }
+            ]
+        ),
+    )
+    monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
+    litellm.in_memory_llm_clients_cache.flush_cache()
+    yield TestClient(app)
+    litellm.in_memory_llm_clients_cache.flush_cache()
+
+
+@pytest.mark.parametrize("endpoint", ("/v1/systemone", "/systemone"))
+def test_proxy_decisions_route_returns_answers_and_cost(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+    endpoint: str,
+) -> None:
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = client.post(endpoint, json=_REQUEST)
+
+    assert response.status_code == 200, response.text
+    assert response.json()["answers"] == _RESPONSE["answers"]
+    assert "_hidden_params" not in response.json()
+    perplexity_cost: Final = litellm.model_cost["perplexity/pplx-decider-v1-27b"]
+    expected_cost: Final = _INPUT_TOKENS * float(perplexity_cost["input_cost_per_token"]) + _OUTPUT_TOKENS * float(
+        perplexity_cost["output_cost_per_token"]
+    )
+
+    assert expected_cost > 0
+    assert float(response.headers["x-litellm-response-cost"]) == pytest.approx(expected_cost)
+    assert upstream.called
+    assert json.loads(upstream.calls[0].request.content) == {
+        "model": "pplx-decider-v1-27b",
+        "state": {"source": "proxy-test"},
+        "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    }
+    assert upstream.calls[0].request.headers["authorization"] == "Bearer test-key"
+
+
+def test_proxy_decisions_dispatches_typesafe_deployment(
+    client: TestClient,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    router: Final = litellm.Router(
+        model_list=[
+            {
+                "model_name": "jev",
+                "litellm_params": {
+                    "model": "typesafe/jev-latest",
+                    "api_key": "k",
+                },
+            }
+        ]
+    )
+    monkeypatch.setattr(litellm.proxy.proxy_server, "llm_router", router)
+    upstream: Final = respx_mock.post("https://api.typesafe.ai/v1/systemone").respond(json=_RESPONSE)
+
+    response: Final = client.post(
+        "/v1/systemone",
+        json={
+            "model": "jev",
+            "state": {"source": "proxy-test"},
+            "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        },
+    )
+
+    assert response.status_code == 200, response.text
+    assert response.json()["answers"] == _RESPONSE["answers"]
+    assert upstream.called
+    assert json.loads(upstream.calls[0].request.content) == {
+        "model": "jev-latest",
+        "state": {"source": "proxy-test"},
+        "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    }
+
+
+def test_proxy_decisions_sends_the_env_key_to_the_deployment_api_base(
+    client: TestClient,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setenv("PERPLEXITYAI_API_KEY", "server-key")
+    monkeypatch.delenv("PERPLEXITY_API_KEY", raising=False)
+    router: Final = litellm.Router(
+        model_list=[
+            {
+                "model_name": "decider",
+                "litellm_params": {
+                    "model": "perplexity/pplx-decider-v1-27b",
+                    "api_base": "https://egress.example/perplexity",
+                },
+            }
+        ]
+    )
+    monkeypatch.setattr(litellm.proxy.proxy_server, "llm_router", router)
+    upstream: Final = respx_mock.post("https://egress.example/perplexity/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = client.post("/v1/systemone", json=_REQUEST)
+
+    assert response.status_code == 200, response.text
+    assert upstream.call_count == 1
+    assert upstream.calls[0].request.headers["authorization"] == "Bearer server-key"
+
+
+def test_proxy_decisions_unknown_model_is_a_client_error(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+) -> None:
+    response: Final = client.post(
+        "/v1/systemone",
+        json={
+            "model": "missing-model",
+            "state": "review",
+            "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        },
+    )
+
+    assert 400 <= response.status_code < 500, response.text
+    assert len(respx_mock.calls) == 0
+
+
+@pytest.mark.parametrize(
+    "request_body",
+    (
+        {
+            "model": "decider",
+            "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        },
+        {
+            "model": "decider",
+            "state": {"source": "proxy-test"},
+        },
+    ),
+    ids=("missing_state", "missing_questions"),
+)
+def test_proxy_decisions_missing_required_field_is_a_client_error(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+    request_body: Mapping[str, object],
+) -> None:
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = client.post("/v1/systemone", json=request_body)
+
+    assert response.status_code == 400, response.text
+    assert not upstream.called
+
+
+def test_proxy_decisions_dispatches_strands_decider(
+    client: TestClient,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.delenv("STRANDS_DECIDER_API_BASE", raising=False)
+    monkeypatch.delenv("STRANDS_DECIDER_API_KEY", raising=False)
+    router: Final = litellm.Router(
+        model_list=[
+            {
+                "model_name": "strands",
+                "litellm_params": {
+                    "model": "strands_decider/strands-decider-2B-hobson-v19",
+                    "api_base": "https://strands.example",
+                },
+            }
+        ]
+    )
+    monkeypatch.setattr(litellm.proxy.proxy_server, "llm_router", router)
+    upstream: Final = respx_mock.post("https://strands.example/v1/systemone").respond(json=_STRANDS_RESPONSE)
+
+    response: Final = client.post(
+        "/v1/systemone",
+        json={
+            "model": "strands",
+            "state": {"source": "proxy-test"},
+            "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+        },
+    )
+
+    assert response.status_code == 200, response.text
+    assert response.json()["answers"] == _STRANDS_RESPONSE["answers"]
+    assert upstream.called
+    assert json.loads(upstream.calls[0].request.content) == {
+        "model": "strands-decider-2B-hobson-v19",
+        "state": {"source": "proxy-test"},
+        "questions": {"is_defect": {"type": "noul", "instructions": "Is this a defect?"}},
+    }
+    assert "authorization" not in upstream.calls[0].request.headers
+
+
+def test_proxy_decisions_without_model_uses_the_proxy_default_model(
+    client: TestClient,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setattr(litellm.proxy.proxy_server, "user_model", "decider")
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = client.post(
+        "/v1/systemone", json={key: value for key, value in _REQUEST.items() if key != "model"}
+    )
+
+    assert response.status_code == 200, response.text
+    assert response.json()["answers"] == _RESPONSE["answers"]
+    assert upstream.called
+    assert json.loads(upstream.calls[0].request.content)["model"] == "pplx-decider-v1-27b"
+
+
+_OPENAI_FORMAT_REQUEST: Final[Mapping[str, object]] = {
+    "model": "decider",
+    "input": [
+        {
+            "role": "user",
+            "content": [
+                {"type": "input_text", "text": "The package arrived with a broken screen."},
+                {"type": "input_text", "text": "I want a refund."},
+            ],
+        },
+        {"role": "user", "content": "Order 1234."},
+    ],
+    "questions": [
+        {"type": "predicate", "name": "damaged", "instructions": "Does the customer report a damaged item?"},
+        {
+            "type": "choice",
+            "instructions": "Should we refund?",
+            "choices": [{"value": True, "description": "Refund now"}, {"value": "escalate"}],
+        },
+        {
+            "type": "score",
+            "name": "severity",
+            "instructions": "How severe is the issue?",
+            "levels": [{"label": "minor"}, {"label": "major", "description": "Product unusable"}],
+        },
+        {"type": "predicate", "name": "fraud", "instructions": "Is this fraud?"},
+    ],
+    "safety_identifier": "end-user-1",
+}
+_SYSTEMONE_ANSWERS_FOR_OPENAI_REQUEST: Final[Mapping[str, object]] = {
+    "model": "pplx-decider-v1-27b",
+    "answers": {
+        "0": {"type": "noul", "noul": 0.95},
+        "1": {"type": "choice", "choice": "true", "confidence": 0.8, "probabilities": {"true": 0.9, "escalate": 0.1}},
+        "2": {
+            "type": "score",
+            "score": 0.7,
+            "confidence": 0.6,
+            "legend": {"0": "minor", "1": "major: Product unusable"},
+            "probabilities": {"0": 0.3, "1": 0.7},
+        },
+    },
+    "usage": {"input_tokens": _INPUT_TOKENS, "output_tokens": _OUTPUT_TOKENS},
+}
+
+_OPENAI_FORMAT_ANSWERS: Final = [
+    {"type": "predicate", "name": "damaged", "probability": 0.95},
+    {
+        "type": "choice",
+        "name": None,
+        "choice": True,
+        "probabilities": [{"value": True, "probability": 0.9}, {"value": "escalate", "probability": 0.1}],
+        "confidence": 0.8,
+    },
+    {
+        "type": "score",
+        "name": "severity",
+        "score": 0.7,
+        "probabilities": [
+            {"value": 0, "label": "minor", "probability": 0.3},
+            {"value": 1, "label": "major", "probability": 0.7},
+        ],
+        "confidence": 0.6,
+    },
+    {"type": "refusal", "name": "fraud"},
+]
+
+
+@pytest.mark.parametrize("endpoint", ("/v1/decisions", "/decisions"))
+def test_openai_format_decisions_translate_through_systemone(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+    endpoint: str,
+) -> None:
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(
+        json=_SYSTEMONE_ANSWERS_FOR_OPENAI_REQUEST
+    )
+
+    response: Final = client.post(endpoint, json=_OPENAI_FORMAT_REQUEST)
+
+    assert response.status_code == 200, response.text
+    assert json.loads(upstream.calls[0].request.content) == {
+        "model": "pplx-decider-v1-27b",
+        "state": "The package arrived with a broken screen.\n\nI want a refund.\n\nOrder 1234.",
+        "questions": {
+            "0": {"type": "noul", "instructions": "Does the customer report a damaged item?"},
+            "1": {
+                "type": "choice",
+                "instructions": "Should we refund?",
+                "criteria": {"true": "Refund now", "escalate": None},
+            },
+            "2": {
+                "type": "score",
+                "instructions": "How severe is the issue?",
+                "criteria": ["minor", "major: Product unusable"],
+            },
+            "3": {"type": "noul", "instructions": "Is this fraud?"},
+        },
+    }
+    body: Final = response.json()
+    assert body["model"] == _SYSTEMONE_ANSWERS_FOR_OPENAI_REQUEST["model"]
+    assert body["answers"] == _OPENAI_FORMAT_ANSWERS
+    assert body["usage"] == {
+        "input_tokens": _INPUT_TOKENS,
+        "input_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 0},
+        "output_tokens": _OUTPUT_TOKENS,
+        "output_tokens_details": {"reasoning_tokens": 0},
+        "total_tokens": _INPUT_TOKENS + _OUTPUT_TOKENS,
+    }
+    assert float(response.headers["x-litellm-response-cost"]) > 0
+
+
+@pytest.mark.parametrize(
+    ("endpoint", "request_body", "upstream_response"),
+    (
+        ("/v1/systemone", _REQUEST, _RESPONSE),
+        ("/v1/decisions", _OPENAI_FORMAT_REQUEST, _SYSTEMONE_ANSWERS_FOR_OPENAI_REQUEST),
+    ),
+    ids=("systemone", "openai_format"),
+)
+def test_decisions_return_guardrail_information_when_requested(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+    endpoint: str,
+    request_body: Mapping[str, object],
+    upstream_response: Mapping[str, object],
+) -> None:
+    respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=upstream_response)
+
+    response: Final = client.post(endpoint, json={**request_body, "include_guardrail_response": True})
+
+    assert response.status_code == 200, response.text
+    assert response.json()["guardrail_information"] == []
+
+
+@pytest.mark.parametrize(
+    "request_body",
+    (
+        _REQUEST,
+        {
+            "model": "decider",
+            "input": "review",
+            "questions": [
+                {
+                    "type": "choice",
+                    "instructions": "Pick one",
+                    "choices": [{"value": True}, {"value": "true"}],
+                }
+            ],
+        },
+        {
+            "model": "decider",
+            "input": [
+                {"role": "user", "content": [{"type": "input_image", "image_url": "data:image/png;base64,AA=="}]}
+            ],
+            "questions": [{"type": "predicate", "instructions": "Is this a defect?"}],
+        },
+    ),
+    ids=("systemone_body", "colliding_choice_values", "image_input"),
+)
+def test_openai_format_decisions_rejects_bodies_it_cannot_translate(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+    request_body: Mapping[str, object],
+) -> None:
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = client.post("/v1/decisions", json=request_body)
+
+    assert response.status_code == 400, response.text
+    assert not upstream.called
+
+
+@pytest.mark.parametrize("endpoint", ("/v1/systemone", "/v1/decisions"))
+@pytest.mark.parametrize("raw_body", (b"", b"{not json"), ids=("empty", "malformed"))
+def test_a_body_that_is_not_json_is_a_client_error(
+    client: TestClient,
+    respx_mock: respx.MockRouter,
+    endpoint: str,
+    raw_body: bytes,
+) -> None:
+    upstream: Final = respx_mock.post("https://api.perplexity.ai/v1/decisions").respond(json=_RESPONSE)
+
+    response: Final = client.post(endpoint, content=raw_body, headers={"Content-Type": "application/json"})
+
+    assert response.status_code == 400, response.text
+    assert not upstream.called
+
+
+def test_openai_format_decisions_reach_an_openai_deployment_unchanged_including_images(
+    client: TestClient,
+    monkeypatch: pytest.MonkeyPatch,
+    respx_mock: respx.MockRouter,
+) -> None:
+    monkeypatch.setattr(litellm, "api_base", None)
+    monkeypatch.delenv("OPENAI_BASE_URL", raising=False)
+    monkeypatch.delenv("OPENAI_API_BASE", raising=False)
+    monkeypatch.setattr(
+        litellm.proxy.proxy_server,
+        "llm_router",
+        litellm.Router(
+            model_list=[{"model_name": "decider", "litellm_params": {"model": "openai/gpt-6-luna", "api_key": "k"}}]
+        ),
+    )
+    image_message: Final = {
+        "type": "message",
+        "role": "user",
+        "content": [{"type": "input_image", "image_url": "data:image/png;base64,AA==", "detail": "low"}],
+    }
+    request_body: Final = {**_OPENAI_FORMAT_REQUEST, "input": [*_OPENAI_FORMAT_REQUEST["input"], image_message]}
+    cached_tokens: Final = 128
+    upstream_usage: Final = {
+        "input_tokens": _INPUT_TOKENS,
+        "input_tokens_details": {"cached_tokens": cached_tokens, "cache_write_tokens": 0},
+        "output_tokens": _OUTPUT_TOKENS,
+        "output_tokens_details": {"reasoning_tokens": 0},
+        "total_tokens": _INPUT_TOKENS + _OUTPUT_TOKENS,
+    }
+    upstream: Final = respx_mock.post("https://api.openai.com/v1/decisions").respond(
+        json={"model": "gpt-6-luna", "answers": _OPENAI_FORMAT_ANSWERS, "usage": upstream_usage}
+    )
+
+    response: Final = client.post("/v1/decisions", json=request_body)
+
+    assert response.status_code == 200, response.text
+    assert json.loads(upstream.calls[0].request.content) == {
+        "model": "gpt-6-luna",
+        "input": [
+            {
+                "type": "message",
+                "role": "user",
+                "content": [
+                    {"type": "input_text", "text": "The package arrived with a broken screen."},
+                    {"type": "input_text", "text": "I want a refund."},
+                ],
+            },
+            {"type": "message", "role": "user", "content": "Order 1234."},
+            image_message,
+        ],
+        "questions": _OPENAI_FORMAT_REQUEST["questions"],
+        "safety_identifier": "end-user-1",
+    }
+    body: Final = response.json()
+    assert body["answers"] == _OPENAI_FORMAT_ANSWERS
+    assert body["usage"] == upstream_usage
+    luna_cost: Final = litellm.model_cost["gpt-6-luna"]
+    expected_cost: Final = (
+        (_INPUT_TOKENS - cached_tokens) * float(luna_cost["input_cost_per_token"])
+        + cached_tokens * float(luna_cost["cache_read_input_token_cost"])
+        + _OUTPUT_TOKENS * float(luna_cost["output_cost_per_token"])
+    )
+    assert expected_cost > 0
+    assert float(response.headers["x-litellm-response-cost"]) == pytest.approx(expected_cost)
+
+
+def _decisions_feature() -> LazyFeature:
+    return next(feature for feature in LAZY_FEATURES if feature.name == "decisions")
+
+
+def _serving_endpoint(bare: FastAPI, path: str) -> object:
+    scope: Final = {"type": "http", "method": "POST", "path": path, "root_path": "", "query_string": b"", "headers": ()}
+    return next(
+        route.endpoint for route in bare.routes if isinstance(route, APIRoute) and route.matches(scope)[0] is Match.FULL
+    )
+
+
+def test_a_config_pass_through_at_v1_decisions_keeps_its_route_and_the_native_api_serves_decisions(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    monkeypatch.delenv("LITELLM_DISABLE_LAZY_ROUTES", raising=False)
+
+    async def pass_through() -> dict[str, str]:
+        return {"served_by": "pass-through"}
+
+    bare: Final = FastAPI()
+    attach_lazy_features(bare, (_decisions_feature(),))
+    SafeRouteAdder.add_api_route_if_not_exists(bare, "/v1/decisions", pass_through, ["POST"])
+    with TestClient(bare) as client:
+        assert client.post("/v1/decisions", json={"model": "gpt-6-luna"}).json() == {"served_by": "pass-through"}
+    assert _serving_endpoint(bare, "/v1/decisions") is pass_through
+    assert _serving_endpoint(bare, "/decisions") is decisions
+    assert _serving_endpoint(bare, "/v1/systemone") is systemone
+    assert _serving_endpoint(bare, "/systemone") is systemone
+
+
+def test_with_lazy_routes_disabled_a_config_pass_through_at_v1_decisions_still_wins(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    monkeypatch.setenv("LITELLM_DISABLE_LAZY_ROUTES", "true")
+
+    async def pass_through() -> dict[str, str]:
+        return {"served_by": "pass-through"}
+
+    @asynccontextmanager
+    async def loads_the_config(app_: FastAPI) -> AsyncGenerator[None]:
+        assert SafeRouteAdder.add_api_route_if_not_exists(app_, "/v1/decisions", pass_through, ["POST"]), (
+            "the native route registered at startup must not block the config pass-through"
+        )
+        yield
+
+    bare: Final = FastAPI(lifespan=loads_the_config)
+    attach_lazy_features(bare, (_decisions_feature(),))
+    with TestClient(bare) as client:
+        assert client.post("/v1/decisions", json={"model": "gpt-6-luna"}).json() == {"served_by": "pass-through"}
+    assert _serving_endpoint(bare, "/v1/decisions") is pass_through
+    assert _serving_endpoint(bare, "/decisions") is decisions

tests/unit/proxy/pass_through_endpoints/test_pass_through_endpoints.py

@@ -9,15 +9,16 @@ from collections.abc import Callable, Mapping
 from contextlib import ExitStack, contextmanager
 from dataclasses import dataclass
 from io import BytesIO
-from types import MappingProxyType, SimpleNamespace
+from types import MappingProxyType, ModuleType, SimpleNamespace
 from typing import Final
 from unittest.mock import AsyncMock, MagicMock, patch
 
 import httpx
 import pytest
 import respx
-from fastapi import HTTPException, Request, Response, UploadFile
+from fastapi import APIRouter, FastAPI, HTTPException, Request, Response, UploadFile
 from fastapi.responses import StreamingResponse
+from fastapi.testclient import TestClient
 from pydantic import TypeAdapter, ValidationError
 from starlette.datastructures import FormData, Headers, QueryParams
 from starlette.datastructures import UploadFile as StarletteUploadFile
@@ -27,12 +28,14 @@ from litellm._logging import verbose_proxy_logger
 from litellm.constants import DEFAULT_REQUEST_TIMEOUT_SECONDS
 from litellm.integrations.custom_logger import CustomLogger
 from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
+from litellm.proxy._lazy_features import LazyFeature, attach_lazy_features
 from litellm.proxy._types import ProxyException, UserAPIKeyAuth
 from litellm.proxy.pass_through_endpoints.pass_through_endpoints import (
     DEFAULT_PASS_THROUGH_REQUEST_TIMEOUT_SECONDS,
     LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY,
     HttpPassThroughEndpointHelpers,
     InitPassThroughEndpointHelpers,
+    SafeRouteAdder,
     _registered_pass_through_routes,
     _truncate_upstream_error_body,
     _with_trace_context,
@@ -8218,3 +8221,31 @@ async def test_pass_throughs_stay_open_while_a_db_sync_reads_the_database(tmp_pa
     await sync
 
     assert (config_during_sync.status_code, db_during_sync.status_code) == (200, 200)
+
+
+def _lazy_feature(monkeypatch: pytest.MonkeyPatch, name: str, path: str) -> LazyFeature:
+    async def served() -> dict[str, str]:
+        return {"served_by": name}
+
+    router: Final = APIRouter()
+    router.add_api_route(path, served, methods=["POST"])
+    module: Final = ModuleType(f"tests.unit.proxy.pass_through_endpoints.lazy_fixture_{name}")
+    module.router = router  # pyright: ignore[reportAttributeAccessIssue]  # fixture module built at test time
+    monkeypatch.setitem(sys.modules, module.__name__, module)
+    return LazyFeature(name=name, module_path=module.__name__, path_prefixes=(path,))
+
+
+def test_a_pass_through_added_after_a_lazy_feature_loaded_takes_over_its_path(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.delenv("LITELLM_DISABLE_LAZY_ROUTES", raising=False)
+
+    async def pass_through() -> dict[str, str]:
+        return {"served_by": "pass-through"}
+
+    app: Final = FastAPI()
+    attach_lazy_features(app, (_lazy_feature(monkeypatch, "decider", "/v1/decider"),))
+    with TestClient(app) as client:
+        assert client.post("/v1/decider").json() == {"served_by": "decider"}
+        assert SafeRouteAdder.add_api_route_if_not_exists(app, "/v1/decider", pass_through, ["POST"])
+        assert client.post("/v1/decider").json() == {"served_by": "pass-through"}
+        assert not SafeRouteAdder.add_api_route_if_not_exists(app, "/v1/decider", pass_through, ["POST"])
+        assert client.post("/v1/decider").json() == {"served_by": "pass-through"}

tests/unit/proxy/public_endpoints/test_public_endpoints.py

@@ -411,11 +411,13 @@ ADD_MODEL_UNLISTED_PROVIDERS: Final = frozenset(
         "sagemaker_nova",
         "scaleway",
         "stability",
+        "strands_decider",
         "synthetic",
         "tencent",
         "tensormesh",
         "text-completion-inception",
         "transcribe",
+        "typesafe",
         "valkey",
         "xiaomi_mimo",
         "zai",

tests/unit/test_utils.py

@@ -1022,6 +1022,8 @@ def test_aaamodel_prices_and_context_window_json_is_valid():
                             "/v1/videos",
                             "/vertex_ai/live",
                             "/v1/listen",
+                            "/v1/systemone",
+                            "/v1/decisions",
                             "/v1beta/interactions",
                         ],
                     },

ui/litellm-dashboard/src/lib/http/schema.d.ts

@@ -4568,6 +4568,23 @@ export interface paths {
         patch?: never;
         trace?: never;
     };
+    "/decisions": {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        get?: never;
+        put?: never;
+        /** Decisions */
+        post: operations["decisions_decisions_post"];
+        delete?: never;
+        options?: never;
+        head?: never;
+        patch?: never;
+        trace?: never;
+    };
     "/delete/allowed_ip": {
         parameters: {
             query?: never;
@@ -15989,6 +16006,23 @@ export interface paths {
         patch?: never;
         trace?: never;
     };
+    "/systemone": {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        get?: never;
+        put?: never;
+        /** Systemone */
+        post: operations["systemone_systemone_post"];
+        delete?: never;
+        options?: never;
+        head?: never;
+        patch?: never;
+        trace?: never;
+    };
     "/tag/daily/activity": {
         parameters: {
             query?: never;
@@ -19755,6 +19789,23 @@ export interface paths {
         patch?: never;
         trace?: never;
     };
+    "/v1/decisions": {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        get?: never;
+        put?: never;
+        /** Decisions */
+        post: operations["decisions_v1_decisions_post"];
+        delete?: never;
+        options?: never;
+        head?: never;
+        patch?: never;
+        trace?: never;
+    };
     "/v1/embeddings": {
         parameters: {
             query?: never;
@@ -22061,6 +22112,23 @@ export interface paths {
         patch?: never;
         trace?: never;
     };
+    "/v1/systemone": {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        get?: never;
+        put?: never;
+        /** Systemone */
+        post: operations["systemone_v1_systemone_post"];
+        delete?: never;
+        options?: never;
+        head?: never;
+        patch?: never;
+        trace?: never;
+    };
     "/v1/threads": {
         parameters: {
             query?: never;
@@ -28022,7 +28090,7 @@ export interface components {
          * CallTypes
          * @enum {string}
          */
-        CallTypes: "embedding" | "aembedding" | "completion" | "acompletion" | "atext_completion" | "text_completion" | "image_generation" | "aimage_generation" | "image_edit" | "aimage_edit" | "moderation" | "amoderation" | "atranscription" | "transcription" | "aspeech" | "speech" | "rerank" | "arerank" | "search" | "asearch" | "_arealtime" | "_aresponses_websocket" | "create_batch" | "acreate_batch" | "aretrieve_batch" | "retrieve_batch" | "acancel_batch" | "cancel_batch" | "pass_through_endpoint" | "anthropic_messages" | "aanthropic_messages" | "get_assistants" | "aget_assistants" | "create_assistants" | "acreate_assistants" | "delete_assistant" | "adelete_assistant" | "acreate_thread" | "create_thread" | "aget_thread" | "get_thread" | "a_add_message" | "add_message" | "aget_messages" | "get_messages" | "arun_thread" | "run_thread" | "arun_thread_stream" | "run_thread_stream" | "afile_retrieve" | "file_retrieve" | "afile_delete" | "file_delete" | "afile_list" | "file_list" | "acreate_file" | "create_file" | "afile_content" | "file_content" | "create_fine_tuning_job" | "acreate_fine_tuning_job" | "create_video" | "acreate_video" | "video_generation" | "avideo_generation" | "avideo_retrieve" | "video_retrieve" | "avideo_content" | "video_content" | "video_remix" | "avideo_remix" | "video_list" | "avideo_list" | "video_retrieve_job" | "avideo_retrieve_job" | "video_delete" | "avideo_delete" | "video_create_character" | "avideo_create_character" | "video_get_character" | "avideo_get_character" | "video_edit" | "avideo_edit" | "video_extension" | "avideo_extension" | "vector_store_file_create" | "avector_store_file_create" | "vector_store_file_list" | "avector_store_file_list" | "vector_store_file_retrieve" | "avector_store_file_retrieve" | "vector_store_file_content" | "avector_store_file_content" | "vector_store_file_update" | "avector_store_file_update" | "vector_store_file_delete" | "avector_store_file_delete" | "vector_store_create" | "avector_store_create" | "vector_store_search" | "avector_store_search" | "ingest" | "aingest" | "query" | "aquery" | "create_interaction" | "acreate_interaction" | "create_container" | "acreate_container" | "list_containers" | "alist_containers" | "retrieve_container" | "aretrieve_container" | "delete_container" | "adelete_container" | "list_container_files" | "alist_container_files" | "upload_container_file" | "aupload_container_file" | "create_sandbox" | "acreate_sandbox" | "delete_sandbox" | "adelete_sandbox" | "run_code" | "arun_code" | "code_interpreter_tool" | "acode_interpreter_tool" | "acancel_fine_tuning_job" | "cancel_fine_tuning_job" | "alist_fine_tuning_jobs" | "list_fine_tuning_jobs" | "aretrieve_fine_tuning_job" | "retrieve_fine_tuning_job" | "responses" | "aresponses" | "alist_input_items" | "llm_passthrough_route" | "allm_passthrough_route" | "generate_content" | "agenerate_content" | "generate_content_stream" | "agenerate_content_stream" | "ocr" | "aocr" | "call_mcp_tool" | "list_mcp_tools" | "asend_message" | "send_message" | "acreate_skill";
+        CallTypes: "embedding" | "aembedding" | "completion" | "acompletion" | "atext_completion" | "text_completion" | "image_generation" | "aimage_generation" | "image_edit" | "aimage_edit" | "moderation" | "amoderation" | "atranscription" | "transcription" | "aspeech" | "speech" | "rerank" | "arerank" | "search" | "asearch" | "decisions" | "adecisions" | "_arealtime" | "_aresponses_websocket" | "create_batch" | "acreate_batch" | "aretrieve_batch" | "retrieve_batch" | "acancel_batch" | "cancel_batch" | "pass_through_endpoint" | "anthropic_messages" | "aanthropic_messages" | "get_assistants" | "aget_assistants" | "create_assistants" | "acreate_assistants" | "delete_assistant" | "adelete_assistant" | "acreate_thread" | "create_thread" | "aget_thread" | "get_thread" | "a_add_message" | "add_message" | "aget_messages" | "get_messages" | "arun_thread" | "run_thread" | "arun_thread_stream" | "run_thread_stream" | "afile_retrieve" | "file_retrieve" | "afile_delete" | "file_delete" | "afile_list" | "file_list" | "acreate_file" | "create_file" | "afile_content" | "file_content" | "create_fine_tuning_job" | "acreate_fine_tuning_job" | "create_video" | "acreate_video" | "video_generation" | "avideo_generation" | "avideo_retrieve" | "video_retrieve" | "avideo_content" | "video_content" | "video_remix" | "avideo_remix" | "video_list" | "avideo_list" | "video_retrieve_job" | "avideo_retrieve_job" | "video_delete" | "avideo_delete" | "video_create_character" | "avideo_create_character" | "video_get_character" | "avideo_get_character" | "video_edit" | "avideo_edit" | "video_extension" | "avideo_extension" | "vector_store_file_create" | "avector_store_file_create" | "vector_store_file_list" | "avector_store_file_list" | "vector_store_file_retrieve" | "avector_store_file_retrieve" | "vector_store_file_content" | "avector_store_file_content" | "vector_store_file_update" | "avector_store_file_update" | "vector_store_file_delete" | "avector_store_file_delete" | "vector_store_create" | "avector_store_create" | "vector_store_search" | "avector_store_search" | "ingest" | "aingest" | "query" | "aquery" | "create_interaction" | "acreate_interaction" | "create_container" | "acreate_container" | "list_containers" | "alist_containers" | "retrieve_container" | "aretrieve_container" | "delete_container" | "adelete_container" | "list_container_files" | "alist_container_files" | "upload_container_file" | "aupload_container_file" | "create_sandbox" | "acreate_sandbox" | "delete_sandbox" | "adelete_sandbox" | "run_code" | "arun_code" | "code_interpreter_tool" | "acode_interpreter_tool" | "acancel_fine_tuning_job" | "cancel_fine_tuning_job" | "alist_fine_tuning_jobs" | "list_fine_tuning_jobs" | "aretrieve_fine_tuning_job" | "retrieve_fine_tuning_job" | "responses" | "aresponses" | "alist_input_items" | "llm_passthrough_route" | "allm_passthrough_route" | "generate_content" | "agenerate_content" | "generate_content_stream" | "agenerate_content_stream" | "ocr" | "aocr" | "call_mcp_tool" | "list_mcp_tools" | "asend_message" | "send_message" | "acreate_skill";
         /** CallbackDelete */
         CallbackDelete: {
             /** Callback Name */
@@ -56443,6 +56511,26 @@ export interface operations {
             };
         };
     };
+    decisions_decisions_post: {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        requestBody?: never;
+        responses: {
+            /** @description Successful Response */
+            200: {
+                headers: {
+                    [name: string]: unknown;
+                };
+                content: {
+                    "application/json": unknown;
+                };
+            };
+        };
+    };
     delete_allowed_ip_delete_allowed_ip_post: {
         parameters: {
             query?: never;
@@ -70997,6 +71085,26 @@ export interface operations {
             };
         };
     };
+    systemone_systemone_post: {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        requestBody?: never;
+        responses: {
+            /** @description Successful Response */
+            200: {
+                headers: {
+                    [name: string]: unknown;
+                };
+                content: {
+                    "application/json": unknown;
+                };
+            };
+        };
+    };
     get_tag_daily_activity_tag_daily_activity_get: {
         parameters: {
             query?: {
@@ -76098,6 +76206,26 @@ export interface operations {
             };
         };
     };
+    decisions_v1_decisions_post: {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        requestBody?: never;
+        responses: {
+            /** @description Successful Response */
+            200: {
+                headers: {
+                    [name: string]: unknown;
+                };
+                content: {
+                    "application/json": unknown;
+                };
+            };
+        };
+    };
     embeddings_v1_embeddings_post: {
         parameters: {
             query?: never;
@@ -79230,6 +79358,26 @@ export interface operations {
             };
         };
     };
+    systemone_v1_systemone_post: {
+        parameters: {
+            query?: never;
+            header?: never;
+            path?: never;
+            cookie?: never;
+        };
+        requestBody?: never;
+        responses: {
+            /** @description Successful Response */
+            200: {
+                headers: {
+                    [name: string]: unknown;
+                };
+                content: {
+                    "application/json": unknown;
+                };
+            };
+        };
+    };
     create_threads_v1_threads_post: {
         parameters: {
             query?: never;