Defer the breaking error policy from #46121. Adapt the #46777 count settings flow to attempt a labelled local estimate for provider failures across providers and configured count models.
Keep provider errors when local counting is disabled or cannot estimate the request, and test the shared behavior across HTTP error statuses.
Keep #43900, #45301, #46121, #46784, #46759, #46767, #46772, #46815, #47057 and #46777. Port only the Gemini prerequisites from #46430.
Remove the deferred Bedrock Mantle behavior and unrelated picks from the final tree. Restore the PDF test provider fixture and deployment, and retain required test imports.
Hand-ported to stable/1.104.x from main (bedrock_runtime_peer, provider, _cache_control_marks_support verbatim, graceful_stop_seconds at the line's 30s stop wait).
Backport of #47064 to stable/1.104.x.
Cherry-picked from ae6f7f6613 (main).
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #46430 to stable/1.104.x.
Cherry-picked from 08a24d391a (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #46121 to stable/1.104.x.
Cherry-picked from 719c44db3f (main).
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Backport of #45463 to stable/1.104.x.
Cherry-picked from d34ad35281 (main).
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Backport of #45301 to stable/1.104.x.
Cherry-picked from 50ab9a3211 (main).
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Backport of #43900 to stable/1.104.x.
Cherry-picked from 6d73fa6b49 (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Backport of #43417 to stable/1.104.x.
Cherry-picked from 8e6d99d74a (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The tests picked from #47067 annotate module constants with Final, which main's copy imports and this line's did not, so the module failed to import on Python 3.12.
Backport of #47067 to stable/1.104.x.
Cherry-picked from e979bd0ecf (main).
Conflict: main renamed the model access helper to check_model_access_helper, while stable/1.104.x still names it _check_model_access_helper, so the new owning-team check calls it by the line's name.
Line adaptation: the line's test_auth_utils.py had no module-level Router import, which main's copy has and the picked helper uses, so it is added.
The decision model guardrail backport (#45900) added supported_decisions_providers to
litellm.decisions.main and imported it from there in get_guardrail_ui_settings. The
decisions backport (#45906) then picked #45130, which rewrites litellm.decisions.main
and moves the provider list into litellm.decisions.call as _supported_providers, so the
import broke and GET /guardrails/ui/add_guardrail_settings raises ImportError. The Admin
UI's Add Provider Guardrail wizard gets no providers as a result
Match main by naming it supported_decisions_providers in litellm.decisions.call and
pointing the endpoint and its test there, the same fix rc/1.105.0 got in fcbb02231b
Ports the pytest_runtest_call hook from tests/unit/conftest.py on main (#44146). Without it the asystemone call type test picks up adecisions callbacks queued by earlier tests in the same worker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
schema.d.ts is regenerated with npm run gen:api instead of hand-merged, the Strands logo the Add Model list imports is copied from main, and the picked tests get the import order and Final import this line's ruff asks for
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: run tests/unit/systemone in the core-utils unit shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(sdk): keep litellm.decisions(state=...) working with a DeprecationWarning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: make tests/unit/systemone a package
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(sdk): surface the litellm.decisions(state=...) deprecation as a UserWarning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 25d9b05386)
PR #45667 removed openrouter/typesafe/jev-1.13 from the cost map after #45640 moved OpenRouter decisions billing to usage.cost. The OpenAI-format wire test still read the removed row and raised KeyError on CircleCI integration-providers
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 5894968c5d)
* docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI
Add summaries, descriptions, inlined request body schemas with examples and typed 200 responses to the
decision routes and their aliases, so the OpenAPI document and the lazy snapshot show both request formats
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap _inlined schema walk depth for the recursion check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit bc2cc42340)
* feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN
A strands_decider deployment whose api_base is a bedrock-agentcore runtime ARN now
calls InvokeAgentRuntime on that runtime, signed with SigV4 from the deployment's aws_*
params or the default chain, reuses one session per runtime, and maps the runtime's
200 error envelopes to 400, 503, or 500
* ci: run the strands_decider unit tests in the llms job
* fix(strands_decider): take the AgentCore session id from the deployment, never the caller
A caller could pick the runtime session through extra_headers, and every new id starts another microVM. The session id now comes from the deployment's agentcore_runtime_session_id (banned in request bodies) or the ARN default, and caller-supplied host, x-amz-*, and x-amzn-bedrock-agentcore-runtime-* headers are not forwarded
* fix(strands_decider): map runtime error replies only on AgentCore ARN calls
* fix(strands_decider): sign plain decisions calls inline, off the AWS signing executor
* test(strands_decider): cover AgentCore runtime ARN calls in integration wire tests
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit ed9c31cfa8)
* feat(decisions): add the OpenAI Decisions spec types and the System One translation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): share one DecisionsModel config and require model and usage on responses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): rename the shared pydantic parent to DecisionsObjectBase
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): map OpenRouter connection failures to APIConnectionError and drop explanatory docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): switch /v1/decisions to the OpenAI Decisions schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): build the usage the OpenAI response schema now requires
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lens): ask signal questions through the OpenAI Decisions schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): refuse safety_identifier for System One providers unless drop_params
* feat(decisions): refuse safety_identifier on providers that cannot take it unless drop_params drops it
* test(decisions): cover hosted_vllm in the safety_identifier and per-provider wire tests
* fix(decisions): check a System One request's safety_identifier against the provider too
* fix(decisions): drop a non-string safety_identifier under drop_params
A malformed safety_identifier now follows the drop_params convention in both body shapes: it answers 400 without drop_params and is dropped before the provider call with it. A string identifier on OpenAI stays on the wire either way.
* fix(decisions): let /v1/decisions drop a non-string safety_identifier under drop_params
The route checked the whole body before routing, so a non-string
safety_identifier answered 400 even when the deployment or the body set
drop_params. The route now leaves that field to the Decisions call, which
knows every drop_params source
* refactor(decisions): return the unsupported safety_identifier as a value and raise it in _prepare_call
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 546402c98c)
* feat(azure_ai): support Microsoft-Decision-1 on the decisions API
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(azure_ai): strip project path and API suffix together in decisions base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 2fcc780498)
* feat(decisions): add hosted_vllm provider
Adds HostedVLLMDecisionsConfig so hosted_vllm/<served model> works on
POST /v1/systemone and POST /v1/decisions against a vLLM server that
serves the System One body (vllm-project/vllm#59299, after v0.31.0).
vLLM only supports choice questions and rejects others, so the base
decisions config gains a per-provider health_check_questions attribute
and the evaluation-mode health probe sends a choice question for
hosted_vllm while every other provider keeps the noul probe.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): move the hosted_vllm placeholder key to constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 61e5f2dd3c)
#45214 routed both decision formats through the shared decisions IR in
litellm/llms/base_llm/decisions/transformation.py and litellm/types/decisions.py.
That left litellm/llms/base_llm/decisions/systemone.py (to_system_one_request,
question_keys, to_decisions_response, the SystemOne* models and their adapter)
and litellm/types/openai_decisions.py with no importer outside their own unit
tests, so both modules and both test files go.
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 0e48048bd5)
* feat(ui): add TypeSafe and Strands Decider to the Add Model provider list
* test(public_endpoints): type the provider fields helper against the route's model
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 3a3b8cad80)
