scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-16 04:29:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 139

fix(oci): drop non-function tools under drop_params and gate reasoning_effort on declared levels (#40732)

Browse source 
* fix(oci): skip non-function tools and gate reasoning_effort on models that reject it

Fixes #31449.

Two OCI GENERIC request-building defects that made Grok-on-OCI unusable
for tool-using coding-agent clients (Codex over the Responses API, Claude
Code over the Anthropic Messages API):

1. `adapt_tool_definition_to_oci_standard` raised `OCIError("OCI only
   supports function tools")` for any tool whose type is not "function",
   aborting the whole request when a client mixed a built-in/server tool
   in with its function tools. Non-function tools are now skipped with a
   warning; if none remain, `tools`/`toolChoice` are omitted from the
   request instead of sending an empty array.

2. The GENERIC param map forwarded `reasoning_effort` to every non-Cohere
   model, but OCI rejects it for all served xAI Grok models
   (`Model grok-4.20 does not support parameter reasoningEffort`) and for
   OpenAI's non-reasoning models (`Unrecognized request argument supplied:
   reasoning_effort` on gpt-4o / gpt-4.1); gpt-5, the o-series, gpt-oss-*
   and Meta/Google models accept it. `_model_supports_reasoning_effort`
   encodes that, `get_supported_openai_params` stops advertising the param
   for those models, and `map_openai_params` follows the standard
   unsupported-param contract (dropped under drop_params, clear error
   otherwise) so `drop_params` / `additional_drop_params` now work as
   documented instead of the 400 surfacing from the service.

Behaviour verified against OCI GenAI in us-chicago-1 (2026-09-11) per model.
The two existing reasoning_effort normalisation tests move from a Grok
model to openai.gpt-5, which is the case they exercise. 21 new tests; the
OCI suite is 406 passed.

* fix(oci): drive reasoning_effort gating from the model catalog and drop tool_choice for skipped tools

Review follow-ups on #40732:

- `_model_supports_reasoning_effort` now reads the `supports_reasoning` flag
  of the model's `oci/` catalog entry through `get_model_info` instead of
  hardcoding xAI / gpt-oss branches. Catalogued models without the flag
  (xAI Grok, Meta Llama) no longer receive `reasoningEffort`; models absent
  from the catalog keep the historical pass-through.
- Catalog: flag `oci/google.gemini-2.5-{flash,pro,flash-lite}` as reasoning
  models and add `oci/openai.gpt-oss-120b` / `oci/openai.gpt-oss-20b`
  (pricing from Oracle's public price list, 128K context per OCI docs).
- `_model_uses_max_completion_tokens` routes on the `openai.*` prefix only.
  OCI-hosted Gemini accepts `maxCompletionTokens` but ignores it (the output
  cap silently disappears) and Llama rejects it with 400, so the catalog's
  reasoning flag must not drive that switch.
- After tool adaptation, a `FUNCTION` tool_choice naming a tool that was
  skipped (built-in / server tools) is dropped with a warning instead of
  being sent to OCI, which rejects the unmatched choice.

Verified live against OCI GenAI (us-chicago-1): Grok with mixed tools plus a
forced built-in now completes and still calls the function tool; Gemini and
gpt-oss forward reasoningEffort with maxTokens honoured; Llama drops it under
drop_params; gpt-5 keeps maxCompletionTokens.

* style(oci): use an immutable set for the tool-name lookup

Keeps the type-discipline gate (LIT002, mutable collections) within its
budget; the lookup is read-only.

* fix(oci): fail closed when a forced tool_choice names a tool OCI will not receive

Review follow-ups on #40732:

- When tool_choice forces a function that is not among the adapted tools
  (a skipped built-in or a misspelled name), send `toolChoice: NONE` instead
  of removing the choice. Falling back to the service default would let the
  model call any other supplied function even though the caller forced a
  specific one, which an agent processing untrusted content must not do.
- Move the #31449 regression tests into the mapped test module
  (`test_oci_chat_transformation.py`) instead of a separate file.

Verified live against OCI GenAI: Grok and Llama with a forced skipped
built-in now complete with no tool call; a forced kept function is still
honoured.

* test(oci): point reasoning_effort comments at the in-module gating tests

The referenced module was folded into this file.

* fix(oci): drop non-function tools only under drop_params and gate reasoning_effort on declared levels

Non-function tools now follow the unsupported-param contract: a 400 naming
drop_params without it, and dropped with a warning under it, with a forced
dropped tool turning tool calls off. reasoning_effort is gated on the model's
reasoning_effort_levels catalog entry, so OCI Llama keeps forwarding it and the
Grok 4.20 models OCI serves, now in the catalog with their Oracle prices, stop
sending a param the service rejects

* fix(oci): gate reasoning_effort on oci/xai.grok-4.20, which rejects it

* fix(oci): disable tool calls when tool_choice forces a dropped custom tool by name

* fix(oci): drop tool_choice that forces a dropped tool and keep a kept function's name

* fix(oci): keep Gemini 2.5 Flash Lite unflagged for reasoning so released proxies keep its max_tokens cap

* fix(oci): answer 400 when a request depends on a dropped tool and leave Gemini max_tokens routing to its own PR

Under drop_params, non-function tools still drop when function tools remain and
tool_choice does not force a dropped tool. A request whose every tool is dropped,
or whose tool_choice forces one, now answers 400 instead of a tool-less reply the
model can fill with invented results, or a swap to another tool.

The maxTokens routing change for OCI Gemini goes back to base, since it is
outside this fix and an open PR carries it.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
This commit is contained in:
Fede Kamelhar 2026-10-15 18:20:02 -04:00 • committed by GitHub
parent ba3fce2bda
commit 68759a239f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 503 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

98
litellm/llms/oci/chat/transformation.py
View file

@ -14,8 +14,10 @@ from collections.abc import AsyncIterator, Callable, Iterator, Mapping
from typing import TYPE_CHECKING, Any, Final
import httpx
from pydantic import TypeAdapter, ValidationError
import litellm
from litellm._logging import verbose_logger
from litellm.constants import DEFAULT_OCI_CHAT_MAX_TOKENS
from litellm.litellm_core_utils.logging_utils import track_llm_api_timing
from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
@ -49,6 +51,7 @@ from litellm.llms.oci.common_utils import (
sign_oci_request,
validate_oci_environment,
)
from litellm.router_utils.reasoning_effort_capability import declared_reasoning_efforts_for_model
from litellm.types.llms.oci import (
CohereChatRequest,
OCIChatRequestPayload,
@ -182,6 +185,75 @@ def _normalize_tool_choice(selected_params: dict) -> None:
)
_TOOL_CHOICE_MODES: Final = frozenset({"auto", "none", "required", "any"})
_TOOLS_ADAPTER: Final = TypeAdapter(list[dict[str, object]])
_JSON_OBJECT_ADAPTER: Final = TypeAdapter(dict[str, object])
def _as_json_object(value: object) -> Mapping[str, object] | None:
try:
return _JSON_OBJECT_ADAPTER.validate_python(value)
except ValidationError:
return None
def _forced_tool_name(tool_choice: object) -> str | None:
if isinstance(tool_choice, str):
return None if tool_choice.lower() in _TOOL_CHOICE_MODES else tool_choice
choice: Final = _as_json_object(tool_choice)
if choice is None:
return None
name: Final = (_as_json_object(choice.get("function")) or choice).get("name")
if isinstance(name, str):
return name
choice_type: Final = choice.get("type")
if isinstance(choice_type, str) and choice_type.lower() not in _TOOL_CHOICE_MODES:
return choice_type
return None
def _tool_identifiers(tool: Mapping[str, object]) -> tuple[object, ...]:
nested: Final = _as_json_object(tool.get(str(tool.get("type")))) or {}
return (tool.get("type"), tool.get("name"), nested.get("name"))
def _without_non_function_tools(params: Mapping[str, object], drop_params: bool) -> Mapping[str, object]:
if params.get("tools") is None:
return params
try:
tools: Final = _TOOLS_ADAPTER.validate_python(params["tools"])
except ValidationError:
return params
dropped: Final = tuple(tool for tool in tools if tool.get("type") != "function")
if not dropped:
return params
dropped_types: Final = sorted(str(tool.get("type")) for tool in dropped)
if not drop_params:
raise OCIError(
status_code=400,
message=f"OCI only supports function tools, got {dropped_types}; set drop_params=True to skip them",
)
kept: Final = [tool for tool in tools if tool.get("type") == "function"]
if not kept:
raise OCIError(
status_code=400,
message=f"OCI only supports function tools, and every tool in this request is one of {dropped_types}",
)
forced: Final = _forced_tool_name(params.get("tool_choice"))
forces_dropped_tool: Final = (
forced is not None
and any(forced in _tool_identifiers(tool) for tool in dropped)
and not any(forced in _tool_identifiers(tool) for tool in kept)
)
if forces_dropped_tool:
raise OCIError(
status_code=400,
message=f"OCI only supports function tools, and tool_choice forces `{forced}`, one of {dropped_types}",
)
verbose_logger.warning("OCI only supports function tools; dropping tools of type %s", dropped_types)
return {**params, "tools": kept}
def _normalize_response_format(selected_params: dict, vendor: OCIVendors) -> None:
rf: Final = selected_params.get("responseFormat")
if not isinstance(rf, dict) or "type" not in rf:
@ -229,6 +301,11 @@ def _normalize_response_format(selected_params: dict, vendor: OCIVendors) -> Non
selected_params["responseFormat"] = {"type": "JSON_OBJECT" if fmt == "JSON" else fmt}
def _model_supports_reasoning_effort(model: str) -> bool:
name: Final = model[4:] if model.lower().startswith("oci/") else model
return declared_reasoning_efforts_for_model(name, "oci") != ()
def get_vendor_from_model(model: str) -> OCIVendors:
"""Return the OCI vendor enum for a model name.
@ -322,7 +399,11 @@ class OCIChatConfig(BaseConfig):
# honoured and advertising it would be misleading. Callers that gate on
# this list strip n=1 (a no-op, matching what map_openai_params does);
# callers that bypass it have n=1 dropped there. Both paths converge.
return [key for key, value in param_map.items() if value]
return [
key
for key, value in param_map.items()
if value and (key != "reasoning_effort" or _model_supports_reasoning_effort(model))
]
def map_openai_params(
self,
@ -337,8 +418,21 @@ class OCIChatConfig(BaseConfig):
self.openai_to_oci_cohere_param_map if vendor == OCIVendors.COHERE else self.openai_to_oci_generic_param_map
)
for key, value in {**non_default_params, **optional_params}.items():
request_params: Final = _without_non_function_tools(
{**non_default_params, **optional_params}, drop_params or litellm.drop_params
)
for key, value in request_params.items():
alias = param_map.get(key)
if key == "reasoning_effort" and alias and not _model_supports_reasoning_effort(model):
if drop_params or litellm.drop_params:
continue
raise OCIError(
status_code=400,
message=(
f"param `reasoning_effort` is not supported on OCI model `{model}` "
"(the model rejects reasoningEffort); set drop_params=True to omit it"
),
)
if alias is False:
# max_retries is a litellm-level control param (litellm applies
# retries itself); it is never a generation param OCI accepts, so

93
litellm/model_prices_and_context_window_backup.json
View file

@ -44194,6 +44194,7 @@
"max_tokens": 131072,
"mode": "chat",
"output_cost_per_token": 1.5e-05,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_response_schema": false
@ -44210,6 +44211,72 @@
"supports_function_calling": true,
"supports_response_schema": false
},
"oci/xai.grok-4.20-0309-non-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true
},
"oci/xai.grok-4.20-0309-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_reasoning": true
},
"oci/xai.grok-4.20-non-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true
},
"oci/xai.grok-4.20-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_reasoning": true
},
"oci/xai.grok-code-fast-1": {
"input_cost_per_token": 5e-06,
"litellm_provider": "oci",
@ -44267,6 +44334,32 @@
"supports_response_schema": true,
"supports_vision": true
},
"oci/openai.gpt-oss-120b": {
"input_cost_per_token": 1.5e-07,
"litellm_provider": "oci",
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 6e-07,
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_reasoning": true
},
"oci/openai.gpt-oss-20b": {
"input_cost_per_token": 7e-08,
"litellm_provider": "oci",
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 3e-07,
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_reasoning": true
},
"oci/cohere.embed-english-v3.0": {
"input_cost_per_token": 1e-07,
"litellm_provider": "oci",

93
model_prices_and_context_window.json
View file

@ -44194,6 +44194,7 @@
"max_tokens": 131072,
"mode": "chat",
"output_cost_per_token": 1.5e-05,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_response_schema": false
@ -44210,6 +44211,72 @@
"supports_function_calling": true,
"supports_response_schema": false
},
"oci/xai.grok-4.20-0309-non-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true
},
"oci/xai.grok-4.20-0309-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_reasoning": true
},
"oci/xai.grok-4.20-non-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true
},
"oci/xai.grok-4.20-reasoning": {
"cache_read_input_token_cost": 2e-07,
"cache_read_input_token_cost_above_200k_tokens": 4e-07,
"input_cost_per_token": 1.25e-06,
"input_cost_per_token_above_200k_tokens": 2.5e-06,
"litellm_provider": "oci",
"max_input_tokens": 1000000,
"max_output_tokens": 1000000,
"max_tokens": 1000000,
"mode": "chat",
"output_cost_per_token": 2.5e-06,
"output_cost_per_token_above_200k_tokens": 5e-06,
"reasoning_effort_levels": [],
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_reasoning": true
},
"oci/xai.grok-code-fast-1": {
"input_cost_per_token": 5e-06,
"litellm_provider": "oci",
@ -44267,6 +44334,32 @@
"supports_response_schema": true,
"supports_vision": true
},
"oci/openai.gpt-oss-120b": {
"input_cost_per_token": 1.5e-07,
"litellm_provider": "oci",
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 6e-07,
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_reasoning": true
},
"oci/openai.gpt-oss-20b": {
"input_cost_per_token": 7e-08,
"litellm_provider": "oci",
"max_input_tokens": 128000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 3e-07,
"source": "https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/pricing",
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_reasoning": true
},
"oci/cohere.embed-english-v3.0": {
"input_cost_per_token": 1e-07,
"litellm_provider": "oci",

6
tests/integration/test_oci_integration.py
View file

@ -389,10 +389,8 @@ async def test_async_tool_use(m: _M, oci_params):
# Reasoning-effort tests (reasoning models only)
# ---------------------------------------------------------------------------
# Reasoning model that accepts the `reasoningEffort` parameter on OCI.
# Not every reasoning model does — xai.grok-4-fast-reasoning, for example,
# rejects it with a 400.
_REASONING_MODEL = "xai.grok-3-mini"
# Reasons at LOW and accepts reasoningEffort NONE on OCI (gpt-oss and Gemini Pro reject NONE, Grok 4.20 the param).
_REASONING_MODEL = "google.gemini-2.5-flash"
@pytest.mark.parametrize("effort", ["low", "medium", "high"])

221
tests/unit/llms/oci/chat/test_oci_chat_transformation.py
View file

@ -1078,15 +1078,19 @@ class TestOCIReasoningEffort:
return body["chatRequest"]
def test_reasoning_effort_lowercase_uppercased(self):
# gpt-5 accepts reasoningEffort; OCI's Grok 4.20 models reject it
# (see TestReasoningEffortGating at the end of this module).
chat_request = self._build_chat_request(
"xai.grok-4-fast-reasoning",
"openai.gpt-5",
{"reasoning_effort": "low"},
)
assert chat_request.get("reasoningEffort") == "LOW"
def test_reasoning_effort_disable_mapped_to_none(self):
# gpt-5 accepts reasoningEffort; OCI's Grok 4.20 models reject it
# (see TestReasoningEffortGating at the end of this module).
chat_request = self._build_chat_request(
"xai.grok-4-fast-reasoning",
"openai.gpt-5",
{"reasoning_effort": "disable"},
)
assert chat_request.get("reasoningEffort") == "NONE"
@ -1947,3 +1951,216 @@ class TestOCIStreamWrapperTerminalChunk:
)
assert [chunk.choices[0].finish_reason for chunk in chunks] == [None, "stop"]
# ---------------------------------------------------------------------------
# BerriAI/litellm#31449: mixed tool lists and reasoning_effort gating
#
# 1. Non-function tools answer 400 without ``drop_params`` and are dropped
# with it, unless the request depends on them: when no function tool is
# left, or ``tool_choice`` forces a dropped tool, it still answers 400.
# 2. ``reasoning_effort`` is withheld from OCI models whose catalog entry
# declares no reasoning effort levels (xAI Grok 4.20), so ``drop_params``
# works and callers get a clear error otherwise.
# ---------------------------------------------------------------------------
from litellm.llms.oci.chat.transformation import get_vendor_from_model
FUNCTION_TOOL = {
"type": "function",
"function": {
"name": "get_weather",
"description": "Weather for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}},
},
}
BUILTIN_TOOL = {"type": "web_search_preview"}
SERVER_TOOL = {"type": "computer_use_preview", "display_width": 1024}
FORCE_BUILTIN = {"type": "function", "function": {"name": "web_search_preview"}}
FORCE_FUNCTION = {"type": "function", "function": {"name": "get_weather"}}
GPT_OSS_MODELS = ["openai.gpt-oss-120b", "openai.gpt-oss-20b"]
def _oci_tool_params(model, tools, tool_choice, drop_params):
request = {"tools": tools} if tool_choice is None else {"tools": tools, "tool_choice": tool_choice}
config = OCIChatConfig()
mapped = config.map_openai_params(
non_default_params=request,
optional_params={},
model=model,
drop_params=drop_params,
)
return config._get_optional_params(get_vendor_from_model(model), mapped, model=model)
class TestNonFunctionTools:
@pytest.mark.parametrize("model", ["xai.grok-4.20", "cohere.command-latest"])
@pytest.mark.parametrize("non_function_tool", [BUILTIN_TOOL, SERVER_TOOL, {"name": "untyped"}])
def test_rejected_without_drop_params(self, monkeypatch, model, non_function_tool):
monkeypatch.setattr(litellm, "drop_params", False)
with pytest.raises(OCIError, match="drop_params") as exc_info:
_oci_tool_params(model, [non_function_tool, FUNCTION_TOOL], None, drop_params=False)
assert exc_info.value.status_code == 400
def test_global_drop_params_drops_them(self, monkeypatch):
monkeypatch.setattr(litellm, "drop_params", True)
params = _oci_tool_params("xai.grok-4.20", [BUILTIN_TOOL, FUNCTION_TOOL], "auto", drop_params=False)
assert [tool.name for tool in params["tools"]] == ["get_weather"]
@pytest.mark.parametrize("model", ["xai.grok-4.20", "cohere.command-latest"])
def test_drop_params_keeps_function_tools_only(self, model):
params = _oci_tool_params(model, [BUILTIN_TOOL, FUNCTION_TOOL, SERVER_TOOL], None, drop_params=True)
assert [tool.name for tool in params["tools"]] == ["get_weather"]
def test_drop_params_keeps_unforced_tool_choice(self):
params = _oci_tool_params("xai.grok-4.20", [BUILTIN_TOOL, FUNCTION_TOOL], "required", drop_params=True)
assert params["toolChoice"] == {"type": "REQUIRED"}
@pytest.mark.parametrize("tool_choice", [None, "auto", "required"])
def test_drop_params_rejects_a_request_with_no_function_tool_left(self, tool_choice):
with pytest.raises(OCIError, match="every tool") as exc_info:
_oci_tool_params("xai.grok-4.20", [BUILTIN_TOOL, SERVER_TOOL], tool_choice, drop_params=True)
assert exc_info.value.status_code == 400
@pytest.mark.parametrize(
"tool_choice",
[FORCE_BUILTIN, "web_search_preview", {"type": "web_search_preview"}],
)
def test_forcing_a_dropped_tool_is_rejected(self, tool_choice):
with pytest.raises(OCIError, match="tool_choice forces `web_search_preview`") as exc_info:
_oci_tool_params("xai.grok-4.20", [BUILTIN_TOOL, FUNCTION_TOOL], tool_choice, drop_params=True)
assert exc_info.value.status_code == 400
@pytest.mark.parametrize(
("dropped_tool", "tool_choice"),
[
({"type": "bash_20250124", "name": "bash"}, {"type": "function", "function": {"name": "bash"}}),
({"type": "custom", "custom": {"name": "code_exec"}}, "code_exec"),
({"type": "custom", "custom": {"name": "code_exec"}}, {"type": "function", "function": {"name": "code_exec"}}),
],
)
def test_forcing_a_dropped_tool_by_name_is_rejected(self, dropped_tool, tool_choice):
with pytest.raises(OCIError, match="tool_choice forces") as exc_info:
_oci_tool_params("xai.grok-4.20", [dropped_tool, FUNCTION_TOOL], tool_choice, drop_params=True)
assert exc_info.value.status_code == 400
@pytest.mark.parametrize("tool_choice", [FORCE_FUNCTION, "get_weather"])
def test_forcing_a_kept_tool_is_preserved(self, tool_choice):
params = _oci_tool_params("xai.grok-4.20", [BUILTIN_TOOL, FUNCTION_TOOL], tool_choice, drop_params=True)
assert params["toolChoice"] == {"type": "FUNCTION", "name": "get_weather"}
def test_forcing_a_kept_function_sharing_a_dropped_tool_name_is_preserved(self):
web_search_function = {**FUNCTION_TOOL, "function": {**FUNCTION_TOOL["function"], "name": "web_search"}}
force_web_search = {"type": "function", "function": {"name": "web_search"}}
params = _oci_tool_params(
"xai.grok-4.20", [{"type": "web_search"}, web_search_function], force_web_search, drop_params=True
)
assert [tool.name for tool in params["tools"]] == ["web_search"]
assert params["toolChoice"] == {"type": "FUNCTION", "name": "web_search"}
def test_misspelled_function_choice_is_left_for_the_service(self):
params = _oci_tool_params("xai.grok-4.20", [BUILTIN_TOOL, FUNCTION_TOOL], "get_weathr", drop_params=True)
assert params["toolChoice"] == {"type": "FUNCTION", "name": "get_weathr"}
def test_function_only_tools_are_untouched_without_drop_params(self, monkeypatch):
monkeypatch.setattr(litellm, "drop_params", False)
params = _oci_tool_params("xai.grok-4.20", [FUNCTION_TOOL], FORCE_FUNCTION, drop_params=False)
assert [tool.name for tool in params["tools"]] == ["get_weather"]
assert params["toolChoice"] == {"type": "FUNCTION", "name": "get_weather"}
SERVED_GROK_4_20_MODELS = [
"xai.grok-4.20-reasoning",
"xai.grok-4.20-non-reasoning",
"xai.grok-4.20-0309-reasoning",
"xai.grok-4.20-0309-non-reasoning",
]
class TestReasoningEffortGating:
@pytest.mark.parametrize("model", SERVED_GROK_4_20_MODELS + ["xai.grok-4.20"])
def test_models_declaring_no_levels_are_gated(self, local_model_cost_map, model):
supported = OCIChatConfig().get_supported_openai_params(model)
assert "reasoning_effort" not in supported
assert "tools" in supported
@pytest.mark.parametrize(
"model",
[
"meta.llama-3.3-70b-instruct",
"meta.llama-4-scout-17b-16e-instruct",
"xai.grok-4.20-multi-agent",
"openai.gpt-5",
"openai.gpt-oss-120b",
"google.gemini-2.5-flash",
"google.gemini-2.5-flash-lite",
],
)
def test_models_declaring_nothing_are_forwarded(self, local_model_cost_map, model):
assert "reasoning_effort" in OCIChatConfig().get_supported_openai_params(model)
def test_models_absent_from_catalog_are_forwarded(self, local_model_cost_map):
assert "oci/meta.llama-9-future-instruct" not in litellm.model_cost
assert "reasoning_effort" in OCIChatConfig().get_supported_openai_params("meta.llama-9-future-instruct")
def test_gate_is_catalog_driven(self, local_model_cost_map, monkeypatch):
entry = {**litellm.model_cost["oci/meta.llama-3.3-70b-instruct"], "reasoning_effort_levels": []}
monkeypatch.setitem(litellm.model_cost, "oci/meta.llama-3.3-70b-instruct", entry)
assert "reasoning_effort" not in OCIChatConfig().get_supported_openai_params("meta.llama-3.3-70b-instruct")
@pytest.mark.parametrize("model", SERVED_GROK_4_20_MODELS)
def test_reasoning_effort_dropped_under_drop_params(self, local_model_cost_map, model):
result = OCIChatConfig().map_openai_params(
non_default_params={"reasoning_effort": "low", "temperature": 0.1},
optional_params={},
model=model,
drop_params=True,
)
assert "reasoningEffort" not in result
assert "reasoning_effort" not in result
assert result["temperature"] == 0.1
def test_reasoning_effort_raises_without_drop_params_for_gated_grok(self, local_model_cost_map, monkeypatch):
monkeypatch.setattr(litellm, "drop_params", False)
with pytest.raises(OCIError, match=r"reasoning_effort.*xai\.grok-4\.20-reasoning"):
OCIChatConfig().map_openai_params(
non_default_params={"reasoning_effort": "low"},
optional_params={},
model="xai.grok-4.20-reasoning",
drop_params=False,
)
@pytest.mark.parametrize(
"model", ["openai.gpt-5", "google.gemini-2.5-flash", "openai.gpt-oss-120b", "meta.llama-3.3-70b-instruct"]
)
def test_reasoning_effort_forwarded_when_not_gated(self, local_model_cost_map, model):
result = OCIChatConfig().map_openai_params(
non_default_params={"reasoning_effort": "low"},
optional_params={},
model=model,
drop_params=False,
)
assert result["reasoningEffort"] == "low"
def _oci_chat_request(model, optional_params):
return OCIChatConfig().transform_request(
model=model,
messages=[{"role": "user", "content": "hi"}],
optional_params={**BASE_OCI_PARAMS, **optional_params},
litellm_params={},
headers={},
)["chatRequest"]
class TestOpenAIPrefixMaxTokensKey:
@pytest.mark.parametrize("model", GPT_OSS_MODELS)
def test_gpt_oss_keeps_max_tokens(self, local_model_cost_map, model):
chat_request = _oci_chat_request(model, {"max_tokens": 64})
assert chat_request["maxTokens"] == 64
assert "maxCompletionTokens" not in chat_request
def test_gpt5_still_routes_to_max_completion_tokens(self, local_model_cost_map):
chat_request = _oci_chat_request("openai.gpt-5", {"max_tokens": 64})
assert chat_request["maxCompletionTokens"] == 64
assert "maxTokens" not in chat_request