Commit graph

5 commits

Author SHA1 Message Date
Fede Kamelhar
68759a239f
fix(oci): drop non-function tools under drop_params and gate reasoning_effort on declared levels (#40732)
* fix(oci): skip non-function tools and gate reasoning_effort on models that reject it

Fixes #31449.

Two OCI GENERIC request-building defects that made Grok-on-OCI unusable
for tool-using coding-agent clients (Codex over the Responses API, Claude
Code over the Anthropic Messages API):

1. `adapt_tool_definition_to_oci_standard` raised `OCIError("OCI only
   supports function tools")` for any tool whose type is not "function",
   aborting the whole request when a client mixed a built-in/server tool
   in with its function tools. Non-function tools are now skipped with a
   warning; if none remain, `tools`/`toolChoice` are omitted from the
   request instead of sending an empty array.

2. The GENERIC param map forwarded `reasoning_effort` to every non-Cohere
   model, but OCI rejects it for all served xAI Grok models
   (`Model grok-4.20 does not support parameter reasoningEffort`) and for
   OpenAI's non-reasoning models (`Unrecognized request argument supplied:
   reasoning_effort` on gpt-4o / gpt-4.1); gpt-5, the o-series, gpt-oss-*
   and Meta/Google models accept it. `_model_supports_reasoning_effort`
   encodes that, `get_supported_openai_params` stops advertising the param
   for those models, and `map_openai_params` follows the standard
   unsupported-param contract (dropped under drop_params, clear error
   otherwise) so `drop_params` / `additional_drop_params` now work as
   documented instead of the 400 surfacing from the service.

Behaviour verified against OCI GenAI in us-chicago-1 (2026-09-11) per model.
The two existing reasoning_effort normalisation tests move from a Grok
model to openai.gpt-5, which is the case they exercise. 21 new tests; the
OCI suite is 406 passed.

* fix(oci): drive reasoning_effort gating from the model catalog and drop tool_choice for skipped tools

Review follow-ups on #40732:

- `_model_supports_reasoning_effort` now reads the `supports_reasoning` flag
  of the model's `oci/` catalog entry through `get_model_info` instead of
  hardcoding xAI / gpt-oss branches. Catalogued models without the flag
  (xAI Grok, Meta Llama) no longer receive `reasoningEffort`; models absent
  from the catalog keep the historical pass-through.
- Catalog: flag `oci/google.gemini-2.5-{flash,pro,flash-lite}` as reasoning
  models and add `oci/openai.gpt-oss-120b` / `oci/openai.gpt-oss-20b`
  (pricing from Oracle's public price list, 128K context per OCI docs).
- `_model_uses_max_completion_tokens` routes on the `openai.*` prefix only.
  OCI-hosted Gemini accepts `maxCompletionTokens` but ignores it (the output
  cap silently disappears) and Llama rejects it with 400, so the catalog's
  reasoning flag must not drive that switch.
- After tool adaptation, a `FUNCTION` tool_choice naming a tool that was
  skipped (built-in / server tools) is dropped with a warning instead of
  being sent to OCI, which rejects the unmatched choice.

Verified live against OCI GenAI (us-chicago-1): Grok with mixed tools plus a
forced built-in now completes and still calls the function tool; Gemini and
gpt-oss forward reasoningEffort with maxTokens honoured; Llama drops it under
drop_params; gpt-5 keeps maxCompletionTokens.

* style(oci): use an immutable set for the tool-name lookup

Keeps the type-discipline gate (LIT002, mutable collections) within its
budget; the lookup is read-only.

* fix(oci): fail closed when a forced tool_choice names a tool OCI will not receive

Review follow-ups on #40732:

- When tool_choice forces a function that is not among the adapted tools
  (a skipped built-in or a misspelled name), send `toolChoice: NONE` instead
  of removing the choice. Falling back to the service default would let the
  model call any other supplied function even though the caller forced a
  specific one, which an agent processing untrusted content must not do.
- Move the #31449 regression tests into the mapped test module
  (`test_oci_chat_transformation.py`) instead of a separate file.

Verified live against OCI GenAI: Grok and Llama with a forced skipped
built-in now complete with no tool call; a forced kept function is still
honoured.

* test(oci): point reasoning_effort comments at the in-module gating tests

The referenced module was folded into this file.

* fix(oci): drop non-function tools only under drop_params and gate reasoning_effort on declared levels

Non-function tools now follow the unsupported-param contract: a 400 naming
drop_params without it, and dropped with a warning under it, with a forced
dropped tool turning tool calls off. reasoning_effort is gated on the model's
reasoning_effort_levels catalog entry, so OCI Llama keeps forwarding it and the
Grok 4.20 models OCI serves, now in the catalog with their Oracle prices, stop
sending a param the service rejects

* fix(oci): gate reasoning_effort on oci/xai.grok-4.20, which rejects it

* fix(oci): disable tool calls when tool_choice forces a dropped custom tool by name

* fix(oci): drop tool_choice that forces a dropped tool and keep a kept function's name

* fix(oci): keep Gemini 2.5 Flash Lite unflagged for reasoning so released proxies keep its max_tokens cap

* fix(oci): answer 400 when a request depends on a dropped tool and leave Gemini max_tokens routing to its own PR

Under drop_params, non-function tools still drop when function tools remain and
tool_choice does not force a dropped tool. A request whose every tool is dropped,
or whose tool_choice forces one, now answers 400 instead of a tool-less reply the
model can fill with invented results, or a swap to another tool.

The maxTokens routing change for OCI Gemini goes back to base, since it is
outside this fix and an open PR carries it.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-15 22:20:02 +00:00
devin-ai-integration[bot]
e2971e0af4
refactor(llms): expose public names for private provider helpers (#45037)
* refactor(litellm): migrate private usage in llms

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(litellm): preserve Bedrock batch signature marker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(litellm): migrate llms private usage symbols

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(litellm): preserve UUID Watsonx project IDs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(litellm): retarget llms mocks to public names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(litellm): limit llms changes to renames and forwarders

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(litellm): keep GCS mock client patching private Vertex auth methods

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(litellm): assert forwarder arguments and type forwarder signatures

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 08:49:25 -07:00
shrey-berri
e494723105
fix(params): validate stream_chunk_size once, before any provider call (#43222)
* fix(params): validate stream_chunk_size once and carry it as typed control options

Checks stream_chunk_size at the top of completion() and acompletion(), accepts
digit strings, returns a 400 naming the param unless drop_params is set, and
stores the checked value under _litellm_control. Bedrock Converse and Invoke
read it from litellm_params; the Bedrock-only checker and the dead Invoke pops
are gone. Owned-kwarg filtering now runs through one helper everywhere.

Refs LIT-8317

* test(bedrock): drop tests for the removed stream_chunk_size_from helper

Refs LIT-8317

* fix(params): check stream_chunk_size before the MCP gateway branch

Refs LIT-8317

* fix(params): return assert_never in the exhaustive control-options match

Refs LIT-8317

* fix(params): address council review of the control options change

Read all_litellm_params live so names registered after import stay
LiteLLM-owned, make litellm_params a required keyword on the stream
wrapper hooks, give digit strings and ints the same 18-digit range,
share the default-chunking test table, test the Responses bridge through
litellm.responses, and revert formatting-only churn in existing tests.

Refs LIT-8317

* fix(params): address the second council review of control options

Keep the Responses bridge on its original all_litellm_params forwarding,
narrow _int_from_decimal_string inline so it type-checks, bound nested
huge ints in the error message, store _litellm_control only when a value
is set, simplify the parser to its single field, drop the one-caller
wrapper, and tighten the tests.

Refs LIT-8317

* fix(params): keep the 18-digit length check on stream_chunk_size strings

A 19-character string with leading zeros such as 0000000000000000001 would
otherwise pass as 1, although the rule and the error message say at most
18 digits.

Refs LIT-8317

* test(params): tidy control options tests after council sign-off

Move the Responses bridge test into the existing bridge test file, drop the
rebind test that pinned an implementation detail, assert through
stored_control_options instead of the storage key, and cover
drop_params="true" through Bedrock streaming.

Refs LIT-8317

* test(params): wrap a chunking test row that went past 120 characters

Refs LIT-8317
2026-09-26 23:01:20 +00:00
yuneng
e4a58ef91a test(unit): make every tests/unit directory a package so pytest collection is unique
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 11:50:59 +00:00
yuneng
443c9f8385 test: migrate nvidia, oci, ocr, oobabooga and openai legacy tests to tests/unit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 08:09:49 +00:00