scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-27 01:22:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 81

feat(oci): add reasoning_effort passthrough — only true missing primitive

Browse source 
OCI's GenericChatRequest exposes a reasoningEffort field
(NONE/MINIMAL/LOW/MEDIUM/HIGH) that's the single biggest cost knob for
reasoning-capable models on the service:

  - GPT-5 family
  - Gemini 2.5
  - Grok reasoning variants (3-mini, 4-fast, 4.20)
  - Cohere Command-A-Reasoning

Setting reasoning_effort=LOW typically cuts reasoning-token spend 5-10×
vs the default. Without exposing this, litellm users had no way to tune
cost-vs-quality on these models.

The other GenericChatRequest fields (verbosity, parallel_tool_calls,
logit_bias, n, metadata, web_search_options, prediction) are not
exposed because they are not missing primitives — they either duplicate
prompt-engineering, framework-level controls, or are too niche to
justify the maintenance surface. We only ship what users genuinely
can't accomplish another way.

Excluded from the Cohere v1 param map: CohereChatRequest has no
reasoningEffort field, and Cohere reasoning models
(cohere.command-a-reasoning) use COHEREV2 which is a separate request
type not covered by this PR.

Verified live: GPT-5.5 + reasoning_effort="HIGH" sends
{"reasoningEffort": "HIGH"} on the wire and OCI accepts the request.
This commit is contained in:
Federico Kamelhar 2026-05-06 13:54:44 -04:00
parent a09f629a5c
commit 05cb73c16a
2 changed files with 10 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
litellm/llms/oci/chat/transformation.py
View file

@ -132,6 +132,7 @@ class OCIChatConfig(BaseConfig):
"logit_bias": "logitBias",
"n": "numGenerations",
"presence_penalty": "presencePenalty",
"reasoning_effort": "reasoningEffort",
"seed": "seed",
"stop": "stop",
"tool_choice": "toolChoice",
@ -150,14 +151,17 @@ class OCIChatConfig(BaseConfig):
"response_format": "responseFormat",
}
# Cohere param map differs from GENERIC in three ways:
# Cohere param map differs from GENERIC in four ways:
# - tool_choice is unsupported
# - stop sequences key is "stopSequences" not "stop"
# - n (numGenerations) is GENERIC-only
# - reasoning_effort is GENERIC-only (CohereChatRequest v1 has no
# reasoningEffort field; Cohere reasoning models like
# command-a-reasoning use COHEREV2 which is a separate request type)
self.openai_to_oci_cohere_param_map = {
k: ("stopSequences" if k == "stop" else v)
for k, v in self.openai_to_oci_generic_param_map.items()
if k not in ("tool_choice", "max_retries", "n")
if k not in ("tool_choice", "max_retries", "n", "reasoning_effort")
}
def get_supported_openai_params(self, model: str) -> List[str]:

4
litellm/types/llms/oci.py
View file

@ -103,6 +103,10 @@ class OCIChatRequestPayload(BaseModel):
seed: Optional[int] = None
frequencyPenalty: Optional[float] = None
presencePenalty: Optional[float] = None
# Reasoning-token budget knob (OCI: NONE/MINIMAL/LOW/MEDIUM/HIGH).
# Honoured by GPT-5 family, Gemini 2.5, Grok reasoning variants,
# Cohere Command-A-Reasoning. Ignored by non-reasoning models.
reasoningEffort: Optional[str] = None
responseFormat: Optional[Dict[str, Any]] = None
toolChoice: Optional[Union[str, Dict[str, Any]]] = None
logitBias: Optional[Dict[str, Any]] = None