mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-12 08:42:18 +00:00
fix(codex): treat a client_version too long to parse as an old Codex and keep undescribed stock levels
A client_version past Python's integer digit limit made the Codex catalog answer 500. It now gets the levels every Codex parses. A stock Codex level with no description validates again, as on base, and lite codex treats undecodable codex --version output as an unknown version
This commit is contained in:
mateo-berri
parent
a0282e0643
commit
62e21014b5
5 changed files with 34 additions and 22 deletions
|
|
@ -492,7 +492,7 @@ lite codex exec "summarize the repo"
|
|||
|
||||
Each command resolves your LiteLLM key (logging in via SSO when none is stored and you are at a terminal; otherwise it expects `LITELLM_PROXY_API_KEY` or `--api-key`), checks the key against the proxy so bad credentials fail immediately instead of deep inside the agent, exports the environment variables the agent reads, then replaces itself with the agent process.
|
||||
|
||||
The right variables are picked per agent. Claude Code gets `ANTHROPIC_BASE_URL` (the proxy root, so it appends `/v1/messages`) and `ANTHROPIC_AUTH_TOKEN`, with any stray `ANTHROPIC_API_KEY` cleared so the proxy token wins, and `ENABLE_TOOL_SEARCH=true` (unless you already set it) so Claude Code keeps tool search on even though the base URL is a proxy rather than a first-party Anthropic host. It also gets `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` (again unless you already set it) so Claude Code v2.1.129+ fills its `/model` picker from the proxy's `/v1/models`; Claude Code only lists entries whose id contains `claude` or `anthropic`, so the proxy lists every other group to Claude Code as `claude-router-<UTF-8 hex of the group name>` and marks a group whose input window reaches 1M with `[1m]`, and a request on such an id is served by the group. Older Claude Code versions ignore the variable. Export it as `0` to turn discovery off. Codex and OpenCode get `OPENAI_BASE_URL` (the proxy plus `/v1`) and `OPENAI_API_KEY`. Codex ignores `OPENAI_BASE_URL`, so it is additionally pointed at the proxy through a custom provider passed as `-c` config overrides (HTTP/SSE Responses transport, since the proxy does not speak the Responses WebSocket protocol). OpenCode additionally gets `OPENCODE_CONFIG_CONTENT` holding a generated `litellm` provider (`@ai-sdk/openai-compatible`, the proxy `/v1` URL, `{env:OPENAI_API_KEY}`) with one model entry per chat model your key can see on `/v1/models`, so its model picker mirrors the proxy without a hand-maintained `opencode.json`; OpenCode merges that over your own config files, and if you already export `OPENCODE_CONFIG_CONTENT` yours is left alone. When the list cannot be fetched, `lite opencode` says so on stderr and launches anyway. Codex gets the same list as a catalog file, `$CODEX_HOME/litellm-models.json` (default `~/.codex/`), passed as `-c model_catalog_json=<path>` so `/model` lists exactly the proxy's chat models; a proxy model the installed Codex already knows (`gpt-5.5`, say) keeps that Codex's own entry, reasoning levels and prompt included, and only its place in the picker comes from the proxy, while a model Codex does not know gets the plain entry Codex uses for an unknown `-m` slug. Before launching, `lite codex` asks the installed Codex for its own list and then has it read the written file back (both through `codex debug models`), and when the fetch, either of those or the write fails (Codex releases older than 0.130 have no such command) it says so on stderr and launches with Codex's built-in catalog, leaving a rejected file in place.
|
||||
The right variables are picked per agent. Claude Code gets `ANTHROPIC_BASE_URL` (the proxy root, so it appends `/v1/messages`) and `ANTHROPIC_AUTH_TOKEN`, with any stray `ANTHROPIC_API_KEY` cleared so the proxy token wins, and `ENABLE_TOOL_SEARCH=true` (unless you already set it) so Claude Code keeps tool search on even though the base URL is a proxy rather than a first-party Anthropic host. It also gets `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` (again unless you already set it) so Claude Code v2.1.129+ fills its `/model` picker from the proxy's `/v1/models`; Claude Code only lists entries whose id contains `claude` or `anthropic`, so the proxy lists every other group to Claude Code as `claude-router-<UTF-8 hex of the group name>` and marks a group whose input window reaches 1M with `[1m]`, and a request on such an id is served by the group. Older Claude Code versions ignore the variable. Export it as `0` to turn discovery off. Codex and OpenCode get `OPENAI_BASE_URL` (the proxy plus `/v1`) and `OPENAI_API_KEY`. Codex ignores `OPENAI_BASE_URL`, so it is additionally pointed at the proxy through a custom provider passed as `-c` config overrides (HTTP/SSE Responses transport, since the proxy does not speak the Responses WebSocket protocol). OpenCode additionally gets `OPENCODE_CONFIG_CONTENT` holding a generated `litellm` provider (`@ai-sdk/openai-compatible`, the proxy `/v1` URL, `{env:OPENAI_API_KEY}`) with one model entry per chat model your key can see on `/v1/models`, so its model picker mirrors the proxy without a hand-maintained `opencode.json`; OpenCode merges that over your own config files, and if you already export `OPENCODE_CONFIG_CONTENT` yours is left alone. When the list cannot be fetched, `lite opencode` says so on stderr and launches anyway. Codex gets the same list as a catalog file, `$CODEX_HOME/litellm-models.json` (default `~/.codex/`), passed as `-c model_catalog_json=<path>` so `/model` lists exactly the proxy's chat models; a proxy model the installed Codex already knows (`gpt-5.5`, say) keeps that Codex's own entry, reasoning levels and prompt included, and only its place in the picker comes from the proxy, while a model Codex does not know gets the plain entry Codex uses for an unknown `-m` slug, plus whichever reasoning levels its model group lists on `/model_group/info` the installed Codex can parse. Before launching, `lite codex` asks the installed Codex for its own list and then has it read the written file back (both through `codex debug models`), and when the fetch, either of those or the write fails (Codex releases older than 0.130 have no such command) it says so on stderr and launches with Codex's built-in catalog, leaving a rejected file in place.
|
||||
|
||||
pi ignores base-URL environment variables entirely, so `lite pi` (kept out of the `lite --help` command listing for now, but fully functional) wires it up differently: before handoff it fetches the models your key can use from the proxy's `/v1/models` (plus each model's context window and output cap from `/model_group/info`, when available) and syncs them into a `litellm` provider entry in pi's `~/.pi/agent/models.json` (honoring `PI_CODING_AGENT_DIR`), then starts pi on that provider's first model via an injected `--model litellm/<id>`. Only that one provider entry is rewritten; the rest of the file, including any other custom providers, is left alone. The entry references the key as `$LITELLM_PROXY_API_KEY`, which the wrapper exports for the session, so the token itself never lands on disk and plain `pi` outside the wrapper simply shows the litellm models as unavailable. The entry also turns on pi's session header (`x-session-id`, sent by pi 0.80.7 and later), so every chat turn of one pi session shares one session id in the logs and sticks to one deployment when session affinity is on. A custom `pi --session-id` must be at least 8 letters, digits, `_`, or `-` for the proxy to read it. Your own flags come after the injected pin, so `lite pi --model litellm/<other-id>` wins, and inside the TUI the `/model` picker lists every synced litellm model.
|
||||
|
||||
|
|
|
|||
|
|
@ -863,7 +863,7 @@ def _codex_version(
|
|||
encoding="utf-8",
|
||||
timeout=_CODEX_PREFLIGHT_TIMEOUT_SECONDS,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
except (OSError, subprocess.TimeoutExpired, UnicodeError):
|
||||
return None
|
||||
words: Final = completed.stdout.split()
|
||||
return words[-1] if completed.returncode == 0 and words else None
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ from pathlib import Path
|
|||
from types import MappingProxyType, NoneType
|
||||
from typing import TYPE_CHECKING, Annotated, Final, Literal, TypeAlias
|
||||
|
||||
from packaging.version import InvalidVersion, Version
|
||||
from packaging.version import Version
|
||||
from pydantic import ConfigDict, Field, StringConstraints, TypeAdapter, ValidationError
|
||||
|
||||
from litellm._logging import verbose_proxy_logger
|
||||
|
|
@ -75,10 +75,10 @@ class _ConfiguredTier(LiteLLMBaseModel):
|
|||
_CONFIGURED_TIERS: Final = TypeAdapter(tuple[_TierId | _ConfiguredTier, ...])
|
||||
|
||||
|
||||
class CodexReasoningLevel(LiteLLMBaseModel):
|
||||
class CodexFallbackReasoningLevel(LiteLLMBaseModel):
|
||||
"""A `ReasoningEffortPreset` as Codex reads it: picking it sends `effort` as the request's reasoning effort."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="allow")
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
effort: str
|
||||
description: str
|
||||
|
|
@ -105,7 +105,7 @@ class CodexFallbackModel(LiteLLMBaseModel):
|
|||
display_name: str
|
||||
description: NoneType = None
|
||||
default_reasoning_level: str | None = Field(default=None, exclude_if=lambda level: level is None)
|
||||
supported_reasoning_levels: tuple[CodexReasoningLevel, ...] = ()
|
||||
supported_reasoning_levels: tuple[CodexFallbackReasoningLevel, ...] = ()
|
||||
shell_type: Literal["unified_exec"] = "unified_exec"
|
||||
visibility: Literal["list"] = "list"
|
||||
supported_in_api: Literal[True] = True
|
||||
|
|
@ -131,6 +131,12 @@ class CodexStockUpgrade(LiteLLMBaseModel):
|
|||
model: str
|
||||
|
||||
|
||||
class CodexReasoningLevel(LiteLLMBaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="allow")
|
||||
|
||||
effort: str
|
||||
|
||||
|
||||
class CodexStockModel(LiteLLMBaseModel):
|
||||
"""One `ModelInfo` entry as Codex ships it or prints it from `codex debug models`.
|
||||
|
||||
|
|
@ -284,13 +290,13 @@ def _codex_can_drive(model_id: str, mode: str | None) -> bool:
|
|||
def _parses_custom_efforts(client_version: str | None) -> bool:
|
||||
try:
|
||||
return client_version is not None and Version(client_version) >= _FIRST_CODEX_WITH_CUSTOM_EFFORTS
|
||||
except InvalidVersion:
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
def codex_reasoning_levels(
|
||||
efforts: Sequence[str], client_version: str | None
|
||||
) -> tuple[tuple[CodexReasoningLevel, ...], str | None]:
|
||||
) -> tuple[tuple[CodexFallbackReasoningLevel, ...], str | None]:
|
||||
"""The reasoning levels a fallback entry offers for a model group's efforts, and its default level.
|
||||
|
||||
Codex before 0.138.0 fails the whole catalog on an effort it has no name for, so for it, and for
|
||||
|
|
@ -305,7 +311,7 @@ def codex_reasoning_levels(
|
|||
return (), None
|
||||
nearest: Final = nearest_declared_reasoning_effort(_PREFERRED_DEFAULT_EFFORT, offered, "down")
|
||||
levels: Final = tuple(
|
||||
CodexReasoningLevel(effort=effort, description=f"Sends reasoning_effort={effort} upstream")
|
||||
CodexFallbackReasoningLevel(effort=effort, description=f"Sends reasoning_effort={effort} upstream")
|
||||
for effort in offered
|
||||
)
|
||||
if _CODEX_UNSET_EFFORT in offered:
|
||||
|
|
|
|||
|
|
@ -1056,8 +1056,9 @@ class TestCodexModelSync:
|
|||
("codex-cli", ["low", "high"]),
|
||||
(None, ["low", "high"]),
|
||||
(OSError("codex vanished"), ["low", "high"]),
|
||||
(UnicodeDecodeError("utf-8", b"\xff", 0, 1, "invalid start byte"), ["low", "high"]),
|
||||
],
|
||||
ids=["current", "first-custom", "pre-custom", "pre-release", "no-number", "failed", "unrunnable"],
|
||||
ids=["current", "first-custom", "pre-custom", "pre-release", "no-number", "failed", "unrunnable", "undecodable"],
|
||||
)
|
||||
def test_unknown_model_offers_the_group_efforts_the_installed_codex_parses(self, tmp_path, version, levels):
|
||||
_, result = self._sync(
|
||||
|
|
|
|||
|
|
@ -156,6 +156,7 @@ _ABSENT = object()
|
|||
(("low", "high", "max"), "0.137.9", ["low", "high"], "low"),
|
||||
(("low", "high", "max"), "0.138.0-alpha.2", ["low", "high"], "low"),
|
||||
(("minimal", "max"), "not-a-version", ["minimal"], "minimal"),
|
||||
pytest.param(("minimal", "max"), "9" * 5000, ["minimal"], "minimal", id="version-past-int-digit-limit"),
|
||||
(("low", "turbo"), None, ["low"], "low"),
|
||||
(("max",), "0.137.0", [], _ABSENT),
|
||||
],
|
||||
|
|
@ -171,14 +172,21 @@ def test_unknown_model_offers_the_group_efforts_its_codex_release_parses(efforts
|
|||
assert entry.get("default_reasoning_level", _ABSENT) == default
|
||||
|
||||
|
||||
def test_a_model_codex_knows_keeps_its_own_reasoning_levels():
|
||||
stock = CodexStockModel(
|
||||
slug="gpt-5.5",
|
||||
display_name="GPT-5.5",
|
||||
priority=3,
|
||||
visibility="list",
|
||||
default_reasoning_level="medium",
|
||||
supported_reasoning_levels=[{"effort": "medium", "description": "Codex's own medium"}],
|
||||
@pytest.mark.parametrize(
|
||||
"levels",
|
||||
[[{"effort": "medium", "description": "Codex's own medium"}], [{"effort": "medium"}]],
|
||||
ids=["described", "undescribed"],
|
||||
)
|
||||
def test_a_model_codex_knows_keeps_its_own_reasoning_levels(levels):
|
||||
stock = CodexStockModel.model_validate(
|
||||
{
|
||||
"slug": "gpt-5.5",
|
||||
"display_name": "GPT-5.5",
|
||||
"priority": 3,
|
||||
"visibility": "list",
|
||||
"default_reasoning_level": "medium",
|
||||
"supported_reasoning_levels": levels,
|
||||
}
|
||||
)
|
||||
|
||||
(entry,) = json.loads(
|
||||
|
|
@ -190,10 +198,7 @@ def test_a_model_codex_knows_keeps_its_own_reasoning_levels():
|
|||
).json
|
||||
)["models"]
|
||||
|
||||
assert (entry["default_reasoning_level"], entry["supported_reasoning_levels"]) == (
|
||||
"medium",
|
||||
[{"effort": "medium", "description": "Codex's own medium"}],
|
||||
)
|
||||
assert (entry["default_reasoning_level"], entry["supported_reasoning_levels"]) == ("medium", levels)
|
||||
|
||||
|
||||
def test_known_model_keeps_codex_stock_entry_listed_under_the_served_id():
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue