fix(cli): read the Codex catalog back before launch and cover every ModelInfo schema
Some checks failed
ai-gateway image / ai-gateway release image (push) Waiting to run
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled

Codex 0.130 and 0.145 require supports_reasoning_summaries and
supports_parallel_tool_calls on every catalog entry, so a catalog written
for 0.154 made those releases exit at startup with a parse error. Every
field some release since 0.105.0 deserializes without a default is now
written, with Codex's own fallback values, and the catalog is read back
once through the installed binary (`codex debug models`) before launch.
A Codex that rejects it, or one older than 0.130 with no such command,
gets the skip notice and launches on its built-in catalog instead.
This commit is contained in:
mateo-berri 2026-09-12 15:55:03 -07:00
parent 68e6089950
commit 222f283c93
3 changed files with 190 additions and 22 deletions

View file

@ -490,7 +490,7 @@ lite codex exec "summarize the repo"
Each command resolves your LiteLLM key (logging in via SSO when none is stored and you are at a terminal; otherwise it expects `LITELLM_PROXY_API_KEY` or `--api-key`), checks the key against the proxy so bad credentials fail immediately instead of deep inside the agent, exports the environment variables the agent reads, then replaces itself with the agent process.
The right variables are picked per agent. Claude Code gets `ANTHROPIC_BASE_URL` (the proxy root, so it appends `/v1/messages`) and `ANTHROPIC_AUTH_TOKEN`, with any stray `ANTHROPIC_API_KEY` cleared so the proxy token wins, and `ENABLE_TOOL_SEARCH=true` (unless you already set it) so Claude Code keeps tool search on even though the base URL is a proxy rather than a first-party Anthropic host. It also gets `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` (again unless you already set it) so Claude Code v2.1.129+ fills its `/model` picker from the proxy's `/v1/models`; Claude Code only lists entries whose id contains `claude` or `anthropic`, so the proxy lists every other group to Claude Code as `claude-router-<UTF-8 hex of the group name>` and marks a group whose input window reaches 1M with `[1m]`, and a request on such an id is served by the group. Older Claude Code versions ignore the variable. Export it as `0` to turn discovery off. Codex and OpenCode get `OPENAI_BASE_URL` (the proxy plus `/v1`) and `OPENAI_API_KEY`. Codex ignores `OPENAI_BASE_URL`, so it is additionally pointed at the proxy through a custom provider passed as `-c` config overrides (HTTP/SSE Responses transport, since the proxy does not speak the Responses WebSocket protocol). OpenCode additionally gets `OPENCODE_CONFIG_CONTENT` holding a generated `litellm` provider (`@ai-sdk/openai-compatible`, the proxy `/v1` URL, `{env:OPENAI_API_KEY}`) with one model entry per chat model your key can see on `/v1/models`, so its model picker mirrors the proxy without a hand-maintained `opencode.json`; OpenCode merges that over your own config files, and if you already export `OPENCODE_CONFIG_CONTENT` yours is left alone. When the list cannot be fetched, `lite opencode` says so on stderr and launches anyway.
The right variables are picked per agent. Claude Code gets `ANTHROPIC_BASE_URL` (the proxy root, so it appends `/v1/messages`) and `ANTHROPIC_AUTH_TOKEN`, with any stray `ANTHROPIC_API_KEY` cleared so the proxy token wins, and `ENABLE_TOOL_SEARCH=true` (unless you already set it) so Claude Code keeps tool search on even though the base URL is a proxy rather than a first-party Anthropic host. It also gets `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` (again unless you already set it) so Claude Code v2.1.129+ fills its `/model` picker from the proxy's `/v1/models`; Claude Code only lists entries whose id contains `claude` or `anthropic`, so the proxy lists every other group to Claude Code as `claude-router-<UTF-8 hex of the group name>` and marks a group whose input window reaches 1M with `[1m]`, and a request on such an id is served by the group. Older Claude Code versions ignore the variable. Export it as `0` to turn discovery off. Codex and OpenCode get `OPENAI_BASE_URL` (the proxy plus `/v1`) and `OPENAI_API_KEY`. Codex ignores `OPENAI_BASE_URL`, so it is additionally pointed at the proxy through a custom provider passed as `-c` config overrides (HTTP/SSE Responses transport, since the proxy does not speak the Responses WebSocket protocol). OpenCode additionally gets `OPENCODE_CONFIG_CONTENT` holding a generated `litellm` provider (`@ai-sdk/openai-compatible`, the proxy `/v1` URL, `{env:OPENAI_API_KEY}`) with one model entry per chat model your key can see on `/v1/models`, so its model picker mirrors the proxy without a hand-maintained `opencode.json`; OpenCode merges that over your own config files, and if you already export `OPENCODE_CONFIG_CONTENT` yours is left alone. When the list cannot be fetched, `lite opencode` says so on stderr and launches anyway. Codex gets the same list as a catalog file, `$CODEX_HOME/litellm-models.json` (default `~/.codex/`), passed as `-c model_catalog_json=<path>` so `/model` lists exactly the proxy's chat models; before launching, `lite codex` has the installed Codex read that file back (`codex debug models`), and when the fetch, the write or that read-back fails (Codex releases older than 0.130 have no such command) it says so on stderr and launches with Codex's built-in catalog, leaving the rejected file in place.
pi ignores base-URL environment variables entirely, so `lite pi` (kept out of the `lite --help` command listing for now, but fully functional) wires it up differently: before handoff it fetches the models your key can use from the proxy's `/v1/models` (plus each model's context window and output cap from `/model_group/info`, when available) and syncs them into a `litellm` provider entry in pi's `~/.pi/agent/models.json` (honoring `PI_CODING_AGENT_DIR`), then starts pi on that provider's first model via an injected `--model litellm/<id>`. Only that one provider entry is rewritten; the rest of the file, including any other custom providers, is left alone. The entry references the key as `$LITELLM_PROXY_API_KEY`, which the wrapper exports for the session, so the token itself never lands on disk and plain `pi` outside the wrapper simply shows the litellm models as unavailable. Your own flags come after the injected pin, so `lite pi --model litellm/<other-id>` wins, and inside the TUI the `/model` picker lists every synced litellm model.

View file

@ -69,6 +69,7 @@ CODEX_PROXY_PROVIDER: Final = "litellm"
CODEX_HOME_ENV: Final = "CODEX_HOME"
CODEX_MODEL_CATALOG_FILENAME: Final = "litellm-models.json"
_CODEX_BASE_INSTRUCTIONS_PATH: Final = Path(__file__).with_name("codex_base_instructions.md")
_CODEX_PREFLIGHT_TIMEOUT_SECONDS: Final = 10.0
class AgentRunError(Exception):
@ -399,9 +400,11 @@ class _CodexTruncationPolicy(BaseModel):
class _CodexModel(BaseModel):
"""One `ModelInfo` entry of a Codex model catalog.
Every field Codex's deserializer has no default for is spelled out here; the
values match the fallback metadata Codex uses today for a model slug it
does not know, so picking a proxy model behaves the same as `codex -m` did.
Every field that some Codex release since `model_catalog_json` appeared
(0.105.0) deserializes without a default is spelled out here, so one catalog
parses on all of them; the values match the fallback metadata Codex uses for
a model slug it does not know, so picking a proxy model behaves the same as
`codex -m` did.
"""
slug: str
@ -415,6 +418,8 @@ class _CodexModel(BaseModel):
availability_nux: None = None
upgrade: None = None
support_verbosity: Literal[False] = False
supports_reasoning_summaries: Literal[False] = False
supports_parallel_tool_calls: Literal[False] = False
default_verbosity: None = None
apply_patch_tool_type: None = None
truncation_policy: _CodexTruncationPolicy = _CodexTruncationPolicy()
@ -470,12 +475,48 @@ def _replace_file(path: Path, text: str) -> None:
raise
def _codex_catalog_rejection(
binary: str,
override: str,
env: Mapping[str, str],
*,
run: Callable[..., subprocess.CompletedProcess[str]],
) -> str | None:
"""Why the installed Codex refuses the catalog, or None once it reads the file back.
`codex debug models` parses the catalog the way a launch does, so a Codex
whose ModelInfo schema disagrees with the one written here fails now, with
the sync skipped, instead of exiting on startup. Releases before 0.130.0
have no `debug models` and fail the same way. A batch shim goes through
cmd.exe exactly as the launch will.
"""
name: Final = os.path.basename(binary)
command: Final = _windows_command(binary, (binary, "-c", override, "debug", "models"))
try:
completed: Final = run(
command,
env=dict(env),
stdin=subprocess.DEVNULL,
capture_output=True,
text=True,
timeout=_CODEX_PREFLIGHT_TIMEOUT_SECONDS,
)
except (OSError, subprocess.TimeoutExpired) as e:
return f"`{name} debug models` failed: {e}"
if completed.returncode == 0:
return None
lines: Final = completed.stderr.strip().splitlines()
return f"`{name} debug models` exited {completed.returncode}: {lines[0] if lines else 'no output'}"
def codex_model_sync_args(
base_env: Mapping[str, str],
base_url: str,
api_key: str,
*,
binary: str = "codex",
get: Callable[..., requests.Response] = requests.get,
run: Callable[..., subprocess.CompletedProcess[str]] = subprocess.run,
home: Callable[[], Path] = Path.home,
instructions_path: Path = _CODEX_BASE_INSTRUCTIONS_PATH,
) -> ModelSyncArgs | ModelSyncSkipped:
@ -483,9 +524,11 @@ def codex_model_sync_args(
Codex has no env or inline equivalent of OPENCODE_CONFIG_CONTENT: the catalog
must be a file, so it is written under $CODEX_HOME (default ~/.codex) and
atomically replaced on every launch. The key never lands in the file. A
failed fetch, read or write is reported rather than raised: Codex still
launches with its built-in catalog and takes a proxy model by name via -m.
atomically replaced on every launch, then read back once through the Codex
at `binary` before it is handed over. The key never lands in the file. A
failed fetch, read, write or read-back is reported rather than raised: Codex
still launches with its built-in catalog and takes a proxy model by name via
-m, and a rejected file stays on disk to be looked at.
"""
listing: Final = _fetch_model_listing(base_url, api_key, get=get)
if isinstance(listing, ModelSyncSkipped):
@ -502,32 +545,39 @@ def codex_model_sync_args(
_replace_file(path, catalog)
except OSError as e:
return ModelSyncSkipped(f"could not write {path}: {e}")
return ModelSyncArgs(("-c", f"model_catalog_json={json.dumps(str(path))}"))
override: Final = f"model_catalog_json={json.dumps(str(path))}"
rejection: Final = _codex_catalog_rejection(binary, override, base_env, run=run)
if rejection is not None:
return ModelSyncSkipped(rejection)
return ModelSyncArgs(("-c", override))
def agent_model_sync_env(
command: str,
binary: str,
base_env: Mapping[str, str],
base_url: str,
api_key: str,
skip_verify: bool,
*,
get: Callable[..., requests.Response] = requests.get,
run: Callable[..., subprocess.CompletedProcess[str]] = subprocess.run,
) -> ModelSyncResult:
"""Extra env or args an agent needs to see the proxy's model list.
OpenCode takes it as env, Codex as a `-c` override; Claude Code discovers
models itself through CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY.
skip_verify means the caller wants no pre-launch proxy call at all, so the
listing is skipped too rather than hanging on an offline proxy.
binary is the resolved path the launch will run (`codex.cmd` on a Windows
npm install). OpenCode takes the list as env, Codex as a `-c` override that
binary has read back first; Claude Code discovers models itself through
CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY. skip_verify means the caller
wants no pre-launch proxy call at all, so the listing is skipped too rather
than hanging on an offline proxy.
"""
agent: Final = os.path.basename(command)
agent: Final = os.path.splitext(os.path.basename(binary))[0]
if agent not in ("opencode", "codex"):
return _NO_EXTRA_ENV
if skip_verify:
return ModelSyncSkipped(f"{_SKIP_VERIFY_FLAG} was passed")
if agent == "codex":
return codex_model_sync_args(base_env, base_url, api_key, get=get)
return codex_model_sync_args(base_env, base_url, api_key, binary=binary, get=get, run=run)
return opencode_model_sync_env(base_env, base_url, api_key, get=get)
@ -682,7 +732,7 @@ def run_agent(
verify(base_url, api_key)
env_before_sync: Final = base_env if base_env is not None else os.environ
synced: Final = sync_models(command[0], env_before_sync, base_url, api_key, skip_verify)
synced: Final = sync_models(binary, env_before_sync, base_url, api_key, skip_verify)
if isinstance(synced, ModelSyncSkipped):
warn(f"litellm: not syncing {display_name} models from the proxy: {synced.reason}")

View file

@ -1,6 +1,7 @@
import inspect
import json
import os
import subprocess
import sys
from pathlib import Path
from unittest.mock import patch
@ -56,6 +57,17 @@ class _Recorder:
return self.returns
class _FakeRun:
def __init__(self, returncode=0, stderr=""):
self.returncode = returncode
self.stderr = stderr
self.calls = []
def __call__(self, args, **kwargs):
self.calls.append((args, kwargs))
return subprocess.CompletedProcess(args, self.returncode, "", self.stderr)
class _FakeJsonResponse:
def __init__(self, status_code, payload=None):
self.status_code = status_code
@ -365,7 +377,7 @@ class TestCodexModelSync:
def _row(model_id, **extra):
return {"id": model_id, "object": "model", "created": 1, "owned_by": "openai", **extra}
def _sync(self, listing, codex_home, base_url="http://localhost:4000/"):
def _sync(self, listing, codex_home, base_url="http://localhost:4000/", run=None):
captured = {}
def fake_get(url, headers, timeout):
@ -373,7 +385,13 @@ class TestCodexModelSync:
captured["headers"] = headers
return _FakeResponse(200, listing)
result = codex_model_sync_args({"CODEX_HOME": str(codex_home)}, base_url, "sk-key", get=fake_get)
result = codex_model_sync_args(
{"CODEX_HOME": str(codex_home)},
base_url,
"sk-key",
get=fake_get,
run=_FakeRun() if run is None else run,
)
return captured, result
@staticmethod
@ -411,6 +429,8 @@ class TestCodexModelSync:
assert entry["truncation_policy"] == {"mode": "bytes", "limit": 10000}
assert entry["experimental_supported_tools"] == []
assert entry["support_verbosity"] is False
assert entry["supports_reasoning_summaries"] is False
assert entry["supports_parallel_tool_calls"] is False
for nullable in ("description", "availability_nux", "upgrade", "default_verbosity", "apply_patch_tool_type"):
assert nullable in entry and entry[nullable] is None
assert entry["base_instructions"].startswith("You are a coding agent running in the Codex CLI")
@ -457,6 +477,7 @@ class TestCodexModelSync:
"http://localhost:4000",
"sk-key",
get=lambda *a, **k: _FakeResponse(200, self._listing(self._row("m"))),
run=_FakeRun(),
home=lambda: tmp_path,
)
assert self._catalog_path(result) == str(tmp_path / ".codex" / "litellm-models.json")
@ -510,17 +531,108 @@ class TestCodexModelSync:
assert isinstance(result, ModelSyncSkipped)
assert reason in result.reason
@pytest.mark.parametrize("command", ["codex", "/opt/bin/codex"])
def test_codex_syncs_through_the_agent_dispatch(self, tmp_path, command):
@pytest.mark.parametrize("binary", ["codex", "/opt/bin/codex", "codex.cmd", "/c/npm/codex.CMD"])
def test_codex_syncs_through_the_agent_dispatch_with_the_binary_it_will_run(self, tmp_path, binary):
run = _FakeRun()
result = agent_model_sync_env(
command,
binary,
{"CODEX_HOME": str(tmp_path)},
"http://localhost:4000",
"sk-key",
False,
get=lambda *a, **k: _FakeResponse(200, self._listing(self._row("m"))),
run=run,
)
assert self._catalog_path(result) == str(tmp_path / "litellm-models.json")
assert binary in run.calls[0][0]
def test_opencode_dispatch_never_runs_codex(self):
def boom(*a, **k):
raise AssertionError("only the Codex sync reads its catalog back")
result = agent_model_sync_env(
"opencode",
{},
"http://localhost:4000",
"sk-key",
False,
get=lambda *a, **k: _FakeResponse(200, self._listing(self._row("m"))),
run=boom,
)
assert "OPENCODE_CONFIG_CONTENT" in result
def test_catalog_is_read_back_through_codex_before_launch(self, tmp_path):
run = _FakeRun()
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=run)
path = self._catalog_path(result)
assert len(run.calls) == 1
command, options = run.calls[0]
assert command == ("codex", "-c", f"model_catalog_json={json.dumps(path)}", "debug", "models")
assert options["env"] == {"CODEX_HOME": str(tmp_path)}
assert options["stdin"] is subprocess.DEVNULL
assert options["capture_output"] is True
assert options["text"] is True
assert options["timeout"] == 10
def test_codex_rejecting_the_catalog_skips_the_sync_and_keeps_the_file(self, tmp_path):
stderr = (
"Error: failed to parse model_catalog_json path `/home/me/.codex/litellm-models.json` as JSON: "
"missing field `supports_parallel_tool_calls` at line 1 column 21648\n"
)
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=_FakeRun(1, stderr))
assert isinstance(result, ModelSyncSkipped)
assert result.reason == (
"`codex debug models` exited 1: Error: failed to parse model_catalog_json path "
"`/home/me/.codex/litellm-models.json` as JSON: missing field `supports_parallel_tool_calls` "
"at line 1 column 21648"
)
assert (tmp_path / "litellm-models.json").exists()
def test_codex_without_debug_models_skips_the_sync(self, tmp_path):
stderr = "error: unrecognized subcommand 'models'\n\nUsage: codex debug [OPTIONS] <COMMAND>\n"
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=_FakeRun(2, stderr))
assert isinstance(result, ModelSyncSkipped)
assert result.reason == "`codex debug models` exited 2: error: unrecognized subcommand 'models'"
def test_codex_failing_silently_is_reported(self, tmp_path):
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=_FakeRun(1))
assert isinstance(result, ModelSyncSkipped)
assert result.reason == "`codex debug models` exited 1: no output"
@pytest.mark.parametrize(
"error", [OSError("codex vanished"), subprocess.TimeoutExpired("codex", 10)], ids=["oserror", "timeout"]
)
def test_unrunnable_preflight_is_reported_not_raised(self, tmp_path, error):
def failing_run(*a, **k):
raise error
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=failing_run)
assert isinstance(result, ModelSyncSkipped)
assert result.reason.startswith("`codex debug models` failed: ")
assert str(error) in result.reason
def test_windows_shim_preflight_goes_through_cmd_exe(self, tmp_path):
shim = _WINDOWS_CLAUDE_CMD.replace("claude", "codex")
run = _FakeRun()
result = codex_model_sync_args(
{"CODEX_HOME": str(tmp_path)},
"http://localhost:4000",
"sk-key",
binary=shim,
get=lambda *a, **k: _FakeResponse(200, self._listing(self._row("m"))),
run=run,
)
override = f"model_catalog_json={json.dumps(self._catalog_path(result))}"
doubled = override.replace('"', '""')
assert run.calls[0][0] == f'{_CMD_PREFIX}""{shim}" "-c" "{doubled}" "debug" "models""'
def test_default_binary_is_codex_on_path(self):
assert _default_of(codex_model_sync_args, "binary") == "codex"
def test_default_runner_is_subprocess_run(self):
assert _default_of(codex_model_sync_args, "run") is subprocess.run
assert _default_of(agent_model_sync_env, "run") is subprocess.run
def test_skip_verify_keeps_the_launch_offline(self):
def boom(*a, **k):
@ -593,7 +705,13 @@ class TestRunAgent:
launcher=lambda p, a, e: order.append("launch"),
)
assert order == ["verify", "sync", "launch"]
assert calls["args"] == ("opencode", {"HOME": "/home/me"}, "http://localhost:4000", "sk-key", False)
assert calls["args"] == (
"/usr/local/bin/opencode",
{"HOME": "/home/me"},
"http://localhost:4000",
"sk-key",
False,
)
def test_unreachable_proxy_is_not_asked_for_models(self):
def failing_verify(*a):