mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
feat(cli): keep the installed Codex's entries for proxy models it already knows
`lite codex` now asks the installed Codex for its own model list through `codex debug models` before writing the catalog. A proxy model whose id matches a stock Codex slug keeps that Codex's entry (reasoning levels, base instructions, context window and the rest) and only its picker position, visibility and upgrade nudge come from the proxy. Unknown slugs still get the plain entry built from the bundled base instructions. The catalog directory is created before the stock call so a fresh CODEX_HOME does not make Codex refuse to run
This commit is contained in:
mateo-berri
parent
222f283c93
commit
5b9153f5ea
3 changed files with 263 additions and 60 deletions
|
|
@ -490,7 +490,7 @@ lite codex exec "summarize the repo"
|
|||
|
||||
Each command resolves your LiteLLM key (logging in via SSO when none is stored and you are at a terminal; otherwise it expects `LITELLM_PROXY_API_KEY` or `--api-key`), checks the key against the proxy so bad credentials fail immediately instead of deep inside the agent, exports the environment variables the agent reads, then replaces itself with the agent process.
|
||||
|
||||
The right variables are picked per agent. Claude Code gets `ANTHROPIC_BASE_URL` (the proxy root, so it appends `/v1/messages`) and `ANTHROPIC_AUTH_TOKEN`, with any stray `ANTHROPIC_API_KEY` cleared so the proxy token wins, and `ENABLE_TOOL_SEARCH=true` (unless you already set it) so Claude Code keeps tool search on even though the base URL is a proxy rather than a first-party Anthropic host. It also gets `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` (again unless you already set it) so Claude Code v2.1.129+ fills its `/model` picker from the proxy's `/v1/models`; Claude Code only lists entries whose id contains `claude` or `anthropic`, so the proxy lists every other group to Claude Code as `claude-router-<UTF-8 hex of the group name>` and marks a group whose input window reaches 1M with `[1m]`, and a request on such an id is served by the group. Older Claude Code versions ignore the variable. Export it as `0` to turn discovery off. Codex and OpenCode get `OPENAI_BASE_URL` (the proxy plus `/v1`) and `OPENAI_API_KEY`. Codex ignores `OPENAI_BASE_URL`, so it is additionally pointed at the proxy through a custom provider passed as `-c` config overrides (HTTP/SSE Responses transport, since the proxy does not speak the Responses WebSocket protocol). OpenCode additionally gets `OPENCODE_CONFIG_CONTENT` holding a generated `litellm` provider (`@ai-sdk/openai-compatible`, the proxy `/v1` URL, `{env:OPENAI_API_KEY}`) with one model entry per chat model your key can see on `/v1/models`, so its model picker mirrors the proxy without a hand-maintained `opencode.json`; OpenCode merges that over your own config files, and if you already export `OPENCODE_CONFIG_CONTENT` yours is left alone. When the list cannot be fetched, `lite opencode` says so on stderr and launches anyway. Codex gets the same list as a catalog file, `$CODEX_HOME/litellm-models.json` (default `~/.codex/`), passed as `-c model_catalog_json=<path>` so `/model` lists exactly the proxy's chat models; before launching, `lite codex` has the installed Codex read that file back (`codex debug models`), and when the fetch, the write or that read-back fails (Codex releases older than 0.130 have no such command) it says so on stderr and launches with Codex's built-in catalog, leaving the rejected file in place.
|
||||
The right variables are picked per agent. Claude Code gets `ANTHROPIC_BASE_URL` (the proxy root, so it appends `/v1/messages`) and `ANTHROPIC_AUTH_TOKEN`, with any stray `ANTHROPIC_API_KEY` cleared so the proxy token wins, and `ENABLE_TOOL_SEARCH=true` (unless you already set it) so Claude Code keeps tool search on even though the base URL is a proxy rather than a first-party Anthropic host. It also gets `CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` (again unless you already set it) so Claude Code v2.1.129+ fills its `/model` picker from the proxy's `/v1/models`; Claude Code only lists entries whose id contains `claude` or `anthropic`, so the proxy lists every other group to Claude Code as `claude-router-<UTF-8 hex of the group name>` and marks a group whose input window reaches 1M with `[1m]`, and a request on such an id is served by the group. Older Claude Code versions ignore the variable. Export it as `0` to turn discovery off. Codex and OpenCode get `OPENAI_BASE_URL` (the proxy plus `/v1`) and `OPENAI_API_KEY`. Codex ignores `OPENAI_BASE_URL`, so it is additionally pointed at the proxy through a custom provider passed as `-c` config overrides (HTTP/SSE Responses transport, since the proxy does not speak the Responses WebSocket protocol). OpenCode additionally gets `OPENCODE_CONFIG_CONTENT` holding a generated `litellm` provider (`@ai-sdk/openai-compatible`, the proxy `/v1` URL, `{env:OPENAI_API_KEY}`) with one model entry per chat model your key can see on `/v1/models`, so its model picker mirrors the proxy without a hand-maintained `opencode.json`; OpenCode merges that over your own config files, and if you already export `OPENCODE_CONFIG_CONTENT` yours is left alone. When the list cannot be fetched, `lite opencode` says so on stderr and launches anyway. Codex gets the same list as a catalog file, `$CODEX_HOME/litellm-models.json` (default `~/.codex/`), passed as `-c model_catalog_json=<path>` so `/model` lists exactly the proxy's chat models; a proxy model the installed Codex already knows (`gpt-5.5`, say) keeps that Codex's own entry, reasoning levels and prompt included, and only its place in the picker comes from the proxy, while a model Codex does not know gets the plain entry Codex uses for an unknown `-m` slug. Before launching, `lite codex` asks the installed Codex for its own list and then has it read the written file back (both through `codex debug models`), and when the fetch, either of those or the write fails (Codex releases older than 0.130 have no such command) it says so on stderr and launches with Codex's built-in catalog, leaving a rejected file in place.
|
||||
|
||||
pi ignores base-URL environment variables entirely, so `lite pi` (kept out of the `lite --help` command listing for now, but fully functional) wires it up differently: before handoff it fetches the models your key can use from the proxy's `/v1/models` (plus each model's context window and output cap from `/model_group/info`, when available) and syncs them into a `litellm` provider entry in pi's `~/.pi/agent/models.json` (honoring `PI_CODING_AGENT_DIR`), then starts pi on that provider's first model via an injected `--model litellm/<id>`. Only that one provider entry is rewritten; the rest of the file, including any other custom providers, is left alone. The entry references the key as `$LITELLM_PROXY_API_KEY`, which the wrapper exports for the session, so the token itself never lands on disk and plain `pi` outside the wrapper simply shows the litellm models as unavailable. Your own flags come after the injected pin, so `lite pi --model litellm/<other-id>` wins, and inside the TUI the `/model` picker lists every synced litellm model.
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ from typing import Final, Literal, TypeAlias
|
|||
|
||||
import click
|
||||
import requests
|
||||
from pydantic import BaseModel, TypeAdapter, ValidationError
|
||||
from pydantic import BaseModel, ConfigDict, TypeAdapter, ValidationError
|
||||
|
||||
from .auth import CliContextObj, context_secret_vault, get_stored_api_key, login
|
||||
from .claude_settings import ClaudeSettingsError, install_statusline_script
|
||||
|
|
@ -398,13 +398,13 @@ class _CodexTruncationPolicy(BaseModel):
|
|||
|
||||
|
||||
class _CodexModel(BaseModel):
|
||||
"""One `ModelInfo` entry of a Codex model catalog.
|
||||
"""One `ModelInfo` entry of a Codex model catalog for a model the installed Codex does not know.
|
||||
|
||||
Every field that some Codex release since `model_catalog_json` appeared
|
||||
(0.105.0) deserializes without a default is spelled out here, so one catalog
|
||||
parses on all of them; the values match the fallback metadata Codex uses for
|
||||
a model slug it does not know, so picking a proxy model behaves the same as
|
||||
`codex -m` did.
|
||||
a model slug it does not know, so picking such a proxy model behaves the
|
||||
same as `codex -m` did.
|
||||
"""
|
||||
|
||||
slug: str
|
||||
|
|
@ -428,31 +428,77 @@ class _CodexModel(BaseModel):
|
|||
base_instructions: str
|
||||
|
||||
|
||||
class _StockCodexUpgrade(BaseModel):
|
||||
model_config = ConfigDict(extra="allow")
|
||||
|
||||
model: str
|
||||
|
||||
|
||||
class _StockCodexModel(BaseModel):
|
||||
"""One `ModelInfo` entry as the installed Codex prints it from `codex debug models`.
|
||||
|
||||
Only the fields the sync rewrites are named; everything else that release
|
||||
knows about the model (its reasoning levels, prompt, tool support) rides
|
||||
along untouched, whatever the release's schema.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="allow")
|
||||
|
||||
slug: str
|
||||
priority: int
|
||||
visibility: str
|
||||
upgrade: _StockCodexUpgrade | None = None
|
||||
|
||||
|
||||
class _StockCodexCatalog(BaseModel):
|
||||
models: tuple[_StockCodexModel, ...]
|
||||
|
||||
|
||||
class _CodexCatalog(BaseModel):
|
||||
models: tuple[_CodexModel, ...]
|
||||
models: tuple[_CodexModel | _StockCodexModel, ...]
|
||||
|
||||
|
||||
def codex_model_catalog(models: Sequence[ListedModel], instructions: str) -> str | None:
|
||||
def _codex_catalog_entry(
|
||||
priority: int,
|
||||
listed: ListedModel,
|
||||
stock: _StockCodexModel | None,
|
||||
served: frozenset[str],
|
||||
instructions: str,
|
||||
) -> _CodexModel | _StockCodexModel:
|
||||
if stock is None:
|
||||
return _CodexModel(
|
||||
slug=listed.id,
|
||||
display_name=listed.id,
|
||||
priority=priority,
|
||||
context_window=listed.max_input_tokens,
|
||||
base_instructions=instructions,
|
||||
)
|
||||
upgrade: Final = stock.upgrade if stock.upgrade is not None and stock.upgrade.model in served else None
|
||||
return stock.model_copy(update={"priority": priority, "visibility": "list", "upgrade": upgrade})
|
||||
|
||||
|
||||
def codex_model_catalog(
|
||||
models: Sequence[ListedModel], stock: Sequence[_StockCodexModel], instructions: str
|
||||
) -> str | None:
|
||||
"""The `model_catalog_json` body listing the proxy's chat models, or None if there are none.
|
||||
|
||||
Codex refuses an empty catalog, hence None instead of `{"models": []}`.
|
||||
Passing a catalog replaces Codex's built-in one, so every entry carries the
|
||||
same base instructions Codex itself uses, otherwise the agent would run
|
||||
without a system prompt.
|
||||
Passing a catalog replaces Codex's built-in one, so a proxy model the
|
||||
installed Codex knows keeps that Codex's own entry and the proxy only
|
||||
decides its place in the picker: the listing orders it, lists it even when
|
||||
Codex hides it, and keeps Codex's upgrade nudge only when the model it
|
||||
points at is served too. A model Codex does not know gets the fallback
|
||||
entry, with the same base instructions Codex itself uses so the agent never
|
||||
runs without a system prompt.
|
||||
"""
|
||||
chat_models: Final = _chat_models(models)
|
||||
if not chat_models:
|
||||
return None
|
||||
served: Final = frozenset(m.id for m in chat_models)
|
||||
known: Final = MappingProxyType({m.slug: m for m in stock})
|
||||
catalog: Final = _CodexCatalog(
|
||||
models=tuple(
|
||||
_CodexModel(
|
||||
slug=m.id,
|
||||
display_name=m.id,
|
||||
priority=index,
|
||||
context_window=m.max_input_tokens,
|
||||
base_instructions=instructions,
|
||||
)
|
||||
for index, m in enumerate(chat_models)
|
||||
_codex_catalog_entry(index, m, known.get(m.id), served, instructions) for index, m in enumerate(chat_models)
|
||||
)
|
||||
)
|
||||
return catalog.model_dump_json()
|
||||
|
|
@ -465,7 +511,6 @@ def codex_model_catalog_path(env: Mapping[str, str], *, home: Callable[[], Path]
|
|||
|
||||
|
||||
def _replace_file(path: Path, text: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.NamedTemporaryFile("w", encoding="utf-8", dir=path.parent, delete=False) as tmp:
|
||||
_ = tmp.write(text)
|
||||
try:
|
||||
|
|
@ -475,23 +520,23 @@ def _replace_file(path: Path, text: str) -> None:
|
|||
raise
|
||||
|
||||
|
||||
def _codex_catalog_rejection(
|
||||
def _codex_debug_models(
|
||||
binary: str,
|
||||
override: str,
|
||||
args: Sequence[str],
|
||||
env: Mapping[str, str],
|
||||
*,
|
||||
run: Callable[..., subprocess.CompletedProcess[str]],
|
||||
) -> str | None:
|
||||
"""Why the installed Codex refuses the catalog, or None once it reads the file back.
|
||||
) -> str | ModelSyncSkipped:
|
||||
"""What `codex debug models` prints with `args` in front, or why the installed Codex could not run it.
|
||||
|
||||
`codex debug models` parses the catalog the way a launch does, so a Codex
|
||||
whose ModelInfo schema disagrees with the one written here fails now, with
|
||||
the sync skipped, instead of exiting on startup. Releases before 0.130.0
|
||||
have no `debug models` and fail the same way. A batch shim goes through
|
||||
The command prints the catalog Codex would launch with, without touching
|
||||
the network, so it lists the installed Codex's own models and parses a
|
||||
catalog override the way a launch does. Releases before 0.130.0 have no
|
||||
such command and are reported the same way. A batch shim goes through
|
||||
cmd.exe exactly as the launch will.
|
||||
"""
|
||||
name: Final = os.path.basename(binary)
|
||||
command: Final = _windows_command(binary, (binary, "-c", override, "debug", "models"))
|
||||
command: Final = _windows_command(binary, (binary, *args, "debug", "models"))
|
||||
try:
|
||||
completed: Final = run(
|
||||
command,
|
||||
|
|
@ -502,11 +547,25 @@ def _codex_catalog_rejection(
|
|||
timeout=_CODEX_PREFLIGHT_TIMEOUT_SECONDS,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired) as e:
|
||||
return f"`{name} debug models` failed: {e}"
|
||||
return ModelSyncSkipped(f"`{name} debug models` failed: {e}")
|
||||
if completed.returncode == 0:
|
||||
return None
|
||||
return completed.stdout
|
||||
lines: Final = completed.stderr.strip().splitlines()
|
||||
return f"`{name} debug models` exited {completed.returncode}: {lines[0] if lines else 'no output'}"
|
||||
detail: Final = lines[0] if lines else "no output"
|
||||
return ModelSyncSkipped(f"`{name} debug models` exited {completed.returncode}: {detail}")
|
||||
|
||||
|
||||
def _stock_codex_models(
|
||||
binary: str, env: Mapping[str, str], *, run: Callable[..., subprocess.CompletedProcess[str]]
|
||||
) -> tuple[_StockCodexModel, ...] | ModelSyncSkipped:
|
||||
printed: Final = _codex_debug_models(binary, (), env, run=run)
|
||||
if isinstance(printed, ModelSyncSkipped):
|
||||
return printed
|
||||
try:
|
||||
return _StockCodexCatalog.model_validate_json(printed).models
|
||||
except ValidationError as e:
|
||||
name: Final = os.path.basename(binary)
|
||||
return ModelSyncSkipped(f"`{name} debug models` printed no model catalog: {e.errors()[0]['msg']}")
|
||||
|
||||
|
||||
def codex_model_sync_args(
|
||||
|
|
@ -524,11 +583,13 @@ def codex_model_sync_args(
|
|||
|
||||
Codex has no env or inline equivalent of OPENCODE_CONFIG_CONTENT: the catalog
|
||||
must be a file, so it is written under $CODEX_HOME (default ~/.codex) and
|
||||
atomically replaced on every launch, then read back once through the Codex
|
||||
at `binary` before it is handed over. The key never lands in the file. A
|
||||
failed fetch, read, write or read-back is reported rather than raised: Codex
|
||||
still launches with its built-in catalog and takes a proxy model by name via
|
||||
-m, and a rejected file stays on disk to be looked at.
|
||||
atomically replaced on every launch. The Codex at `binary` first lists its
|
||||
own models, so the ones the proxy serves keep that Codex's entries, and then
|
||||
reads the file back once before it is handed over. The key never lands in
|
||||
the file. A failed fetch, read, listing, write or read-back is reported
|
||||
rather than raised: Codex still launches with its built-in catalog and takes
|
||||
a proxy model by name via -m, and a rejected file stays on disk to be looked
|
||||
at.
|
||||
"""
|
||||
listing: Final = _fetch_model_listing(base_url, api_key, get=get)
|
||||
if isinstance(listing, ModelSyncSkipped):
|
||||
|
|
@ -537,18 +598,25 @@ def codex_model_sync_args(
|
|||
instructions: Final = instructions_path.read_text(encoding="utf-8")
|
||||
except OSError as e:
|
||||
return ModelSyncSkipped(f"could not read {instructions_path}: {e}")
|
||||
catalog: Final = codex_model_catalog(listing, instructions)
|
||||
path: Final = codex_model_catalog_path(base_env, home=home)
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
except OSError as e:
|
||||
return ModelSyncSkipped(f"could not write {path}: {e}")
|
||||
stock: Final = _stock_codex_models(binary, base_env, run=run)
|
||||
if isinstance(stock, ModelSyncSkipped):
|
||||
return stock
|
||||
catalog: Final = codex_model_catalog(listing, stock, instructions)
|
||||
if catalog is None:
|
||||
return ModelSyncSkipped(f"{base_url.rstrip('/')}/v1/models lists no chat models")
|
||||
path: Final = codex_model_catalog_path(base_env, home=home)
|
||||
try:
|
||||
_replace_file(path, catalog)
|
||||
except OSError as e:
|
||||
return ModelSyncSkipped(f"could not write {path}: {e}")
|
||||
override: Final = f"model_catalog_json={json.dumps(str(path))}"
|
||||
rejection: Final = _codex_catalog_rejection(binary, override, base_env, run=run)
|
||||
if rejection is not None:
|
||||
return ModelSyncSkipped(rejection)
|
||||
read_back: Final = _codex_debug_models(binary, ("-c", override), base_env, run=run)
|
||||
if isinstance(read_back, ModelSyncSkipped):
|
||||
return read_back
|
||||
return ModelSyncArgs(("-c", override))
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -57,14 +57,109 @@ class _Recorder:
|
|||
return self.returns
|
||||
|
||||
|
||||
_STOCK_REASONING_LEVELS = [
|
||||
{"effort": "low", "description": "Fast responses with lighter reasoning"},
|
||||
{"effort": "medium", "description": "Balances speed and reasoning depth for everyday tasks"},
|
||||
{"effort": "high", "description": "Greater reasoning depth for complex problems"},
|
||||
]
|
||||
|
||||
_STOCK_MODELS = {
|
||||
"gpt-5.6-terra": {
|
||||
"slug": "gpt-5.6-terra",
|
||||
"display_name": "GPT-5.6 Terra",
|
||||
"description": "Balanced agentic coding model for everyday work.",
|
||||
"default_reasoning_level": "medium",
|
||||
"supported_reasoning_levels": _STOCK_REASONING_LEVELS,
|
||||
"shell_type": "unified_exec",
|
||||
"visibility": "list",
|
||||
"supported_in_api": True,
|
||||
"priority": 7,
|
||||
"availability_nux": None,
|
||||
"upgrade": None,
|
||||
"base_instructions": "You are Codex, a coding agent based on GPT-5.6.",
|
||||
"apply_patch_tool_type": "freeform",
|
||||
"supports_parallel_tool_calls": True,
|
||||
"context_window": 272000,
|
||||
"comp_hash": "terra-hash",
|
||||
},
|
||||
"gpt-5.5": {
|
||||
"slug": "gpt-5.5",
|
||||
"display_name": "GPT-5.5",
|
||||
"description": "Frontier model for complex coding, research, and real-world work.",
|
||||
"default_reasoning_level": "medium",
|
||||
"supported_reasoning_levels": _STOCK_REASONING_LEVELS,
|
||||
"shell_type": "unified_exec",
|
||||
"visibility": "list",
|
||||
"supported_in_api": True,
|
||||
"priority": 12,
|
||||
"availability_nux": None,
|
||||
"upgrade": None,
|
||||
"base_instructions": "You are Codex, a coding agent based on GPT-5.",
|
||||
"apply_patch_tool_type": "freeform",
|
||||
"supports_parallel_tool_calls": True,
|
||||
"context_window": 272000,
|
||||
"comp_hash": "gpt-5.5-hash",
|
||||
},
|
||||
"gpt-5.4": {
|
||||
"slug": "gpt-5.4",
|
||||
"display_name": "GPT-5.4",
|
||||
"description": "Strong model for everyday coding.",
|
||||
"default_reasoning_level": "medium",
|
||||
"supported_reasoning_levels": _STOCK_REASONING_LEVELS,
|
||||
"shell_type": "unified_exec",
|
||||
"visibility": "hide",
|
||||
"supported_in_api": True,
|
||||
"priority": 16,
|
||||
"availability_nux": None,
|
||||
"upgrade": {
|
||||
"model": "gpt-5.6-terra",
|
||||
"migration_markdown": "GPT-5.4 is no longer available. Switch to GPT-5.6 Terra to continue.",
|
||||
"retirement_at": "2026-08-31T19:00:00Z",
|
||||
},
|
||||
"base_instructions": "You are Codex, a coding agent based on GPT-5.",
|
||||
"apply_patch_tool_type": "freeform",
|
||||
"supports_parallel_tool_calls": True,
|
||||
"context_window": 272000,
|
||||
"comp_hash": "gpt-5.4-hash",
|
||||
},
|
||||
"codex-auto-review": {
|
||||
"slug": "codex-auto-review",
|
||||
"display_name": "Codex Auto Review",
|
||||
"description": None,
|
||||
"supported_reasoning_levels": [],
|
||||
"shell_type": "unified_exec",
|
||||
"visibility": "hide",
|
||||
"supported_in_api": False,
|
||||
"priority": 43,
|
||||
"availability_nux": None,
|
||||
"upgrade": None,
|
||||
"base_instructions": "You are Codex, reviewing a change.",
|
||||
"apply_patch_tool_type": None,
|
||||
"supports_parallel_tool_calls": True,
|
||||
"context_window": 272000,
|
||||
"comp_hash": "review-hash",
|
||||
},
|
||||
}
|
||||
|
||||
_STOCK_CATALOG = json.dumps({"models": list(_STOCK_MODELS.values())})
|
||||
|
||||
|
||||
class _FakeRun:
|
||||
def __init__(self, returncode=0, stderr=""):
|
||||
"""A `codex` that prints `stock` from a bare `debug models` and answers a catalog override with `returncode`.
|
||||
|
||||
`stock=None` is a Codex with no `debug models` at all: every call answers with `returncode` and `stderr`.
|
||||
"""
|
||||
|
||||
def __init__(self, returncode=0, stderr="", stock=_STOCK_CATALOG):
|
||||
self.returncode = returncode
|
||||
self.stderr = stderr
|
||||
self.stock = stock
|
||||
self.calls = []
|
||||
|
||||
def __call__(self, args, **kwargs):
|
||||
self.calls.append((args, kwargs))
|
||||
if self.stock is not None and "model_catalog_json=" not in str(args):
|
||||
return subprocess.CompletedProcess(args, 0, self.stock, "")
|
||||
return subprocess.CompletedProcess(args, self.returncode, "", self.stderr)
|
||||
|
||||
|
||||
|
|
@ -415,10 +510,36 @@ class TestCodexModelSync:
|
|||
assert "sk-key" not in text
|
||||
catalog = json.loads(text)
|
||||
assert [m["slug"] for m in catalog["models"]] == ["gpt-5.5", "claude-opus-4-7"]
|
||||
assert [m["display_name"] for m in catalog["models"]] == ["gpt-5.5", "claude-opus-4-7"]
|
||||
assert [m["display_name"] for m in catalog["models"]] == ["GPT-5.5", "claude-opus-4-7"]
|
||||
assert [m["priority"] for m in catalog["models"]] == [0, 1]
|
||||
|
||||
def test_every_entry_has_the_fields_codex_requires(self, tmp_path):
|
||||
def _entries(self, codex_home):
|
||||
return {m["slug"]: m for m in json.loads((codex_home / "litellm-models.json").read_text())["models"]}
|
||||
|
||||
def test_known_model_keeps_the_installed_codex_entry(self, tmp_path):
|
||||
self._sync(self._listing(self._row("gpt-5.5", mode="chat")), tmp_path)
|
||||
assert self._entries(tmp_path)["gpt-5.5"] == {**_STOCK_MODELS["gpt-5.5"], "priority": 0}
|
||||
|
||||
def test_hidden_stock_model_is_listed_when_the_proxy_serves_it(self, tmp_path):
|
||||
self._sync(self._listing(self._row("gpt-5.4")), tmp_path)
|
||||
entry = self._entries(tmp_path)["gpt-5.4"]
|
||||
assert entry["visibility"] == "list"
|
||||
assert entry["upgrade"] is None
|
||||
assert entry["supported_reasoning_levels"] == _STOCK_REASONING_LEVELS
|
||||
|
||||
def test_stock_upgrade_nudge_survives_when_its_target_is_listed(self, tmp_path):
|
||||
self._sync(self._listing(self._row("gpt-5.4"), self._row("gpt-5.6-terra")), tmp_path)
|
||||
entries = self._entries(tmp_path)
|
||||
assert entries["gpt-5.4"]["upgrade"] == _STOCK_MODELS["gpt-5.4"]["upgrade"]
|
||||
assert [entries["gpt-5.4"]["priority"], entries["gpt-5.6-terra"]["priority"]] == [0, 1]
|
||||
|
||||
def test_unparseable_stock_catalog_is_reported(self, tmp_path):
|
||||
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=_FakeRun(stock="not json"))
|
||||
assert isinstance(result, ModelSyncSkipped)
|
||||
assert result.reason.startswith("`codex debug models` printed no model catalog: ")
|
||||
assert not (tmp_path / "litellm-models.json").exists()
|
||||
|
||||
def test_unknown_model_gets_the_fields_codex_requires(self, tmp_path):
|
||||
_, result = self._sync(self._listing(self._row("m")), tmp_path)
|
||||
entry = json.loads((tmp_path / "litellm-models.json").read_text())["models"][0]
|
||||
|
||||
|
|
@ -435,12 +556,17 @@ class TestCodexModelSync:
|
|||
assert nullable in entry and entry[nullable] is None
|
||||
assert entry["base_instructions"].startswith("You are a coding agent running in the Codex CLI")
|
||||
|
||||
def test_context_window_comes_from_max_input_tokens(self, tmp_path):
|
||||
listing = self._listing(self._row("big", max_input_tokens=400000), self._row("unknown"))
|
||||
_, result = self._sync(listing, tmp_path)
|
||||
models = {m["slug"]: m for m in json.loads((tmp_path / "litellm-models.json").read_text())["models"]}
|
||||
def test_context_window_comes_from_max_input_tokens_for_unknown_models_only(self, tmp_path):
|
||||
listing = self._listing(
|
||||
self._row("big", max_input_tokens=400000),
|
||||
self._row("unknown"),
|
||||
self._row("gpt-5.5", max_input_tokens=400000),
|
||||
)
|
||||
self._sync(listing, tmp_path)
|
||||
models = self._entries(tmp_path)
|
||||
assert models["big"]["context_window"] == 400000
|
||||
assert models["unknown"]["context_window"] is None
|
||||
assert models["gpt-5.5"]["context_window"] == 272000
|
||||
|
||||
def test_non_chat_models_are_left_out(self, tmp_path):
|
||||
listing = self._listing(
|
||||
|
|
@ -544,7 +670,8 @@ class TestCodexModelSync:
|
|||
run=run,
|
||||
)
|
||||
assert self._catalog_path(result) == str(tmp_path / "litellm-models.json")
|
||||
assert binary in run.calls[0][0]
|
||||
assert len(run.calls) == 2
|
||||
assert all(binary in command for command, _ in run.calls)
|
||||
|
||||
def test_opencode_dispatch_never_runs_codex(self):
|
||||
def boom(*a, **k):
|
||||
|
|
@ -561,19 +688,21 @@ class TestCodexModelSync:
|
|||
)
|
||||
assert "OPENCODE_CONFIG_CONTENT" in result
|
||||
|
||||
def test_catalog_is_read_back_through_codex_before_launch(self, tmp_path):
|
||||
def test_codex_lists_its_own_models_then_reads_the_catalog_back_before_launch(self, tmp_path):
|
||||
run = _FakeRun()
|
||||
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=run)
|
||||
path = self._catalog_path(result)
|
||||
|
||||
assert len(run.calls) == 1
|
||||
command, options = run.calls[0]
|
||||
assert command == ("codex", "-c", f"model_catalog_json={json.dumps(path)}", "debug", "models")
|
||||
assert options["env"] == {"CODEX_HOME": str(tmp_path)}
|
||||
assert options["stdin"] is subprocess.DEVNULL
|
||||
assert options["capture_output"] is True
|
||||
assert options["text"] is True
|
||||
assert options["timeout"] == 10
|
||||
assert [command for command, _ in run.calls] == [
|
||||
("codex", "debug", "models"),
|
||||
("codex", "-c", f"model_catalog_json={json.dumps(path)}", "debug", "models"),
|
||||
]
|
||||
for _, options in run.calls:
|
||||
assert options["env"] == {"CODEX_HOME": str(tmp_path)}
|
||||
assert options["stdin"] is subprocess.DEVNULL
|
||||
assert options["capture_output"] is True
|
||||
assert options["text"] is True
|
||||
assert options["timeout"] == 10
|
||||
|
||||
def test_codex_rejecting_the_catalog_skips_the_sync_and_keeps_the_file(self, tmp_path):
|
||||
stderr = (
|
||||
|
|
@ -591,9 +720,12 @@ class TestCodexModelSync:
|
|||
|
||||
def test_codex_without_debug_models_skips_the_sync(self, tmp_path):
|
||||
stderr = "error: unrecognized subcommand 'models'\n\nUsage: codex debug [OPTIONS] <COMMAND>\n"
|
||||
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=_FakeRun(2, stderr))
|
||||
run = _FakeRun(2, stderr, stock=None)
|
||||
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=run)
|
||||
assert isinstance(result, ModelSyncSkipped)
|
||||
assert result.reason == "`codex debug models` exited 2: error: unrecognized subcommand 'models'"
|
||||
assert len(run.calls) == 1
|
||||
assert not (tmp_path / "litellm-models.json").exists()
|
||||
|
||||
def test_codex_failing_silently_is_reported(self, tmp_path):
|
||||
_, result = self._sync(self._listing(self._row("m")), tmp_path, run=_FakeRun(1))
|
||||
|
|
@ -625,7 +757,10 @@ class TestCodexModelSync:
|
|||
)
|
||||
override = f"model_catalog_json={json.dumps(self._catalog_path(result))}"
|
||||
doubled = override.replace('"', '""')
|
||||
assert run.calls[0][0] == f'{_CMD_PREFIX}""{shim}" "-c" "{doubled}" "debug" "models""'
|
||||
assert [command for command, _ in run.calls] == [
|
||||
f'{_CMD_PREFIX}""{shim}" "debug" "models""',
|
||||
f'{_CMD_PREFIX}""{shim}" "-c" "{doubled}" "debug" "models""',
|
||||
]
|
||||
|
||||
def test_default_binary_is_codex_on_path(self):
|
||||
assert _default_of(codex_model_sync_args, "binary") == "codex"
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue