mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-09 03:18:44 +00:00
3f5ed5a9c8
* refactor(e2e/claude_code): align proxy env names with the rest of tests/e2e
Every claude_code compat cell used to read its own `LITELLM_PROXY_BASE_URL` and `LITELLM_PROXY_API_KEY` and duplicate the same 12-line "missing env, hard fail" block. The rest of `tests/e2e/` reads `LITELLM_PROXY_URL` and `LITELLM_MASTER_KEY` from `e2e_config.py`, so anyone standing up a live proxy for one suite had to export a second spelling for claude_code, and every cell repeated the same boilerplate.
Centralize the resolution in `claude_code/_env.py`. `resolve_proxy()` prefers the suite-wide `LITELLM_PROXY_URL` / `LITELLM_MASTER_KEY` names and falls back to the legacy pair so existing CI wiring on stage keeps working during the roll-out. `require_proxy(compat_result)` is the one-liner cells call to bind `(base_url, api_key)` or hard-fail with a message that names both spellings.
55 cell files, `_basic_messaging.py`, and the driver's own unit-test fixture now go through the helper. `run_compat.sh` accepts either spelling and normalizes to the primary names before invoking pytest. `cron_vm/run_daily.sh` exports the primary names when launching pytest.
`_pr_gate_unit_tests/test_env_resolution.py` pins the resolution rules so a future edit cannot silently reintroduce the drift: primary names win on tie, legacy names still resolve when primary is unset, mixed URL-primary key-legacy still resolves, empty-string exports are treated as unset, `require_proxy` names both spellings in its error message.
Net diff: 71 files, +370/-1240.
* fix(e2e): anchor claude_code Bash pin at parents[1] so container run collects
`test_bash_tool_restrictions.py` derived `REPO_ROOT = Path(__file__).resolve().parents[4]` and then joined `tests/e2e/claude_code/<feature>`. That works locally, but the stage container mounts tests/e2e/ at /app/e2e/, so parents[4] resolves to filesystem root and the `_bash_cells()` assertion looks for `/tests/e2e/claude_code/tool_use` — a path that doesn't exist. Collection interrupts before any test runs, so the entire e2e suite appears broken.
Fix: `CLAUDE_CODE_DIR = Path(__file__).resolve().parents[1]` resolves to the sibling `claude_code/` dir in either layout, and the `relative_to(REPO_ROOT)` calls become `relative_to(CLAUDE_CODE_DIR)` so test IDs and error messages read the same.
Adds `test_claude_code_dir_anchor_is_layout_independent` as a regression pin: it checks the anchor lands on a directory named `claude_code` that contains this test file, which would fail under the old parents[4] anchor when run from /app/e2e/.
* feat(e2e/claude_code): register compat deployments via /model/new from a session fixture
Every compat cell hardcodes a virtual model name like `claude-sonnet-4-6` or `claude-sonnet-4-6-bedrock-invoke` and hits the proxy expecting it to be routable. On stage those live in the deployed model_list; locally the `docker-config.yaml` under tests/e2e/ only declares one of them, so anything past haiku 400s with `Invalid model name`.
`claude_code/test_config.yaml` is the ground-truth compat matrix config the deployment already uses. `_compat_models.py` loads it, normalizes the yaml keys pydantic would silently drop (vertex_ai_* → vertex_*), and selects the subset whose provider credentials are present in the environment. An autouse session fixture in `conftest.py` POSTs each selected deployment to `/model/new`, blocks until it is servable on the data plane, and tears them all down on session exit. Skips silently when the proxy env is unset so pure-unit runs stay hermetic.
`test_compat_models.py` pins the invariants that keep this safe. Every cell-referenced name must have a yaml entry (drift check catches a cell probing a name the fixture never registered); the yaml has no unused declarations; the fixture registers exactly 15 deployments (3 tiers × 5 provider surfaces); vertex_ai_* yaml keys populate the pydantic body's vertex_* fields (they got silently dropped historically); Azure needs both AZURE_FOUNDRY_* env vars; Bedrock lifts creds from the ambient AWS chain; Vertex needs both the yaml refs AND ambient GCP credentials.
* refactor(e2e/claude_code): inject env + runner instead of monkeypatching
`require_proxy` and `_basic_messaging.run_basic_messaging_cell` now take the env mapping (and the CLI runner) as constructor-style arguments with `os.environ` and `run_claude_models_parallel` as defaults. Tests exercise the branching by passing dicts and callables directly, so `monkeypatch.setenv` and `monkeypatch.setattr(_basic_messaging, "run_claude_models_parallel", ...)` are gone from every unit test in this refactor's blast radius.
`test_env_resolution.py` drops the `monkeypatch.setenv`/`delenv` fixtures and passes `env={...}` dicts to `require_proxy`. Added a new pinned check that a successful resolution leaves `compat_result` untouched, and split the "unset env" test into three explicit shapes (empty, primary-only, legacy-only) so a regression that swaps the precedence rule can no longer hide behind a single monkeypatched fixture.
`test_basic_messaging.py` (driver) replaces the `_install_fake_runner(monkeypatch, ...)` helper with `_make_fake_runner(...)` that returns a `(callable, captured_dict)` pair the test passes in via the helper's new `runner=` kwarg. Also drops the autouse `_proxy_env` fixture in favor of a module-level `_PROXY_ENV` dict each test wires through the helper's new `env=` kwarg. Added a regression pin that a missing-env call hard-fails without ever invoking the runner (so the guard order stays correct).
`test_run_daily_pytest_scrubs_env.py` updates its pin to assert the new suite-wide env spellings (`LITELLM_PROXY_URL` / `LITELLM_MASTER_KEY`) instead of the legacy `LITELLM_PROXY_BASE_URL` / `LITELLM_PROXY_API_KEY` that `run_daily.sh` used to export.
* handwrote rules
59 lines
2.4 KiB
Python
59 lines
2.4 KiB
Python
"""passthrough x Azure (Microsoft Foundry).
|
|
|
|
Drive the real `claude` CLI in foundry mode (CLAUDE_CODE_USE_FOUNDRY=1)
|
|
with ANTHROPIC_FOUNDRY_BASE_URL aimed at the proxy's `/azure`
|
|
passthrough route. The CLI POSTs `/v1/messages` with the model in the
|
|
JSON body -- unlike the bedrock/vertex modes there is no model segment
|
|
in the URL, so the proxy's router-alias resolution cannot engage and
|
|
the `/azure` route falls back to its env-configured target: the proxy
|
|
must set AZURE_API_BASE to the Foundry resource's Anthropic surface
|
|
(`https://<resource>.services.ai.azure.com/anthropic`) and
|
|
AZURE_API_KEY to the Foundry key (see
|
|
cron_vm/litellm-compat-matrix.env.example). The model ids are the
|
|
Foundry deployment names, which this matrix provisions to match the
|
|
Anthropic ids.
|
|
|
|
The (feature, provider) for this cell is inferred from the file path by
|
|
`tests/e2e/claude_code/conftest.py`:
|
|
|
|
tests/e2e/claude_code/passthrough/test_azure.py
|
|
^^^^^^^^^^^ ^^^^^
|
|
feature_id provider
|
|
|
|
Wiring verified live at authoring time: through `{proxy}/azure` the
|
|
Foundry Anthropic surface accepted the `api-key` / `Authorization:
|
|
Bearer` headers the fallback sends (a bogus key 401s, the real key
|
|
proceeds to deployment lookup), so a red cell here means missing
|
|
AZURE_API_BASE/AZURE_API_KEY on the proxy, missing Foundry deployments
|
|
for the three tiers, or a genuine forwarding gap -- not an auth-scheme
|
|
mismatch.
|
|
|
|
Known-red at authoring time against a healthy Foundry resource: the
|
|
`/azure` fallback assembles only its own auth headers and drops the
|
|
rest of the client's headers, including the `anthropic-version` header
|
|
the CLI sends, and Foundry's Anthropic surface rejects the request
|
|
with 400 "anthropic-version: header is required" (the same request
|
|
sent directly to Foundry with that header succeeds). This cell stays
|
|
red until that forwarding gap is fixed, which is precisely the class
|
|
of bug the row exists to surface.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from claude_code._passthrough import foundry_extra_env, run_passthrough_cell
|
|
|
|
AZURE_MODELS = [
|
|
"claude-haiku-4-5",
|
|
"claude-sonnet-4-5",
|
|
"claude-opus-4-7",
|
|
]
|
|
|
|
|
|
def test_passthrough_azure(compat_result):
|
|
"""Drive the `claude` CLI through `{proxy}/azure` and assert a reply."""
|
|
run_passthrough_cell(
|
|
compat_result=compat_result,
|
|
models=AZURE_MODELS,
|
|
prompt="Reply with the single word 'pong' and nothing else.",
|
|
build_extra_env=foundry_extra_env,
|
|
)
|