fix(cron_vm): greptile — scrub provider secrets from pytest invocation env

Wrap the pytest call in run_daily.sh in `env -i` with the same minimal
allowlist the PR-gate already uses, mirroring the CircleCI step. The
systemd EnvironmentFile injects ANTHROPIC_API_KEY /
AWS_BEARER_TOKEN_BEDROCK / VERTEXAI_* / AZURE_FOUNDRY_* /
AGENT_SHIN_GITHUB_TOKEN / GITHUB_TOKEN into the script for the proxy to
consume; pytest inherits them by default but only needs the loopback
proxy URL/key. Scrubbing them closes two paths: PR-controlled test code
reading them out of os.environ, and model-directed Read tool calls
reaching /proc/<pytest-pid>/environ during PDF/vision cells. Add a pin
test mirroring the existing version-probe one.
This commit is contained in:
mateo-berri 2026-05-19 06:40:17 +00:00
parent a253bd940b
commit 3e7ecdd152
No known key found for this signature in database
2 changed files with 127 additions and 1 deletions

View file

@ -0,0 +1,95 @@
"""Pin: the cron `pytest` invocation must run under `env -i`.
The systemd service `litellm-compat-matrix.service` loads provider
credentials (`ANTHROPIC_API_KEY`, `AWS_BEARER_TOKEN_BEDROCK`,
`AZURE_FOUNDRY_API_KEY`, `VERTEXAI_*`) and the agent-shin GitHub token
(`AGENT_SHIN_GITHUB_TOKEN`) into `run_daily.sh`'s environment from
`/etc/litellm-compat-matrix.env`. Pytest only needs to talk to the
loopback proxy at `127.0.0.1:${PROXY_PORT}` and has no legitimate reason
to see provider creds in its own `os.environ`. Leaving them in would
let a test under `tests/claude_code/` read them via `os.environ` and
exfiltrate them, and would also let a model-directed `Read` tool call
during a PDF/vision cell reach `/proc/<pytest-pid>/environ`. The
PR-gate's pytest step in `.circleci/config.yml` already runs under
`env -i`; this pin enforces the same scrub on the cron path.
"""
from __future__ import annotations
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[3]
RUN_DAILY = REPO_ROOT / "tests" / "claude_code" / "cron_vm" / "run_daily.sh"
def _pytest_invocation_block() -> str:
"""Return only the executable lines around the pytest invocation.
Comment text in run_daily.sh explains *why* certain credential
names must not appear, so a naïve substring scan over the whole
region would false-positive on the rationale itself. Strip lines
whose first non-space character is `#`.
"""
body = RUN_DAILY.read_text()
start = body.index('log "running pytest"')
end = body.index("PYTEST_EXIT=$?", start)
return "\n".join(
line for line in body[start:end].splitlines()
if line.lstrip()[:1] != "#"
)
def test_pytest_invocation_wraps_in_env_i() -> None:
block = _pytest_invocation_block()
assert "env -i" in block, (
"run_daily.sh: the pytest invocation must run under `env -i` so "
"PR-controlled test code under tests/claude_code/ cannot read "
"provider/agent-shin credentials out of the systemd service "
"environment, and so a model-directed `Read` tool call cannot "
"reach /proc/<pytest-pid>/environ to pull them out."
)
assert block.index("env -i") < block.index('"${WORKTREE_UV}" run pytest'), (
"run_daily.sh: `env -i` must precede the pytest invocation; "
"otherwise pytest inherits the full credential-bearing env."
)
def test_pytest_invocation_env_i_excludes_provider_secrets() -> None:
block = _pytest_invocation_block()
for forbidden in (
"ANTHROPIC_API_KEY",
"AWS_BEARER_TOKEN_BEDROCK",
"AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY",
"VERTEXAI_CREDENTIALS",
"VERTEXAI_PROJECT",
"VERTEXAI_LOCATION",
"AZURE_FOUNDRY_API_KEY",
"AZURE_FOUNDRY_API_BASE",
"GITHUB_TOKEN",
"AGENT_SHIN_GITHUB_TOKEN",
):
assert forbidden not in block, (
f"run_daily.sh: the pytest-step `env -i` allowlist must not "
f"pass {forbidden} through. Found it inside the pytest "
f"invocation block."
)
def test_pytest_invocation_passes_proxy_url_and_key_explicitly() -> None:
block = _pytest_invocation_block()
assert "LITELLM_PROXY_BASE_URL=" in block, (
"run_daily.sh: the pytest `env -i` block must still pass "
"LITELLM_PROXY_BASE_URL so the test suite knows where to find "
"the loopback proxy."
)
assert "LITELLM_PROXY_API_KEY=" in block, (
"run_daily.sh: the pytest `env -i` block must still pass "
"LITELLM_PROXY_API_KEY so the test suite can authenticate to "
"the loopback proxy."
)
assert "COMPAT_RESULTS_PATH=" in block, (
"run_daily.sh: the pytest `env -i` block must still pass "
"COMPAT_RESULTS_PATH so the conftest writes the per-cell "
"tagged-union artifact to the script-managed path."
)

View file

@ -351,9 +351,40 @@ fi
log "running pytest"
set +e
# Pytest only needs to talk to the loopback proxy at 127.0.0.1:${PROXY_PORT}
# — it has no legitimate reason to see ANTHROPIC_API_KEY /
# AWS_BEARER_TOKEN_BEDROCK / VERTEXAI_* / AZURE_FOUNDRY_* /
# AGENT_SHIN_GITHUB_TOKEN / GITHUB_TOKEN in its own env. The systemd
# unit's EnvironmentFile injects all of those into this script for the
# proxy to consume, and pytest inherits them by default. Wrap the
# invocation in `env -i` so:
#
# 1. test code under tests/claude_code/ (or anything it imports)
# cannot read provider/agent-shin creds out of `os.environ` and
# exfiltrate them via an outbound call from inside a conftest hook
# or a fixture (a sibling vector to the model-controlled Bash/Read
# concern handled by `cli_driver.py`'s own env scrub);
# 2. a model-directed `Read` tool call during a PDF/vision cell
# cannot reach /proc/<pytest-pid>/environ and pull the creds out
# of the parent process the way it can today;
# 3. this matches the PR-gate pytest step in `.circleci/config.yml`,
# which already runs under `env -i` with the same minimal
# allowlist.
#
# `cli_driver.py` re-allowlists its own subset (PATH/USER/LOGNAME/etc.)
# when spawning the `claude` binary, so the CLI still finds Node + the
# claude shim on PATH and gets a fresh isolated HOME per invocation.
(
cd "${WORKTREE}" \
&& LITELLM_PROXY_BASE_URL="http://127.0.0.1:${PROXY_PORT}" \
&& env -i \
PATH="${PATH}" \
HOME="${HOME}" \
USER="${USER:-mateo}" \
TERM="${TERM:-dumb}" \
LANG="${LANG:-C.UTF-8}" \
LC_ALL="${LC_ALL:-}" \
TMPDIR="${TMPDIR:-/tmp}" \
LITELLM_PROXY_BASE_URL="http://127.0.0.1:${PROXY_PORT}" \
LITELLM_PROXY_API_KEY="${PROXY_API_KEY}" \
COMPAT_RESULTS_PATH="${RESULTS_JSON}" \
"${WORKTREE_UV}" run pytest "${PYTEST_ARGS[@]}"