A test's JUnit report says what happened to it, never what it was about.
`@pytest.mark.covers("cell.id")` is a registry key, not a description: it
cannot answer "which tests drive /v1/responses on Anthropic", and nothing
in the report says where a failing test actually died.
Two halves, deliberately separated, both riding out as JUnit <property>
entries the downstream emitter already knows how to read.
DECLARED - `@meta(Subject(...))` from the new tests/e2e/e2e_metadata.py.
One frozen dataclass, every field a closed enum (domain, route, provider,
model, capabilities, mode), so a typo is a basedpyright error at the call
site rather than a property that silently never appears. Serialization is
one pass over `dataclasses.asdict`, so a new scalar field needs no
serializer edit; `capabilities` is deduped and sorted at declaration so
committed run artifacts diff cleanly whatever order a test spelled it in.
Empty fields emit nothing - the suite does not pad every testcase with
five empty entries.
RECORDED - `@step("POST /chat/completions")` on harness helpers, never on
tests. Each call appends its label to the running test's user_properties
in call order, so the list IS the test's user story and cannot drift from
what the test did. The label is recorded BEFORE the wrapped call, so a
helper that raises still leaves its own label last: a failing test's last
step is where it died. Consecutive duplicates collapse and the log caps at
50, so a poll loop is one beat of the story rather than fifty.
Steps cannot be attached where the other properties are -
`pytest_collection_modifyitems` runs before any test body, so the recorder
is empty there. They attach from the existing `pytest_runtest_makereport`
wrapper on the call phase, which is what puts them on failures too, and an
autouse fixture empties the log at setup. The attach drops any prior step
entries first, because the suite runs `--reruns 1` and a retry would
otherwise stack a second copy of the story behind the first.
`covers` is untouched: the marker is separate because a dataclass passed
to `covers` would be dropped silently by `dedupe_covers` and would hard-
fail collection in tests/integration/conftest.py. The fixed
package/covers/source prefix stays byte-identical, and `@meta` goes BELOW
`@covers` so `Item.location` still anchors at the first decorator and
every `source` deep link keeps pointing where it pointed.
`Provider` mirrors litellm's `LlmProviders` values instead of importing
them, so nothing here - the module or its call sites - imports litellm.
tests/e2e is a black-box HTTP suite that is copied to the runner image on
its own, so a `from litellm...` at the top of a test module would make the
package a COLLECTION-time dependency: where it is absent, every test in
the suite errors out before running rather than importing slowly. The
mirror cannot drift silently - `TestProviderMirrorsLitellm` asserts every
value is a real `LlmProviders` value wherever litellm is importable, and
skips where it is not, which is the property it is guarding.
A declared `model` names the constant the test drives, never a copy of its
value: `CHEAP_ANTHROPIC_MODEL` and `CHEAP_OPENAI_MODEL` are env-overridable
(`E2E_CHEAP_ANTHROPIC_MODEL`, `E2E_CHEAP_OPENAI_MODEL`), so a hardcoded
default would have reported a model the run never touched. The same holds
for a file's own `BACKEND`/`MODEL` constant, where the copy was merely
waiting to drift.
Pilot: tests/e2e/quota_management, all 85 tests annotated and its three
clients plus cost_rows @step-decorated, to prove the API against real
tests rather than a toy. The rest of the suite is a later backfill.
Verified: 42 harness unit tests in test_junit_properties.py (19 new), 550
harness unit tests green, basedpyright over tests/e2e at the same 29
pre-existing errors as origin/main (all in the untouched mcp/
oauth_chat_client.py), ruff clean, the coverage-registry collector
byte-identical before and after (473/586), and the test-quality gate OK
against origin/main. Collection was also run with the litellm package
blocked at the import hook: 1339/1350 collected either way, the one error
being the pre-existing missing `httpx2` in mcp/. The live e2e tests need a
deployed proxy and real provider keys and were not run.
* fix(key_generate): use user's budget for UI session personal keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Update litellm/proxy/management_endpoints/key_management_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(user_update): evict cached user on max_budget change so the personal key ceiling refreshes on every worker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus-berri <jesus@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Driving a real Claude Code 2.1.278 through the proxy, and a direct call to
api.anthropic.com, both show these two fields are JSON arrays on the wire rather
than objects. The request carries safeguards as
[{"type": "dangerous_tool_use", "classifier_context": {...}}] under beta
dangerous-tool-use-2026-09-03, and the 200 comes back with safeguard_results as
[{"type": "dangerous_tool_use", "status": {"type": "available", "tool_uses": {...}}}].
No runtime change: the request filter matches on TypedDict keys and never inspects
the value. The test fixtures move to the captured shapes so the regression tests
pin what the client and the provider actually exchange.
Groq lists qwen/qwen3.6-27b for shutdown on 2026-09-14. Adds the six regional
Bedrock qwen.qwen3-next-80b-a3b entries priced per AWS's published regional
rates (absorbs #42191) with a regression test that the regional entry is used
instead of the US rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Buildkite ephemeral stack runs the gateway in another pod, so it cannot reach the pytest host's provider edge. The GitHub changed-e2e lane runs gateways on the runner and sets E2E_PROVIDER_EDGE_HOST_REACHABLE
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Native Anthropic Messages requests derived their allowlist from
AnthropicMessagesRequestOptionalParams, which lacked safeguards, and the
shared beta-header filter dropped betas unknown to the provider mapping
even when the upstream is api.anthropic.com itself. Claude Code auto mode
then saw no safeguard_results and fell back to billed classifier calls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
