Budgets reset at midnight in the configured timezone with no way to control
the time of day, so a drained daily budget surfaces as an overnight incident.
Add a litellm_settings.budget_reset_time option (e.g. "12:00") that shifts
day/week/month resets to a configurable wall-clock time in the existing
timezone, so the end of the budget window lands during business hours.
The reset time is parsed once into an immutable BudgetResetSettings and
injected into the reset job (constructor) and computation, rather than read
from a module-level global at call time. A malformed value fails fast at
startup. Sub-day durations ignore the offset. Unset preserves midnight resets.
Google added a queued value to Interaction.status in the live Interactions
OpenAPI spec, so the compliance canary test_status_enum_values started
failing on every open PR. The exact-match assertion is deliberate; it is
how we find out the spec moved, so this adds the new value rather than
loosening the check, and mirrors it into the generated Status enums so
InteractionStatus stays truthful.
* fix(model_armor): sanitize error details by default
Generated with AI
Co-Authored-By: Claude Code
* fix(model_armor): sanitize handler-raised HTTP errors and redact scanned content in guardrail logging
The async HTTP handler raises MaskedHTTPStatusError on any non-2xx via
raise_for_status, so the non-200 branch in make_model_armor_request never ran
against a live API and the raw upstream body reached callers and logs. Catch
the raised error and build the sanitized detail from the response status
Replace the empty-dict guardrail logging payload with field-level redaction of
the keys that echo scanned content (text, sanitizedText, findings) so guardrail
traces keep filter states and block reasons while scanned content stays out
Restore the upstream status code in the sanitized error detail, read guardrail
metadata from the same key the hooks write, and keep guardrail_status within
its typed literal values
* fix(model_armor): bound redactor recursion depth and allowlist it in the recursion detector
_redact_scanned_content walks provider JSON bounded by _REDACT_MAX_DEPTH=20 and
fails closed by returning the redaction sentinel at the cap
* fix(model_armor): honor fail_on_error for upstream API failures
API failures now raise a dedicated ModelArmorAPIError so hooks can tell them
apart from content-block HTTPExceptions; fail_on_error=False lets the request
proceed on a Model Armor outage again while fail-closed configs get the same
sanitized 400 as before
Also addresses review notes: sanitize_error_detail constructor annotation
matches the nullable config field, redaction is owned by the metadata write
sites so _process_response no longer re-applies it, and the request and
response debug log branches move into helpers
* test(model_armor): cover fail_on_error routing on during-call, post-call, streaming, and file-scan paths
* chore: remove accidentally committed pytest cache files
* fix(model_armor): keep sanitize_error_detail coerced across in-memory config reloads
update_in_memory_litellm_params assigns raw LitellmParams fields, so a hot
reloaded config carrying an explicit null would silently disable sanitization;
re-apply the only-explicit-False-opts-out coercion after the update
* fix(model_armor): redact matched malicious URIs and reuse the shared recursion depth constant
maliciousUriMatchedItems echoes the caller-supplied URL including path and
query, so it joins the scanned-content key set; the redactor depth cap now
comes from DEFAULT_MAX_RECURSE_DEPTH in litellm constants instead of a local
literal
* fix(model_armor): keep API failures out of the intervention trace status
Fail-closed upstream failures re-raise ModelArmorAPIError instead of
converting to HTTPException(400), so the shared guardrail logging keeps
recording them as guardrail_failed_to_respond while content blocks stay
guardrail_intervened. Callers see the same 500 shape as before this PR,
with the sanitized message
* chore(model_armor): drop explanatory comment per repository comment policy
---------
Co-authored-by: eugene-yao-zocdoc <eugene.yao@zocdoc.com>
Adds 'agentcore' to SearchProviders, backed by an AgentCore Gateway
web-search connector target (MCP tools/call over Streamable HTTP).
Web Search on Amazon Bedrock AgentCore is an AWS-managed web index
(GA June 2026). Exposing it as a native search provider lets Bedrock
users enable Claude Code / Anthropic-native WebSearch through
websearch_interception with a pure-YAML config and AWS-native auth,
keeping the whole search path inside AWS.
Implementation:
- New AgentCoreSearchConfig (litellm/llms/bedrock/search/) reusing
BaseAWSLLM credential resolution. Auth follows the gateway's inbound
authorizer type: AWS_IAM gateways get a SigV4-signed request
(explicit aws_access_key_id/aws_secret_access_key params or the
default credential chain); CUSTOM_JWT gateways get an OAuth2 bearer
token via api_key / AGENTCORE_GATEWAY_TOKEN
- SigV4 signing region is derived from the gateway URL so callers
don't need aws_region_name to match their default region
- Adds an optional sign_request() hook to BaseSearchConfig (no-op by
default) and teaches the search HTTP handler to send a signed body
verbatim, mirroring the existing anthropic_messages/chat pattern
- Handles both plain-JSON and SSE-framed MCP responses, propagates
MCP errors, truncates queries to the 200-char gateway limit
Tested:
- 13 unit tests: payload/signing, explicit AKSK passthrough, bearer
token via api_key and env, query truncation, SSE frames, MCP error
propagation, region derivation
- Verified end-to-end against real AWS_IAM and CUSTOM_JWT gateways,
including full Claude Code CLI WebSearch round-trips through the
proxy with websearch_interception
* adding deepkeep as custom guardrail
* adding deepkeep as a custom guardrail
* adding deepkeep as a custom guardrail (hooks)
* adding litellm/proxy/_experimental/out/ to .gitignore
* adding deepkeep as custom guardrail in litellm
* removing sentinel_fortress
* comparing schema.prisma files
* fix(deepkeep): address greptile review comments
- extra_headers: fix type annotation (list -> Dict[str, str]) and actually
merge them into _build_request_headers() so user-configured headers
reach the DeepKeep API
- user_api_key_hash: only fall back to user_api_key_token when no
explicit hash is already set, avoiding silent overwrite
- apply_guardrail: preserve tool_calls and structured_messages in the
return value so downstream callers don't lose that content
Adds tests for all four fixes.
* fix(deepkeep): address greptile review comments
- extra_headers: fix type annotation (list -> Dict[str, str]) and actually
merge them into _build_request_headers() so user-configured headers
reach the DeepKeep API
- user_api_key_hash: only fall back to user_api_key_token when no
explicit hash is already set, avoiding silent overwrite
- apply_guardrail: preserve tool_calls and structured_messages in the
return value so downstream callers don't lose that content
Adds tests for all four fixes.
* fix: add missing __init__.py and allowlist entries for upstream merge
- tests/test_litellm/proxy/client/__init__.py: fixes pytest collection
collision with tests/test_litellm/models/test_models.py (same basename)
- tests/test_litellm/models/__init__.py: same fix
- backend/routes/allowlist.py: add /config_overrides/ and /v1/unified_access_group
prefixes for new routes added by upstream
* fix(ui/tests): resolve frontend-lint failures in new test files
- useLogDetails.test.ts: add Wrapper.displayName, replace 'null as any'
with null, type resolveCall promise resolver properly
- usePaginatedDailyActivity.test.ts: remove unused waitFor import,
add Wrapper.displayName, change Record<string,any> to Record<string,unknown>
- UsageViewSelect.adminFiltering.test.tsx: replace all props:any with
explicit SelectProps/BadgeProps/SelectOption types, replace (X as any).displayName
with direct X.displayName assignment
no-explicit-any count: 2034 (budget: 2040). Prettier check: clean.
* fix(ui): sync proxy/_experimental/out/ exactly to upstream
245 stale JS chunk files from earlier merges were left in the out/
directory but had been deleted in upstream. The Docker image in CI is
built by copying this directory verbatim, so the stale artifacts caused
the SERVER_ROOT_PATH redirect E2E to fail.
Synced by: git checkout upstream/litellm_internal_staging -- out/ (adds
new files) + git rm on every file present in HEAD but absent from
upstream.
* Update litellm/proxy/guardrails/guardrail_hooks/deepkeep/deepkeep.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(makefile): fall back to upstream/litellm_internal_staging for strict-budget gate
origin/litellm_internal_staging exists on BerriAI's CI but not on forks
that use a different remote name (e.g. Azure DevOps as origin). Fall
back to upstream/litellm_internal_staging when the origin ref is absent.
* linter reformat
* fix(deepkeep): apply guardrail tool/tool_call redactions from API response
When DeepKeep returns GUARDRAIL_INTERVENED with redacted tools or
tool_calls, the previous code ignored those redactions and forwarded
the original (potentially sensitive) values to the model — a guardrail
bypass for content embedded in tool schemas or function arguments.
Fix: prefer response_json["tools"] / response_json["tool_calls"] when
present, falling back to the originals only when the guardrail did not
return replacements — consistent with the existing pattern for texts and
images.
Refactor _build_return_inputs() into a private static helper to keep
apply_guardrail() under the PLR0915 statement limit (50).
Adds test_apply_guardrail_applies_tool_redactions_from_response to
assert that redacted tool payloads from the API response are used.
* Update litellm/proxy/guardrails/guardrail_hooks/deepkeep/deepkeep.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(lint): move base-ref fallback into ruff_strict_gate.py; revert Makefile
The previous Makefile fix had a shell bug: 'git rev-parse --verify'
writes the resolved SHA to stdout, so the $$(...) substitution captured
both the SHA and the echo output, handing '--base <sha>\norigin/...' as
two tokens to the Python script, causing exit code 1 in CI.
Fix: revert Makefile to its original single-line invocation and add
_resolve_base() to ruff_strict_gate.py. The function checks whether the
requested ref resolves; if not, it tries the 'upstream/' equivalent
before falling back to the original ref (letting git emit a clear error).
Behaviour in BerriAI CI: origin/litellm_internal_staging resolves → used
as before, no change.
Behaviour on forks with a different 'origin': falls back to
upstream/litellm_internal_staging transparently.
* fix(lint): fix UP006/UP045/F401 in changed files; add depth guard to check_any_discipline
- Replace Dict/List/Optional/Tuple typing imports with built-in equivalents
(UP006, UP045) across files touched in this PR diff, then clean up
the now-unused typing imports (F401).
- Add _MAX_CONTAINS_ANY_DEPTH guard to check_any_discipline.contains_any()
to prevent RecursionError on deeply-nested mypy types.
* fix(lint): resolve all three CI lint job failures
1. lint (ruff_strict_gate) — UP006/UP045/F401 violations introduced on
changed lines. Fixed Dict/List/Optional/Tuple → built-in equivalents
across every file in the PR diff; cleaned up now-unused typing imports.
2. any-discipline — RecursionError in check_any_discipline.contains_any()
on deeply-nested mypy types. Upstream fixed this by converting to an
iterative stack-based algorithm (merged). Also added deepkeep.py to
any-discipline-budget.json via 'make lint-any-budget-update' so the
new file's Any count is baselined instead of failing against the
zero-baseline default.
3. basedpyright reportMissingParameterType — **kwargs in DeepKeepGuardrail
__init__ lacked a type annotation. Added **kwargs: Any.
* Update litellm/deepkeep_tilt_config.yaml
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(lint): black reformat after merge
* fix(deepkeep): honour empty-list replacements in _build_return_inputs
When DeepKeep returns GUARDRAIL_INTERVENED with an intentional empty
replacement (e.g. texts:[], tool_calls:[]) the previous truthiness check
treated [] as absent and forwarded the original content downstream —
a guardrail bypass for any case where the firewall wants to fully clear
a field.
Fix: replace all response_json.get(field) truthiness checks with
'is not None' comparisons so that an empty list is respected as a
deliberate replacement. Applies to texts, images, tools, tool_calls,
and the original-input fallback guards.
Adds test_apply_guardrail_honours_empty_list_replacements.
* fix(test): replace live httpbin.org call with mocked transport in test_pass_through_with_httpbin_redirect
Root cause of OOM: the test made a real HTTP request to https://httpbin.org
inside a pytest-xdist worker. Under memory pressure the worker's httpx client
and redirect-following logic allocated enough virtual memory to trip the OOM
killer (confirmed by ulimit -v 16GB reproducing the crash with 'node down: Not
properly terminated' on this exact test).
Fix: replace the real network call with a custom httpx.AsyncBaseTransport that
returns a pre-built 302 -> 200 response sequence in-memory. The test now runs
hermetically with no network dependency and no excess memory allocation.
ulimit -v 16GB: 24,284 passed (0 crashes) after this fix.
* fix: merge upstream/litellm_internal_staging (197 commits), resolve conflicts
7 conflicts resolved:
- 6 Python files: upstream added new code with old-style typing (Optional,
Dict, List) on lines where we had ruff-fixed modern syntax (str | None,
dict, list). Took upstream's version then re-ran ruff UP006/UP045/F401
--fix to keep both the new content and ruff compliance.
- test_openapi_compliance.py: upstream replaced 'role' with 'steps' in
output_fields and updated the spec comment. Took upstream's version.
Also: added _resolve_base() fallback to type_check_gate.py and removed
the hard 'git fetch origin litellm_internal_staging' from the Makefile's
lint-basedpyright target (same pattern as ruff_strict_gate.py fix).
* fix: merge upstream (41 commits), resolve .gitignore conflict, fix BLE001
- .gitignore: upstream removed package.json/out/ ignore entries; took theirs
- deepkeep.py: added '# noqa: BLE001' on catch-all Exception handler
(BLE001 rule newly enforced in ruff-strict-budget)
- type_check_gate.py: added _resolve_base() fallback for basedpyright gate
- Makefile: removed hard 'git fetch origin' from lint-basedpyright target
* fix: merge upstream (57 commits), resolve conflicts
- Makefile: upstream added lint-fetch-base target; made it tolerant of
missing origin/litellm_internal_staging (git fetch || true)
- test_websearch_chat_completion.py: took upstream's new assertions and
skipif marker
- anthropic_cache_control_hook.py: upstream added new code using List/Dict/Tuple
which were undefined after our earlier UP006 cleanup; replaced with
built-in list/dict/tuple
* fix(coverage): revert ruff UP006/UP045 changes on upstream files
The previous ruff fixes (Dict→dict, Optional→X|None) on 7 upstream files
added ~500 changed lines of pure type-annotation no-ops to our PR diff.
codecov/patch penalised these uncovered lines, dropping patch coverage
to 51.35% (target 61.83%).
Fix: revert these files to exactly match upstream/litellm_internal_staging.
The ruff_strict_gate still passes because the violations exist equally in
both the base and HEAD (total == base_count → no breach).
* fix: merge upstream (130 commits), resolve Makefile + base_email conflicts
- Makefile: upstream changed lint deps to $(LINT_DEP_INSTALL)/$(LINT_DEP_BASE);
kept our --base removal (handled by _resolve_base in Python scripts)
- base_email.py: took upstream's dedup cache addition
- deepkeep.py: ruff format after merge
* chore: remove lint/format-only changes and non-feature files
Revert all lint-infra and black/ruff-reformat-only changes back to
upstream/litellm_internal_staging so the PR diff shows only the DeepKeep
guardrail feature:
- Makefile, scripts/ruff_strict_gate.py, scripts/type_check_gate.py
(lint-gate infra)
- credential_migration.py + enterprise/* + assorted test files
(black-reformat / xdist test-isolation drift)
- backend/routes/allowlist.py (merge glue)
Remove non-feature local artifacts: build-and-push.sh,
deepkeep_tilt_config.yaml, stray __init__.py collision shims, and
unrelated UI test files.
* fix(lint): add reason to BLE001 noqa to satisfy type-discipline gate (LIT003)
The type-discipline budget ratcheted LIT003's ceiling to 292 as upstream
fixed reasonless suppressions, so our '# noqa: BLE001' (code but no
reason) tipped the total to 293 and failed CI. Add a reason per the
required '# noqa: CODE # <reason>' shape.
* fix(deepkeep): apply structured_messages redactions returned by the guardrail API
_build_return_inputs dropped any structured_messages the DeepKeep API returned and
always forwarded the original input, so redactions on that field never took effect.
Check the response first, same as texts/images/tools/tool_calls
* chore(ui): drop redundant preserve prop from the guardrail form
preserve defaults to true in rc-field-form (isMergedPreserve falls back to true when
unset), so the explicit prop changed nothing and only widened this PR's blast radius
to every guardrail provider in the shared form
* fix(deepkeep): stop extra_headers list from crashing the guardrail call and name the real firewall id config key
litellm_params.extra_headers is a list of header names to forward, so passing it
straight into dict.update raised ValueError and, under fail_closed, took the request
down with it. Only merge mapping values and warn otherwise
The docstring example and the missing-secret error both said firewall_id, but
initialize_guardrail only reads deepkeep_firewall_id, so anyone following them
had their value silently ignored
* refactor(proxy): drop normalize_callback change; split to its own PR (#33905)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yaniv Israel <yaniv@deepkeep.ai>
Co-authored-by: DK-yaniv <164404355+DK-yaniv@users.noreply.github.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The MCP OAuth DCR relay's client-facing /register response handed back LiteLLM's own
<base>/callback as the client's redirect_uris on every arm except the true bridge relay.
A spec-compliant OAuth 2.1 DCR client (e.g. Open WebUI over Streamable HTTP) adopts that
value for its subsequent /authorize calls, so /callback redirects to itself; the second
hit carries the client's opaque state, fails to decrypt, and surfaces as the LIT-4197
"oauth_state ... Incorrect padding" error, making pass-through MCP OAuth unusable against
real DCR-capable upstreams.
The short-circuit arm keeps registering the gateway callback upstream (unchanged) and now
echoes the client's own redirect_uris back to the client across all register arms. Since
the client then authorizes with its own separate-origin redirect, the /authorize rejection
hint now points operators to MCP_TRUSTED_REDIRECT_ORIGINS, the mechanism a legitimate
cross-origin OAuth client needs (auto-trusting a DCR-registered redirect would reintroduce
the VERIA-57 open-redirect vector, since dynamic registration is unauthenticated).
temp_budget_increase was only applied on the DB-fetch path of _user_api_key_auth_builder, so a key served from the auth cache reverted to its original max_budget and was wrongly blocked with BudgetExceededError once spend crossed the original budget while staying under the effective budget.
Move _update_key_budget_with_temp_budget_increase out of the DB-only branch so it runs for every resolved token regardless of source. The cache stores the original budget and each cache hit returns a fresh model_copy(), so this never double-applies.
Fixes#25760
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The JWT first-login upsert in get_user_object creates the user row by
merging default_internal_user_params straight into table.create, so a
configured budget_duration landed with budget_reset_at NULL. The reset
sweep now heals such rows (PR #33623), but until the next sweep the row
shows a null reset time and its first window starts at the sweep instead
of one full duration after creation. Compute budget_reset_at at creation
like every other write path (/user/new, UI SSO, /key/generate, /team/new)
already does
Continues #29762. Response models (Message, Choices, Usage) delete unset
optional fields in __init__ so model_dump matches the OpenAI spec. Each delete
routed through pydantic's BaseModel.__delattr__, whose per-call
ModelMetaclass.__getattr__ lookup and _check_frozen dominate construction. When
the target is a declared field already present in __dict__ on a non-frozen
model, delete it with object.__delattr__ directly; that is exactly what pydantic
2.13 does for that case, minus the metaclass getattr and the frozen check. It
falls back to the previous super().__delattr__ path for extras, private
attributes, cached properties and missing names, so behavior is unchanged.
Co-authored-by: Jay Gowdy <jgowdy@godaddy.com>
Internal users seeded from default_internal_user_params (SSO/JWT first-login
upsert, or /user/new without an explicit budget_reset_at) get budget_duration
set but budget_reset_at = NULL. The ResetBudgetJob user/team queries filter on
{"budget_reset_at": {"lt": now}}, which never matches NULL, so these rows are
never reset: their spend accumulates for the lifetime of the row and silently
exceeds max_budget with no periodic reset.
The budget-table query already handles this by OR-ing in a
{budget_reset_at IS NULL AND budget_duration IS NOT NULL} branch. Apply the
same pattern to the user and team reset queries in PrismaClient.get_data.
Adds a regression test asserting both the user and team reset queries select
NULL-budget_reset_at rows that have a budget_duration.
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Ran the before/after proof live against Vertex Claude (global endpoint,
project vertex-check-481318): base transform 400s an unflagged model
(claude-opus-4-7) on a mid-conversation role:system reminder, the fix
hoists it to a 200, and a flagged model (claude-opus-4-8) keeps the
reminder in messages with cache_read held at 15615 across the reminder
turn. Flip both vertex.mid_conversation_system rows to fail_before_fix:
proven.
Azure AI Foundry and Vertex AI serve Claude on the first-party Anthropic
Messages contract, which was verified live to be byte-identical to
api.anthropic.com: a leading role:"system" entry in messages is rejected on
every model ("messages.0: use the top-level 'system' parameter"), and a
mid-conversation role:"system" reminder is accepted in place on Claude 4.8+/5
but 400s on Claude 4.7 and older ("role 'system' is not supported on this
model"). This is the same contract Bedrock Invoke already handles model-aware
(PRs #32578/#32831/#32882); Vertex and Azure did no hoisting at all, so a Claude
Code session on an older Vertex/Azure Claude model hard-400s on its reminder
turns, and the only thing sparing 4.8+/5 was that nothing was hoisted
Extract Bedrock's model-gated normalization into the shared
AnthropicMessagesConfig base as _normalize_system_role_messages and call it from
the Vertex and Azure messages configs. Flagged models (4.8+/5) hoist only the
leading run of system entries and keep mid-conversation reminders in place so
the top-level system prefix stays byte-identical and the prompt cache is
preserved; unflagged models hoist every system entry so the request returns a
completion instead of a 400
Add supports_mid_conversation_system to the azure_ai and vertex_ai Claude 4.8+/5
cost-map entries. Exact cost-map hits win over the claude-mid-conversation-system
fallback rule, so without the explicit flag those models would be treated as
unsupported and hoist every reminder, collapsing the prompt cache (the exact
customer regression). A per-provider test guards this so future 4.8+/5 entries
cannot silently miss the flag
Closes the Vertex/Azure gap from the customer RCA
Continues #29761. Delta.__init__ set roughly ten attributes through pydantic's
__setattr__ and then deleted the five OpenAI omits on every chunk. Those keys
are extra fields (extra='allow'), so this builds __pydantic_extra__ and
__pydantic_fields_set__ directly after the parent init instead of round-tripping
each field through __setattr__/__delattr__. The resulting __dict__,
__pydantic_extra__, __pydantic_fields_set__ and model_dump output (including
exclude_unset, which the streaming path relies on) are byte-identical to the
previous behavior; a serialization-contract test locks that. A TYPE_CHECKING
block re-declares the extra attributes with their concrete types so type
checkers still see delta.content and friends.
Co-authored-by: Jay Gowdy <jgowdy@godaddy.com>
Staging now contains #33153, whose final rounds made _extract_upstream_auth_failure a thin delegate
to upstream_auth_challenge and introduced the response-level iterator this branch predates. The
resolution completes the consolidation both branches were converging on: iter_exception_tree
(faults/traversal.py) is the one tree walk, _iter_upstream_responses is rebuilt on top of it instead
of carrying a second copy of the traversal, the manager keeps the delegate, and the semantic filter
port from this branch stands. Test conflicts were append-append and both sides are kept
* test(e2e): cover passthrough headers, batch assume-role, gemini, vllm, bedrock guardrails, batch rate-limit mapping
Add parent-package e2e suites for the six feature gaps: pass-through header forwarding via /config/pass_through_endpoint, Bedrock batch STS assume-role, Gemini chat + files, hosted_vllm batch/files, Bedrock guardrail pre_call blocks (plus restored content-filter team opt-out), and OpenAI batch RPM 429 body mapping. Registry cells and LiteLLMParamsBody/TeamMetadata fields updated so markers collect cleanly.
* test(e2e): cover LIT-4587 gaps for redis, responses, tpm cache, apply_guardrail, langfuse
Adds customer-shaped live e2e for apply_guardrail, responses store+metadata TTL,
TPM excluding cached tokens, redis-backed RPM, redis circuit-breaker path,
Langfuse spend, Cohere chat, virtual-key auth, file content download, hosted_vllm
chat, and Nova Sonic realtime. Registry cells updated for the new markers.
* test(e2e): drive LIT-4587 gap suites on Anthropic to avoid Gemini quota flakes
Redis RPM, circuit-breaker path, virtual-key auth, responses metadata, and
Langfuse driver models now use Anthropic haiku so local runs stay green when
Gemini daily quota is exhausted.
* test(e2e): drop Langfuse spend suite; feature is being deprecated
Remove test_langfuse_e2e.py, logging.langfuse registry cells, and the
langfuse-only conftest driver/credentials fixtures.
* test(e2e): fold provider/batch feature tests into their endpoint suites
Keep the e2e layout endpoint- and suite-scoped instead of one file per
provider or feature
Move the virtual-key auth case into access_control/test_access_control_e2e.py
as TestVirtualKeyAuth (replacing an incomplete stub) and drop the standalone
test_virtual_key_auth_e2e.py
Fold the five per-file batch suites (file content, RPM 429 mapping, Bedrock
assume-role, Gemini files, hosted_vllm batch) into batches/test_batches_e2e.py.
The hosted_vllm batch case is skipped for now since it needs a live vLLM server
(HOSTED_VLLM_API_BASE) the e2e environment does not provision; it and the
gemini-files and RPM-mapping cases reference LIT-3382 / LIT-3266 where relevant
Merge the cohere, gemini and hosted_vllm chat cases into
llm_translation/test_chat_completions_regression_e2e.py so /chat/completions
coverage lives in one endpoint file, and repoint the coverage_registry source
fields to the new homes
Move the shared CacheControl / TextBlock / RichMessage request blocks into the
root models.py (re-exported from endpoints_client) so quota_management can use
them without a cross-suite import, which also clears the basedpyright errors in
test_tpm_excludes_cached_tokens_e2e.py; type the httpbin echo body in
test_passthrough_headers_e2e.py with a pydantic model to drop the Any-typed
json.loads path
* test(e2e): address review feedback and re-home virtual-key coverage
Replace the tautological Bedrock assume-role batch id assertion (`startswith(...)
or batch.id`, always true) with a managed-id shape check, since the unified
target_model_names path re-encodes the id rather than returning a raw ARN
Raise the batch RPM-mapping test's rpm_limit above one so the file upload can no
longer consume the key's sole request unit before batch create runs; the batch
create then clears the generic per-request limiter and the batch limiter is what
returns the "Batch rate limit exceeded" body the assertions check
Set exercised_on to [] on the pass-through header test; it drives a pass-through
endpoint, not /chat/completions
Move the virtual-key valid_allows / invalid_denied cells from other.yaml to
mgmt.yaml as mgmt.virtual_key.* so TestVirtualKeyAuth rolls up under Management,
and point its covers marker at the new ids
* test(e2e): add reliability suite covering fallback, timeout, and cache behavior
* test(e2e): move reliability suite under router and drive it with real deployments
* test(e2e): make the router complexity fixture opt-in so reliability tests can coexist
* fix(cache): make in-memory and disk increments atomic
* refactor(cache): narrow in-memory increment lock scope
* fix(cache): address follow-up review on increment tests/types
* fix(cache): refresh atomic increment coverage
* test(cache): widen increment race window with non-zero _SlowInt seed
The zero seed was falsy, so InMemoryCache.increment_cache's `get_cache(...) or 0`
and DiskCache.get_cache's truthiness guard both discarded the _SlowInt before
__add__ could run, leaving the sleep-based window-widening inert. Seed a non-zero
value and return _SlowInt from __add__ so the sleep fires on every read-modify-write
in both backends, making the concurrency regression deterministic.
* test(cache): cover InMemoryCache.async_increment delegation
Add a focused async test asserting async_increment accumulates through the
locked sync path, exercising the previously uncovered delegation line.
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
