* test(integration): streamed Bedrock Messages usage cost equals the recorded spend (Pylon #6667)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): echoed cost-map model info is not persisted as deployment overrides (Pylon #6844)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): reset sweep runs on one pod per tick while replicas share the lease (Pylon #6521)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock post-call guardrail scans streamed Anthropic Messages tool use without 500 (Pylon #6503)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): realtime cached audio tokens bill at the audio cache-read rate (Pylon #6704)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): legacy GET /spend/logs returns at most the 10000 most recent rows and flags truncation (Pylon #6752)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): itemize Responses API cache write tokens as cache creation cost (Pylon #6454)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): migration entrypoint deploys pending migrations before proxy startup (Pylon #6649)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): opted-in team keys stop at the owner's personal budget (Pylon #6641)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): guardrail information stays in the spend log when the caller sends metadata on /v1/messages (Pylon #6614)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): key model allowlist is enforced on Bedrock passthrough routes (Pylon #6419)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): JWT mapped key backfills a null user email from token claims (Pylon #6266)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): scheduled budget reset recovers from a transient DB transport failure (Pylon #6582)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): team key lists models granted through a team access group (Pylon #6044)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): JWT subject without team claim lands in the configured default team (Pylon #5895)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): end-user spend lands for a key without user_id when the auth cache is Redis (Pylon #6021)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stale-low redis counter still blocks team member over budget (Pylon #5824)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prompt-carrying spend rows are written in byte-bounded statements (Pylon #6083)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): logs UI session_total_spend sums every round of a multi-round session (Pylon #5928)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): config.yaml guardrails are served by the guardrail usage detail and overview (Pylon #5813)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): plain chat request skips the object permission lookup (Pylon #5965)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): register july accounting regression contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): isolate cost map override clear on owned proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): make reset lease claim and db relay refusal deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): poll pg_stat settle, bound unbanned relay refusals, clear reset lease on teardown
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bound relay refusals so the budget sweep can reconnect
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): saving echoed model_info never persists cost map pricing as a deployment override (Pylon #6870)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): budget_duration change on /budget/update recomputes budget_reset_at (Pylon #6913)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): count_tokens on a budgeted key reserves no budget and a later completion still succeeds (Pylon #6966)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): /cost/estimate reports configured prices for a deployment absent from the cost map (Pylon #7014)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cache the team member default budget in Redis as JSON (Pylon #7180)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): aggregated team daily activity reports whole-range team spend in one page (Pylon #7224)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed daily user rollup commits are retried so spend report and daily activity agree (Pylon #7268)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): CLI session token without org_id is charged to and capped by the team organization budget (Pylon #7291)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gemini passthrough success releases its budget reservation from the spend counter (Pylon #7295)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): batch retrieval spend row sums reasoning tokens and counts output and error file failures (Pylon #7341)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): uncostable batches retire from the cost poll page so newer batches are costed (Pylon #7342)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): charge a team member added without any budget on its membership row (Pylon #7363)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed dispatched requests keep estimated input tokens in spend logs (Pylon #7519)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock passthrough converse guardrail ignores tool definitions (Pylon #7524)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): explicit null budget_duration on /team/new is not replaced by default_team_params (Pylon #7536)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): PATCH /organization/update with a null limit clears it (Pylon #7577)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): ultrafast service_tier bills ultrafast rates without leaking pricing fields upstream (Pylon #7587)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep the selected model in the response and spend log for an Azure Model Router alias (Pylon #7636)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): disconnected Bedrock /v1/messages stream still bills terminal usage (Pylon #7685)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks cached prompt tokens bill at cache rates (Pylon #7738)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): completed batch spend row records reasoning tokens and error file failures (Pylon #7928)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bill OCR annotation pages at annotation_cost_per_page (Pylon #7958)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): in-flight count tokens request reserves no key budget so a completion still reaches the provider (Pylon #7307)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed /v1/responses success callbacks keep provider response headers (Pylon #7775)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)"
This reverts commit 910348be7a.
* test(integration): reconcile contracts manifest for bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): isolate proxy config writes in bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): address review feedback on budget reset bounds and callback batch accumulation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): avoid rebinding the cache identity accumulator
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert forwarded messages per cache identity call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): make budget reset and team default tests deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
tests/integration/conftest.py imported the bare `integration` package. Because
tests/__init__.py and tests/integration/__init__.py both exist, pytest's default
prepend import mode puts only the repo root on sys.path, so that name resolved
only under the PYTHONPATH that tests/integration/run.py injects. Every other
invocation died at conftest import with
ModuleNotFoundError: No module named 'integration' and exit 4, including the
command test_oci_integration.py documents in its own docstring.
The imports now use the tests.integration._support path that pytest actually
resolves, matching the 120 other `from tests.` imports in the suite. run.py's
PYTHONPATH still works because it already puts the repo root on the path.
tests/code_coverage_tests/test_integration_suite_imports.py collects every file
under tests/integration with PYTHONPATH scrubbed and asserts a non-zero
collection count, so an unresolvable import fails the code-quality job instead
of only the developers who run these files by hand. CI runs the three
pre-existing files through the allowlist rather than executing them, which is
why nothing caught this.
