Commit graph

3 commits

Author SHA1 Message Date
devin-ai-integration[bot]
65468219db
test(integration): regression tests for August cost tracking and budgeting bugs (#42622)
Some checks are pending
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
Terraform Modules / fmt, validate, test (aws) (push) Waiting to run
Terraform Modules / fmt, validate, test (gcp) (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Waiting to run
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Waiting to run
* test(integration): saving echoed model_info never persists cost map pricing as a deployment override (Pylon #6870)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): budget_duration change on /budget/update recomputes budget_reset_at (Pylon #6913)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): count_tokens on a budgeted key reserves no budget and a later completion still succeeds (Pylon #6966)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): /cost/estimate reports configured prices for a deployment absent from the cost map (Pylon #7014)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cache the team member default budget in Redis as JSON (Pylon #7180)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): aggregated team daily activity reports whole-range team spend in one page (Pylon #7224)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): failed daily user rollup commits are retried so spend report and daily activity agree (Pylon #7268)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): CLI session token without org_id is charged to and capped by the team organization budget (Pylon #7291)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): gemini passthrough success releases its budget reservation from the spend counter (Pylon #7295)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): batch retrieval spend row sums reasoning tokens and counts output and error file failures (Pylon #7341)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): uncostable batches retire from the cost poll page so newer batches are costed (Pylon #7342)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): charge a team member added without any budget on its membership row (Pylon #7363)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): failed dispatched requests keep estimated input tokens in spend logs (Pylon #7519)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bedrock passthrough converse guardrail ignores tool definitions (Pylon #7524)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): explicit null budget_duration on /team/new is not replaced by default_team_params (Pylon #7536)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): PATCH /organization/update with a null limit clears it (Pylon #7577)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): ultrafast service_tier bills ultrafast rates without leaking pricing fields upstream (Pylon #7587)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep the selected model in the response and spend log for an Azure Model Router alias (Pylon #7636)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): disconnected Bedrock /v1/messages stream still bills terminal usage (Pylon #7685)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): databricks cached prompt tokens bill at cache rates (Pylon #7738)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): completed batch spend row records reasoning tokens and error file failures (Pylon #7928)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bill OCR annotation pages at annotation_cost_per_page (Pylon #7958)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): in-flight count tokens request reserves no key budget so a completion still reaches the provider (Pylon #7307)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): streamed /v1/responses success callbacks keep provider response headers (Pylon #7775)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Revert "test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)"

This reverts commit 910348be7a.

* test(integration): reconcile contracts manifest for bundled regression tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): isolate proxy config writes in bundled regression tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): address review feedback on budget reset bounds and callback batch accumulation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): avoid rebinding the cache identity accumulator

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert forwarded messages per cache identity call

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): make budget reset and team default tests deterministic

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 04:13:03 +00:00
devin-ai-integration[bot]
153e5ed185
fix(fal_ai): reuse the status client and headers on the video result probe (#42511)
* fix(fal_ai): reuse the status client and headers on the video result probe

The Fal result GET issued after a COMPLETED status poll built its own default
client and only carried Authorization and Content-Type, so an injected client,
a request-level ssl_verify and extra_headers were honored on the status GET but
not on the result GET, and a transport failure on that probe escaped as a 500.
The handler now hands the selected sync or async client to the provider status
transform, Fal reuses it with the full validated header set, and a probe
transport error stays non-terminal like the existing 429 and 5xx handling

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(fal_ai): mark the result probe header dict as mutable-ok for the type discipline gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(fal_ai): move the result probe repro to tests/integration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): restore the shared e2e helpers to the merge base

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 13:12:51 -07:00
Yuneng Jiang
7679e42736
test: cover provider wire contracts, streaming and recovery
Adds owned local TCP peers, a Redis process helper and SigV4 helpers to
tests/integration, and integration contracts for Anthropic and Bedrock wire
shapes, Bedrock role configuration, S3 wire, observed routing, Redis
recovery and stream contracts. Consolidates the four commits previously
stacked on litellm_integration_accounting onto its main-merged tip
2026-09-16 12:48:30 -07:00