litellm/tests/integration/observability
devin-ai-integration[bot] 65468219db
Some checks are pending
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
Terraform Modules / fmt, validate, test (aws) (push) Waiting to run
Terraform Modules / fmt, validate, test (gcp) (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Waiting to run
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Waiting to run
test(integration): regression tests for August cost tracking and budgeting bugs (#42622)
* test(integration): saving echoed model_info never persists cost map pricing as a deployment override (Pylon #6870)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): budget_duration change on /budget/update recomputes budget_reset_at (Pylon #6913)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): count_tokens on a budgeted key reserves no budget and a later completion still succeeds (Pylon #6966)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): /cost/estimate reports configured prices for a deployment absent from the cost map (Pylon #7014)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cache the team member default budget in Redis as JSON (Pylon #7180)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): aggregated team daily activity reports whole-range team spend in one page (Pylon #7224)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): failed daily user rollup commits are retried so spend report and daily activity agree (Pylon #7268)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): CLI session token without org_id is charged to and capped by the team organization budget (Pylon #7291)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): gemini passthrough success releases its budget reservation from the spend counter (Pylon #7295)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): batch retrieval spend row sums reasoning tokens and counts output and error file failures (Pylon #7341)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): uncostable batches retire from the cost poll page so newer batches are costed (Pylon #7342)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): charge a team member added without any budget on its membership row (Pylon #7363)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): failed dispatched requests keep estimated input tokens in spend logs (Pylon #7519)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bedrock passthrough converse guardrail ignores tool definitions (Pylon #7524)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): explicit null budget_duration on /team/new is not replaced by default_team_params (Pylon #7536)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): PATCH /organization/update with a null limit clears it (Pylon #7577)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): ultrafast service_tier bills ultrafast rates without leaking pricing fields upstream (Pylon #7587)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep the selected model in the response and spend log for an Azure Model Router alias (Pylon #7636)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): disconnected Bedrock /v1/messages stream still bills terminal usage (Pylon #7685)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): databricks cached prompt tokens bill at cache rates (Pylon #7738)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): completed batch spend row records reasoning tokens and error file failures (Pylon #7928)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bill OCR annotation pages at annotation_cost_per_page (Pylon #7958)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): in-flight count tokens request reserves no key budget so a completion still reaches the provider (Pylon #7307)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): streamed /v1/responses success callbacks keep provider response headers (Pylon #7775)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Revert "test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)"

This reverts commit 910348be7a.

* test(integration): reconcile contracts manifest for bundled regression tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): isolate proxy config writes in bundled regression tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): address review feedback on budget reset bounds and callback batch accumulation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): avoid rebinding the cache identity accumulator

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert forwarded messages per cache identity call

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): make budget reset and team default tests deterministic

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 04:13:03 +00:00
..
test_callback_delivery.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_guardrail_effects.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_otel_text_completion_choices.py fix(otel): keep text completion choice fields beside the synthesized message (#42537) 2026-09-22 14:24:32 -07:00