scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 35

test(e2e): drop xfail markers for the now-fixed team-budget-JSON and custom-pricing-leak bugs (#31249)

Browse source 
Both tests were xfail(strict=True) for known proxy bugs: /team/new writing
budget_limits as a raw list (Prisma 500) and custom per-token pricing leaking into
the shared cost map for sibling deployments. Both are fixed, so the tests pass and
strict mode reports the unexpected pass as a failure. Remove the markers (as their
reasons instructed) so they run as plain regression guards; docstrings updated to
describe the regression each now pins.
This commit is contained in:
mubashir1osmani 2026-06-24 18:51:10 -07:00 • committed by GitHub
parent fa307fe9e5
commit 1ff1557b96
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 6 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
tests/e2e/budgets/test_team_multi_window_budget_e2e.py
View file

@ -7,11 +7,10 @@ the tight window's cap is exceeded, then - once the 30s elapses and the reset jo
and calls flow again. This exercises the reset_budget_windows TEAM branch (raw SQL over
LiteLLM_TeamTable.budget_limits, the literal #25109 path), which had no live coverage.
Fails at team creation today: /team/new writes the raw window list straight to the
Json? column, where Prisma rejects it (500), unlike the key path and /team/update which
json.dumps it first. Marked xfail(strict=True) so the suite stays green while the bug
persists and flips to a failure the moment the write is fixed and the marker should be
removed.
This also guards the /team/new write path: it must json.dumps the window list into
the Json? column. A raw list there made Prisma reject the create with a 500 (the key
path and /team/update already json.dumps first); a regression would fail team creation
here.
"""
import time
@ -33,12 +32,6 @@ def _call(client: BudgetClient, key: str):
return client.chat(key, "claude-haiku-4-5", f"team-window {unique_marker()}", max_tokens=16)
@pytest.mark.xfail(
strict=True,
reason="known proxy bug: /team/new writes budget_limits straight to the Json? "
"column and Prisma rejects it (500), unlike the key path and /team/update which "
"json.dumps first; remove this marker once that write is fixed",
)
def test_team_short_window_blocks_then_resets(client: BudgetClient, resources: ResourceManager) -> None:
team_id = client.create_team(
alias=f"e2e-team-window-{unique_marker()}",

11
tests/e2e/llm_translation/test_custom_pricing_e2e.py
View file

@ -10,9 +10,8 @@ back here from the same config file. Three behaviors are checked independently:
- reporting: /model/info surfaces those rates for the model
- isolation: gemini-2.5-flash shares the same underlying gemini/gemini-2.5-flash
but sets no override, so it must keep its own price; an override that leaks into
the shared cost map misprices it. This fails on a real proxy gap today, so it is
marked xfail(strict=True): the suite stays green while the leak persists and
flips to a failure the moment isolation is fixed and the marker should be removed.
the shared cost map misprices it. A regression that reintroduces that leak makes
the sibling's rate match the custom one and fails the isolation check.
"""
import time
@ -192,12 +191,6 @@ def test_model_info_reports_custom_pricing(client: PassthroughClient) -> None:
)
@pytest.mark.xfail(
strict=True,
reason="known proxy bug: a deployment's custom per-token pricing leaks into the "
"shared cost map for sibling deployments of the same underlying model; remove "
"this marker once isolation is fixed",
)
def test_custom_pricing_is_isolated_from_sibling_deployment(
client: PassthroughClient,
) -> None: