litellm/tests/integration/pricing
yuneng-jiang b18188dee2
test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984)
* test(e2e): retire the quiet_stack lock

quiet_stack held an exclusive host-wide lock for six tests, draining every
other xdist worker while they ran. On the full e2e lane that left 35-38% of
worker time idle. Each of those tests now lives where it can run without
pausing the suite

- the two memory tests move to perf/ behind the perf marker, so the perf lane
  runs them on its own fresh stack. Their refusing deployments use a literal
  key because the perf stack carries no provider keys
- the stored-request size check from the memory test becomes an integration
  test on an owned proxy that stores prompts
- the discount and margin spend-row tests are dropped from e2e. Their
  breakdown assertions join the existing integration pricing test, which
  already owns the discount and margin repricing flow
- the Bedrock guardrail cost test stays in e2e without the pricing reset
  fixture, since nothing else on the stack writes pricing config now
- the MCP semantic tool filter test turns on a proxy-wide setting, so it gets
  a new proxy_wide_config opt-in marker and runs after the main run

* test(e2e): register the claude_code deployments as one batch

The claude_code session fixture registered its 24 deployments one at a time on every xdist worker, and each one waited out the 15 s propagation window on its own, about 6 minutes per worker. ProxyClient.register_models posts the whole batch, polls each model until every replica serves it, then waits out propagation once from the last write. A model that fails to post or never becomes servable comes back as ModelNotRegistered, and an unservable one is deleted, so the fixture keeps reporting per-deployment failures the way it did
2026-11-04 09:55:53 -08:00
..
test_bundled_cost_map_price_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_configured_prices.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_cost_margin_discount.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
test_databricks_cache_pricing.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_model_listing_token_limits.py fix(router): keep a deployment's token limits off the shared backend cost map key (#49952) 2026-11-03 16:14:44 -08:00
test_ocr_page_pricing.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_off_peak_pricing.py test(integration): cover off-peak pricing on a live proxy 2026-09-19 00:10:22 +00:00
test_opencode_go_reported_cost_wire.py test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
test_per_second_pricing.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_price_data_reload_prices.py fix(proxy): omit cost headers for unpriced responses (#49467) 2026-11-03 12:54:08 -08:00
test_price_precedence.py Format database and accounting integration tests 2026-09-14 21:24:20 -07:00
test_realtime_cached_audio_pricing.py fix(cost): apply a deployment's pricing override to realtime sessions (#43114) 2026-09-24 21:03:59 -07:00
test_sail_service_tier_pricing.py test: move Sail, Weave and Presidio e2e tests to integration tests (#49533) 2026-10-31 18:31:01 -07:00
test_search_mode_pricing.py fix(search): price Tavily advanced and Linkup deep searches at their own cost map entries (#47411) 2026-10-19 23:32:31 -07:00
test_service_tier_pricing.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00