litellm/scripts
ryan-crabbe-berri ee4547d41f feat(ci): fail the build on a test that catches its own AssertionError
A failing `assert` raises AssertionError, which is an Exception, so an
`except Exception` in the same test catches it, the handler returns, and
pytest records a pass on exactly the regression the assertion guards. TQ009
finds the 29 tests in the suite shaped that way and seeds the ceiling at the
23 left after this branch.

Six of them were in tests/test_litellm/. Narrowing the try to the call and
moving the assertions below it turned two of them red:

- test_sentry_sample_rate asserted SENTRY_API_SAMPLE_RATE was written back
  to os.environ, which set_callbacks never does. It now reads the rate off
  the initialised client, which is where the value actually lands.
- test_convert_tool_response_with_url_image downloaded from a host that no
  longer resolves, and the product swallows a fetch failure, so the skip the
  author wrote could never fire and the assertion failure it caught instead
  read as a network skip. The download is now served from a stubbed
  transport, so the media path runs offline and deterministically.

Also stops two sentry tests leaking a MagicMock into sys.modules, and derives
the budget-coverage test's rule set from the checker instead of a literal
list that goes stale on every new rule.
2026-08-26 18:50:43 -07:00
..
adaptive_router_demo feat: commit new adaptive routing 2026-04-18 21:29:39 -07:00
health_check feat(rate-limiter): allow opting out of v3 TPM reservation and Redis circuit breaker (#30211) 2026-06-11 10:34:26 -07:00
benchmark_anthropic_messages_perf.py perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths (#28289) 2026-05-23 12:15:59 -07:00
benchmark_chat_completions_perf.py perf: eliminate per-request callback scanning on proxy hot path (#27858) 2026-05-14 09:28:31 -07:00
benchmark_mock.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
benchmark_model_response_creator.py fix: improve bedrock streaming hot path perf (#28720) 2026-05-28 11:31:37 -07:00
benchmark_proxy_vs_provider.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
benchmark_streaming_chunk_overhead.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
budget_ratchet_check.py feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt (#37588) 2026-08-20 10:08:49 -07:00
check_test_quality.py feat(ci): fail the build on a test that catches its own AssertionError 2026-08-26 18:50:43 -07:00
check_type_discipline.py perf(ci): fan the budget checkers out across cores (#37784) 2026-08-22 22:44:58 -07:00
create_litellm_branch.ps1 feat: add script to create branches with litellm_ prefix (#17606) 2025-12-06 10:41:39 -08:00
create_litellm_branch.sh enhance: create_litellm_branch tool to be more robust (#17874) 2025-12-12 05:35:50 -08:00
create_team_key_and_submit_guardrail.sh feat(guardrails): team-based guardrail registration and approval workflow (#22459) 2026-03-02 22:06:49 -08:00
eval_compression.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
gate_slot_lock.py refactor(make): stop queueing bootstrap for a machine-wide gate slot 2026-08-15 21:53:28 +00:00
install-cli.sh fix(install): pass an explicit Python version request to uv tool install 2026-07-26 21:02:35 -07:00
install.sh fix(install): pass an explicit Python version request to uv tool install 2026-07-26 21:02:35 -07:00
install_git_hooks.sh build(lint): rename make pre-commit to make check with a working-tree fallback 2026-08-08 03:25:35 -07:00
mock_bedrock_passthrough_target.py Refactor Bedrock response stream shape handling (#27257) 2026-05-06 17:39:38 -07:00
mock_grayswan_timeout_server.py implement failopen option default to True on grayswan guardrail (#18266) 2026-01-06 15:17:05 +05:30
mutation_report.py fix(ci): stop the mutation report publishing a score it never measured (#37825) 2026-08-21 20:15:52 -07:00
pre_commit_lint.sh feat(scripts): queue heavy gates behind a machine-wide slot lock 2026-08-14 17:22:32 -07:00
prisma_generate_if_needed.py fix(lint): generate the prisma client into the gate-owned venv 2026-08-06 01:54:26 -07:00
ruff_strict_gate.py feat(scripts): queue heavy gates behind a machine-wide slot lock 2026-08-14 17:22:32 -07:00
test_agent_mcp_endpoints.sh Agents - assign tools (#22064) 2026-02-25 11:44:30 -08:00
test_guardrails_register_endpoints.sh feat(guardrails): team-based guardrail registration and approval workflow (#22459) 2026-03-02 22:06:49 -08:00
test_quality_gate.py feat(ci): freeze the conftest save/restore inventory so it can only shrink (#37621) 2026-08-20 21:39:59 +00:00
test_tool_allowlist_script.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
tpm_headline_test.sh fix: atomic TPM rate limit (#27001) 2026-05-05 16:58:07 -07:00
type_check_gate.py feat(scripts): queue heavy gates behind a machine-wide slot lock 2026-08-14 17:22:32 -07:00
type_discipline_gate.py feat(scripts): queue heavy gates behind a machine-wide slot lock 2026-08-14 17:22:32 -07:00
verify_adaptive_router.py feat: add adaptive routing to litellm 2026-04-18 16:35:17 -07:00
with_dashboard_node.sh fix(bootstrap): fail fast when nvm cannot activate the pinned node 2026-08-04 21:18:58 -07:00