- scripts/run_benchmark_rps_suite.py: run benchmark at 1k, 2k, ..., 10k RPS sequentially
- Each run writes to benchmark_results_<rps>rps.txt (e.g. benchmark_results_1000rps.txt)
- First run waits for proxy; runs 2–10 use --no-wait. --output-dir, --requests supported
- proxy_cli: LITELLM_AUTO_BENCHMARK_RPS_SUITE=1 runs suite when proxy is up
- Optional: LITELLM_AUTO_BENCHMARK_OUTPUT_DIR, LITELLM_AUTO_BENCHMARK_REQUESTS
- Wait for proxy health (LITELLM_PROXY_URL base + /health) then run benchmark
- Save stdout to --output file (default benchmark_results_YYYYMMDD_HHMMSS.txt)
- Env: LITELLM_PROXY_URL, PROVIDER_URL; pass-through args (e.g. --rps-control, --requests)
- --no-wait to skip health wait
- Add --rps-control N to cap requests started per second (no concurrency cap)
- Add make_request_with_rps_limit() and _rps_refill_task() for rate limiting
- benchmark_endpoint() accepts rps_control; when set, ignores max_concurrent
- --requests still controls total requests; works with --rps-control
- Document RPS example in docstring and --help epilog
- Add health_check_client.py for monitoring model availability
- Add health_check_client_README.md with usage documentation
- Add health_check_requirements.txt for dependencies
- Add run_parallel_health_checks.ps1 (PowerShell version)
- Add run_parallel_health_checks.sh (Bash version)
- Organize all scripts under scripts/health_check/ directory
* fix(oldteams.tsx): allow org admin to create team on ui
* fix(oldteams.tsx): show org admin a dropdown of allowed orgs for team creation
* docs(access_control.md): cleanup doc
* feat(ibm_guardrails/): initial commit adding support for ibm guardrails on litellm
allows user to use self-hosted ibm guardrails
* feat(ibm_detector.py): working detector
* docs(ibm_guardrails.md): document new ibm guardrails
* fix: fix linting errors
Replace iter_lines()/aiter_lines() with iter_text()/aiter_text() using explicit
UTF-8 encoding to handle non-ASCII characters like µ in streaming responses.
- Added utf8_iter_lines() and utf8_aiter_lines() helper functions
- Ensures proper UTF-8 decoding of streaming response content
- Added comprehensive tests for Unicode character handling
Fixes#12660
