scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36
litellm/tests/load_tests
History
Exact
Cursor Agent 02e6b59abd perf: guard orjson import with fallback, pre-parse httpx URLs, fix docstring 
Addresses three review concerns and adds httpx URL caching:

1. safe_json_dumps.py: Guard orjson import with try/except fallback to
   stdlib json. This module is on the core SDK import path via
   _logging.py — unconditional orjson import would break plain
   'pip install litellm' (non-proxy) users.

2. router.py print_deployment: Update docstring to accurately describe
   the reduced return shape (model_name + litellm_params only).

3. run_perf_comparison.sh: Fix locustfile reference to use the correct
   locustfile_perf.py instead of locustfile.py.

4. httpx URL pre-parsing (~7.8us -> ~0.4us per request, 19x speedup):
   Add _parse_url() with LRU cache (maxsize=64) that pre-parses URL
   strings into httpx.URL objects. Applied to all HTTP methods (GET,
   POST, PUT, PATCH, DELETE) in both AsyncHTTPHandler and HTTPHandler.
   Eliminates regex-heavy re.finditer inside httpx._urlparse on every
   request — confirmed as a GIL hotspot in py-spy thread dumps.

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-08 01:41:03 +00:00
..
compare_perf_results.py test: add locust benchmark comparison tooling for perf analysis 2026-03-08 01:00:05 +00:00
compare_results.py feat: add Rust sidecar binary for high-performance HTTP forwarding 2026-03-07 19:04:10 +00:00
loadtest_config.yaml feat: add Rust sidecar binary for high-performance HTTP forwarding 2026-03-07 19:04:10 +00:00
loadtest_config_perf.yaml test: add locust benchmark comparison tooling for perf analysis 2026-03-08 01:00:05 +00:00
loadtest_config_sidecar.yaml feat: add Rust sidecar binary for high-performance HTTP forwarding 2026-03-07 19:04:10 +00:00
locustfile.py feat: add Rust sidecar binary for high-performance HTTP forwarding 2026-03-07 19:04:10 +00:00
locustfile_perf.py test: add locust benchmark comparison tooling for perf analysis 2026-03-08 01:00:05 +00:00
memory_leak_utils.py Add memory leak detection tests with CI integration (#18881) 2026-01-09 17:36:10 -08:00
mock_openai_server.py feat: add Rust sidecar binary for high-performance HTTP forwarding 2026-03-07 19:04:10 +00:00
run_loadtest.sh feat: add Rust sidecar binary for high-performance HTTP forwarding 2026-03-07 19:04:10 +00:00
run_perf_comparison.sh perf: guard orjson import with fallback, pre-parse httpx URLs, fix docstring 2026-03-08 01:41:03 +00:00
test_datadog_load_test.py test: update tests to new deployment model (#10142) 2025-04-18 14:22:12 -07:00
test_langsmith_load_test.py fix langsmith load tests 2024-09-11 21:19:03 -07:00
test_linear_memory_growth.py Add memory leak detection tests with CI integration (#18881) 2026-01-09 17:36:10 -08:00
test_memory_usage.py (perf) Fix memory leak on /completions route (#8551) 2025-02-14 18:58:16 -08:00
test_otel_load_test.py test: update tests to new deployment model (#10142) 2025-04-18 14:22:12 -07:00
test_vertex_embeddings_load_test.py test: test_embedding_performance 2025-05-14 21:31:07 -07:00
test_vertex_load_tests.py vertex testing use pathrise-convert-1606954137718 2025-01-05 14:00:17 -08:00
vertex_key.json ci/cd update vertex acct 2025-01-05 13:43:32 -08:00