Commit graph

11 commits

Author SHA1 Message Date
Alexsander Hamir
ab2980e092 Add cProfile prof files and problem_tracker.md 2026-02-09 16:35:40 -08:00
Alexsander Hamir
7100365f53 Add concurrency test results (1-10-30-50-100) and analysis to problem_tracker 2026-02-09 15:20:18 -08:00
Alexsander Hamir
56f4dc50f7 Add cache OFF test results and analysis showing no performance impact
- Run test with cache disabled using local LLM mock

- Compare cache ON vs OFF: identical performance (~15s avg, <2% diff)

- Update problem_tracker.md with key findings summary

- Update test_config_123.yaml to use REDIS_PORT env var
2026-02-09 13:31:59 -08:00
Alexsander Hamir
918f25b0f5 Add local LLM cache-on test and update problem_tracker
- problem_tracker: add user none + cache ON run (local LLM), results table and note
- user_none_1_request_per_user_local_llm_cache_on.txt: 100 req, 100 users, cache ON
2026-02-09 12:30:36 -08:00
Alexsander Hamir
f01669ccde Add warmup option and validation for latency testing
- measure_latency.py: add --warmup, --warmup-verbose; print warmup latency per user
- auth_checks.py: add LITELLM_DEBUG_END_USER_CACHE for cache HIT/MISS logging
- problem_tracker: warmup findings (created mode, no latency improvement)
2026-02-09 11:53:52 -08:00
Alexsander Hamir
7615085e67 Add created mode to latency test and capture happy-path results
- measure_latency.py: add --user-mode created, --key-mode per_user
  Pre-creates end users via POST /customer/new and keys via /key/generate
- problem_tracker: add created-mode scenarios, checkboxes, and analysis
  Created mode (pre-existing users) shows better latency than random
  (non-existent users) at 10 req/user: 1.2s avg vs 1.5s, 10% vs 33% above 1s
2026-02-09 11:24:35 -08:00
Alexsander Hamir
611582287c measure_latency: add created user mode, require user-mode and key-mode
- Add --user-mode created to pre-create end users via /customer/new (requires --key-mode per_user)
- Remove defaults for --user-mode and --key-mode; both are now required
- Update problem_tracker.md Next Steps and Findings
2026-02-09 11:15:12 -08:00
Alexsander Hamir
56141a399e Add user-mode analysis: end_user lookups contribute to baseline latency 2026-02-09 10:22:30 -08:00
Alexsander Hamir
2c5a987a0a Improve baseline latency spikes section in problem tracker 2026-02-09 09:56:38 -08:00
Alexsander Hamir
847fd5e752 Update problem tracker 2026-02-06 09:06:31 -08:00
Alexsander Hamir
b1909e13ae Problem Tracker
This document trackers the progress on this issue until resolution
2026-02-06 09:02:27 -08:00