- Add test_memory_baseline_100k for testing at scale (100,000 requests)
- Increase MEMORY_LIMIT from 50 MB to 60 MB for all router tests
- Memory limit adjustment accounts for observed usage patterns in 30k test
- Increase batch size from 20 to 100 requests (matches proxy test pattern)
- Add memory tracking and detailed logging throughout test execution
- Add periodic memory checks every 10 batches
- Direct task creation without intermediate validation for maximum speed
- Explicit cleanup of responses after each batch
- Average 300-400 req/s (was ~50-100 req/s before)
- Increase memory limit from 40 MB to 50 MB to account for larger batch size
- 1k test now completes in ~6.5 seconds (was ~20+ seconds)
- Add test_memory_baseline_30k to Router tests
- Add test_proxy_memory_baseline_30k and test_proxy_memory_baseline_250k to Proxy tests
- Increase proxy tests timeout from 30m to 45m to accommodate 250k test (~16 min runtime)
- Update memory_leak_tests job to use test_router_acompletion_memory_growth.py (renamed)
- Add new step to run proxy server memory growth tests
- Test both Router (1k, 2k, 4k, 10k) and Proxy (1k, 10k, 20k) in CI
- Reduce timeout from 60m to 30m per section for faster feedback
- Remove 30k router test from CI to keep runtime reasonable
- Rename test_linear_memory_growth.py to test_router_acompletion_memory_growth.py for clarity
- Add new test_proxy_chat_completions_memory_growth.py to test full proxy server memory usage
- Implement dynamic port allocation (18888-18919) in mock_server fixture for parallel test execution
- Add httpx client diagnostics to differentiate test artifacts from actual proxy memory growth
- Increase MEMORY_LIMIT to 170 MB based on empirical testing showing ~174 MB stable usage at scale
- Add aggressive cleanup routines with detailed logging to ensure accurate memory measurements
- Fix Windows Unicode compatibility by removing emoji characters from test output
- Tests now properly detect slow memory accumulation (~170 KB per 1000 requests) in proxy server
* fix: align max_tokens with max_output_tokens for consistency
Fixed inconsistent max_tokens definitions in model_prices_and_context_window.json.
According to LiteLLM convention, max_tokens should equal max_output_tokens when available.
Models fixed:
- deepseek-chat: 131072 → 8192 (now equals max_output_tokens)
- dashscope/qwen-flash: 1000000 → 32768 (now equals max_output_tokens)
- databricks/databricks-gemma-3-12b: 128000 → 32000 (now equals max_output_tokens)
This ensures consistency across all providers where max_tokens represents
the maximum number of tokens that can be generated in the output.
* fix: align max_tokens with max_output_tokens for 244 models
- Fix 244 models where max_tokens != max_output_tokens
- Add test to validate max_tokens consistency and prevent regressions
According to model_prices_and_context_window.json spec:
- max_tokens is a LEGACY parameter
- Should always equal max_output_tokens when both are present
This ensures consistency across all model definitions.
