Alexsander Hamir
1de09f3c02
Refactor measure_latency.py: SimulatedUser dataclass and helper functions
...
- Add SimulatedUser dataclass (idx, num_requests, headers, payload)
- Extract build_simulated_users, run_user, print_summary
- Extract compute_request_distribution, setup_keys_and_end_users,
print_run_banner, run_all_users
- Simplify main to orchestrate helpers
- Preserve all existing functionality
2026-02-09 11:30:42 -08:00
Alexsander Hamir
7615085e67
Add created mode to latency test and capture happy-path results
...
- measure_latency.py: add --user-mode created, --key-mode per_user
Pre-creates end users via POST /customer/new and keys via /key/generate
- problem_tracker: add created-mode scenarios, checkboxes, and analysis
Created mode (pre-existing users) shows better latency than random
(non-existent users) at 10 req/user: 1.2s avg vs 1.5s, 10% vs 33% above 1s
2026-02-09 11:24:35 -08:00
Alexsander Hamir
611582287c
measure_latency: add created user mode, require user-mode and key-mode
...
- Add --user-mode created to pre-create end users via /customer/new (requires --key-mode per_user)
- Remove defaults for --user-mode and --key-mode; both are now required
- Update problem_tracker.md Next Steps and Findings
2026-02-09 11:15:12 -08:00
Alexsander Hamir
bcd6da2ea9
Add latency test output and end_user proxy logs
2026-02-09 11:13:43 -08:00
Alexsander Hamir
cb0bf2fbf8
Add user-mode latency run outputs (callbacks on/off, none/sequential/random)
2026-02-09 10:22:56 -08:00
Alexsander Hamir
56141a399e
Add user-mode analysis: end_user lookups contribute to baseline latency
2026-02-09 10:22:30 -08:00
Alexsander Hamir
d778c5e821
Add --user-mode for measure_latency.py: none, sequential, random
2026-02-09 10:13:45 -08:00
Alexsander Hamir
2c5a987a0a
Improve baseline latency spikes section in problem tracker
2026-02-09 09:56:38 -08:00
Alexsander Hamir
3feee539c5
Latency issue measurements
...
Wether callbacks are on or not the issue is the same, the first request
of a user is expensive, and it gets worse if many users are making their
first request at the same time
2026-02-09 09:54:08 -08:00
Alexsander Hamir
3f663aea8c
Merge branch 'main' into litellm_perf_test_0004
2026-02-09 09:27:44 -08:00
Sameer Kankute
136fc698ef
Merge pull request #20601 from Harshit28j/litellm_fix_budget_model_v2
...
fix conflicts with main- (this PR is from upstream/main)
2026-02-09 20:08:27 +05:30
Sameer Kankute
6b2bcdb870
Merge pull request #20483 from BerriAI/litellm_completion_websearch
...
[Feat] Chat completion - Add Websearch support using LiteLLM /search (using web search interception hook)
2026-02-09 17:52:52 +05:30
Sameer Kankute
6158e46f00
Merge pull request #20747 from BerriAI/litellm_image_gen_bas_model_fix
...
Fix: base_model name for body and deplyment name in URL
2026-02-09 17:51:41 +05:30
Sameer Kankute
2d18ae4f9e
Fix mypy issues
2026-02-09 17:44:39 +05:30
Sameer Kankute
125e11d36e
Fix mypy issues
2026-02-09 17:43:52 +05:30
Sameer Kankute
5693e2c785
Merge pull request #20752 from BerriAI/litellm_fix_cicd_9_feb
...
Fix: get_supported_anthropic_messages_params
2026-02-09 17:38:39 +05:30
Sameer Kankute
5702cc7e13
Fix: get_supported_anthropic_messages_params
2026-02-09 17:38:03 +05:30
Sameer Kankute
0b5cb47c03
fix: Missing return statement for async streaming
2026-02-09 17:34:48 +05:30
Sameer Kankute
e5f41ba054
Merge pull request #20733 from BerriAI/litellm_v1_messages_claude_4_6
...
[Feat]Add new claude 4-6 feat for v1/messages
2026-02-09 17:26:40 +05:30
Sameer Kankute
5611974228
Fix : litellm/tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.py
2026-02-09 17:18:05 +05:30
Sameer Kankute
ef55d37bf0
Merge branch 'main' into litellm_v1_messages_claude_4_6
2026-02-09 17:14:36 +05:30
Sameer Kankute
30d17c29e4
handle when litellm_parrams might be none
2026-02-09 17:13:39 +05:30
Sameer Kankute
3cf109ed0c
Merge pull request #20745 from BerriAI/litellm_vercel_ai_models
...
Add new vercel ai anthropic models
2026-02-09 17:07:56 +05:30
Sameer Kankute
23088f86bd
Add response schema for vercel ai sonnet 4.5
2026-02-09 17:07:36 +05:30
Sameer Kankute
493eaa6200
Merge pull request #20748 from BerriAI/litellm_anthropic_output_config
...
Add output_config as supported param
2026-02-09 17:04:08 +05:30
Sameer Kankute
a322db02ab
Merge pull request #20696 from BerriAI/litellm_oss_staging_02_08_2026
...
Litellm oss staging 02 08 2026
2026-02-09 16:59:54 +05:30
Sameer Kankute
285c00ac20
Merge pull request #20710 from tsachis/fix-video-list-pagination-id-encoding
...
Fix video list pagination cursors not encoded with provider metadata
2026-02-09 16:41:52 +05:30
Sameer Kankute
1b2278951d
Add output_config as supported param
2026-02-09 16:39:21 +05:30
Sameer Kankute
f929461fc6
Merge pull request #20702 from emerzon/fix/issue-20698-stream-chunk-thinking-blocks
...
fix(streaming): preserve interleaved thinking/redacted_thinking blocks
2026-02-09 16:32:43 +05:30
Sameer Kankute
d35691aa0c
Fix: base_model name for body and deplyment name in URL
2026-02-09 16:25:46 +05:30
Sameer Kankute
6c4d6bb15e
Add new vercel ai anthropic models
2026-02-09 16:00:01 +05:30
Sameer Kankute
2f33445054
Merge pull request #20738 from BerriAI/main
...
merge main
2026-02-09 15:07:39 +05:30
Sameer Kankute
a924a07255
Merge pull request #20736 from CAFxX/verificationtoken-index
...
add missing indexes on VerificationToken table
2026-02-09 14:37:51 +05:30
Sameer Kankute
c48986ba8d
Fix double json import
2026-02-09 14:21:43 +05:30
Sameer Kankute
1f04115fb0
Fix: is_web_search_tool_chat_completion
2026-02-09 13:59:05 +05:30
Sameer Kankute
7fa4d090ec
Add doc for chat completion web search
2026-02-09 13:51:26 +05:30
Sameer Kankute
4e94ecb08d
Add tests for WebSearch interception with chat completions API
2026-02-09 13:41:29 +05:30
Sameer Kankute
3d49388d8e
Fix structured response of tool call
2026-02-09 13:36:01 +05:30
Carlo Alberto Ferraris
248fe65736
add missing indexes on VerificationToken table
2026-02-09 16:59:28 +09:00
Sameer Kankute
319453d059
Add documentation for Fast Mode
2026-02-09 11:39:35 +05:30
Sameer Kankute
c3b1c0a590
Add fast mode for other providers
2026-02-09 11:36:10 +05:30
Sameer Kankute
b822e2e0ff
Add support for fast param
2026-02-09 11:28:00 +05:30
Sameer Kankute
29e6efade9
Add inference_geo as supported messages param
2026-02-09 10:57:04 +05:30
Sameer Kankute
20440bcadc
Add inference based costing
2026-02-09 10:51:50 +05:30
Sameer Kankute
3307f3d1c6
Add inference_geo as supported messages param
2026-02-09 10:47:13 +05:30
Sameer Kankute
d41df6053a
Add all new feat for v1/messages
2026-02-09 10:35:11 +05:30
Sameer Kankute
8cd8a01d5a
Add compaction for vertex ai
2026-02-09 10:18:19 +05:30
Sameer Kankute
aaa48f8ad2
Merge pull request #20712 from emerzon/fix/responses-tool-call-delta-index-mapping
...
fix(responses): preserve tool call argument deltas when streaming id is omitted
2026-02-09 08:43:04 +05:30
Emerson Gomes
cf17a440cd
fix(responses): guard ambiguous tool-call index reuse
2026-02-08 08:53:17 -06:00
Emerson Gomes
68d788c84d
fix(responses): preserve streamed tool deltas when id is omitted
2026-02-08 08:48:39 -06:00
