scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-22 00:31:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 51
31938 commits 2566 branches 1413 tags 3.2 GiB
Commit graph

31938 commits

This branch
This branch
All branches
Author SHA1 Message Date
Alexsander Hamir
ab2980e092 Add cProfile prof files and problem_tracker.md 2026-02-09 16:35:40 -08:00
Alexsander Hamir
7100365f53 Add concurrency test results (1-10-30-50-100) and analysis to problem_tracker 2026-02-09 15:20:18 -08:00
Alexsander Hamir
56f4dc50f7 Add cache OFF test results and analysis showing no performance impact 
- Run test with cache disabled using local LLM mock

- Compare cache ON vs OFF: identical performance (~15s avg, <2% diff)

- Update problem_tracker.md with key findings summary

- Update test_config_123.yaml to use REDIS_PORT env var
 2026-02-09 13:31:59 -08:00
Alexsander Hamir
a0baf27c6c Use os.environ for sensitive config in test_config_123.yaml 
- database_url, master_key, cache_params (host/password), model api_key
- SLACK_WEBHOOK_URL, Langfuse keys (commented)
- Requires DATABASE_URL, MASTER_KEY, MODEL_API_KEY, REDIS_HOST, REDIS_PASSWORD
 2026-02-09 12:35:36 -08:00
Alexsander Hamir
4932aff55c Redact sensitive values in test_config_123.yaml 
- database_url: replace with REDACTED placeholders
- master_key: REDACTED_MASTER_KEY
- cache_params: host and password redacted
- litellm_params api_key: REDACTED_MODEL_API_KEY (all model entries)
- Langfuse keys in comments: REDACTED
 2026-02-09 12:33:42 -08:00
Alexsander Hamir
918f25b0f5 Add local LLM cache-on test and update problem_tracker 
- problem_tracker: add user none + cache ON run (local LLM), results table and note
- user_none_1_request_per_user_local_llm_cache_on.txt: 100 req, 100 users, cache ON
 2026-02-09 12:30:36 -08:00
Alexsander Hamir
f01669ccde Add warmup option and validation for latency testing 
- measure_latency.py: add --warmup, --warmup-verbose; print warmup latency per user
- auth_checks.py: add LITELLM_DEBUG_END_USER_CACHE for cache HIT/MISS logging
- problem_tracker: warmup findings (created mode, no latency improvement)
 2026-02-09 11:53:52 -08:00
Alexsander Hamir
1de09f3c02 Refactor measure_latency.py: SimulatedUser dataclass and helper functions 
- Add SimulatedUser dataclass (idx, num_requests, headers, payload)
- Extract build_simulated_users, run_user, print_summary
- Extract compute_request_distribution, setup_keys_and_end_users,
  print_run_banner, run_all_users
- Simplify main to orchestrate helpers
- Preserve all existing functionality
 2026-02-09 11:30:42 -08:00
Alexsander Hamir
7615085e67 Add created mode to latency test and capture happy-path results 
- measure_latency.py: add --user-mode created, --key-mode per_user
  Pre-creates end users via POST /customer/new and keys via /key/generate
- problem_tracker: add created-mode scenarios, checkboxes, and analysis
  Created mode (pre-existing users) shows better latency than random
  (non-existent users) at 10 req/user: 1.2s avg vs 1.5s, 10% vs 33% above 1s
 2026-02-09 11:24:35 -08:00
Alexsander Hamir
611582287c measure_latency: add created user mode, require user-mode and key-mode 
- Add --user-mode created to pre-create end users via /customer/new (requires --key-mode per_user)
- Remove defaults for --user-mode and --key-mode; both are now required
- Update problem_tracker.md Next Steps and Findings
 2026-02-09 11:15:12 -08:00
Alexsander Hamir
bcd6da2ea9 Add latency test output and end_user proxy logs 2026-02-09 11:13:43 -08:00
Alexsander Hamir
cb0bf2fbf8 Add user-mode latency run outputs (callbacks on/off, none/sequential/random) 2026-02-09 10:22:56 -08:00
Alexsander Hamir
56141a399e Add user-mode analysis: end_user lookups contribute to baseline latency 2026-02-09 10:22:30 -08:00
Alexsander Hamir
d778c5e821 Add --user-mode for measure_latency.py: none, sequential, random 2026-02-09 10:13:45 -08:00
Alexsander Hamir
2c5a987a0a Improve baseline latency spikes section in problem tracker 2026-02-09 09:56:38 -08:00
Alexsander Hamir
3feee539c5 Latency issue measurements 
Wether callbacks are on or not the issue is the same, the first request
of a user is expensive, and it gets worse if many users are making their
first request at the same time
 2026-02-09 09:54:08 -08:00
Alexsander Hamir
3f663aea8c Merge branch 'main' into litellm_perf_test_0004 2026-02-09 09:27:44 -08:00
Sameer Kankute
136fc698ef
Merge pull request #20601 from Harshit28j/litellm_fix_budget_model_v2 
fix conflicts with main- (this PR is from upstream/main)
 2026-02-09 20:08:27 +05:30
Sameer Kankute
6b2bcdb870
Merge pull request #20483 from BerriAI/litellm_completion_websearch 
[Feat] Chat completion - Add Websearch support using LiteLLM /search (using web search interception hook)
 2026-02-09 17:52:52 +05:30
Sameer Kankute
6158e46f00
Merge pull request #20747 from BerriAI/litellm_image_gen_bas_model_fix 
Fix: base_model name for body and deplyment name in URL
 2026-02-09 17:51:41 +05:30
Sameer Kankute
2d18ae4f9e Fix mypy issues 2026-02-09 17:44:39 +05:30
Sameer Kankute
125e11d36e Fix mypy issues 2026-02-09 17:43:52 +05:30
Sameer Kankute
5693e2c785
Merge pull request #20752 from BerriAI/litellm_fix_cicd_9_feb 
Fix: get_supported_anthropic_messages_params
 2026-02-09 17:38:39 +05:30
Sameer Kankute
5702cc7e13 Fix: get_supported_anthropic_messages_params 2026-02-09 17:38:03 +05:30
Sameer Kankute
0b5cb47c03 fix: Missing return statement for async streaming 2026-02-09 17:34:48 +05:30
Sameer Kankute
e5f41ba054
Merge pull request #20733 from BerriAI/litellm_v1_messages_claude_4_6 
[Feat]Add new claude 4-6 feat for v1/messages
 2026-02-09 17:26:40 +05:30
Sameer Kankute
5611974228 Fix : litellm/tests/test_litellm/llms/bedrock/chat/invoke_transformations/test_bedrock_chat_invoke_transformations_anthropic_claude3_transformation.py 2026-02-09 17:18:05 +05:30
Sameer Kankute
ef55d37bf0
Merge branch 'main' into litellm_v1_messages_claude_4_6 2026-02-09 17:14:36 +05:30
Sameer Kankute
30d17c29e4 handle when litellm_parrams might be none 2026-02-09 17:13:39 +05:30
Sameer Kankute
3cf109ed0c
Merge pull request #20745 from BerriAI/litellm_vercel_ai_models 
Add new vercel ai anthropic models
 2026-02-09 17:07:56 +05:30
Sameer Kankute
23088f86bd Add response schema for vercel ai sonnet 4.5 2026-02-09 17:07:36 +05:30
Sameer Kankute
493eaa6200
Merge pull request #20748 from BerriAI/litellm_anthropic_output_config 
Add output_config as supported param
 2026-02-09 17:04:08 +05:30
Sameer Kankute
a322db02ab
Merge pull request #20696 from BerriAI/litellm_oss_staging_02_08_2026 
Litellm oss staging 02 08 2026
 2026-02-09 16:59:54 +05:30
Sameer Kankute
285c00ac20
Merge pull request #20710 from tsachis/fix-video-list-pagination-id-encoding 
Fix video list pagination cursors not encoded with provider metadata
 2026-02-09 16:41:52 +05:30
Sameer Kankute
1b2278951d Add output_config as supported param 2026-02-09 16:39:21 +05:30
Sameer Kankute
f929461fc6
Merge pull request #20702 from emerzon/fix/issue-20698-stream-chunk-thinking-blocks 
fix(streaming): preserve interleaved thinking/redacted_thinking blocks
 2026-02-09 16:32:43 +05:30
Sameer Kankute
d35691aa0c Fix: base_model name for body and deplyment name in URL 2026-02-09 16:25:46 +05:30
Sameer Kankute
6c4d6bb15e Add new vercel ai anthropic models 2026-02-09 16:00:01 +05:30
Sameer Kankute
2f33445054
Merge pull request #20738 from BerriAI/main 
merge main
 2026-02-09 15:07:39 +05:30
Sameer Kankute
a924a07255
Merge pull request #20736 from CAFxX/verificationtoken-index 
add missing indexes on VerificationToken table
 2026-02-09 14:37:51 +05:30
Sameer Kankute
c48986ba8d Fix double json import 2026-02-09 14:21:43 +05:30
Sameer Kankute
1f04115fb0 Fix: is_web_search_tool_chat_completion 2026-02-09 13:59:05 +05:30
Sameer Kankute
7fa4d090ec Add doc for chat completion web search 2026-02-09 13:51:26 +05:30
Sameer Kankute
4e94ecb08d Add tests for WebSearch interception with chat completions API 2026-02-09 13:41:29 +05:30
Sameer Kankute
3d49388d8e Fix structured response of tool call 2026-02-09 13:36:01 +05:30
Carlo Alberto Ferraris
248fe65736
add missing indexes on VerificationToken table 2026-02-09 16:59:28 +09:00
Sameer Kankute
319453d059 Add documentation for Fast Mode 2026-02-09 11:39:35 +05:30
Sameer Kankute
c3b1c0a590 Add fast mode for other providers 2026-02-09 11:36:10 +05:30
Sameer Kankute
b822e2e0ff Add support for fast param 2026-02-09 11:28:00 +05:30
Sameer Kankute
29e6efade9 Add inference_geo as supported messages param 2026-02-09 10:57:04 +05:30