AlexsanderHamir
18c5ec8ff7
Add missing env key
2025-10-16 11:33:04 -07:00
Alexsander Hamir
170519057e
Merge pull request #15576 from BerriAI/litellm_remove_deepcopy
...
perf(router): use shallow copy instead of deepcopy for model aliases
2025-10-15 17:26:02 -07:00
AlexsanderHamir
294fb94b18
perf(router): use shallow copy instead of deepcopy for model aliases
...
Replace copy.deepcopy() with dict.copy() in _get_all_deployments when
creating model aliases.
Safe because:
1. Only modifies top-level 'model_name' field (isolated by shallow copy)
2. Nested dicts (litellm_params, model_info) are never modified after return
3. When model_alias=None, returns original dict with NO copy, proving
callers expect nested structures to be read-only
4. Key insight: Code that needs to modify nested structures does deepcopy FIRST.
This proves the contract is: \"treat returned deployments as read-only for
nested fields.\" (see line 6894: copy.deepcopy before modifying litellm_params)
Performance: ~10-100x faster than deepcopy on nested dict structures.
Tests: All 114 router unit tests pass.
2025-10-15 16:25:37 -07:00
AlexsanderHamir
d094a334d9
add: unit test
2025-10-14 16:38:04 -07:00
Ishaan Jaff
65163c7ccb
[Fix] GEMINI - CLI - add google_routes to llm_api_routes ( #15500 )
...
* fix: add google_routes to llm_api_routes
* test: test_virtual_key_llm_api_routes_allows_google_routes
2025-10-14 13:57:39 -07:00
AlexsanderHamir
5bffa58ee0
feat(ssl): add configurable ECDH curve for TLS performance
...
Add `ssl_ecdh_curve` setting to configure TLS key exchange curve.
Allows disabling PQC on OpenSSL 3.x for better performance.
Configurable via SDK (litellm.ssl_ecdh_curve), YAML (litellm_settings),
or env var (SSL_ECDH_CURVE). Common curves: X25519, prime256v1, secp384r1.
2025-10-14 13:57:39 -07:00
Krrish Dholakia
b0d963cc0c
docs(index.md): bump rc
2025-10-14 13:57:39 -07:00
AlexsanderHamir
fcb85f8856
perf(router): optimize timing functions in completion hot path
...
Replace time.time() with more appropriate timing functions for better
performance and reliability:
- Use time.perf_counter() for duration measurements in acompletion(),
_acompletion(), and async_get_available_deployment()
- Use time.monotonic() for timeout calculations in scheduler methods
(schedule_acompletion and _schedule_factory)
Benefits:
- 30-40% faster timing calls (~300ns savings per call)
- time.monotonic() provides reliable timeouts unaffected by system
clock changes (NTP adjustments, DST, manual time changes)
- time.perf_counter() offers highest resolution for performance metrics
- Follows Python best practices for timing operations
2025-10-14 13:57:39 -07:00
Krrish Dholakia
cb29e33cad
docs: fix doc
2025-10-14 13:57:39 -07:00
Dhruv Yadav
b57406e53e
add tests for openrouter cost tracking
2025-10-14 13:57:39 -07:00
Dhruv Yadav
09662b5081
direct cost calculation from openrouter
2025-10-14 13:57:39 -07:00
AlexsanderHamir
2e57d19a55
update benchmarks
2025-10-14 13:57:39 -07:00
AlexsanderHamir
6df051e87b
docs: update benchmark results with improved infrastructure
...
- Update to 2 instance baseline (1035 RPS @ 200ms median)
- Add 4 instance results (1170 RPS @ 100ms median)
- Update machine specs to 4 CPU / 8GB RAM
- Update Locust settings to 1000 users
2025-10-14 13:57:39 -07:00
kowyo
92bd238b87
fix comment
2025-10-14 13:57:39 -07:00
kowyo
6896c60b5b
fix: only use think level for gpt-oss model
2025-10-14 13:57:39 -07:00
Kowyo
9557a37d37
fix: update 'think' parameter assignment to use provided value in transformation.py
2025-10-14 13:57:39 -07:00
Kowyo
e30b716ba8
fix: add 'think' parameter handling in ollama_chat.py
2025-10-14 13:57:39 -07:00
Kowyo
4f33bc4e46
fix(ollama/chat): 'think' param handling
2025-10-14 13:57:39 -07:00
mubashir1osmani
6424480571
add ecs to docs
2025-10-14 13:57:39 -07:00
huangyafei
9a980f36d4
Add anthropic/claude-sonnet-4.5 to OpenRouter cost map
2025-10-14 13:57:39 -07:00
Krish Dholakia
ff20f8402a
Merge pull request #15456 from BerriAI/litellm_staging_branch_10_11_2025_p1
...
Litellm staging branch 10 11 2025 p1
2025-10-12 22:01:45 -07:00
Krrish Dholakia
0ffc81f010
fix: fix reformatting
2025-10-12 22:01:16 -07:00
Krrish Dholakia
1949436047
test: update test
2025-10-12 21:58:06 -07:00
Krish Dholakia
04dc9d091c
Merge branch 'main' into litellm_staging_branch_10_11_2025_p1
2025-10-12 21:55:44 -07:00
Ishaan Jaffer
e761665f22
docs fix
2025-10-11 18:21:35 -07:00
Imad Saddik
133189e191
Fixed a few typos ( #15267 )
2025-10-11 17:53:42 -07:00
Hampus Näsström
e650f39821
Reduce claude-4-sonnet max_output_tokens to 64k ( #15409 )
...
Claude 4 Sonnet doesn't support more than 64k output tokens: https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/claude/sonnet-4
2025-10-11 17:51:27 -07:00
Copilot
f5359ba007
Fix apply_guardrail endpoint returning raw string instead of ApplyGuardrailResponse ( #15436 )
...
* Initial plan
* Fix apply_guardrail endpoint to return ApplyGuardrailResponse
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
2025-10-11 17:50:37 -07:00
Alexsander Hamir
300a142926
[Add]: perf summary ( #15458 )
...
* add: perf summary
* update docs
* fix: focus on p99
2025-10-11 17:38:13 -07:00
Krrish Dholakia
78e2274381
fix(pass_through_endpoints.py): use path instead of endpoint
...
includes the mapped route
2025-10-11 17:04:56 -07:00
jlan-nl
328505dcda
Fix prices ( #15374 )
...
Co-authored-by: IQHL (Hans Jacob Landelius) <iqhl@novnordisk.com>
2025-10-11 16:53:55 -07:00
Ishaan Jaffer
087bd2d306
docs fix
2025-10-11 16:49:11 -07:00
Krrish Dholakia
3ee3cb945d
fix(team_endpoints.py): document new param
2025-10-11 16:45:55 -07:00
Krrish Dholakia
a71329fbe1
fix: fix linting errors
2025-10-11 16:42:22 -07:00
Krrish Dholakia
2266b272aa
fix(proxy/_types.py): fix default to be none
2025-10-11 16:41:03 -07:00
Krrish Dholakia
23870cbda0
fix(pass_through_endpoints.py): fix typo
2025-10-11 16:38:32 -07:00
Ishaan Jaffer
1ccd949495
docs fix
2025-10-11 16:34:47 -07:00
Ishaan Jaffer
b9fef02f04
docs v1.78.0
2025-10-11 16:33:46 -07:00
Ishaan Jaffer
7288f4355f
docs fix
2025-10-11 16:24:21 -07:00
Ishaan Jaffer
54b8bde60c
docs fix
2025-10-11 16:23:46 -07:00
Krrish Dholakia
a7456ab21e
refactor: refactor to cut down large functions
2025-10-11 16:18:37 -07:00
Ishaan Jaffer
8d29fc8cdd
docs
2025-10-11 16:12:00 -07:00
Krrish Dholakia
6bd722bba4
fix: always retain config models
2025-10-11 16:09:33 -07:00
Krrish Dholakia
8eb58db989
fix: fix linting error
2025-10-11 15:42:53 -07:00
Krrish Dholakia
090a2c0cfe
fix: remove unused uuid import
2025-10-11 15:38:28 -07:00
Krrish Dholakia
f96f2106f7
fix(pass_through_endpoints.py): fix check for mapped routes
2025-10-11 15:32:48 -07:00
Krish Dholakia
8c4c4acda2
Merge pull request #15457 from BerriAI/litellm_docs_10_11_2025
...
Litellm docs 10 11 2025
2025-10-11 15:16:13 -07:00
Krrish Dholakia
3f921d8dbe
docs(index.md): document mcp oauth support
2025-10-11 15:14:15 -07:00
Krrish Dholakia
9b5943ae50
docs: cleanup docs
2025-10-11 15:09:24 -07:00
Krrish Dholakia
fdaa163758
docs(claude_responses_api.md): update docs with mcp server tutorial
2025-10-11 15:07:54 -07:00
