rohan.w.charles
bc88dbfbc9
fix: implement mid-stream fallback handling based on user preference
2026-01-14 16:28:51 +05:30
rohan.w.charles
e70b1bbfad
fix: add handling for user-disabled mid-stream fallbacks
2026-01-14 16:04:36 +05:30
Sameer Kankute
b2d4f67e17
Merge pull request #19023 from eagle-p/feat/guardrail-clean-error-message
...
fix(guardrails): use clean error messages for blocked requests
2026-01-14 14:03:16 +05:30
Sameer Kankute
11cc8dc9d2
Merge pull request #19037 from Jetemple/json-log-fix
...
fix: enable JSON logging via configuration and add regression test
2026-01-14 13:51:36 +05:30
YutaSaito
8d204f0a84
Merge pull request #19060 from BerriAI/litellm_fix_anthropic-during-call-error
...
[fix] anthropic during call error
2026-01-14 15:13:09 +09:00
YutaSaito
5de0e97539
Merge pull request #19063 from BerriAI/litellm_docs_auth_message_ui
...
[docs] auth message UI
2026-01-14 15:09:37 +09:00
Yuta Saito
16ebfb0d5c
docs: add mcp faq
2026-01-14 15:08:31 +09:00
Yuta Saito
2d3c292cd8
chore: Clarify MCP OAuth token handling in UI
2026-01-14 14:31:18 +09:00
yuneng-jiang
39f56abd8d
Merge pull request #18878 from BerriAI/litellm_hakhundov_contribution
...
[Fix] Feature Flag for Fix SCIM GET /Users error and enforce SCIM 2.0 compliance
2026-01-13 20:46:28 -08:00
Yuta Saito
82ed6283fa
test: reorganize unified guardrail tests into nested classes
2026-01-14 13:38:43 +09:00
Yuta Saito
c5ced033c9
fix: anthropic during call guardrail error
2026-01-14 13:37:01 +09:00
yuneng-jiang
d5f05f6c23
Merge pull request #19055 from BerriAI/litellm_ui_usage_model_act_fix
...
[Fix] UI - Usage: Model Activity Chart Y Axis
2026-01-13 19:53:08 -08:00
Cesar Garcia
2b79d37e39
feat: Add Cerebras zai-glm-4.7 model support and deprecate zai-glm-4.6 ( #18864 )
...
- Add cerebras/zai-glm-4.7 with same specs as 4.6 (128K context, $2.25/M input, $2.75/M output)
- Mark cerebras/zai-glm-4.6 with deprecation_date: 2026-01-20
- Both models support function calling, reasoning, and tool choice
2026-01-13 19:45:02 -08:00
YutaSaito
95b419d65b
Merge pull request #18938 from BerriAI/litellm_fix_generate_two_telemetry
...
[fix] generate two telemetry in responses
2026-01-14 12:35:23 +09:00
Ishaan Jaffer
fd46140fef
add call flow
2026-01-13 19:19:19 -08:00
Ishaan Jaffer
cf82236176
docs fix
2026-01-13 19:18:36 -08:00
Ishaan Jaff
d1acfcdb28
[Docs] Add Litellm architecture md doc ( #19057 )
...
* v0 of this
* remove bloat
* bring back older diagram
* fix
* fix
* docs fix
* fix rate limiting
* docs fix
* fix
* doc fix
* docs fix
* add SDK level fixes
* docs fix
2026-01-13 19:05:12 -08:00
Ishaan Jaff
1b00576711
[Feat] New Model - Azure Model Router on LiteLLM AI Gateway ( #19054 )
...
* fix - azure model router integration
* fix:_check_provider_match
* fix:_get_response_model
* tests azure model router
* test_azure_ai_model_router_streaming_model_in_chunk
* fix LlmProviders.AZURE.value
* test_azure_ai_model_router_streaming_cost_with_stream_options
* def test_get_model_from_chunks_azure_model_router():
* _get_model_from_chunks
* docs azure model router
* azure model router
2026-01-13 18:31:43 -08:00
yuneng-jiang
851006790c
Fix model list not showing
2026-01-13 18:16:26 -08:00
Yuta Saito
a1bceb8403
fix: anthropic Extra inputs are not permitted error
2026-01-14 11:15:56 +09:00
Yuta Saito
2814d3dd3d
test: cover success handler sync/async logging branches
2026-01-14 10:52:14 +09:00
YutaSaito
a66e007574
Merge pull request #19051 from BerriAI/litellm_fix_mcp-rest-auth-checks
...
[fix] mcp rest auth checks
2026-01-14 10:39:28 +09:00
Yuta Saito
c3d0dd6897
fix: add aresponses litellm_params
2026-01-14 10:30:56 +09:00
Yuta Saito
b71e30a8ac
fix: mastake env variable name
2026-01-14 10:30:55 +09:00
Yuta Saito
fde68e4a29
fix: mastake env variable name
2026-01-14 10:30:55 +09:00
Yuta Saito
bc02d462ff
fix: add call type aresponses
2026-01-14 10:30:55 +09:00
Yuta Saito
6ca9282857
refactor: make variable
2026-01-14 10:30:55 +09:00
yuneng-jiang
7c57138398
Remove stack from AreaChart
2026-01-13 17:28:02 -08:00
Alexsander Hamir
15c3bc219b
[Refactor] Add CI enforcement for O(1) operations in _get_model_cost_key to prevent performance regressions ( #19052 )
...
* Optimize _get_model_cost_key to avoid expensive scans
- Remove expensive O(n) scan fallback that was causing 42.87% CPU overhead
- Only scan when size mismatch detected (O(1) check)
- Add warning in docstring: Only O(1) lookup operations are acceptable
- Clean up comments to be more concise
- Keep stale entry rebuild for pop() case (only triggers when stale entry found)
This fixes the performance issue where the scan was being triggered on every
failed lookup, causing severe CPU overhead during router operations.
* Add code quality check to enforce O(1) operations in _get_model_cost_key
- Add check_get_model_cost_key_performance.py to statically analyze _get_model_cost_key
- Detects O(n) operations (loops, comprehensions, problematic function calls)
- Recursively checks called functions to find nested O(n) operations
- Allows conditional O(n) rebuilds in helper functions (_rebuild_model_cost_lowercase_map, _handle_stale_map_entry_rebuild, _handle_new_key_with_scan)
* Integrate _get_model_cost_key performance check into CI pipeline
- Add check_get_model_cost_key_performance.py to check_code_and_doc_quality job
- Ensures O(1) requirement is enforced in CI to prevent performance regressions
* Remove unused performance test and clean up utils.py
- Remove test_get_model_info_performance.py (no longer needed)
- Remove extra blank line in utils.py
* Document allowed helper functions and exception process in _get_model_cost_key
- Add documentation listing allowed helper functions with O(n) operations
- Explain why these are acceptable (conditionally called)
- Add instructions for adding new exceptions to check_get_model_cost_key_performance.py
* Fix docstring detection and type checker error in performance check
- Add proper docstring tracking to skip docstring content (fixes false positive for 'map' in docstring)
- Add None check for docstring_quote to fix type checker error
- Restore _handle_new_key_with_scan to allowed_helpers list
* Remove check_get_model_cost_key_performance from CI pipeline
- Temporarily remove the performance check from CI to avoid blocking builds
* Restore performance check and remove memory leak tests from CI
- Add back check_get_model_cost_key_performance.py to CI pipeline
- Remove memory_leak_tests job that was causing port conflicts
* Remove extra blank line in CI config
2026-01-13 17:08:03 -08:00
yuneng-jiang
2ea6fcb584
Merge pull request #19050 from BerriAI/litellm_ui_usage_top_x
...
[Feature] UI - Usage: Allow Top Virtual Keys and Models to Show More Entries
2026-01-13 17:06:07 -08:00
Raghav Jhavar
272a48d880
[bug fix] do not fallback to token counter if disable_token_counter is enabled ( #19041 )
...
* do not fallback to token counter if disable_token_counter is enabled, and return errors instead
* add exceptions and exception utils to map the same as /v1/chat/completions
* use safe_json_loads
2026-01-13 16:53:38 -08:00
Yuta Saito
b54b12d0cc
fix: lint
2026-01-14 09:53:25 +09:00
yuneng-jiang
c05190f5b1
Merge remote-tracking branch 'origin' into litellm_hakhundov_contribution
2026-01-13 16:27:48 -08:00
yuneng-jiang
70987ffdc8
Merge pull request #19047 from BerriAI/litellm_usage_column_names
...
[Fix] UI - Usage: Team ID and Team Name in Export Report
2026-01-13 16:26:26 -08:00
yuneng-jiang
212eec5a6c
Merge pull request #18997 from BerriAI/litellm_ui_key_generate_readable
...
[Feature] UI - Simplify Key Generate Permission Error
2026-01-13 16:26:15 -08:00
yuneng-jiang
52d5eccf3d
Merge pull request #19010 from BerriAI/litellm_ui_filter_components
...
[Refactor] UI - User and Team Table Filters to Reusable Component
2026-01-13 16:26:07 -08:00
yuneng-jiang
e6ca97a9a5
top model show N
2026-01-13 16:21:14 -08:00
yuneng-jiang
ddf2b24901
Top virtual keys show N select
2026-01-13 15:50:34 -08:00
Alexsander Hamir
a1dd3ead4d
[Perf] Remove bottleneck causing high CPU usage & overhead under heavy load ( #19049 )
2026-01-13 15:22:09 -08:00
Yuta Saito
df37770a70
test: add permission test
2026-01-14 07:58:21 +09:00
yuneng-jiang
308d24d067
Resolve Team ID and Team name in export
2026-01-13 14:42:28 -08:00
Cesar Garcia
a445d8a4b4
fix(pricing): correct cache_read pricing for gemini-2.5-pro models ( #18157 )
...
- Fix cache_read_input_token_cost: 3.125e-07 → 1.25e-07 ($0.125/1M)
- Add cache_read_input_token_cost_above_200k_tokens: 2.5e-07 ($0.25/1M)
Models updated:
- gemini-2.5-pro
- gemini-2.5-pro-exp-03-25
- gemini-2.5-pro-preview-03-25
- gemini-2.5-pro-preview-05-06
- gemini-2.5-pro-preview-06-05
- gemini-2.5-pro-preview-tts
- gemini/gemini-2.5-pro
- gemini/gemini-2.5-pro-preview-03-25
- gemini/gemini-2.5-pro-preview-05-06
- gemini/gemini-2.5-pro-preview-06-05
- gemini/gemini-2.5-pro-preview-tts
Pricing source: https://ai.google.dev/gemini-api/docs/pricing
2026-01-14 04:09:24 +05:30
nulone
478bdcb60b
fix(model_prices): sync DeepSeek chat/reasoner to V3.2 pricing ( #18884 )
2026-01-14 03:52:50 +05:30
Cesar Garcia
d03c5017ff
fix: correct context window sizes for GPT-5 model variants ( #18928 )
...
* fix: correct context window sizes for GPT-5 model variants
Updates max_input_tokens for GPT-5, GPT-5.1, and GPT-5.2 model variants
to match OpenAI's official specifications, resolving issue #18927 .
Changes:
- GPT-5.1 (base, codex variants): 272k → 400k tokens
- GPT-5.1-chat variants: 272k → 128k tokens (with max_output 16,384)
- GPT-5 (base): 272k → 400k tokens
- GPT-5-chat: 272k → 128k tokens (with max_output 16,384)
- GPT-5-codex: 272k → 400k tokens
- GPT-5-mini: 272k → 400k tokens
- GPT-5-nano: 272k → 400k tokens
- GPT-5-pro: 272k → 400k tokens (max_output 272k)
- GPT-5 dated versions (2025-08-07): 272k → 400k tokens
Affected providers: OpenAI, Azure (all regions), OpenRouter
Fixes #18927
* fix: correct Azure GPT-5 context window limits to match Azure docs
Azure OpenAI has different limits than OpenAI for GPT-5 models.
Changes:
- Azure GPT-5 models: max_input_tokens 400k → 272k (Azure limit)
- Azure GPT-5 Pro: max_output_tokens 272k → 128k (Azure limit)
- OpenAI GPT-5 models: remain at 400k (correct)
- OpenRouter models: remain at 400k (routes to OpenAI)
Azure docs specify 272k input + 128k output = 400k total context.
OpenAI allows full 400k input + 128k output.
* fix: correct Azure GPT-5 max_input_tokens to 272k
Azure has explicit input limit of 272k tokens (not 400k like OpenAI).
Context window 400k = 272k input + 128k output for Azure.
OpenAI allows flexible input up to 400k (context - output).
2026-01-14 03:49:47 +05:30
Robin
b7c5662273
Fix: update novita models prices ( #19005 )
...
* feat: ci
* feat: fix novita models prices
2026-01-14 03:30:19 +05:30
Yuta Saito
22aad95bb1
fix: rest_endpoints own allowed_mcp_servers for MCP calls
2026-01-14 06:39:46 +09:00
Jack Temple
9e08c2207f
fix: enable JSON logging via configuration and add regression test
2026-01-13 09:38:19 -07:00
Sameer Kankute
93203cda7c
Merge pull request #19027 from BerriAI/litellm_add_0_budget_model_bypass
...
[Feat] Add support for 0 cost models
2026-01-13 18:05:36 +05:30
Sameer Kankute
e98c2e4425
Merge pull request #19012 from BerriAI/litellm_fix_model_deployment_routing
...
Fix: Model matching priority in configuration
2026-01-13 17:55:01 +05:30
Sameer Kankute
bb0ab38636
Merge pull request #19007 from BerriAI/litellm_fix_header_forwarding_passthrough
...
Fix: Header forwarding in bedrock passthrough
2026-01-13 17:50:14 +05:30
