Alexsander Hamir
278cafcf94
Optimize _get_model_cost_key to avoid expensive scans
...
- Remove expensive O(n) scan fallback that was causing 42.87% CPU overhead
- Only scan when size mismatch detected (O(1) check)
- Add warning in docstring: Only O(1) lookup operations are acceptable
- Clean up comments to be more concise
- Keep stale entry rebuild for pop() case (only triggers when stale entry found)
This fixes the performance issue where the scan was being triggered on every
failed lookup, causing severe CPU overhead during router operations.
2026-01-13 16:16:44 -08:00
Alexsander Hamir
a1dd3ead4d
[Perf] Remove bottleneck causing high CPU usage & overhead under heavy load ( #19049 )
2026-01-13 15:22:09 -08:00
Cesar Garcia
a445d8a4b4
fix(pricing): correct cache_read pricing for gemini-2.5-pro models ( #18157 )
...
- Fix cache_read_input_token_cost: 3.125e-07 → 1.25e-07 ($0.125/1M)
- Add cache_read_input_token_cost_above_200k_tokens: 2.5e-07 ($0.25/1M)
Models updated:
- gemini-2.5-pro
- gemini-2.5-pro-exp-03-25
- gemini-2.5-pro-preview-03-25
- gemini-2.5-pro-preview-05-06
- gemini-2.5-pro-preview-06-05
- gemini-2.5-pro-preview-tts
- gemini/gemini-2.5-pro
- gemini/gemini-2.5-pro-preview-03-25
- gemini/gemini-2.5-pro-preview-05-06
- gemini/gemini-2.5-pro-preview-06-05
- gemini/gemini-2.5-pro-preview-tts
Pricing source: https://ai.google.dev/gemini-api/docs/pricing
2026-01-14 04:09:24 +05:30
nulone
478bdcb60b
fix(model_prices): sync DeepSeek chat/reasoner to V3.2 pricing ( #18884 )
2026-01-14 03:52:50 +05:30
Cesar Garcia
d03c5017ff
fix: correct context window sizes for GPT-5 model variants ( #18928 )
...
* fix: correct context window sizes for GPT-5 model variants
Updates max_input_tokens for GPT-5, GPT-5.1, and GPT-5.2 model variants
to match OpenAI's official specifications, resolving issue #18927 .
Changes:
- GPT-5.1 (base, codex variants): 272k → 400k tokens
- GPT-5.1-chat variants: 272k → 128k tokens (with max_output 16,384)
- GPT-5 (base): 272k → 400k tokens
- GPT-5-chat: 272k → 128k tokens (with max_output 16,384)
- GPT-5-codex: 272k → 400k tokens
- GPT-5-mini: 272k → 400k tokens
- GPT-5-nano: 272k → 400k tokens
- GPT-5-pro: 272k → 400k tokens (max_output 272k)
- GPT-5 dated versions (2025-08-07): 272k → 400k tokens
Affected providers: OpenAI, Azure (all regions), OpenRouter
Fixes #18927
* fix: correct Azure GPT-5 context window limits to match Azure docs
Azure OpenAI has different limits than OpenAI for GPT-5 models.
Changes:
- Azure GPT-5 models: max_input_tokens 400k → 272k (Azure limit)
- Azure GPT-5 Pro: max_output_tokens 272k → 128k (Azure limit)
- OpenAI GPT-5 models: remain at 400k (correct)
- OpenRouter models: remain at 400k (routes to OpenAI)
Azure docs specify 272k input + 128k output = 400k total context.
OpenAI allows full 400k input + 128k output.
* fix: correct Azure GPT-5 max_input_tokens to 272k
Azure has explicit input limit of 272k tokens (not 400k like OpenAI).
Context window 400k = 272k input + 128k output for Azure.
OpenAI allows flexible input up to 400k (context - output).
2026-01-14 03:49:47 +05:30
Robin
b7c5662273
Fix: update novita models prices ( #19005 )
...
* feat: ci
* feat: fix novita models prices
2026-01-14 03:30:19 +05:30
Sameer Kankute
93203cda7c
Merge pull request #19027 from BerriAI/litellm_add_0_budget_model_bypass
...
[Feat] Add support for 0 cost models
2026-01-13 18:05:36 +05:30
Sameer Kankute
e98c2e4425
Merge pull request #19012 from BerriAI/litellm_fix_model_deployment_routing
...
Fix: Model matching priority in configuration
2026-01-13 17:55:01 +05:30
Sameer Kankute
bb0ab38636
Merge pull request #19007 from BerriAI/litellm_fix_header_forwarding_passthrough
...
Fix: Header forwarding in bedrock passthrough
2026-01-13 17:50:14 +05:30
Sameer Kankute
54f6f55c98
Merge pull request #18873 from BerriAI/litellm_staging_01_09_2026
...
staging 01/09/2025
2026-01-13 17:31:44 +05:30
Sameer Kankute
f2cb861d6a
Merge pull request #18976 from BerriAI/litellm_staging_12_19_2025
...
Staging 12/19/2025 - implement failopen option default to True on grayswan guardrail (#18266 )
2026-01-13 17:00:36 +05:30
Sameer Kankute
de6330b6b6
Fix test_async_otel_callback[False]
2026-01-13 16:59:17 +05:30
Sameer Kankute
1932d03aed
Add docs on Zero-Cost Models
2026-01-13 16:44:02 +05:30
Sameer Kankute
762a3ef090
Add support for 0 cost models
2026-01-13 16:39:57 +05:30
Sameer Kankute
d656f01bc9
Merge pull request #19009 from Dima-Mediator/fix-image-tokens-spend-logging
...
Fix image tokens spend logging for /images/generations
2026-01-13 15:03:37 +05:30
Sameer Kankute
5349d8922c
Merge pull request #19003 from BerriAI/litellm_add_azure_ai_claude_opus
...
Add pricing of azure_ai/claude-opus-4-5
2026-01-13 13:52:47 +05:30
YutaSaito
b3e126222f
Merge pull request #19013 from BerriAI/litellm_test_comment_out_flaky
...
[test] temporarily disable flaky responses_id_security tests
2026-01-13 15:52:57 +09:00
Yuta Saito
2c8ac2c3f1
test: temporarily disable flaky responses_id_security tests
2026-01-13 15:51:37 +09:00
Sameer Kankute
ecb3959c3c
Merge pull request #18208 from Chesars/fix/case-insensitive-model-cost-lookup
...
fix: case-insensitive model cost map lookup
2026-01-13 11:53:43 +05:30
Sameer Kankute
dfece51f8c
Fix: Model matching priority in configuration
2026-01-13 11:44:48 +05:30
Sameer Kankute
005541075b
Fix: Header forwarding in bedrock passthrough
2026-01-13 09:45:14 +05:30
Dima-Mediator
7c61933bc5
Fix image tokens spend logging for /images/generations
2026-01-12 23:07:08 -05:00
Sameer Kankute
5a51b74658
Add pricing of azure_ai/claude-opus-4-5
2026-01-13 09:15:05 +05:30
Ishaan Jaff
a1bba8c99b
[Feat] MSFT SSO - allow overriding env var attribute names ( #18998 )
...
* add MSFT SSO constants
* fix MSFT SSO env vars
* test_microsoft_sso_handler_openid_from_response_with_custom_attributes
2026-01-12 18:56:35 -08:00
Ishaan Jaffer
0feedfdf3d
ui new build
2026-01-12 18:55:18 -08:00
Ishaan Jaffer
dd959790bb
fix build
2026-01-12 18:53:42 -08:00
Ishaan Jaff
c4e6ae4d9e
[Feat] UI Feedback Form - why LiteLLM ( #18999 )
...
* init survey prompt
* init survey modal
* init Survey Modal
* POST feedback hook
* survey Modal
* add other
* in product survey fixes
* fix survey prompt
* fix survey
2026-01-12 18:48:17 -08:00
Sameer Kankute
a727aa9980
Merge pull request #18340 from Point72/ephrimstanley/fix-batch
...
Fix batch deletion and retrieve
2026-01-13 08:13:43 +05:30
YutaSaito
5bbd22070b
Merge pull request #18996 from BerriAI/litellm_release
...
bump: version 1.80.15 → 1.80.16
2026-01-13 11:27:35 +09:00
Ishaan Jaff
21d611554b
[Fix] MSFT SSO - allow setting custom MSFT Base URLs ( #18977 )
...
* fix TestCustomMicrosoftSSO
* init CustomMicrosoftSSO
* use CustomMicrosoftSSO
* docs fix
* docs fix
2026-01-12 18:26:53 -08:00
Yuta Saito
f149491498
bump: version 1.80.15 → 1.80.16
2026-01-13 11:21:57 +09:00
yuneng-jiang
c126cfd8db
Merge pull request #18994 from BerriAI/litellm_ui_key_refresh_fix
...
[Fix] UI - Refetch Keys after Key Create
2026-01-12 17:56:47 -08:00
YutaSaito
474107da0c
Merge pull request #18987 from BerriAI/litellm_fix_security_test
...
[fix] security test
2026-01-13 10:41:13 +09:00
YutaSaito
9114e44b49
Merge pull request #18993 from BerriAI/litellm_fix_remove_flaky_test
...
[test] remove flaky azure oidc embedding test
2026-01-13 10:37:30 +09:00
yuneng-jiang
b610dad3b8
fixing build
2026-01-12 17:34:47 -08:00
Yuta Saito
a57f1e2e08
test: remove flaky azure oidc embedding test
2026-01-13 10:34:01 +09:00
yuneng-jiang
7d468bc9e3
refetch after key create
2026-01-12 17:31:37 -08:00
Yuta Saito
4daac9e332
chore: add ALLOWED_CVES
2026-01-13 10:30:34 +09:00
Yuta Saito
54b21cabf7
chore: add config option
2026-01-13 10:19:12 +09:00
Yuta Saito
a0e0026df0
chore: document temporary grype ignore for CVE-2019-1010022
2026-01-13 10:09:23 +09:00
yuneng-jiang
9f9e5dd337
Merge pull request #18916 from BerriAI/litellm_ui_org_filters_1
...
[Feature] UI - Organization Table Filters
2026-01-12 15:57:37 -08:00
yuneng-jiang
c75a370619
Merge pull request #18910 from BerriAI/litellm_org_filtering_search
...
[Feature] /organization/list Query Params
2026-01-12 15:57:26 -08:00
yuneng-jiang
fb420a5de8
Merge pull request #18985 from BerriAI/litellm_e2e_neon_script_1
...
[Infra] UI - E2E Tests: Neon E2E DB Script
2026-01-12 15:56:22 -08:00
Ishaan Jaffer
f5fc38dcf9
docs litellm cloud
2026-01-12 13:55:56 -08:00
yuneng-jiang
685437c9cc
Adding help scripts for neon
2026-01-12 13:51:09 -08:00
YutaSaito
cc1da800f0
Merge pull request #18983 from BerriAI/litellm_fix_helm_chart_testing
...
[test] helm chart testing
2026-01-13 06:47:49 +09:00
Yuta Saito
b6bb90f252
test: helm chart testing
...
load local Docker image for helm chart tests
2026-01-13 06:40:57 +09:00
YutaSaito
9caf685f1e
Merge pull request #18940 from BerriAI/litellm_fix_extra_headers
...
[fix] forward MCP extra headers case-insensitively
2026-01-13 06:03:19 +09:00
Ephrim Stanley
99cb59c2d2
Add end to end integration tests for batches
2026-01-12 13:58:47 -05:00
Ephrim Stanley
2763b91960
Add end to end integration tests for batches
2026-01-12 13:55:27 -05:00
