Commit graph

14665 commits

Author SHA1 Message Date
mateo-berri
c330d466f3 fix(anthropic): normalize oauth authorization header to one canonical casing 2026-08-26 11:45:51 -07:00
Mateo Wang
767e6015af
Merge pull request #34658 from BerriAI/litellm_azure_realtime_entra_id_auth
fix(azure/realtime): authenticate realtime websocket with Azure AD token when no api-key
2026-08-26 11:44:38 -07:00
Mateo Wang
2d3144c6c8
Merge pull request #38368 from BerriAI/litellm_fix_bedrock_mantle_gpt55_gpt54_context_window
fix(model_prices): raise bedrock_mantle gpt-5.5 and gpt-5.4 max_input_tokens to Mantle's enforced 1050000
2026-08-26 11:42:47 -07:00
mateo-berri
dbe52a80c1 test(cost): wrap overlong us data residency test declaration 2026-08-26 11:42:28 -07:00
mateo-berri
e97a84afcf fix(model_prices): add above-272k priority rates to azure us/eu gpt-5.6 entries 2026-08-26 11:40:19 -07:00
yuneng-jiang
7bc80994b7
Merge pull request #38305 from BerriAI/litellm_/testing-strategy-audit-c39e33
fix(ci): let the mutation workflow find covered lines so it generates mutants
2026-08-26 11:34:08 -07:00
mateo-berri
9c38f6f125 test(tencent): drive thinking tests off the real cost map instead of patched internals 2026-08-26 11:34:06 -07:00
yuneng-jiang
b03e913ccf
Merge pull request #38315 from BerriAI/litellm_cost_estimate_pricing_edges
test(cost-estimate): pin the prices and period totals /cost/estimate returns
2026-08-26 11:32:22 -07:00
mateo-berri
8307be68c9 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr38100_tencent_thinking 2026-08-26 11:26:58 -07:00
mateo-berri
229e136783 fix(mcp): honor admin-entered OAuth URLs on authorize after issuer yield
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-26 11:26:54 -07:00
Mateo Wang
ca6387fae7
Merge pull request #38271 from daniel-meismer-zocdoc/litellm_mcp_bridge_provider_token_lifetime
fix(mcp): preserve provider access token lifetime
2026-08-26 11:25:55 -07:00
Mateo Wang
def5ca6d68
Merge pull request #38299 from BerriAI/litellm_fix_vertex_pt_custom_auth_google_token
fix(proxy): keep the caller's Google token on credential-less Vertex passthrough under custom auth
2026-08-26 11:24:56 -07:00
yuneng-jiang
309da5e70e
Merge branch 'litellm_internal_staging' into litellm_/testing-strategy-audit-c39e33 2026-08-26 11:24:55 -07:00
mateo-berri
a0d1fef89d fix(anthropic): scale cache costs by fast mode and trust served speed 2026-08-26 11:24:12 -07:00
mateo-berri
8ff832901b fix(health): accept image_edit mode on /health/test_connection 2026-08-26 11:18:50 -07:00
mateo-berri
2068066d69 fix(anthropic): detect oauth Authorization header case-insensitively 2026-08-26 11:16:17 -07:00
Devin AI
f3c1e2e2a7 fix(guardrails): forward aws_external_id when the bedrock guardrail assumes a role
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 18:12:15 +00:00
Mateo Wang
47cfd3f5ca
Merge pull request #38371 from BerriAI/litellm_fix_claude3_1hr_cache_pricing
fix(model_prices): price 1-hour cache writes on claude-3-haiku and claude-3-opus at 2x input
2026-08-26 11:11:46 -07:00
Mateo Wang
a6754fa619
Merge pull request #38272 from daniel-meismer-zocdoc/litellm_hidden_alias_explicit_lookup
fix(router): resolve hidden aliases for explicit lookup
2026-08-26 10:56:26 -07:00
Mateo Wang
d46bcde2b5
Merge pull request #37922 from BerriAI/litellm_techdebt_20260822
chore(typing): roll up the daily tech debt cleanups from Aug 20 to Aug 26
2026-08-26 10:54:50 -07:00
mateo-berri
764048750e fix(mcp): name invalid_token challenges by the caller's requested spelling 2026-08-26 10:52:28 -07:00
mateo-berri
b349b9bf50 fix(pricing): add azure gpt-5.6 cache write rates and correct data zone priority 2026-08-26 10:50:44 -07:00
mateo-berri
6416a97a4d fix(model_prices): price 1-hour cache writes on claude-3-haiku and claude-3-opus at 2x input 2026-08-26 10:48:49 -07:00
mateo-berri
dcffd1da52 refactor(redis): drop docstrings restating the code 2026-08-26 10:45:56 -07:00
mateo-berri
a2cd2d8a4b fix(cost-map): add US data residency uplift to claude-sonnet-4-6 and mythos entries 2026-08-26 10:43:43 -07:00
mateo-berri
a3f654719e fix(proxy): strip the JWT that authenticated on credential-less Vertex passthrough 2026-08-26 10:43:04 -07:00
mateo-berri
b97d5e77eb fix(model_prices): raise bedrock_mantle gpt-5.5 and gpt-5.4 max_input_tokens to Mantle's enforced 1050000 2026-08-26 10:42:00 -07:00
Mateo Wang
95285c3433
Merge pull request #38211 from eugene-yao-zocdoc/litellm_anthropic_responses_strictness_pr
fix(anthropic-responses): preserve structured output strictness
2026-08-26 10:39:45 -07:00
devin-ai-integration[bot]
c11c654b8e
fix(proxy): honor DATABASE_DISABLE_PREPARED_STATEMENTS in componentized entrypoints (#38363)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 10:39:04 -07:00
mateo-berri
0bfc733278 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_techdebt_20260822 2026-08-26 10:35:55 -07:00
mateo-berri
c449f11451 fix(anthropic): detect client credential headers case-insensitively on /v1/messages 2026-08-26 10:35:39 -07:00
Devin AI
07c9812739 fix(model_prices): carry anthropic behavior flags on deepinfra claude entries, move retired together models to deprecated list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 17:18:51 +00:00
Devin AI
d266326a42 fix(model_prices): azure gpt-4.1-nano retirement date, together deprecations, novita gpt-oss-120b vision flag, fireworks deepseek-v4-pro-0813
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-26 16:56:53 +00:00
Devin AI
9300018414 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_bedrock_sol_anthropic_1hr 2026-08-26 16:46:44 +00:00
Mateo Wang
cdb60af024
Merge pull request #38269 from BerriAI/litellm_together_structured_outputs
fix(together_ai): fail open on response_format instead of dropping it for unregistered models
2026-08-26 09:34:56 -07:00
Mateo Wang
c81ceba431
Merge pull request #38232 from BerriAI/litellm_e2e_bedrock_customer_matrix
test(e2e): cover the Bedrock provider-feature cells customers run
2026-08-26 09:34:07 -07:00
ryan-crabbe-berri
597b4bb239
Merge pull request #38327 from BerriAI/litellm_tiered_pricing_rate_fallbacks
test(cost-calc): pin the rate fallbacks inside a tiered-pricing tier
2026-08-26 09:20:56 -07:00
Yuneng Jiang
3fdeb79737
fix(test): keep the mutmut sentinel out of the cleared environment
test_google_login_only_threads_user_code_when_enabled cleared the whole
process environment for the duration of the call. mutmut's trampoline reads
os.environ['MUTANT_UNDER_TEST'] with a bare subscript, so the first
trampolined callee inside the block, _get_cli_sso_flow_or_raise, raised
KeyError. The bare `except Exception: pass` swallowed it and the assertion
then read call_args on a mock that was never called, which is where
"'NoneType' object has no attribute 'kwargs'" came from.

The test only needs the SSO provider variables unset, so it now preserves the
rest of the environment instead of clearing everything. google_login does not
raise here, so the try/except is gone and any future exception propagates; the
added assert turns a silent early return into a readable failure instead of an
AttributeError.

Root cause measured in a trampolined copy of the mutated folder with
MUTANT_UNDER_TEST=stats: the old test fails there with
KeyError: 'MUTANT_UNDER_TEST' inside _mutmut_trampoline, the new one passes.
That was the only test the mutation run could not execute, so the --deselect
comes back out and it counts toward the score again.
2026-08-26 08:19:56 -07:00
ksk2023
a76e224a5f fix(cost_calculator): resolve real cost key when model_name alias contains '/'
When a router-facing model_name alias contains a '/' whose leading segment
is not a registered provider (e.g. 'vertex/claude-opus-5' for deployment
'vertex_ai/claude-opus-5'), _select_model_name_for_cost_calc re-prefixed
it into a non-existent key ('vertex_ai/vertex/claude-opus-5'), so cost
lookup silently priced every streamed request at $0 - token counts were
recorded, no error raised, budgets never tripped.

After prefixing, walk the alias tail and return the first assembly that
exists in litellm.model_cost ('vertex_ai/claude-opus-5'). Provider/region
segments in the head are preserved, and an alias that resolves to no
known key keeps the previous behavior (no crash, legacy double-prefix).

Fixes #38069
2026-08-26 22:06:06 +08:00
Mateo Wang
40423e6ec0
Merge pull request #38325 from BerriAI/litellm_fix_responses_id_stream_route
fix(proxy): encrypt streamed responses ids on /openai/v1/responses and /responses aliases
2026-08-26 02:38:24 -07:00
Mateo Wang
4185c8af07
Merge pull request #38320 from BerriAI/litellm_passthrough_object_ownership
fix(passthrough): record ownership of streamed responses under managed ids
2026-08-26 02:15:12 -07:00
mateo-berri
95f8373e3c test(responses): drive streamed-id regression via production ResponseCompletedEvent shape
The streamed-id regression test built a bare BaseLiteLLMOpenAIResponseObject with a
top-level id, hitting the wrong _encrypt_response_id branch. A real streamed create
emits ResponseCompletedEvent, whose client-visible id lives on event.response.id, so
the test now drives that production event shape and reads collected[0].response.id.
Mutating the alias route gate or disabling the .response.id encryption branch both
fail the test.
2026-08-26 01:52:01 -07:00
Yuneng Jiang
54cbc44705
test(cost-calc): pin the rate fallbacks inside a tiered-pricing tier
_get_tiered_base_costs documents that tiered pricing is all-or-nothing: a
tier is picked from the request's input tokens, and any rate that tier does
not declare falls back to the tier's own input rate so one request is never
priced from two tiers.

Nothing checked that. Every existing tiered test supplies a fully populated
tier, so the fallbacks were never reached: deleting them from the source
left the whole suite green. The fallbacks are not hypothetical either. Of
the 66 tiered rows shipped in model_prices_and_context_window.json, 54
declare no cache-creation rate and 44 declare no cache-read rate, so the
fallback is what prices their cached tokens today.

Adds three tests on the generic path:
  - a tier with no cache rates bills cached and cache-creation tokens at
    that tier's input rate, ignoring the model's top-level cache rates
  - a tier with no above-1hr rate bills 1h cache writes at the tier's
    cache-creation rate rather than zero
  - a tier with no input rate is not a priced tier at all, so the model's
    flat rates still apply instead of billing input at zero

Test-only change, no source touched.
2026-08-26 01:46:29 -07:00
mateo-berri
498ba9dd62 fix(proxy): encrypt streamed responses ids on /openai/v1/responses and /responses aliases
The streaming security hook only encrypted response ids when request_route
matched "/v1/responses" exactly, so streamed creates on the /openai/v1/responses
and /responses aliases leaked the plain managed id. A second virtual key could
GET, continue, and DELETE another key's response. Normalize the route (strip the
provider prefix, accept the /responses alias) before gating, mirroring the
non-streaming hook which has no route gate.
2026-08-26 01:38:13 -07:00
mateo-berri
6a9662a5a8 fix(passthrough): recognize CR-only SSE frame delimiters when minting streamed managed ids 2026-08-26 01:36:18 -07:00
mateo-berri
6386a68c9c fix(router): fail fast on PermissionDeniedError with a single deployment 2026-08-26 01:09:14 -07:00
mateo-berri
e0c101b4da fix(passthrough): record ownership of streamed responses under managed ids 2026-08-26 01:05:15 -07:00
mateo-berri
910c41f154 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_branchless_provider_status_mapping
# Conflicts:
#	tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py
2026-08-26 00:55:42 -07:00
Devin AI
2eedcb62ce Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_techdebt_20260822 2026-08-26 07:54:51 +00:00
Mateo Wang
137311ffd6
Merge pull request #38310 from BerriAI/litellm_fix_anthropic_messages_error_mapping
fix(otel): map /v1/messages provider errors before failure logging
2026-08-26 00:49:17 -07:00