Commit graph

14741 commits

Author SHA1 Message Date
Mateo Wang
b600f02fc2
Merge pull request #34788 from BerriAI/litellm_fix_s3_vectors_search
fix(vector_stores): s3 vectors search router bypass + rag query config drop + ui error swallow
2026-09-02 11:35:56 -07:00
mateo-berri
d7ee215c57 fix(responses): keep namespace tools intact when a guardrail returns them unchanged
Any pre_call guardrail on /v1/responses flattened Codex namespace tools
into ns__member functions and wrote the flattened list back to the
request, so the model called mcp__server__tool with no namespace and
Codex rejected the call as unsupported.

The handler now keeps the client's original tools, hands the guardrail a
deep copy of the flattened ones, and rebuilds data["tools"] by matching
the guardrail's output to the originals by type and name. Unchanged
tools go back as the original objects, a dropped or edited namespace
member changes only that member, and tools the guardrail injects are
still appended.

Fixes #39183
2026-09-02 11:32:04 -07:00
mateo-berri
ef14bed029 fix(hosted_vllm): reject invalid rerank truncation params with a 400 2026-09-02 11:31:19 -07:00
mateo-berri
df24dab7c9 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_azure_ai_reclassify
# Conflicts:
#	tests/test_litellm/test_main.py
2026-09-02 11:31:12 -07:00
Mateo Wang
a43228ef72
Merge pull request #39249 from BerriAI/litellm_router_settings_reject_unknown_keys
fix: apply optional_pre_call_checks and reject unsupported router settings on /config/update
2026-09-02 11:23:22 -07:00
mateo-berri
8d00220ace test(hosted_vllm): annotate rerank truncation test locals as Final 2026-09-02 11:19:20 -07:00
mateo-berri
dfaf235234 fix(bedrock): honor BEDROCK_MANTLE_API_BASE on bedrock/mantle messages and chat URLs 2026-09-02 11:08:45 -07:00
mateo-berri
dbc126cfc9 fix(hosted_vllm): forward truncate_prompt_tokens on rerank requests 2026-09-02 11:07:30 -07:00
mateo-berri
7b942fd983 fix(azure_ai): route audio and realtime calls on Foundry hosts through the Azure OpenAI handlers 2026-09-02 11:05:38 -07:00
mateo-berri
1801fbb1a8 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_converse_legacy_thinking_adaptive 2026-09-02 11:03:02 -07:00
Mateo Wang
4049a075bd
Merge pull request #39170 from BerriAI/litellm_registry_audit_2026_09_01
fix(models): registry audit 2026-09-01: openai realtime and long-context tiers, mistral aliases, voyage, xai, fireworks, together, scaleway, azure ai, govcloud, azure gov, cloudflare whisper, deprecation dates
2026-09-02 11:02:43 -07:00
mateo-berri
dcc2c2ac3a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_techdebt_20260901 2026-09-02 10:59:17 -07:00
mateo-berri
db909620fb Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_lit5443_mantle_chat_aws_creds
Resolves the conflicts in llm_http_handler.py and its test file, and replaces
the mantle test that patched BaseAWSLLM.get_credentials at class level with
one that injects the signer into BedrockMantleChatConfig, which the
test-quality gate's ratcheted TQ008 ceiling on staging now requires
2026-09-02 10:52:27 -07:00
devin-ai-integration[bot]
ffc0a8e428
fix: run access group key sync UPDATEs on the writer, not the read replica (#39128)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 10:51:17 -07:00
mateo-berri
712c3ee4ae Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_messages_passthrough_cache_control_ttl 2026-09-02 10:51:10 -07:00
mateo-berri
0c53961445 fix(proxy-extras): give prisma migrate deploy its own timeout budget 2026-09-02 10:44:51 -07:00
mateo-berri
7a35c34303 fix(models): add the us-gov. geo inference profile keys for Claude Sonnet 5 and Opus 4.8 2026-09-02 10:30:14 -07:00
mateo-berri
53da9bca8e fix(bedrock): drop client_metadata for every converse model 2026-09-02 10:29:44 -07:00
Mateo Wang
2ffe6a1dc8
Merge pull request #39160 from BerriAI/litellm_gemini_thinking_content
fix(gemini): return enabled thinking content by default
2026-09-02 10:22:28 -07:00
Mateo Wang
6ed2cdd428
Merge pull request #39238 from BerriAI/litellm_anthropic_models_display_name
feat(proxy): configurable display_name for the Anthropic-shaped /v1/models listing
2026-09-02 10:18:43 -07:00
Mateo Wang
921c1d1248
Merge pull request #39246 from BerriAI/litellm_fix_e2e_junit_properties_types
test(e2e): read JUnit properties off the real collected pytest Item
2026-09-02 10:18:18 -07:00
Mateo Wang
1710d977bf
Merge pull request #39176 from BerriAI/litellm_rerank_provider_error_body
fix(rerank): map provider errors with the resolved provider on sync and async paths
2026-09-02 10:18:11 -07:00
Mateo Wang
25c5b6ce43
Merge pull request #39069 from BerriAI/litellm_stream_usage_cost_default
feat(streaming): carry final response cost on streamed usage by default
2026-09-02 10:17:51 -07:00
moe-berri
7b919f89a8 fix(router): track routed model in fallback attempts 2026-09-02 10:00:47 -07:00
Mateo Wang
e4b903d2cd
Merge pull request #39194 from BerriAI/litellm_fix_deepseek_ocr_model_prefix
fix(vertex): avoid duplicate DeepSeek OCR model namespace
2026-09-02 09:57:50 -07:00
mateo-berri
c05aef159f Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_strip_client_metadata 2026-09-02 09:44:14 -07:00
mateo-berri
0c7fe53c27 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_registry_audit_2026_09_01 2026-09-02 09:25:36 -07:00
Mateo Wang
55e9e4ce2f
Merge pull request #39340 from BerriAI/litellm_gemini_3_8_flash
feat(gemini): day-0 pricing for gemini-3.8-flash
2026-09-02 09:24:24 -07:00
mateo-berri
9f1c07c2b9 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_messages_passthrough_cache_control_ttl 2026-09-02 09:16:35 -07:00
mateo-berri
69cd1bada6 test(gemini): compare gemini-3.8-flash to 3.7 flash field by field 2026-09-02 08:46:28 -07:00
yassin
bf22e6cd4a Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_azure_ai_reclassify 2026-09-02 15:46:01 +00:00
devin-ai-integration[bot]
2ce4e3f8a9
fix(guardrails): run apply_guardrail-only providers in logging_only mode (#39297)
* fix(guardrails): run apply_guardrail-only providers in logging_only mode

A CustomGuardrail that implements only apply_guardrail inherited the CustomLogger
no-op async_logging_hook, so mode: logging_only never scanned anything and never
recorded guardrail_information. CustomGuardrail.async_logging_hook now routes the
logged request and response through the call type's guardrail translation on
copies and appends the verdict to standard_logging_object.guardrail_information.

Resolves LIT-4876

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): keep logging_only scan copies inside the error boundary and return a fresh logging payload

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(guardrails): cover embedding scan, native-hook bypass, and unmapped call type in logging_only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 08:32:49 -07:00
mateo
da23e0241d fix(models): add cloudflare whisper transcription pricing and pin govcloud pricing tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 15:28:53 +00:00
mateo-berri
6b83b16559 feat(gemini): day-0 pricing for gemini-3.8-flash
Gemini 3.8 Flash launches today with the same promotional pricing, limits,
and thinking settings as Gemini 3.7 Flash, so the gemini/, vertex_ai/, and
bare cost map entries mirror the 3.7 Flash ones. Regression tests lock the
launch prices, the 4096-token cache minimum, and the gemini-3 thought
signature gate in for the new model.
2026-09-02 08:04:14 -07:00
mateo
a7836ede15 fix(models): absorb open registry PRs: govcloud bedrock and mantle, azure gov, openai tiered long-context, scaleway, together qwen3.8, azure ai cache and kimi k2.7 code, azure mai deprecations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 14:48:51 +00:00
mateo
385957e830 fix: reject constructor-managed router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 07:49:25 +00:00
mateo
77aa22e065 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_techdebt_20260901 2026-09-02 07:44:53 +00:00
Devin AI
2d643a8683 chore: merge litellm_internal_staging into litellm_realtime_router_session_model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:37:16 +00:00
mateo
974b331a4d fix: accept persistable router settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 06:15:52 +00:00
mateo-berri
7cde2cd77f test(guardrails): type the presidio sibling test helpers precisely 2026-09-01 22:39:17 -07:00
mateo-berri
c18511be7d fix(guardrails): track and tear down presidio sibling callbacks
initialize_presidio registers up to three callbacks per guardrail but the
registry only kept the first, so deleting or re-syncing the guardrail left
the post_call siblings serving the old config. The initializer now returns
every callback it registered, the registry tracks primary and siblings per
guardrail id, delete purges all of them from every callback list, and
update pushes the new params into each while siblings keep their stage.
2026-09-01 22:28:38 -07:00
mateo-berri
4a5d0b8163 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_headroom_ccr_streaming_responses
# Conflicts:
#	litellm/llms/openai/responses/guardrail_translation/handler.py
#	tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_handler.py
2026-09-01 22:25:59 -07:00
Mateo Wang
31ca4ddf32
Merge pull request #39166 from BerriAI/litellm_bedrock_bearer_token_converse_crash
fix(bedrock): stop Converse crashing on bearer-token auth without SigV4 credentials
2026-09-01 22:24:30 -07:00
James Liounis
e2c3f51c46
fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping (#37883)
* fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping

SearchAPIRouter dropped every parameter configured on a search tool, forwarding
only per-request kwargs. Any tool-level setting (mode, max_results, ...) was
silently lost on the way to the adapter, for every search provider.

Also completes the Parallel AI v1 search surface: after_date, fetch_policy,
location and include_domains now nest under advanced_settings instead of being
sent as unknown top-level fields, responses preserve search_id / session_id /
warnings / raw excerpts, and search cost is derived from the request mode and
the provider's reported usage rather than a single flat rate.

* fix(parallel_ai): stop a caller from pricing its own search request

`_parallel_ai_usage` carries the provider's reported usage into cost
calculation. It was only written when the response contained a usage block, so
a caller could pass `_parallel_ai_usage=[{"name": "sku_search", "count": 0}]`
and, whenever the provider omitted usage, bill $0.00 instead of $0.005 — the
value also reached the upstream request body as an unknown field.

The key is now stripped from inbound params and written unconditionally from
the parsed response, so only the provider can populate it.

* fix(parallel_ai): price fast search mode correctly

* test(parallel_ai): fake search at HTTP boundary

* fix(parallel_ai): tolerate null search result fields

---------

Co-authored-by: khushishelat <shelatkhushi@gmail.com>
2026-09-01 21:46:46 -07:00
moe-berri
6adc14b4b1 fix(router): preserve Claude subagent fallbacks 2026-09-01 21:13:45 -07:00
moe-berri
8a0967443d fix(router): isolate Claude session binding cache 2026-09-01 20:38:13 -07:00
mateo
1d3e26fd98 fix: preserve shared optional callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 03:28:04 +00:00
mateo
70a4f74a0d test: allow callback state fixture mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:56:23 +00:00
mateo
3dea586d3b test: cover runtime callback reconciliation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:48:09 +00:00
mateo
7b86b7f4cd test: isolate router callback state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-02 02:45:45 +00:00