litellm/litellm
Aryan Pardeshi a844dadcd2
feat(gemini): allow disabling thought summaries while keeping thinking on (#36460)
* feat(gemini): allow disabling thought summaries while keeping thinking on

Both reasoning-effort mappers hardcoded includeThoughts to True whenever
thinking was on, so thinking ON with summaries OFF was unreachable even though
that is the google SDK's own default. Summaries are the narration rather than
the reasoning, and for callers that discard them they are pure added latency.

Adds litellm.gemini_include_thoughts_default, defaulting to True so nothing
changes for anyone who does not set it. It gates the includeThoughts=True
branches in the thinkingBudget mapper, the thinkingLevel mapper, and the
anthropic-style thinking param mapper.

The disable and none paths still force includeThoughts=False: with thinking
off there is nothing to summarise.

* test(gemini): assert thought-summary flag through the outbound Gemini request

* feat(gemini): let a request's thought summary ask override the gateway default

thinking.display (summarized, updates, omitted) and a reasoning_effort object carrying summary now decide includeThoughts for that request; gemini_include_thoughts_default applies only when the request says nothing

* feat(gemini): let a thought summary ask on /v1/messages and /v1/responses override the gateway default

* fix(gemini): read a text gateway default as a boolean and leave display omitted to the flag

Normalize gemini_include_thoughts_default through normalize_drop_params so a
YAML or env value written as "false" or "0" turns thought text off and a
null or unknown value keeps the default on. thinking.display "omitted" no
longer counts as a per-request ask, so the gateway flag decides, matching
/v1/messages. Adds proxy integration cells for chat, /v1/messages and
/v1/responses that prove the flag, every per-request ask, streaming, and a
burst across workers.

* test(gemini): match the streamed Responses usage, which omits null token details

* feat(gemini): read the AI SDK reasoning summary alias and thinking.summary as chat asks

* fix(gemini): keep the logged reasoning_effort unchanged and key the response cache on the summary alias

A dict reasoning_effort next to the AI SDK reasoning_summary alias now counts as a summary ask, and the alias is part of the cache key so a plain request and an alias ask never share a reply

* fix(caching): key the response cache on the summary alias wherever the request carries it

The cache key now reads reasoning_summary and reasoningSummary through the same
reader as the summary fold, so an alias inside extra_body gets its own key too

* test: type the new summary-ask unit tests and mark their locals Final

* test: annotate the remaining summary-ask unit tests the PR touches

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-16 09:01:33 +00:00
..
_v2 feat(cache): select Rust caching through explicit cache objects (#43601) 2026-09-29 00:01:44 +00:00
a2a_protocol refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
anthropic_interface refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
assistants refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
batch_completion
batches refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
caching feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
chat_completions refactor(python-bridge): ship a signature base and read the resolved call (#45450) 2026-10-08 17:20:29 -07:00
completion_extras fix(responses): map Anthropic tool search to OpenAI server tool search (#46441) 2026-10-15 05:13:56 +00:00
compression refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
containers refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
decisions fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
embeddings refactor(python-bridge): ship a signature base and read the resolved call (#45450) 2026-10-08 17:20:29 -07:00
endpoints/speech/speech_to_completion_bridge
evals
experimental_mcp_client fix(mcp): resolve /mcp/proxy tool_id from the owning server instead of relisting every server (#46545) 2026-10-15 22:46:50 +00:00
files refactor(types): replace Any with proven types in 24 files (#46663) 2026-10-15 06:10:12 -07:00
fine_tuning
google_genai refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
harness refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
images fix: preserve explicit provider for async image, audio, and realtime routes (#36346) 2026-10-15 05:21:43 -07:00
integrations fix(guardrails): record every content filter match in logging_only and offer the mode in the UI (#46673) 2026-10-16 00:37:03 -07:00
interactions refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
litellm_core_utils feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
llms feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
messages feat(anthropic): generate auto mode safeguard_results with a classifier model on the /v1/messages chat bridge (#45600) 2026-10-14 13:09:11 +00:00
models fix(proxy): allow model instances in LiteLLM_EndUserTable validation (#41677) 2026-10-15 10:57:26 -07:00
ocr refactor(python-bridge): ship a signature base and read the resolved call (#45450) 2026-10-08 17:20:29 -07:00
passthrough fix(bedrock-passthrough)!: return AWS-shaped errors so AWS SDKs read the code and message (#46522) 2026-10-15 22:05:46 -07:00
proxy fix(proxy): track and drain management audit log writes on shutdown (#43594) 2026-10-16 08:41:34 +00:00
proxy_auth
rag refactor(types): replace Any with proven types in 24 files (#46663) 2026-10-15 06:10:12 -07:00
realtime_api fix(bedrock): forward Nova Sonic realtime tool calls with their arguments (#46214) 2026-10-16 01:10:53 +00:00
repositories fix(proxy): reset end-user spend in committed pages so the budget cascade survives a large LiteLLM_EndUserTable (#46502) 2026-10-15 19:47:12 -07:00
rerank_api refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
responses feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
router_strategy refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
router_utils refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
rust_bridge refactor(rust-bridge): let each route host pick the body params (#46787) 2026-10-15 18:27:06 -07:00
sandbox refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
search refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
secret_managers refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
skills refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
systemone feat(typesafe): support Jev 1.14 images, box questions, and long-context headers (#46596) 2026-10-14 22:29:48 -07:00
telemetry refactor: clean up fresh tech debt in per-user Copilot auth, Ternary FOCUS export and telemetry (#45769) 2026-10-15 04:29:27 +00:00
tracing fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
types fix(cost): bill Responses API image generation and usage reported web search as tool usage cost (#41877) 2026-10-15 20:18:17 -07:00
vector_store_files refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
vector_stores refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
videos feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
__init__.py feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
_internal_context.py fix(otel): nest cache spans under their operation and name service spans by purpose (#44150) 2026-10-03 09:20:27 -07:00
_lazy_imports.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_lazy_imports_registry.py feat(typesafe): support Jev 1.14 images, box questions, and long-context headers (#46596) 2026-10-14 22:29:48 -07:00
_logging.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
_redis.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
_redis_credential_provider.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_service_logger.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
_uuid.py
_version.py feat(sdk): build core from an independent packaging manifest (#44340) 2026-10-08 18:40:14 -07:00
anthropic_beta_headers_config.json fix(vertex_ai): forward the inline-tools-2026-09-15 beta to Vertex and Anthropic (#45297) 2026-10-08 12:27:52 -07:00
anthropic_beta_headers_manager.py refactor(types): move SpanAttributes, SpecialHeaders and AllowedModelRegion out of proxy._types (#44717) 2026-10-05 21:40:13 -07:00
blog_posts.json
budget_manager.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
constants.py fix(bedrock-passthrough)!: return AWS-shaped errors so AWS SDKs read the code and message (#46522) 2026-10-15 22:05:46 -07:00
cost.json
cost_calculator.py fix(responses): write one spend-log row per native WebSocket turn (#46768) 2026-10-15 19:05:22 -07:00
exceptions.py fix(streaming): keep provider response headers on SDK streaming errors (#41528) 2026-10-15 08:11:54 -07:00
main.py feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00
model_prices_and_context_window_backup.json fix(vertex-ai): drop cached flex price from gemini 3 preview rows (#46826) 2026-10-15 20:31:10 -07:00
policy_templates_backup.json fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python (#43903) 2026-09-30 22:20:36 +00:00
provider_endpoints_support_backup.json feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
py.typed
router.py fix(router): let an exact base_model widen a deployment's context window (#46197) 2026-10-16 06:37:40 +00:00
scheduler.py fix(scheduler): remove a request's queue entry once it stops waiting (#43061) 2026-10-07 17:34:38 -07:00
setup_wizard.py fix(model_prices): correct Claude Sonnet 5.5 capabilities and provider keys (#43587) 2026-09-28 19:12:31 +00:00
timeout.py
tool_loop.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
utils.py feat(gemini): allow disabling thought summaries while keeping thinking on (#36460) 2026-10-16 09:01:33 +00:00