Both bridges (Responses→CC and CC→Responses) independently encoded the
same field mapping knowledge. This extracts 4 shared mappings into a
single module so future changes only need to happen in one place.
Shared mappings:
- status ↔ finish_reason bidirectional dicts and functions
- response_format ↔ text.format paired conversion functions
- provider_specific_fields normalization helper
- usage field name translation (input_tokens ↔ prompt_tokens, etc.)
No behavioral changes — bridge methods now delegate to the shared module.
The `dimensions` parameter was correctly mapped to `outputDimensionality`
in `optional_params` but never placed in the request body. The Vertex AI
predict endpoint expects it under a `parameters` field.
Add `parameters` dict to `VertexMultimodalEmbeddingRequest` TypedDict and
populate it from `optional_params` in `transform_embedding_request`.
Fixes#24392
Gemini API returns a DOCUMENT modality in promptTokensDetails for PDF
inputs, but the token parser only handled TEXT, IMAGE, AUDIO, and VIDEO.
DOCUMENT tokens were silently dropped, causing cost to be undercounted
by up to 99% for PDF-heavy requests.
Map DOCUMENT tokens to text_tokens since Gemini bills documents at the
text token rate. Applied to all four modality parser loops:
promptTokensDetails, cacheTokensDetails, responseTokensDetails, and
candidatesTokensDetails.
Fixes#24375
Replace _is_gemini_3_model() substring check with a
web_search_billing_unit field in model_prices JSON:
- "per_query": each search query billed individually (Gemini 3.x)
- "per_prompt" (default): flat fee per grounded API call (Gemini 2.x)
Add web_search_billing_unit to 23 Gemini 3.x model entries.
Update docs and tests accordingly.
Add tests for the gpt-5.1/5.2/5.4 reasoning.effort interaction:
- gpt-5.1 with no reasoning allows flexible temperature
- gpt-5.1 with effort='high' drops temperature
- gpt-5.4 with effort='none' allows flexible temperature
- Gemini 2.x charges per grounded prompt (flat $0.035), clamped to 1
regardless of internal query count
- Gemini 3.x charges per search query ($0.014 each)
- Extract web_search_requests from groundingMetadata in non-streaming
responses (parity with streaming path)
- Add search_context_cost_per_query to vertex_ai and base Gemini entries
- Move tests to tests/test_litellm/ (CI directory)
The Gemini web search cost calculator hardcoded $0.035 per request,
which is only correct for Gemini 2.x models. Gemini 3.x models
charge $0.014 per request.
Read from search_context_cost_per_query in model_info (same field
used by Anthropic, OpenAI, and Perplexity) with fallback to the
legacy $0.035 for models not yet updated in the JSON.
Also add search_context_cost_per_query to all 25 Gemini models
that support web search in model_prices_and_context_window.json.
Fixes#24369
The Responses API map_openai_params passed all params through without
applying model-specific validation. GPT-5 models (except gpt-5-chat)
only accept temperature=1 unless reasoning.effort="none" on models
that support it (5.1, 5.2, 5.4).
Reuse the existing OpenAIGPT5Config logic from chat completions to
validate temperature in the Responses API path. With drop_params=True,
unsupported temperature values are silently dropped; without it,
UnsupportedParamsError is raised.
Fixes#16090
The Gemini batch embedding transformation was spreading all
optional_params into the request body via **gemini_params. Params
like max_tokens (injected by add_provider_specific_params_to_optional_params)
would reach the Gemini API and cause a 400 BadRequestError.
Extract _filter_embed_params() that maps dimensions/task_type and
keeps only the fields Gemini embeddings actually accept
(outputDimensionality, taskType, title). Applied to both
transform_openai_input_gemini_content and
transform_openai_input_gemini_embed_content.
This also fixes drop_params: true not preventing the error, since
the param was re-injected after the drop_params check.
Fixes#24293
When the Responses API converts function_call and message output items
into chat completion messages, they can become two consecutive assistant
messages. The Bedrock Converse transformer merges these into one, but
the merge preserves input order — so if function_call came first, the
toolUse block ends up before the text block.
Claude models (Sonnet 4, Haiku 3.5+) reject this ordering with:
"tool_use ids were found without tool_result blocks immediately after"
Add _sort_bedrock_assistant_content_blocks() that reorders content
blocks within assistant messages: reasoningContent → text → toolUse.
Applied in both sync and async Bedrock Converse transformation paths.
Fixes#24361
When Gemini sends tool call arguments in the same streaming chunk as a
content block transition, the Anthropic adapter discarded the
processed_chunk containing the input_json_delta. This caused tool_use
blocks to arrive with empty input: {}.
Queue the processed_chunk alongside the block transition events when it
contains input_json_delta data. Applied to both sync and async paths.
Fixes#24134
When Azure sends stream_options.include_usage=True, it emits an initial
chunk with choices=[] (prompt_filter_results) before the first content
chunk. Previously, LiteLLM inflated this empty-choices chunk with a
default StreamingChoices, which consumed the sent_first_chunk flag and
caused strip_role_from_delta to strip role from the real first chunk.
Additionally, the first real chunk with role='assistant' and content=''
was discarded by is_chunk_non_empty as "empty".
This fix:
- Forwards chunks with choices=[] faithfully (no inflated default)
- Only marks sent_first_chunk for chunks with real choices
- Treats chunks with role in delta as non-empty
- Guards choices[0] access in __next__/__anext__ and stream_chunk_builder
Fixes#24221
Allows wrapping multiple inputs in a nested list to produce a single
combined embedding (text + image = 1 vector). Flat lists continue to
produce separate embeddings per input (OpenAI-compatible default).
Examples:
input=["text", "image"] → 2 separate embeddings
input=[["text", "image"]] → 1 combined embedding
input=[["text", "image"], "x"] → 2 embeddings (1 combined + 1 separate)
When multiple inputs were passed to the Gemini embedding endpoint and any
contained multimodal data (images, audio, etc.), LiteLLM incorrectly used
the `embedContent` endpoint which combines all inputs into a single
aggregated embedding. Now uses `batchEmbedContents` with each input as a
separate request, returning N embeddings for N inputs as expected.
Also fixes hardcoded index=0 in batch embedding responses.
Addresses Greptile feedback that test assertions were weakened when
removing summary: "detailed" expectations — now every default-behavior
test explicitly asserts that "summary" is absent from the result.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(redis): add circuit breaker to RedisCache to fast-fail when Redis is down (#24181)
* feat(redis): add circuit breaker env var constants
* feat(redis): add RedisCircuitBreaker and apply guard decorator to all async ops
* fix(dual_cache): fall back to L1 instead of re-raising on Redis increment failures
* test(caching): add circuit breaker unit tests
* fix(redis): fast-fail concurrent HALF_OPEN probes — only one probe at a time
* fix(dual_cache): return None fallback when in_memory_cache is absent and Redis fails
* test(caching): add regression tests for HALF_OPEN concurrency and None fallback
* Fix blocking sync next in __anext__ (#24177)
* Fix blocking sync next
* Update tests/test_litellm/litellm_core_utils/test_streaming_handler.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix PEP 479 regression in __anext__ sync iterator exhaustion
asyncio.to_thread re-raises thread exceptions inside a coroutine, where
PEP 479 converts StopIteration to RuntimeError before any except clause
can catch it. Add _next_sync_or_exhausted() module-level helper that
catches StopIteration in the thread and returns a sentinel instead, then
raise StopAsyncIteration in the coroutine.
Also rewrites the non-blocking test to use asyncio.gather() instead of
asyncio.create_task() (which returned None on Python 3.9 / pytest-asyncio
in CI), and adds an exhaustion regression test that drains the wrapper
fully and asserts no RuntimeError leaks out.
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: add git-subdir source type to claude-code/plugins API (#24223)
Support a third plugin source type `git-subdir` alongside the existing
`github` and `url` types, as documented in the official Claude Code
plugin marketplaces spec.
New format: {"source": "git-subdir", "url": "...", "path": "subdir/path"}
- Validates url and path fields are present and non-empty
- Rejects absolute paths, '..' segments, backslashes, and percent-encoded
traversal sequences (including double-encoded variants via regex check)
- Extracts path validation into _validate_git_subdir_path() helper
- Updates Pydantic field description to document all three source types
- Adds isValidUrl() check for url/git-subdir source types in the UI form
- Adds "Git Subdir" option to the UI form with a required Path field
- Adds unit tests covering success, update, missing/empty fields,
path traversal variants, and unknown source type
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* [FEAT] add extract_header and extract_footer to Mistral OCR supported params (#24213)
* docs: add git-subdir source type to claude-code plugin marketplace docs (#24289)
* fix(ui): swap J/K keyboard navigation in log details drawer (#24279) (#24286)
J should navigate down (next) and K should navigate up (previous),
matching vim/standard conventions.
* fix: use async_set_cache in user_api_key_auth hot path (#24302)
* fix: use async_set_cache in auth hot path to avoid blocking event loop
* test: assert no blocking set_cache call in _user_api_key_auth_builder
* test: broaden blocking call check to all sync DualCache methods
* test: fix regression test to actually catch blocking cache calls
* fix: ruff lint unused variable + UI build MessageManager error
- litellm/caching/redis_cache.py: remove unused variable 'e' in circuit
breaker exception handler (F841)
- add_plugin_form.tsx: use MessageManager.error() instead of undefined
message.error() for git URL validation
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* docs: add REDIS_CIRCUIT_BREAKER env vars to config_settings reference
Add REDIS_CIRCUIT_BREAKER_FAILURE_THRESHOLD and
REDIS_CIRCUIT_BREAKER_RECOVERY_TIMEOUT to the environment variables
reference table so test_env_keys.py passes.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Vincenzo Barrea <manamana88@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Robert Kirscht <rkirscht242@gmail.com>
Co-authored-by: Imgyu Kim <kimimgo@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
