Commit graph

181 commits

Author SHA1 Message Date
Ishaan Jaffer
e8461b5b97
style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
Ishaan Jaffer
9977e63e3c
Merge remote-tracking branch 'origin/main' into worktree-foamy-jumping-coral 2026-04-15 18:29:55 -07:00
Ishaan Jaffer
f768946549
style: black format test_anthropic_common_utils.py 2026-04-15 18:19:12 -07:00
Sameer Kankute
1a9a31e4a2
Merge pull request #25665 from BerriAI/litellm_oss_staging_04_13_2026_p1
litellm oss staging 04/13/2026
2026-04-14 23:50:08 +05:30
Kris Yang
15245a5eb7
fix: emit input_json_delta for tool args bundled in first streaming chunk (#25533)
* fix: emit input_json_delta for tool args bundled in first streaming chunk

Some providers (xAI, Gemini) include tool_call function arguments in the
same streaming chunk as the function name/id. The AnthropicStreamWrapper
was discarding the trigger chunk entirely when starting a new content
block, which silently dropped the input_json_delta carrying tool
arguments. This caused tool_use blocks to arrive with empty input {}.

Now queue the processed_chunk after content_block_start when it carries
non-empty input_json_delta data. Backward compatible: providers that send
empty arguments in the first chunk (OpenAI-style) are unaffected since
the condition checks for truthy partial_json.

* test: add tests for input_json_delta emission on bundled tool args

Covers the fix for providers (xAI, Gemini) that bundle tool_call
arguments in the same streaming chunk as the function name/id.
Verifies the AnthropicStreamWrapper emits input_json_delta after
content_block_start, and that empty-arg chunks (OpenAI-style) are
unaffected.

* style: apply Black formatting to streaming_iterator.py

* fix: mirror input_json_delta fix to sync __next__ and add sync tests

* test: make no_extra_delta tests assert explicitly instead of passing silently
2026-04-14 23:37:49 +05:30
ishaan-berri
e2fc7d64e8
Merge pull request #25396 from BerriAI/litellm_bedrock-normalize-custom-tool-schema
feat(bedrock): normalize custom tool JSON schema for Invoke and Converse
2026-04-14 10:21:15 -07:00
Sameer Kankute
5670f6c7d4
fix(anthropic): tighten thinking-signature retry (Greptile)
- Omit messages whose list content is empty after stripping thinking blocks
- Retry only on HTTP 400 plus invalid-signature body match
- Return response inline from retry loop; drop unreachable None guard
- Tests: thinking-only turn dropped, non-400 no retry

Made-with: Cursor
2026-04-14 10:03:10 +05:30
Sameer Kankute
c7f7708d27
feat(anthropic): retry /v1/messages after invalid thinking signature
Strip thinking blocks from the request body and retry once when Anthropic returns an invalid thinking signature error (e.g. after credential or deployment change). Applies to all BaseAnthropicMessagesConfig providers (direct Anthropic, Bedrock, Vertex, Azure AI).

Made-with: Cursor
2026-04-14 09:39:39 +05:30
Ishaan Jaffer
9be7b4c07c
test(advisor): add integration tests for full dispatch path, max_uses, provider bypass 2026-04-11 18:16:56 -07:00
Ishaan Jaffer
fa5258466d
test(advisor): add unit tests for max_uses=0, missing model, default fallback 2026-04-11 18:16:56 -07:00
Ishaan Jaffer
844e34b68b
test(advisor): remove live e2e test file (tests run locally via script) 2026-04-11 17:52:06 -07:00
Ishaan Jaffer
742e2fe1aa
test(advisor): add live e2e tests for advisor orchestration against real proxy 2026-04-11 17:46:17 -07:00
Ishaan Jaffer
ce3d039bcd
test(advisor): add unit tests for orchestration loop (mocked backends, 8 tests) 2026-04-11 17:43:26 -07:00
Ishaan Jaffer
318196f793
test(advisor): add tests for auto-strip advisor_tool_result blocks 2026-04-10 13:15:51 -07:00
Ishaan Jaffer
55f0e6605b
test(anthropic): add advisor tool tests for /messages beta header path 2026-04-10 12:39:30 -07:00
Ishaan Jaffer
0f9eba4de0
test(anthropic): add advisor tool transformation tests 2026-04-10 12:39:30 -07:00
Sameer Kankute
ec9102107a
fix(bedrock): custom tool schemas and missing tool names for Converse
Normalize JSON Schema type custom to object for Bedrock invoke and
_bedrock_tools_pt, ensure stable names for tools without name, and
avoid KeyError in the Anthropic messages adapter when translating
tools to OpenAI format for bedrock/converse.

Made-with: Cursor
2026-04-09 10:39:00 +05:30
Krrish Dholakia
f42ffed2bd
Litellm oss staging 04 02 2026 p1 (#25055)
* fix(vertex_ai): support pluggable (executable) credential_source for WIF auth (#24700)

The WIF credential dispatch in load_auth() only handled identity_pool and
aws credential types. When credential_source.executable was present (used
for Azure Managed Identity via Workload Identity Federation), it fell
through to identity_pool.Credentials which rejected it with MalformedError.

Add dispatch to google.auth.pluggable.Credentials for executable-type
credential sources, following the same pattern as the existing identity_pool
and aws helpers.

Fixes authentication for Azure Container Apps → GCP Vertex AI via WIF
with executable credential sources.

* feat(logging): add component and logger fields to JSON logs for 3rd p… (#24447)

* feat(logging): add component and logger fields to JSON logs for 3rd party filtering

* Let user-supplied extra fields win over auto-generated component/logger, tighten test assertions

* Feat - Add organization into the metrics metadata for org_id & org_alias (#24440)

* Add org_id and org_alias label names to Prometheus metric definitions

* Add user_api_key_org_alias to StandardLoggingUserAPIKeyMetadata

* Populate user_api_key_org_alias in pre-call metadata

* Pass org_id and org_alias into per-request Prometheus metric labels

* Add test for org labels on per-request Prometheus metrics

* chore: resolve test mockdata

* Address review: populate org_alias from DB view, add feature flag, use .get() for org metadata

* Add org labels to failure path and verify flag behavior in test

* Fix test: build flag-off enum_values without org fields

* Gate org labels behind feature flag in get_labels() instead of static metric lists

* Scope org label injection to metrics that carry team context, remove orphaned budget label defs, add test teardown

* Use explicit metric allowlist for org label injection instead of team heuristic

* Fix duplicate org label guard, move _org_label_metrics to class constant

* Reset custom_prometheus_metadata_labels after duplicate label assertion

* fix: emit org labels by default, remove flag, fix missing org_alias in all metadata paths

* fix: emit org labels by default, no opt-in flag required

* fix: write org_alias to metadata unconditionally in proxy_server.py

* fix: 429s from batch creation being converted to 500 (#24703)

* add us gov models (#24660)

* add us gov models

* added max tokens

* Litellm dev 04 02 2026 p1 (#25052)

* fix: replace hardcoded url

* fix: Anthropic web search cost not tracked for Chat Completions

The ModelResponse branch in response_object_includes_web_search_call()
only checked url_citation annotations and prompt_tokens_details, missing
Anthropic's server_tool_use.web_search_requests field. This caused
_handle_web_search_cost() to never fire for Anthropic Claude models.

Also routes vertex_ai/claude-* models to the Anthropic cost calculator
instead of the Gemini one, since Claude on Vertex uses the same
server_tool_use billing structure as the direct Anthropic API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix(anthropic): pass logging_obj to client.post for litellm_overhead_time_ms (#24071)

When LITELLM_DETAILED_TIMING=true, litellm_overhead_time_ms was null for
Anthropic because the handler did not pass logging_obj to client.post(),
so track_llm_api_timing could not set llm_api_duration_ms. Pass
logging_obj=logging_obj at all four post() call sites (make_call,
make_sync_call, acompletion, completion). Add test to ensure make_call
passes logging_obj to client.post.

Made-with: Cursor

* sap - add additional parameters for grounding

- additional parameter for grounding added for the sap provider

* sap - fix models

* (sap) add filtering, masking, translation SAP GEN AI Hub modules

* (sap) add tests and docs for new SAP modules

* (sap) add support of multiple modules config

* (sap) code refactoring

* (sap) rename file

* test(): add safeguard tests

* (sap) update tests

* (sap) update docs, solve merge conflict in transformation.py

* (sap) linter fix

* (sap) Align embedding request transformation with current API

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) mock commit

* (sap) run black formater

* (sap) add literals to models, add negative tests, fix test for tool transformation

* (sap) fix formating

* (sap) fix models

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) commit for rerun bot review

* (sap) minor improve

* (sap) fix after bot review

* (sap) lint fix

* docs(sap): update documentation

* fix(sap): change creds priority

* fix(sap): change creds priority

* fix(sap): fix sap creds unit test

* fix(sap): linter fix

* fix(sap): linter fix

* linter fix

* (sap) update logic of fetching creds, add additional tests

* (sap) clean up code

* (sap) fix after review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) add a possibility to put the service key by both variants

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) update test

* (sap) update service key resolve function

* (sap) run black formater

* (sap) fix validate credentials, add negative tests for credential fetching

* (sap) fix validate credentials, add negative tests for credential fetching

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) fix after bot review

* (sap) lint fix

* (sap) lint fix

* feat: support service_tier in gemini

* chore: add a service_tier field mapping from openai to gemini

* fix: use x-gemini-service-tier header in response

* docs: add service_tier to gemini docs

* chore: add defaut/standard mapping, and some tests

* chore: tidying up some case insensitivity

* chore: remove unnecessary guard

* fix: remove redundant test file

* fix: handle 'auto' case-insensitively

* fix: return service_tier on final steamed chunk

* chore: black

* feat: enable supports_service_tier to gemini models

* Fix get_standard_logging_metadata tests

* Fix test_get_model_info_bedrock_models

* Fix test_get_model_info_bedrock_models

* Fix remaining tests

* Fix mypy issues

* Fix tests

* Fix merge conflicts

* Fix code qa

* Fix code qa

* Fix code qa

* Fix greptile review

---------

Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Josh <36064836+J-Byron@users.noreply.github.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Alperen Kömürcü <alperen.koemuercue@sap.com>
Co-authored-by: Vasilisa Parshikova <vasilisa.parshikova@sap.com>
Co-authored-by: Lin Xu <lin.xu03@sap.com>
Co-authored-by: Mark McDonald <macd@google.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
2026-04-08 21:37:10 -07:00
Krrish Dholakia
cb4027531b fix: add explicit "summary" not in result guards to opt-out test paths
Addresses Greptile feedback that test assertions were weakened when
removing summary: "detailed" expectations — now every default-behavior
test explicitly asserts that "summary" is absent from the result.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 14:53:25 -07:00
Krrish Dholakia
0091d048dc fix: make reasoning summary opt-in, fix missing injection path, narrow test exceptions
Address Greptile review feedback:
1. Replace opt-out `disable_default_reasoning_summary` with existing opt-in
   `reasoning_auto_summary` flag — avoids backwards-incompatible change where
   all users routing thinking-enabled requests would silently get a changed
   reasoning_effort shape (string -> dict) on upgrade.
2. Add default summary injection to `_translate_thinking_to_openai` — this path
   was the only one missing it, causing inconsistent behavior for
   litellm.completion() callers using the Anthropic adapter.
3. Narrow `except Exception` to `except (ValueError, TypeError, AttributeError)`
   in tests to avoid masking genuine failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 11:36:26 -07:00
Krish Dholakia
c350d08d66
Merge branch 'main' into litellm_oss_staging_03_05_2026 2026-03-21 10:31:50 -07:00
Sameer Kankute
00ee80e660
Merge branch 'main' into litellm_oss_staging_03_05_2026 2026-03-21 21:14:38 +05:30
Sameer Kankute
5b5c998dbd
Merge branch 'main' into litellm_oss_staging_03_19_2026 2026-03-21 20:31:08 +05:30
Cesar Garcia
ead607a42b
Merge pull request #24072 from Chesars/fix/strict-additional-properties-20997-clean
fix(adapter): add additionalProperties: false for OpenAI strict mode in Anthropic adapter
2026-03-20 22:19:39 -03:00
joereyna
f0e0d98f86 fix(test): mock get_auth_header instead of get_api_key in anthropic file content test 2026-03-20 16:07:09 -07:00
Sameer Kankute
ab8675dd12
Merge branch 'main' into litellm_oss_staging_03_05_2026 2026-03-20 14:50:21 +05:30
Krish Dholakia
3a0652c445
Merge branch 'main' into feat/anthropic-auth-token-and-base-url 2026-03-19 18:41:19 -07:00
Devin Petersohn
f784da41af Fix get_complete_url to honour ANTHROPIC_BASE_URL in experimental passthrough
Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-19 12:37:12 -07:00
Devin Petersohn
b7e2269942 Address review feedback: fix OAuth routing in get_auth_header and self-contained validate_environment
Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-19 12:17:00 -07:00
Devin Petersohn
f415b72bcf feat(anthropic): support ANTHROPIC_AUTH_TOKEN and ANTHROPIC_BASE_URL env vars
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
2026-03-19 10:16:30 -07:00
Sameer Kankute
ecfcf241c6
Merge pull request #24119 from BerriAI/main
merge main
2026-03-19 15:53:32 +05:30
Krish Dholakia
8b4ed363e4
Merge pull request #24070 from xr843/fix/24026-thinking-blocks-null
Fix thinking blocks dropped when thinking field is null
2026-03-18 21:22:37 -07:00
Chesars
286b8d1460 fix: also populate required for all properties in strict mode
OpenAI strict mode requires both additionalProperties:false AND all
property keys in required. Without required, OpenAI rejects the schema
even with additionalProperties:false set.
2026-03-18 23:03:20 -03:00
Chesars
4bd7bdcf43 fix: add additionalProperties: false for OpenAI strict mode in Anthropic adapter
When translating Anthropic output_format to OpenAI response_format,
the adapter sets strict: true but didn't add additionalProperties: false,
which OpenAI requires at every object nesting level. This caused
BadRequestError for structured output requests routed to OpenAI models.

Fixes #20997
2026-03-18 22:32:23 -03:00
xianren
8969a3d176 Fixed thinking blocks dropped when thinking field is null (#24026)
The check `content.get("thinking", None) is not None` incorrectly
drops thinking blocks when the `thinking` key is explicitly null or
absent. Changed to `content.get("type") == "thinking"` to match
the fix already applied in the experimental pass-through path (PR #15501).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 09:10:21 +08:00
Andrzej Pomirski
cf8d1ac521 fix: streaming container_id and consistent Pydantic types in output
- Populate container_id on streaming code_interpreter_results by
  re-emitting at message_delta when container info arrives
- Reconstruct Pydantic OutputCodeInterpreterCall objects from plain
  dicts in _extract_tool_result_output_items so responses_output
  has uniform types across streaming and non-streaming paths
2026-03-18 12:37:13 +01:00
Andrzej Pomirski
d10007cef4 test: add non-bash skip test and mock end-to-end streaming integration test
- test_non_bash_tool_result_skipped: verifies text_editor results produce
  zero code_interpreter_call items
- test_end_to_end_streaming_chunks_to_code_interpreter_output: exercises
  full path from Anthropic SSE chunks through ModelResponseIterator,
  stream_chunk_builder, and _extract_tool_result_output_items without
  a live server
2026-03-18 12:37:13 +01:00
Andrzej Pomirski
5b3e84f383 fix: address remaining review feedback
- Empty stdout/stderr now produces outputs=None (matching OpenAI parity)
  instead of outputs=[{logs:""}], in both streaming and non-streaming paths
- Fix test fixture to use real Anthropic type "bash_code_execution_tool_result"
  instead of "code_execution_tool_result"
- Add test for empty-output → outputs=None behavior
- Add unit tests for _extract_tool_result_output_items: Pydantic objects,
  plain dicts (post-model_dump), empty/missing provider_specific_fields,
  and in-place substitution preserving output ordering
2026-03-18 12:37:13 +01:00
Andrzej Pomirski
2bf8751f6b fix: streaming code_interpreter_results dropped for multiple code executions
stream_chunk_builder uses "last value wins" for list-valued
provider_specific_fields keys. _build_code_interpreter_results was
emitting only new items (incremental), so earlier results were silently
dropped when multiple sequential code executions occurred.

- Emit cumulative list from _build_code_interpreter_results, matching
  web_search_results pattern
- Assemble server_tool_use input from input_json_delta deltas at
  content_block_stop (Anthropic streams input: {} in start block)
- Handle dict items in _extract_tool_result_output_items after
  model_dump() serialization in stream_chunk_builder
- Simplify _merge_provider_specific_fields to last-value-wins for lists,
  matching stream_chunk_builder semantics
2026-03-18 12:37:13 +01:00
Andrzej Pomirski
92b89353ae fix: surface Anthropic code execution results as code_interpreter_call in Responses API
PR #18945 added support for capturing Anthropic server-side tool results
(bash_code_execution_tool_result, etc.) in provider_specific_fields, but
the data never reached the Responses API output because:

1. Non-streaming: provider_specific_fields wasn't copied into _hidden_params
2. Streaming: chunk delta's provider_specific_fields wasn't accumulated
3. Tool results weren't mapped to standard output items

This fix:
- Copies provider_specific_fields to _hidden_params in transform_response()
- Accumulates provider_specific_fields from streaming chunk deltas
- Maps bash_code_execution_tool_result to code_interpreter_call output items
  with code and outputs (matching OpenAI's native shape)
- Removes redundant function_call items for server-side tools
- Adds OutputCodeInterpreterCall type to the output union
2026-03-18 12:37:13 +01:00
yuneng-jiang
dd1a3e15e1
Merge pull request #23526 from Sameerlite/litellm_anthropic-guardrail-tools
fix(anthropic): preserve native tool format when guardrails convert tools for Anthropic Messages API
2026-03-14 09:38:43 -07:00
Chesars
5c1e5c2510 Merge main into litellm_oss_staging_03_05_2026 2026-03-14 00:42:39 -03:00
Chesars
2d33d6496b Merge branch 'upstream/main' into HEAD
# Conflicts:
#	tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py
2026-03-13 22:56:08 -03:00
Sameer Kankute
45ba9e1f7e fix(anthropic): preserve native tool format when guardrails convert tools for Anthropic Messages API
- Keep Anthropic-native tools (tool_search_tool_regex, web_search, bash, etc.) in original format when translating to OpenAI format for guardrails
- Convert guardrail-returned tools back from OpenAI to Anthropic format (type=custom for user tools)
- Add TOOL_SEARCH_TOOL to ANTHROPIC_HOSTED_TOOLS enum; use prefix matching for native tool detection
- Set type=custom explicitly when mapping OpenAI function tools to AnthropicMessagesTool
- Add test for Anthropic native tools with guardrails

Made-with: Cursor
2026-03-13 11:34:18 +05:30
Cursor Agent
9a356644bf
fix(tests): stabilize 3 failing CI tests
1. Add missing __init__.py files in tests/test_litellm/llms/gemini/ and
   subdirectories (realtime/, image_edit/) to fix ModuleNotFoundError
   with pytest-xdist parallel workers.

2. Update test_transform_request_uses_dynamic_max_tokens to use
   claude-3-7-sonnet-20250219 (max_output_tokens=64000) since
   claude-3-5-sonnet-20241022 was removed from model_prices JSON
   during deprecated model cleanup. The test assertion was outdated.

3. Update context caching TTL tests to use gemini-2.5-pro instead of
   gemini-1.5-pro. The old model was removed from model_prices JSON,
   causing supports_system_messages to return False, which prevented
   system_instruction from appearing in the transformation output.

Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
2026-03-13 00:26:31 +00:00
Cursor Agent
e242356570
fix(ci): fix ruff lint errors and 9 failing unit tests on main
Lint fixes (check_code_and_doc_quality job):
- Remove unused variable reasoning_effort in gpt_5_transformation.py (F841)
- Remove unused timezone imports in mcp_server rest_endpoints.py and server.py (F401)
- Remove unused ProxyBaseLLMRequestProcessing import in realtime endpoints.py (F401)
- Add BaseRealtimeHTTPConfig to TYPE_CHECKING block in utils.py (F821)
- Add PLR0915 per-file-ignore for mcp_server/rest_endpoints.py in ruff.toml

Test fixes (litellm_mapped_tests_llms job):
- Gemini video cost tests: pass explicit model_info to video_generation_cost()
  instead of relying on gemini/veo-3.0-generate-preview being in model_prices JSON
- Anthropic max_tokens tests: mock get_max_tokens() to return expected values
  instead of depending on claude-3-5-sonnet-20241022 being in model_prices JSON
- Vertex AI pydantic obj test: update from removed gemini-1.5-pro to gemini-2.5-flash,
  update expected request body to use response_json_schema format
- Vertex AI/Bedrock file_content integration tests: update mocks to target
  base_llm_http_handler.retrieve_file_content (the new code path via
  ProviderConfigManager) instead of the old vertex_ai_files_instance/
  bedrock_files_instance paths

Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
2026-03-12 19:58:43 +00:00
Cesar Garcia
ebe75a1bc6
Merge pull request #16594 from Chesars/feat/anthropic-files-api
feat: Add Anthropic Files API support
2026-03-11 13:45:53 -03:00
Chesars
9eff611b1a feat(anthropic): add Files API support for SDK
Implement Anthropic Files API (upload, retrieve, list, delete, content)
using the BaseFilesConfig provider pattern. Adds multipart form-data
support to BaseLLMHTTPHandler for file uploads.
2026-03-11 12:45:19 -03:00
Peter Dave Hello
3f18cd2fdc
[Docs] Fix "Page Not Found" link for Anthropic endpoint (#23349)
* fix(anthropic): enforce type:'object' on tool input schemas

Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.

The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.

Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.

Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.

Related issues: #12020, #64, #1671

* fix(anthropic): deduplicate tool_result messages by tool_call_id

Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.

This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().

Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).

Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.

Related issues: #11804, #11029, #6836, #1782, #151

* fix: shallow copy input_schema to avoid caller mutation + add mutation guard test

Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call

* feat: add qwen3.5 series for openrouter

* fix: typo on max_output_tokens and max_tokens from qwen3.5 series

* chore: fix

* chore: fix

* [Test] UI - Logs: Add unit tests for 5 untested view_logs components

Add vitest tests for TypeBadges, ErrorViewer, ConfigInfoMessage, TimeCell, and TruncatedValue covering rendering, user interactions, and edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* feat(chat-ui): responses API + MCP tool execution in /chat (#23297)

* feat(ui): add Chat UI v0 — standalone LiteLLM-branded chat window

Adds a full chat UI accessible from the sidebar Chat link (opens in new tab).
- Standalone route at /chat (outside dashboard layout — no Navbar/Sidebar chrome)
- Claude.ai-style layout: model selector top-left, LiteLLM logo center, settings top-right
- Greeting with time-of-day, centered input card, suggestion chips (Write/Learn/Code/Brainstorm)
- Sliding conversation history sidebar with Cmd+K search, rename, delete, date grouping
- localStorage-backed conversation persistence (litellm_chat_history_v1)
- Streaming completions via makeOpenAIChatCompletionRequest with AbortController stop support
- MCP server picker (toggle servers on/off per conversation)
- LiteLLM aesthetic: white/light-gray background, Ant Design blue (#1677ff) primary, system font
- Sidebar2: Chat menu item opens in new tab via window.open

* feat(chat-ui): responses API + MCP tool execution display

- Switch /chat from chat completions to responses API (previous_response_id session chaining)
- Add MCP server picker with search filter in chat input bar
- Show MCP tool call events (list_tools + call_tool) inline in chat via MCPEventsDisplay
- Add tool chip strip showing available tools when MCP servers are selected
- Non-blocking MCP toggle: server added immediately, verification in background (works for no-auth MCPs like deepwiki)
- Add truncateAfterMessage to useChatHistory for edit/retry
- Sync activeConversationId on URL change (fixes stale conversation on new chat)
- Add "Open Chat" shortcut button to sidebar

* fix(chat-ui): switch to responses API, remove dead code, add tests

- Switch handleSend from makeOpenAIChatCompletionRequest to makeOpenAIResponsesRequest with previous_response_id session chaining
- Add responsesSessionId state; reset to null when starting a new conversation
- Remove unused ChatInputBar.tsx and ModelSelector.tsx (dead code)
- Add tests/test_litellm/test_chat_ui_responses_session.py covering previous_response_id forwarding and signature validation

* fix(chat-ui): address greptile review issues

- Reset responsesSessionId when activeConversationId changes (not just on new conversation)
- Wire onMCPEvent callback into makeOpenAIResponsesRequest; render MCPEventsDisplay below messages
- Clear mcpEvents on each new send
- Explicitly filter history to user/assistant roles only (no tool-role casting)
- Remove duplicate "Chat" menu item from sidebar (pinned button serves same purpose)
- Make Sider a flex column so "Open Chat" button actually pins to bottom
- Fix tests to intercept real HTTP requests and assert previous_response_id in body

* fix(chat-ui): address greptile review feedback (greploop iteration 1)

- Fix duplicate context: when responsesSessionId is set, only send the
  new user message as input (prior context is already server-side via
  session chaining). Full history is still sent on the first turn.
- Fix ephemeral MCP events: store events per-message in ChatMessage.mcpEvents
  instead of ephemeral component state. Events now survive across turns
  and render inline below each assistant response via MCPEventsDisplay.
- Remove stale mcpEvents useState and ephemeral panel at bottom of chat.

* fix(chat-ui): address greptile review feedback (greploop iteration 2)

- Fix stale session on edit/retry: derive previousResponseId as null when
  historyOverride is set so edit/retry always starts a fresh Responses API
  session rather than chaining off a now-invalid prior session
- Fix unsafe MCPEvent cast: import MCPEvent directly from MCPEventsDisplay
  into types.ts and type ChatMessage.mcpEvents as MCPEvent[], eliminating
  the bare 'as MCPEvent[]' cast in ChatMessages.tsx

* fix(chat-ui): fix MCPEvent layering, batch localStorage writes, module-level test imports

- Move MCPEvent interface definition into chat/types.ts (single source of truth)
- MCPEventsDisplay.tsx now imports MCPEvent from types.ts instead of defining it locally
- Batch MCP event localStorage writes: accumulate during stream, persist once in finally
- Move test imports to module level per PEP 8 convention

* fix(chat-ui): fix MCPEvent import path and rename truncateFromMessage

- responses_api.tsx now imports MCPEvent directly from chat/types (not via MCPEventsDisplay re-export)
- Remove the now-unnecessary MCPEvent re-export from MCPEventsDisplay.tsx
- Rename truncateAfterMessage → truncateFromMessage: the function removes the target message and all subsequent ones (not just what comes after), so the new name accurately describes the behavior

* fix(responses-api): fix whitespace token filter and MCP server URL construction

- Drop the delta.trim() whitespace filter that was silently swallowing spaces
  and newlines during streaming, causing words to concatenate and paragraphs
  to collapse. Only skip truly empty strings (delta.length > 0).
- Use proxyBaseUrl for MCP server_url construction instead of the hardcoded
  relative path "litellm_proxy/mcp", so non-root deployments route correctly.

* fix(responses-api): use unique server_label per MCP server to prevent tool routing collisions

* fix(chat-ui): move MCPEvent to shared mcp_tools/types, skip partial events on abort

- Move MCPEvent interface to mcp_tools/types.tsx (shared with MCPServer/MCPTool),
  eliminating the playground→chat cross-module dependency. chat/types.ts and
  both playground components now import from mcp_tools/types.
- Only persist accumulated MCP events when the stream completes cleanly; aborted
  or errored turns drop partial events to avoid showing incomplete tool calls.

* fix(responses-api): use server_name for MCP URL routing, fix test path

- Use server_name (not alias) as the URL path segment for MCP server_url;
  alias is a display name that may differ from the registered proxy route.
  URL-encode the path to handle names with spaces/special characters.
- Fix sys.path.insert in tests to use __file__-relative path so tests pass
  regardless of which directory pytest is invoked from.

* fix(chat-ui): fix stale session after failed edit, clean MCP event persistence, unique server_label

- Eagerly call setResponsesSessionId(null) when historyOverride is set so a
  failed/aborted edit does not leave a stale session contaminating the next turn
- Replace abort-signal check with streamCompletedCleanly flag to correctly skip
  MCP event persistence on both abort and non-abort errors (network/API failures)
- Use server_name (unique) as server_label instead of alias to prevent silent
  tool-routing failures when two MCP servers share the same display name

* [Feat] UI - Show logos on MCP Apps page (#23320)

* feat(ui): add MCP server logo support across admin and chat UIs

- New MCPLogoSelector component with grid of well-known logos (GitHub,
  Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
  auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
  graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering

- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
  Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
  GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
  asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
  including /ui/chat/ (absolute paths resolve correctly everywhere)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(codeql): remove ruby from language matrix (#23227)

* Add team-scoped MCP server filtering for key creation and fix UnboundLocalError

When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix cross-team MCP server info disclosure and restricted key bypass

The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix mcp_tool_permissions JSON string deserialization in _resolve_team_allowed_mcp_servers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [Feature] UI - MCP Servers: Add per-server health recheck

Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix Anthropic docs link for beta endpoint

Update the Anthropic /v1/messages beta endpoint docstring to point to
its current pass-through documentation.

This keeps the change scoped to the incorrect URL and avoids changing
unverified wording in the surrounding comment.

---------

Co-authored-by: netbrah <162479981+netbrah@users.noreply.github.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
2026-03-11 20:17:41 +05:30
Sameer Kankute
c1b860b3c1
Revert "fix: strip empty text content blocks in /v1/messages endpoint (#23097)"
This reverts commit 2c738cc939.
2026-03-10 09:53:19 +05:30