scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-02 07:13:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 48
litellm/tests/e2e/llm_translation
History
Exact
Mateo Wang a713ed6bab
fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692) 
* test(e2e): add gemini tool call id length repro for LIT-8912

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them

* fix(gemini): pair a tool result with its call when the client re-adds the thought signature to the call id

Strands, which the AgentCore harness runs, puts provider_specific_fields.thought_signature back on the
assistant tool call id as <id>__thought__<signature> while its tool result keeps the bare id, so the Gemini
conversion found no matching call and answered 500. Ids now match with the signature suffix stripped.

Also adds thought_signature_in_tool_call_id to the litellm_params inventory test and gives the pass-through
tests' mocked logging objects the litellm_params a real Logging object always has.

* fix(gemini): keep the thought signature on streamed Messages and Responses tool calls and accept a string setting

* fix(gemini): read the thought signature back from Responses input and coerce the per-request setting

A Responses function_call input item now keeps its provider_specific_fields, so a
client that sends the item back gets its real thought signature replayed to Gemini.

A per-request thought_signature_in_tool_call_id of "false" or 0 now keeps ids short,
and a value that is not a boolean answers 400, or is dropped under drop_params,
instead of a 500.

The e2e test now uses the sdk fixture and @meta subjects

* fix(caching): key the response cache on thought_signature_in_tool_call_id

* fix(responses): send thought signature fields only with short ids and drop malformed routing kwargs whole

* fix(main): validate routing kwargs after the locals snapshot so fallbacks and model_list calls keep working

The e2e test now uses gemini-3.8-flash with enough max_tokens to finish thinking before the tool call.

* fix(gemini): keep non-chat routes tolerant of a malformed thought_signature_in_tool_call_id, match tool results by exact id first, and keep the deployment's value when drop_params drops a malformed override

* fix(router): validate strict litellm params through a helper so star imports cannot re-bind a public Final

* perf(router): skip strict validation when a request overrides no deployment param

* fix(responses): keep a dict tool call's thought signature on the streamed function_call done item

* refactor(main): return invalid supplemental params as a value and test the streaming tool call with a real response

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-31 03:20:42 -07:00
..
fixtures test(e2e): serve the vision image from our own fixture 2026-08-27 14:29:53 -07:00
realtime fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
conftest.py test(e2e): replace custom endpoints_client with provider SDK clients 2026-07-23 03:43:20 +00:00
conversational_matrix.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
LLM_TRANSLATION_COVERAGE_MATRIX.md fix(proxy): record streamed /v1/responses container ownership before the response.completed frame (#43140) 2026-09-25 13:45:43 -07:00
passthrough_client.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
responses_helpers.py fix(providers): route Azure Responses past dated api versions, count Grok reasoning tokens on azure_ai, refresh provider sets after register_model, fix gpt-6.1-sol Azure rows (#46377) 2026-10-16 15:23:46 -07:00
sdk_clients.py test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg 2026-09-21 12:39:16 -07:00
structured_output.py test(e2e): move live-provider legacy tests into tests/e2e (#44120) 2026-10-02 00:02:18 -07:00
test_anthropic_completion_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_audio_speech_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_audio_transcriptions_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_bedrock_converse_redacted_thinking_e2e.py fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
test_bedrock_guardrail_config_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_bedrock_kimi_k3_messages_e2e.py fix(bedrock): apply the 16 maxTokens floor to Kimi K3 on Converse (#43875) 2026-10-20 13:01:45 +00:00
test_bedrock_mantle_output_limit_e2e.py fix(bedrock_mantle): send max_completion_tokens for gpt-5.6 and gpt-6 chat completions (#43982) 2026-10-11 18:28:48 +00:00
test_bedrock_native_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_bedrock_provider_matrix_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_bedrock_web_search_server_tool_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_cache_control.py test(e2e): assert the Vertex cache-creation billing contract in the prompt caching test (#47027) 2026-10-16 18:51:38 -07:00
test_cache_control_injection_tool_calls_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_completion_prediction_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_chat_completions_contract_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_completions_regression_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_mid_conversation_system_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_stream_contract_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_tool_round_trip_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_completions_endpoint_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_containers_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_conversational_matrix_e2e.py test(e2e): add conversational matrix across chat, messages and responses (#42359) 2026-09-21 22:32:56 -07:00
test_count_tokens_source_e2e.py feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
test_credential_messages_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_custom_pricing_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_deepseek_reasoning_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_embedding_only_supported_endpoints_e2e.py feat(router): opt in to non-chat deployment filtering (#46257) 2026-10-29 08:33:36 -07:00
test_embeddings_endpoint_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_files_batches_contract_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_gemini_tool_call_id_length_e2e.py fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692) 2026-10-31 03:20:42 -07:00
test_gemini_url_context_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_google_interactions_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_google_native_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_image_edits_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_image_generation_e2e.py test(e2e): fix MCP tool search hit matching and the retired Bedrock image model (#46998) 2026-10-16 15:45:39 -07:00
test_messages_azure_foundry_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_bedrock_e2e.py fix(bedrock): keep tool_reference results as text on converse (#44590) 2026-10-09 17:52:17 -07:00
test_messages_beta_headers_e2e.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_messages_count_tokens_bedrock_e2e.py fix(bedrock): count tokens on bedrock-mantle when bedrock-runtime cannot count a Claude model (#45317) 2026-10-08 15:26:49 -07:00
test_messages_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_mid_conversation_system_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_mid_conversation_system_native_providers_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_prompt_caching_and_tool_search_e2e.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_moderations_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_ocr_rust_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_ollama_e2e.py test(e2e): cover ollama and ollama_chat on chat completions, responses and messages (#45079) 2026-10-07 20:17:35 +00:00
test_passthrough_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_passthrough_headers_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_provider_features_e2e.py fix(completion): send prompt_cache_retention to OpenAI instead of silently dropping it (#47480) 2026-10-20 09:41:22 -07:00
test_realtime_http_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_reasoning_effort_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_rerank_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_responses_bridge_streaming_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_responses_codex_tool_search_e2e.py feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972) 2026-10-29 00:29:40 -07:00
test_responses_e2e.py fix(responses): forward top-level moderation param to OpenAI (#48708) 2026-10-28 10:33:11 -07:00
test_responses_retrieve_e2e.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_sagemaker_nova_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_sail_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_together_ai_e2e.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_token_counter_gemini_contents_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_vector_stores_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_vertex_capabilities_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_vertex_passthrough_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00