Commit graph

23 commits

Author SHA1 Message Date
devin-ai-integration[bot]
656c63c0b4
test(translation): add claude-haiku-5-5 vertex ai cases (#47136)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-17 17:53:59 +00:00
devin-ai-integration[bot]
0c3087eb20
test(translation): recapture azure ai claude bases after usage speed field (#47117)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-17 09:30:34 -07:00
devin-ai-integration[bot]
b578dcf278
test(translation): recapture anthropic claude bases after usage speed field (#47116)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-17 09:29:56 -07:00
Kerry Lu
b5ef20e084
test(translation): recapture azure gpt-6-astra content_filters (#46691)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 09:27:01 -07:00
Kerry Lu
7b7604f0ed
test(translation): share rich fixtures across Claude, GPT and Gemini providers (#46585)
* test(integration): add rich anthropic claude haiku 5.5, opus 5.5 and fable 5.1 translation cases

Capture-backed basic cases on /v1/chat/completions, /v1/messages and /v1/responses for claude-haiku-5-5, claude-opus-5-5 and claude-fable-5-1, reusing the shared client templates and ANTHROPIC_MESSAGES_REQUEST/RESPONSE with model-specific id, content and usage overrides. The minimal Opus 5.5 hello cases are replaced by the rich conversation. Sonnet 5.5 and legacy models are unchanged

* test(translation): rich OpenAI captures for gpt-6.1-sol, gpt-6-luna, gpt-6-astra and gpt-5.6-terra

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(translation): rich Azure gpt-6.1-sol, gpt-6-luna, gpt-6-astra and gpt-5.6-terra cases

* test(translation): rich azure_ai Claude cases for haiku-5-5 and fable-5-1

Bind the shared chat_completions, messages and responses client templates and the anthropic provider templates to azure_ai/claude-haiku-5-5 and azure_ai/claude-fable-5-1 from real Azure AI Foundry captures, keeping the Azure dialect exact (/anthropic/v1/messages, x-api-key plus api-key on chat_completions and responses, system as a text block list on /v1/messages). Add prompt_cache_hit variants for both models from the second call inside the cache window. The two deployments need rows in tests/integration/proxy_config.yaml

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(translation): rich Gemini bases for gemini-3.8-flash, gemini-3.1-pro-preview and gemini-4-argon

Add the shared generateContent provider templates in templates/gemini.py from
real captures and bind the three target models on chat completions, messages
and responses with explicit per-model response overrides. gemini-4-argon is
registered in the three basic lists. gemini-3.5-flash is unchanged

* test(translation): enrich vertex_ai claude and gemini cases from live captures

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): rich Bedrock converse, invoke and mantle translation cases from real captures

* test(integration): register the rich translation deployments and follow the rich Gemini 3.8 Flash cases

Add the synthetic deployments the rich provider cases bind (anthropic, openai, azure, azure_ai, gemini, vertex_ai, bedrock converse, invoke and mantle) to tests/integration/proxy_config.yaml, all pointed at the fake provider on 127.0.0.1:8191 with synthetic keys. The Gemini omitted-temperature contract now takes its expected generationConfig and reply text from the rich GEMINI_3_8_FLASH_TEST_CASE literals instead of the retired minimal hello cases

* test(translation): reuse bridged and native response templates

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
2026-10-14 19:53:09 -07:00
devin-ai-integration[bot]
ff8b5d8324
test(translation): add claude-fable-5-1 translation cases (#46581)
* test(translation): add claude-fable-5-1 translation cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(translation): match sibling style in claude-fable-5-1 cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 01:54:42 +00:00
devin-ai-integration[bot]
a1da0ecbce
test(translation): recapture bedrock invoke claude-sonnet-4-6 bases after usage shape change (#46579)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 01:54:32 +00:00
devin-ai-integration[bot]
c9cca25991
test(translation): add gpt-6-astra translation cases (#46580)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 01:52:58 +00:00
devin-ai-integration[bot]
0f1312f39d
test(integration): add claude-haiku-5-5 basic translation cases (#45176)
* test(integration): add claude-haiku-5-5 basic translation cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): match sibling style in claude-haiku-5-5 translation cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 01:04:54 +00:00
Kerry Lu
0c91bd7fdb
test(translation): share rich request and response templates (#46546)
* test(translation): enrich Sonnet 5.5 base conversations

* test(translation): capture prompt caching and explicit reasoning

* refactor(translation): share endpoint request and response templates

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 00:30:36 +00:00
Sukhdeep
87478c7b6d
fix(gemini): do not inject temperature for Gemini 3+ models when omitted (#45107)
* fix(gemini): do not inject temperature for Gemini 3+ models when omitted

map_openai_params added temperature=1.0 to every Gemini 3+ request that did not set one, on both the gemini and vertex_ai routes. Google is deprecating temperature, top_p and top_k for Gemini 3+ and says upcoming models will reject them, so this default turns a request that never set temperature into one that will fail

An omitted temperature now falls back to Google's own default, which their docs give as 1.0 for all Gemini 3 models. Explicit temperature, top_p and top_k are still forwarded

Fixes #38663

* test(gemini): annotate sampling regression tests

* test(gemini): drop the injected temperature from Gemini translation cases

The nine Gemini base cases for chat completions, responses and messages send no temperature, so the provider request no longer carries the removed 1.0 default. Each new body was sent to the real Gemini API for gemini-3.5-flash, gemini-3.8-flash and gemini-3.1-pro-preview and returned 200, and the stored provider responses are unchanged

* test(gemini): drop the default-temperature wording from the thinking level test docstring

* test(gemini): cover omitted and explicit sampling on both Gemini routes

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-12 20:26:49 +00:00
devin-ai-integration[bot]
8080601a95
feat(bedrock): default Grok on Bedrock to native Chat Completions, mint unique tool call ids and drop stop (#45473)
* fix(bedrock): route Grok 4.7 tools with reasoning to native Chat Completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): mint unique tool call ids on native Chat Completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): send a blank text block for Converse tool results with no supported content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(bedrock): default Grok on Bedrock to native Chat Completions and leave Converse unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): gate tool call id minting to Grok on native Chat Completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(bedrock): check the Grok tool call id gate at both call sites

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): remint positional tool call ids on native Chat Completions for every model

AWS returned call_0, call_1 for GPT 5.6 as well as Grok on 2026-10-09 (and unique ids for GPT 5.6 earlier the same day), so the id format is not fixed per model. Drop the Grok-only gate and remint any call_<digits> id on the route

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): drop stop sequences for Grok instead of forwarding them to a 400

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 20:56:03 +00:00
devin-ai-integration[bot]
64cddd6e13
test(integration): basic translation cases for the bedrock_mantle route (#44750)
* test(integration): bedrock_mantle-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): rename translation runner run to assert_translation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): use assert_translation and drop the LIT-9196 skips in the bedrock_mantle basic cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-06 03:17:34 +00:00
devin-ai-integration[bot]
c1d639afff
test(integration): basic translation cases for the vertex_ai route (#44751)
* test(integration): vertex_ai-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): rename translation runner run to assert_translation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): use assert_translation in vertex_ai-route basic translation cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 20:00:19 -07:00
devin-ai-integration[bot]
7cdb3d3760
test(integration): basic translation cases for the bedrock_invoke route (#44752)
* test(integration): bedrock_invoke-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): rename translation runner run to assert_translation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): call assert_translation in the bedrock_invoke basic cases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 19:31:46 -07:00
devin-ai-integration[bot]
8b7b42f3aa
test(integration): basic translation cases for the openai route (#44667)
* test(integration): openai-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): ignore x-stainless headers in translation runner

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): skip openai chat completions basic cases on LIT-9235

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 15:32:50 -07:00
devin-ai-integration[bot]
6a92a49c32
test(integration): basic translation cases for the gemini route (#44662)
* test(integration): gemini-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): raise gemini basic cases to 1024 output tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 15:32:26 -07:00
devin-ai-integration[bot]
4314f3ce0c
test(integration): basic translation cases for the azure route (#44672)
* test(integration): azure-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): use the three-line form for the azure chat LIT-9235 skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 22:24:42 +00:00
devin-ai-integration[bot]
7123484c28
test(integration): basic translation cases for the openai_responses route (#44663)
* test(integration): openai_responses-route basic translation cases on messages, chat completions and responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): skip openai_responses-route chat completions basic cases until LIT-9196 is fixed

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 15:06:10 -07:00
devin-ai-integration[bot]
d23a0636b4
test(integration): azure_ai-route basic translation cases on messages, chat completions and responses (#44658)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 15:01:55 -07:00
devin-ai-integration[bot]
d3be5c3ab7
test(integration): bedrock_converse-route basic translation cases on messages, chat completions and responses (#44646)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 21:39:34 +00:00
devin-ai-integration[bot]
14913548d1
test(integration): anthropic-route basic translation cases for six Claude models on messages, chat completions and responses (#44607)
* test(integration): add anthropic-route /v1/messages basic cases for haiku-4-5, sonnet-5, opus-4-8 and sonnet-5-5

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add anthropic-route /v1/chat/completions and /v1/responses basic cases with generated-field matchers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): skip anthropic-route /v1/responses basic cases on LIT-9231 and expect the echoed request params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): write LiteLLM-generated ids and timestamps as mock.ANY and drop matchers.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 18:24:01 +00:00
devin-ai-integration[bot]
f850b2c324
test(integration): exact four-part translation cases on a shared fake provider and shared YAML deployment (#44451)
* test(integration): exact four-part translation cases on a shared fake provider and shared YAML deployment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): compare every non-transport provider header and check for late provider requests at session end

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): name TranslationTestCase fields after litellm and provider sides and drop regressions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prefix checked TranslationTestCase fields with expected_ and name the fake reply mock_provider_response

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs(integration): name TranslationTestCase fields in the translation README

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add a claude-opus-5-5 base case and deployment next to claude-sonnet-4-6

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): name translation cases <MODEL>_TEST_CASE and document the naming rule

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs(integration): move translation test rules into tests/integration/translation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-04 00:09:58 +00:00