* fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params
A GPT-5 request whose reasoning_effort the model map does not enable for
that model lost the effort with no log line under drop_params. An Azure
deployment name missing from the cost map lost reasoning_effort none this
way and ran at the model's default effort. The gate now logs a warning
naming the model, the level, the map entry it checked, and the
model_info.base_model and model_info.reasoning_effort_levels fixes.
* fix(openai): warn when the Responses route or a deployment's declared levels drop a reasoning effort
The /v1/responses gate dropped a row-disabled GPT-5 effort, and reasoning for an OpenAI model without
supports_reasoning, with no log line, and the router logged a declared-level drop only at INFO. Both
Responses drops now log a WARNING and the router logs its drop at WARNING. The warning now leads with
model_info.reasoning_effort_levels, since base_model only helps a deployment named differently from
its underlying model.
* fix: warn only when a reasoning level is actually dropped, and on generic reasoning_effort drops
* fix: warn when thinking or a Bedrock Converse reasoning level is dropped, and name the router drop a reasoning level
* fix: skip the drop warning when a non-reasoning model is asked for no reasoning, and honor declared levels on Bedrock Responses
* fix: stay silent when reasoning is turned off on /v1/responses and warn on the Bedrock Responses fallback drop
* fix(reasoning): warn when any reasoning-off request is dropped and give each route only the advice it honors
A request to turn reasoning off on a model name the catalog lacks now warns on every route, since it may be a reasoning model. Chat suggests model_info.reasoning_effort_levels and, for a name with no catalog entry, model_info.base_model. OpenAI Responses suggests declared levels only where the config honors them, and Bedrock suggests neither. The Bedrock and Mantle Responses forwarding of declared levels is reverted until it can be proven live.
* fix(reasoning): suggest reasoning_effort_levels and base_model only where the route reads them
The chat gate suggested model_info.reasoning_effort_levels on routes that never pass declared levels
(github_copilot, chatgpt and other OpenAIConfig delegates) and model_info.base_model on OpenAI and
Azure AI chat, whose gate looks up the deployment name rather than base_model. /v1/responses/compact
suggested levels it never reads either.
Declared levels are now None when a caller passes none, so the advice appears only when a caller fed
them, and the base_model advice is limited to the Azure chat gate, the one that looks up base_model.
The openai row of the chat warning test now expects no base_model advice, the Responses warning test
goes through get_optional_params_responses_api the way /v1/responses and compact call it, and the Azure
direct-config test passes declared levels the way get_optional_params does. Long test lines are wrapped.
* fix: read only the built-in catalog when choosing the reasoning drop warning
* test: migrate selected llm translation contracts
* test: migrate anthropic completion translation tests
* test: migrate bedrock and cohere translation tests
* test: tighten bedrock translation migration
* test: migrate azure and openai model translation tests
* test: migrate openai and small-provider translation tests
* test: migrate gemini and vertex translation tests
* test: migrate optional params and rerank translation tests
* test: migrate huggingface, json and openrouter translation tests
* test: fix openrouter integration expectations
* test: migrate databricks, xai, cometapi and fireworks translation tests
* test: migrate ovhcloud, prompt and small provider translation tests
* test: split reasoning effort grid into unit and e2e
* test: migrate router and realtime guardrail translation tests
* test: migrate google interactions translation tests
* test: migrate a2a, evals, skills and hosted vllm translation tests
* test: remove remaining llm translation suite
* test: tighten migrated llm translation unit tests and land missing rows
Restores rows the first pass dropped or weakened (bedrock agentcore and openwebui thinking stream, anthropic prefix, citations and tool_choice, azure agents handler, huggingface bare api_base, optional params num_retries, alternating roles, rerank response type), replaces truthiness and call-count checks with whole-value assertions, and moves tests to the mirror file of the code they cover
* test: move cohere, databricks, fireworks and gemini rows to mirror files over mocked HTTP
Restores three TestCohereV2Transform methods the first pass nested out of collection, replaces direct transform calls with respx-backed completion calls, and drops the databricks and fireworks files that sat outside their mirror paths
* test: close lost-intent gaps found in review of the llm translation migration
Points the hosted-mock guard at the triton test's new integration path, restores exact rerank result assertions for infinity, drops the reasoning e2e fallback that passed for any multi-line answer, and adds the mixtral invoke system prompt variants, eval deletion, and async pdf url download that the legacy suite exercised
* test: assert provider acceptance in the reasoning effort e2e
The live half of the split grid proves each route accepts reasoning_effort. claude-fable-5-1 on the direct route answers without surfacing reasoning content, so a reasoning-presence check either fails on head or needs a fallback that passes for any answer. Wire shape stays covered by the unit grid
* test: cover async and stream-end paths the legacy translation suite exercised
Adds async variants for bedrock converse file and thinking content, bedrock embedding timeout and region resolution, interaction stream termination and success logging, xai reasoning token folding, and bytes chunks with a prefix before data:. Also scopes the vertex safety settings global in test_vertex_safety_settings with monkeypatch so it stops leaking into gemini request bodies on the same worker
* test: type the new llm translation e2e calls and mark moved fixture dirs as packages
* test: name the reasoning effort e2e models as module constants
* test: assert reasoning content, register the new gemini coverage ids and give the guardrail config e2e a unique name
* test: block the prompt the shared guardrail denies and drop the reasoning presence check
* test: drop the duplicate litellm import left by the main sync
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: move the unit half of 126 mixed legacy files into tests/unit
* test: restore litellm globals that moved tests set
* test: finalize migration test cleanup
* test: restore original bodies of moved legacy tests
The move into tests/unit had rewritten 612 test bodies, and some of the rewrites dropped assertions. Each moved test now carries its original body from the legacy file, with only the imports, helpers, fake provider credentials and monkeypatched env it needs to run under tests/unit
test_timeout_streaming goes back to tests/local_testing because it needs the fake OpenAI endpoint server. The image payload fixture moves with its only user, and two tests that leaked global state (a registered model cost entry and queued logging tasks) are now isolated
* test: drop module imports shadowed by restored local imports
* test: assert on LiteLLM output in no-assertion moved tests and isolate leaks
Twenty no-assertion candidates get one assertion on the value LiteLLM returns, with the original lines unchanged. Four tests go back to their legacy files because they only check types or imports, write into the working directory, or cannot assert without a body change
Two moved tests leaked globals into later tests in the same worker, so monkeypatch fixtures now restore the retry-after header parser and the end user cost tracking flags
* test: drain queued logging tasks before the Phoenix span test
The moved Phoenix test counted spans from logging tasks that earlier tests had queued, so the drain fixture moves to tests/unit/conftest.py and both it and the Datadog batch test use it. test_factory_function goes back to its legacy file because its returned wrapper calls the real Assistants API and cannot be asserted on without a body change
---------
Co-authored-by: yuneng <yuneng@berri.ai>
