litellm/tests/unit/llms/openai
paveln83 9fba44d300
fix(openai): cap and bill web search rounds on Responses-pinned chat completions (#51802)
* fix(openai): cap web search agentic loops with max_agentic_loops on OpenAI chat completions

The OpenAI chat dispatcher ran every custom logger's chat agentic loop
without a depth check, and the web search follow-up never carried the
loop depth, so max_agentic_loops was ignored on openai/ deployments.
The dispatcher now skips loggers the shared chat completion loop in
litellm.acompletion already runs, so those go through its depth, cap,
and repeated-call checks once. Loggers that only implement the
chat-specific hooks still run in the OpenAI dispatcher

* fix(openai): honor max_agentic_loops when chat completions bridge to Responses

A gpt-5.4+ chat completion with function tools on api.openai.com is sent to
/v1/responses by the Responses bridge. The Responses handler then ran its own
agentic loop on that call, which never saw the request's max_agentic_loops or
depth, and the shared chat completion loop in litellm.acompletion ran again on
what it returned, so every chat follow-up restarted the Responses loop at depth
0. With max_agentic_loops=1, web search made 8 Responses calls and 7 searches

The async bridge now marks its Responses call as part of a chat completion
turn, and the Responses handler skips its own loop for such calls. The shared
chat loop owns the whole turn, so its depth, cap, and repeated-call checks run
once

* fix(responses): keep search follow-ups on the Responses API for responses/ pinned models

When an async chat completion crosses the Responses bridge, the shared chat
loop now owns the turn and builds each web search follow-up from the model
recorded on the logging object. responses() overwrote that record with the
stripped model, so a model pinned with the responses/ prefix (for example
openai/responses/<model> on a Responses-only gateway) sent its follow-up to
chat completions, and the turn ended on the unexecuted tool call

responses() now leaves the record alone when the chat loop owns the turn, the
same way completion() skips its own record when the reverse bridge owns it

* fix(agentic-loop): bill every round of a non-streamed Responses-bridged chat loop once

A bridged chat request logs its first round before the chat loop runs, so a
follow-up run as an internal sub-call was never billed. Each follow-up of a
non-streamed bridged parent now writes its own spend row, and a failed
follow-up no longer re-bills rounds that already wrote one

* fix(agentic-loop): keep streamed Responses-bridged web search from billing its final round twice

* fix(openai): bill every bridged search round in one row after guardrails

On the proxy a non-streamed Responses-pinned chat request now parks one
spend row that sums every round the loop served, so the row and the
x-litellm-response-cost header show the whole request and nothing is
written before post-call guardrails pass. A streamed request keeps round
one's row and folds every later round into the replayed final row.

The bridge now skips only Responses hooks that run their own chat loop,
so hooks that only handle the Responses API still run.

* fix(openai): price each bridged search round at its own size

Each round of a bridged chat request is now priced on its own usage and the
combined proxy row adds those prices, so summed usage never crosses a
long-context price tier. A streamed replay keeps that total and its cost
breakdown

* fix(agentic-loop): run the chat hook of bridged callbacks that define both agentic hooks

The bridge skipped the Responses hook of a callback that runs its own chat
loop, and the shared loop only asked the general gate, so a custom callback
with both hooks stopped intercepting bridged chat. Ask its chat gate after
the shared loop. Price tier tests now use test-owned gpt-5.6 prices.

* fix(agentic-loop): route bridged chat to the chat hook only for callbacks that opt in

A custom callback that defined both agentic hooks lost its Responses hook on
chat completions bridged to the Responses API, so a chat hook that declined
left the request unhandled, and one that claimed it changed which hook
answered compared with main.

Callbacks now opt in with wants_chat_completion_hooks_on_bridged_chat. Web
search interception sets it, so its loop still runs once on the chat side.
Every other callback keeps main's routing through its Responses hook.

* fix(websearch_interception): keep main's Responses loop for a bridged web_search_options call

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 14:36:10 -05:00
..
chat fix(openai): download PDF URLs off the event loop and transform async chat messages once (#49176) 2026-11-10 16:37:00 +00:00
completion fix(openai): keep the upstream error body on e.response for OpenAI and Azure errors (#49079) 2026-10-29 23:30:02 -07:00
decisions feat(decisions): add Parallel AI web-grounded decisions (#49188) 2026-11-03 04:47:02 +00:00
embeddings test: add package initializers to migrated unit test directories 2026-09-20 11:52:07 +00:00
evals fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
image_generation refactor(images): remove dall-e-2 and dall-e-3 code paths (#48703) 2026-10-28 17:48:09 -07:00
image_variations fix(openai): keep the upstream error body on e.response for OpenAI and Azure errors (#49079) 2026-10-29 23:30:02 -07:00
live refactor(router): delete the python router core, the rust core is the only core (#50178) 2026-11-05 10:40:46 -08:00
realtime feat(realtime)!: support latest OpenAI audio models (#35600) 2026-11-03 07:27:52 +00:00
responses fix(anthropic): close strict JSON schemas on the Messages to Responses bridge (#51337) 2026-11-12 06:15:14 +00:00
speech test: add package initializers to migrated unit test directories 2026-09-20 11:52:07 +00:00
transcriptions fix(cost)!: bill transcriptions by the seconds the provider reports (#50727) 2026-11-09 13:37:55 -08:00
vector_store_files fix(proxy)!: return provider-matching 4xx for the LIT-8062 status code follow-ups (#45537) 2026-10-13 11:42:39 -07:00
vector_stores test: add package initializers to migrated unit test directories 2026-09-20 11:52:07 +00:00
videos feat(videos): add first-class OpenRouter video request fields (#50319) 2026-11-06 16:50:48 -08:00
__init__.py test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
test_cost_calculation.py feat(fal_ai): bill fal video clips by duration and resolution (#51679) 2026-11-13 12:55:55 -08:00
test_data_residency.py feat(openai): bill gov.api.openai.com FedRAMP endpoint with 1.1x uplift (#49951) 2026-11-04 08:57:42 -08:00
test_gpt_reasoning_transformation.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_is_model_gpt_5_model.py refactor(llms): rename model-version transformation files to family names (#48509) 2026-10-26 18:53:16 -07:00
test_o_series_transformation.py fix(openai): stop stripping cache_control from the caller's own messages and tools (#35602) 2026-10-31 00:29:10 -07:00
test_openai.py fix(openai): cap and bill web search rounds on Responses-pinned chat completions (#51802) 2026-11-14 14:36:10 -05:00
test_openai_authentication_error.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_common_utils.py fix(logging): warn when drop_params drops a requested reasoning level on the remaining paths (#48933) 2026-10-30 08:48:43 -07:00
test_openai_empty_response.py fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967) 2026-10-16 18:06:01 -07:00
test_openai_file_content_streaming.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_openai_image_edit_transformation.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_openai_workload_identity.py refactor(router): route the router family through RoutingContext and close the outside surface (#49118) 2026-10-30 19:18:54 -07:00
test_organization_costs.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_use_chat_completions_api_no_leak.py fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967) 2026-10-16 18:06:01 -07:00