litellm/tests/unit/litellm_core_utils
Priyansh Nandwana 67ae3fb527
fix(websearch)!: keep the provider prefix on the agentic follow-up model (#38885)
* fix(websearch): keep the provider prefix on the agentic follow-up model

The agentic follow-up rebuilt its model string with "a slash means it is already
qualified". Handlers are handed the provider-stripped model, and for a provider
that routes through a sub-path the remainder still holds a slash, so
bedrock/mantle/anthropic.claude-sonnet-5 arrived as mantle/anthropic.claude-sonnet-5
and stayed that way. The follow-up acompletion could not resolve a provider from it
and raised "LLM Provider NOT provided".

Prefix unless the model is already qualified with that same provider, in one shared
helper used by both the handler and the websearch patch builder, which carried
copies of the same heuristic. This also fixes openrouter, whose stripped model
keeps a slash for the same reason, and the old startswith test that read openai as
a prefix of openai_like.

Fixes #38829

* test(websearch): cover the agentic follow-up call site

The helper had unit tests but the call site did not, so the two changed lines in
_execute_chat_completion_agentic_plan were uncovered. Drive the real follow-up with
mock_response instead of patching litellm, which keeps the test on the behaviour:
against the old code the sub-path case raises "LLM Provider NOT provided", and the
ordinary-model case passes either way as a guard.

* refactor(websearch): trim the comments on the provider-qualify helper

* refactor(websearch): cut the helper and test docstrings to one line

* fix(websearch): leave a hook-qualified follow-up model on its own provider

A hook that patches in its own model owns that choice, so re-qualifying it sent a
cross-provider follow-up to the original request's provider instead. Only a bare
patched model and the request's provider-stripped model take the prefix now, and
the sibling agentic-loop path resolves the model the same way.

* fix(websearch): fall back to the request model when a patch carries no model

An empty patched model dispatched a provider-only name instead of falling back, and a
provider-stripped sub-path copied into the patch lost its prefix. A patched model now
keeps its own provider only when its leading segment names one.

* fix(websearch): keep the provider prefix on the responses follow-up model too

The responses patch builder still used the old "a slash means already qualified" test, so
a provider-stripped sub-path model reached the follow-up unqualified on that surface. It
now resolves the model the same way the chat completion builder does.

* fix(websearch): read custom_llm_provider without a member call on untyped kwargs

* fix(websearch): type the chat patch builder kwargs instead of working around them

* test(websearch): run the real search path and flatten the qualify helper tests

* fix(websearch): route a hook's model on a JSON-configured provider and keep the request provider on an echoed model

* fix(websearch): leave a follow-up model alone when its prefix already routes to the request provider

* fix(websearch): send the agentic follow-up to the model the caller requested

completion() and responses() record the caller's model before routing strips
it, and the chat and Responses follow-up builders prefer that model. This keeps
bedrock/converse_like/, bedrock/claude_platform/ and openrouter/openrouter/*
deployments on their own route for the follow-up.

* test(websearch): cover the recorded caller model in Headroom's retrieval follow-up

* test(websearch): cover the recorded caller model in Compresr's retrieval follow-up

Also merges the repeated provider-prefix check in core_helpers into one helper
and drops a test docstring

* test(websearch): drive the real search path through a mocked Perplexity HTTP route

* refactor(websearch): read the provider from kwargs inside _caller_model

* test(http-handler): type the agentic follow-up provider tests

* refactor(websearch): record agentic loop params through a Logging method

Moving the recording out of completion() keeps it under basedpyright's
code-flow complexity limit after the merge with main.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 04:47:18 -07:00
..
audio_utils test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
llm_cost_calc fix(cost-map): rolling provider pricing, capability and retirement audit (#48188) 2026-10-28 16:07:22 +00:00
llm_response_utils fix(logging): spilled over PTU requests without base_model log the served model's rates (#47655) 2026-10-20 22:28:19 +00:00
prompt_templates fix(responses): drop a malformed prompt_cache_breakpoint under drop_params on the direct /v1/responses path (#45330) 2026-10-29 03:37:54 -07:00
specialty_caches test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
__init__.py test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
attribute_forwarder.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
conftest.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
event_loop_lag.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
fake_secret_vault.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
messages_with_counts.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_agentic_followup_kwargs.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_anthropic_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_api_route_to_call_types.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_app_crypto.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_audio_utils.py fix(proxy): account for media in combined TPM reservations (#48865) 2026-10-28 12:39:41 -07:00
test_aws_partition.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_batch_utils.py refactor(router): fence the router surface and document the router-core boundary (#48839) 2026-10-29 00:01:52 +00:00
test_bedrock_converse_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_bug_report.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_chat_completion_agentic_loop.py fix(websearch)!: keep the provider prefix on the agentic follow-up model (#38885) 2026-10-29 04:47:18 -07:00
test_classifier_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cli_token_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cloud_storage_security.py fix(proxy)!: let proxy admin keys read raw cloud storage file ids on GET /files/{id}/content (#40621) 2026-10-20 10:58:42 +00:00
test_codestral_provider_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_core_helpers.py fix(websearch)!: keep the provider prefix on the agentic follow-up model (#38885) 2026-10-29 04:47:18 -07:00
test_coroutine_checker.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cost_discount.py feat(cost): support provider/model-pattern keys in cost_discount_config (#43335) 2026-10-12 14:59:18 -07:00
test_custom_logger_registry.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_dd_tracing.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_decisions_pricing.py feat(decisions): bill decisions and systemone from _decisions cost map fields (#45904) 2026-10-12 14:31:12 -07:00
test_decode_special_tokens.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_default_encoding.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_dot_notation_indexing.py refactor(typing): widen delete_nested_value to Mapping and drop two unused Span aliases (#48001) 2026-10-24 19:39:27 +00:00
test_duration_parser.py feat(proxy): report when each per-model budget window resets (#45797) 2026-10-15 10:56:45 -07:00
test_env_utils.py fix(config): treat blank env values as unset so the proxy neither crash-loops nor silently drops a guardrail (#46341) 2026-10-14 12:49:31 -07:00
test_error_normalization.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_exception_mapping_utils.py fix(responses)!: bill partial usage on mid-stream failures and report bridged /v1/messages failures to the proxy (#47537) 2026-10-28 13:32:09 -07:00
test_extract_base64_image.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_fallback_generalizations.py refactor(fallback-generalizations)!: remove the legacy extends and mixed-rule shim (#48737) 2026-10-28 05:17:28 -07:00
test_fallback_utils.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_get_litellm_params.py fix(chatgpt): derive the upstream session id from prompt_cache_key (#42014) 2026-10-28 10:57:51 -07:00
test_get_llm_provider_endpoint_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_get_llm_provider_logic.py fix(proxy): let typesafe/* allowlists cover /typesafe Jev calls and list Jev in the model picker (#47697) 2026-10-28 20:19:58 -07:00
test_get_model_cost_map.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_get_supported_openai_params.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_health_check_helpers.py fix(proxy)!: list every priced Vertex family under a vertex_ai/* grant (#41959) 2026-10-29 00:31:04 -07:00
test_hidden_params.py perf(streaming): cut per-chunk pydantic round trips on openai-compatible streams (#47030) 2026-10-20 02:42:32 +00:00
test_image_handling.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_initialize_dynamic_callback_params.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_internal_call_metadata.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_json_fragment_accumulator.py test(json_fragment_accumulator): assert peak allocation instead of a drain timing ratio (#43370) 2026-10-28 22:04:47 -07:00
test_json_schema_validation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_litellm_logging.py fix(logging): bill each SDK call its own native Bedrock guardrail trace when calls share one metadata dict (LIT-10239) (#48923) 2026-10-28 15:11:26 -07:00
test_llm_judge.py fix(guardrails): resolve decision models in caller team scope (#46085) 2026-10-15 11:43:36 -07:00
test_llm_request_utils.py fix(proxy)!: list every priced Vertex family under a vertex_ai/* grant (#41959) 2026-10-29 00:31:04 -07:00
test_logging_callback_manager.py fix(router): unregister strategy selectors from global callbacks on discard (#45795) 2026-10-21 02:39:36 +00:00
test_logging_utils.py fix(logging): bound data URI regex so base64 truncation stays linear (#45132) 2026-10-07 12:32:04 -07:00
test_logging_worker.py fix(router): preserve native baseline identity and accounting (#44960) 2026-10-07 11:00:31 -07:00
test_max_streaming_duration.py fix(router): log each dropped /v1/messages attempt through its own logging copy and open the retry without its usage (#45194) 2026-10-20 21:46:45 -07:00
test_model_param_helper.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_model_response_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_moderation_standard_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_oauth_token_exchange.py feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158) 2026-10-09 15:19:07 -07:00
test_offline_mode.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_optional_imports.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_path_utils.py refactor(core): move path_utils from proxy common_utils to litellm_core_utils (#45775) 2026-10-10 19:07:22 +00:00
test_private_json.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_affinity.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_specific_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_ptu_pricing.py feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
test_realtime_errors.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_realtime_streaming.py fix(bedrock)!: block realtime replay after an input-first start and reject malformed Nova Sonic session settings (#46936) 2026-10-17 00:29:50 +00:00
test_redact_messages.py feat(bedrock): price native guardrailConfig trace usage into spend and x-litellm-response-cost (#48278) 2026-10-27 12:10:45 -07:00
test_request_timeout_resolver.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_retry_after_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_divide_seconds.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_json_dumps.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_sensitive_data_masker.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_sentry_scrubbing.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_served_output_texts.py fix: handle missing choice indices in streamed output recording (#46947) 2026-10-17 14:17:46 -07:00
test_stream_usage_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_streaming_chunk_builder_cursor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_server_tool_use.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_utils.py fix(token_counter): count OpenAI models newer than tiktoken's table with o200k_base (#48728) 2026-10-29 00:08:12 -07:00
test_streaming_handler.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
test_streaming_overhead.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_text_completion_conversion.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_thread_pool_executor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter.py fix(token_counter)!: keep a loaded Hugging Face tokenizer when a slower download fails (#48916) 2026-10-29 03:50:58 -07:00
test_token_counter_tool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter_tool_data.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_tokenizer.py fix(token_counter)!: keep a loaded Hugging Face tokenizer when a slower download fails (#48916) 2026-10-29 03:50:58 -07:00
test_tool_fields.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_tool_search_spend_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_url_utils.py fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
test_xai_oauth_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00