litellm/tests/unit/litellm_core_utils
devin-ai-integration[bot] b195d15b93
fix(vertex_ai,azure_ai)!: auto-add the compaction, thinking display, and tool change betas on Claude routes (#44747)
* fix(vertex_ai): forward and auto-add the compact-2026-09-04 beta on Claude routes

* test(vertex_ai): prove the compact-2026-09-04 beta reaches the wire through both Claude handlers

* fix(vertex_ai): move a client-sent body compact-2026-09-04 beta to the request header

* refactor(vertex_ai): build the Claude beta set without mutation and inject the client in the messages wire test

* fix(vertex_ai): keep client body betas in the body and answer 400 on a malformed anthropic_beta

* fix(rust): allow the compact-2026-09-04 beta on vertex_ai in the Rust beta table

The beta header config now maps compact-2026-09-04 for vertex_ai, so the Rust
table that mirrors it must stop rejecting that beta there, and the Rust Vertex
messages route adds it for a compaction request like the Python route does.

* fix(vertex_ai): keep client body betas out of the Vertex body next to litellm betas, honor drop_params, and send the compaction beta on count_tokens

Litellm's own betas replace a client's body anthropic_beta again, as on main, so a body beta
Vertex rejects no longer turns a 200 into a 400. A malformed beta drops under drop_params and
names its field otherwise. inline-tools-2026-09-15 joins compact-2026-09-04 as header-only, since
Vertex rejects both in the body. Vertex count_tokens sends compact-2026-09-04 when the request
carries compaction.

* fix(logging): log and bill Anthropic messages compaction responses on the Rust route

The Rust route hands the served Anthropic body to the logger, which validates it
against AnthropicResponse. That model had no compaction content block and no
compaction stop reason, so a compaction reply raised and wrote no spend row.

* fix(vertex_ai,azure_ai): forward compact-2026-09-04 to Azure AI Foundry, merge Vertex body betas, and redact compaction summaries

Azure AI Foundry needs the compact-2026-09-04 header for compaction, a replayed compaction block, and count tokens, so the beta map, the native compaction providers, and the Rust beta table now allow it there. Vertex keeps a client's body betas next to the ones litellm adds, as the Anthropic route does, and turn_off_message_logging now redacts compaction summaries too

* fix(vertex_ai,azure_ai): auto-add thinking display and tool change betas, redact streamed compaction summaries

Folds the header changes from #44103 into this PR. Vertex AI and Azure AI Claude
routes now add thinking-display-updates-2026-08-18 when thinking.display is
"updates" and mid-conversation-tool-changes-2026-07-01 when a tool_addition
block is present, matching the Anthropic and Bedrock routes.

With turn_off_message_logging on, compaction summaries from streamed messages
and streamed chat completions are now redacted in spend logs and callback
payloads, like the non-streamed response already was.

* fix(vertex_ai,azure_ai,rust): add tool change betas on count tokens and Rust messages, bill Rust server tool replies

Vertex and Azure count tokens now add mid-conversation-tool-changes-2026-07-01 when the history carries a tool_addition or tool_removal block, through one shared helper with the compaction beta.

The Rust /v1/messages route adds thinking-display-updates-2026-08-18 and mid-conversation-tool-changes-2026-07-01 when a request uses them, on the providers that accept each beta.

Rust-route logging parses the served body with the Anthropic response parser, so web_search, pause_turn, mcp_tool_use and container_upload replies get a spend row.

Legacy enabled thinking keeps display updates on anthropic too, and the Vertex transform reads drop_params through drops_unsupported_params.

* style(rust): rewrap beta map test line with cargo fmt

* test(rust): expect the display beta on Databricks, which main's beta list now accepts

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-30 09:28:19 -07:00
..
audio_utils
llm_cost_calc fix(cost): finalize synchronous response cost once for header, spend log and rollups (#48993) 2026-10-29 11:46:49 -07:00
llm_response_utils fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
prompt_templates fix(anthropic): send PDF image_url content to Claude as document blocks (#46984) 2026-10-30 02:21:39 -07:00
specialty_caches test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
__init__.py
attribute_forwarder.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
conftest.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
event_loop_lag.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
fake_secret_vault.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
messages_with_counts.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_agentic_followup_kwargs.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_anthropic_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_api_route_to_call_types.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_app_crypto.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_audio_utils.py fix(proxy): account for media in combined TPM reservations (#48865) 2026-10-28 12:39:41 -07:00
test_aws_partition.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_batch_utils.py refactor(router): fence the router surface and document the router-core boundary (#48839) 2026-10-29 00:01:52 +00:00
test_bedrock_converse_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_bug_report.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_chat_completion_agentic_loop.py fix(websearch)!: keep the provider prefix on the agentic follow-up model (#38885) 2026-10-29 04:47:18 -07:00
test_classifier_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cli_token_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cloud_storage_security.py fix(proxy)!: let proxy admin keys read raw cloud storage file ids on GET /files/{id}/content (#40621) 2026-10-20 10:58:42 +00:00
test_codestral_provider_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_core_helpers.py fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
test_coroutine_checker.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cost_discount.py feat(cost): support provider/model-pattern keys in cost_discount_config (#43335) 2026-10-12 14:59:18 -07:00
test_custom_logger_registry.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_dd_tracing.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_decisions_pricing.py feat(decisions): bill decisions and systemone from _decisions cost map fields (#45904) 2026-10-12 14:31:12 -07:00
test_decode_special_tokens.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_default_encoding.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_dot_notation_indexing.py refactor(typing): widen delete_nested_value to Mapping and drop two unused Span aliases (#48001) 2026-10-24 19:39:27 +00:00
test_duration_parser.py feat(proxy): report when each per-model budget window resets (#45797) 2026-10-15 10:56:45 -07:00
test_env_utils.py fix(router): cap the default OTPM reservation and reconcile token counters after each call (#49054) 2026-10-29 20:32:04 -07:00
test_error_normalization.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_exception_mapping_utils.py fix(bedrock): route a Bedrock model id with no Invoke provider to Converse (#43290) 2026-10-30 06:27:59 -07:00
test_extract_base64_image.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_fallback_generalizations.py refactor(fallback-generalizations)!: remove the legacy extends and mixed-rule shim (#48737) 2026-10-28 05:17:28 -07:00
test_fallback_utils.py fix(cache_control): pick the chat-path dialect from the deployment's provider prefix (#45295) 2026-10-30 05:11:42 -07:00
test_get_litellm_params.py fix(chatgpt): derive the upstream session id from prompt_cache_key (#42014) 2026-10-28 10:57:51 -07:00
test_get_llm_provider_endpoint_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_get_llm_provider_logic.py fix(proxy): let typesafe/* allowlists cover /typesafe Jev calls and list Jev in the model picker (#47697) 2026-10-28 20:19:58 -07:00
test_get_model_cost_map.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_get_supported_openai_params.py fix(bedrock): backfill published regional service tier prices (#49113) 2026-10-29 19:46:57 -07:00
test_health_check_helpers.py fix(health): probe only models a partial wildcard serves (#49060) 2026-10-29 23:34:57 -07:00
test_hidden_params.py perf(streaming): cut per-chunk pydantic round trips on openai-compatible streams (#47030) 2026-10-20 02:42:32 +00:00
test_image_handling.py fix(anthropic): send PDF image_url content to Claude as document blocks (#46984) 2026-10-30 02:21:39 -07:00
test_initialize_dynamic_callback_params.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_internal_call_metadata.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_json_fragment_accumulator.py test(json_fragment_accumulator): assert peak allocation instead of a drain timing ratio (#43370) 2026-10-28 22:04:47 -07:00
test_json_schema_validation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_litellm_logging.py fix(vertex_ai,azure_ai)!: auto-add the compaction, thinking display, and tool change betas on Claude routes (#44747) 2026-10-30 09:28:19 -07:00
test_llm_judge.py fix(guardrails): resolve decision models in caller team scope (#46085) 2026-10-15 11:43:36 -07:00
test_llm_request_utils.py fix(health): probe only models a partial wildcard serves (#49060) 2026-10-29 23:34:57 -07:00
test_logging_callback_manager.py fix(router): unregister strategy selectors from global callbacks on discard (#45795) 2026-10-21 02:39:36 +00:00
test_logging_utils.py fix(logging): bound data URI regex so base64 truncation stays linear (#45132) 2026-10-07 12:32:04 -07:00
test_logging_worker.py fix(proxy): persist tag spend and drain success callbacks within the shutdown budget (#48988) 2026-10-29 09:32:18 -07:00
test_max_streaming_duration.py fix(router): log each dropped /v1/messages attempt through its own logging copy and open the retry without its usage (#45194) 2026-10-20 21:46:45 -07:00
test_model_param_helper.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_model_response_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_moderation_standard_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_oauth_token_exchange.py feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158) 2026-10-09 15:19:07 -07:00
test_offline_mode.py feat(proxy): add LITELLM_OFFLINE_MODE, one switch for the fetches LiteLLM makes on its own (#48096) 2026-10-23 16:54:19 -07:00
test_optional_imports.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_path_utils.py refactor(core): move path_utils from proxy common_utils to litellm_core_utils (#45775) 2026-10-10 19:07:22 +00:00
test_private_json.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_affinity.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_specific_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_ptu_pricing.py feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
test_realtime_errors.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_realtime_streaming.py fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
test_redact_messages.py fix(vertex_ai,azure_ai)!: auto-add the compaction, thinking display, and tool change betas on Claude routes (#44747) 2026-10-30 09:28:19 -07:00
test_request_timeout_resolver.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_retry_after_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_divide_seconds.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_json_dumps.py fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
test_sensitive_data_masker.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_sentry_scrubbing.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_served_output_texts.py fix(guardrails): scan and rewrite streamed /v1/completions under post_call guardrails (#49035) 2026-10-29 08:46:49 -07:00
test_stream_usage_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_streaming_chunk_builder_cursor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_server_tool_use.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_utils.py fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
test_streaming_handler.py fix(streaming): convert upstream logprobs so streams with logprobs end with their finish chunk (#33549) 2026-10-30 04:41:02 -07:00
test_streaming_overhead.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_text_completion_conversion.py refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
test_thread_pool_executor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter.py fix(count_tokens): count a history that carries a compaction block (#44784) 2026-10-30 00:11:55 -07:00
test_token_counter_tool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter_tool_data.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_tokenizer.py fix(token_counter)!: keep a loaded Hugging Face tokenizer when a slower download fails (#48916) 2026-10-29 03:50:58 -07:00
test_tool_fields.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_tool_search_spend_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_url_utils.py fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
test_xai_oauth_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00