AmazonConverseConfig._transform_usage only read the aggregate
cacheWriteInputTokens field, so cache_creation_token_details was always
unset for Bedrock Converse responses. calculate_cache_writing_cost bills
the whole cache-write count at the 5m rate whenever that field is None,
so 1-hour TTL cache writes on the standard Bedrock chat path were always
undercounted, even though Bedrock returns the 5m/1h split in
usage.cacheDetails.
Parse cacheDetails (when present) into CacheCreationTokenDetails so the
correct rate applies to each portion. No cacheDetails in the response
(older models/regions) keeps the previous behavior.
Fixes#36760
Co-Authored-By: pi (Claude/GPT via @earendil-works/pi-coding-agent) <noreply@earendil.works>
Python keeps only the last binding for a name, so when a file defines the same
test twice the earlier one is unreachable. pytest cannot collect a function that
no longer exists, so nothing reports it and the file still looks like it covers
the scenario.
These ten are cases where the two definitions have different bodies, meaning a
real test was replaced rather than duplicated. Each is renamed to say what it
actually covers, which makes it reachable again:
- test_gemini_frequency_penalty: the dead copy checks the parameter is listed in
get_supported_openai_params for vertex_ai; the survivor checks get_optional_params
maps a value for gemini. Different function and different provider.
- test_async_log_success_event_adds_to_queue and the failure variant: the dead
copies run without mocking asyncio.create_task, so they exercise the real task
path the survivors mock out.
- test_async_send_batch_triggers_tasks: the dead copy asserts send is not awaited
directly; the survivor asserts create_task was called.
- test_model_id_in_required_metrics: the dead copy checks the model_id label on
twelve further metrics the survivor dropped.
- test_anthropic_messages_pt_file_block_preserves_cache_control: the dead copy
passes model and llm_provider explicitly and uses real base64 PDF content.
- test_translate_streaming_openai_chunk_to_anthropic_with_thinking: the dead copy
covers thinking_delta; the survivor covers signature_delta.
- test_client_initialization and test_client_without_api_key: the dead copies
assert the resource clients are wired with the right base URL and key; the
survivors only construct the object.
- test_client_initialization_strips_trailing_slash: the dead copy constructs
ModelsManagementClient directly rather than going through Client.
Verification: collecting the seven touched files gives 401 node IDs before and
411 after, the ten new names and nothing else, with nothing lost. All ten pass.
Running the touched files in full gives 299 passed, and test_optional_params.py
goes from 111 passed to 112.
Two further shadowed definitions were left alone rather than renamed: the dead
copies of test_prompt_caching and test_cost_calculator_with_base_model_with_router
have no assertions at all, one being a bare pass and the other a lone import, so
restoring them would add tests that cannot fail.
Three groups, all verified by running the suite rather than by inspection.
18 files whose every test function carries an unconditional @pytest.mark.skip,
39 test functions in total. They are collected on every CI run and always skip,
so they advertise coverage the suite does not have. Reasons on the marks include
"AWS Suspended Account", "lakera deprecated their v1 endpoint" and "moved to
using 'otel' for logging"; 26 of the marks predate 2025.
30 test functions with a byte-identical body and identical decorators to a
sibling in the same file and class, differing only in name. Deleting one of each
pair removes no coverage. Four further candidates were excluded because they
override an inherited test, where deleting the override un-shadows the base
class implementation instead of removing a duplicate.
9 test functions that a later definition of the same name shadows, so Python
never binds them and pytest cannot collect them.
One file that is a demo script rather than a test; its own docstring says to run
it with python.
Verification: collecting the 26 edited files gives 2,492 node IDs before and
2,462 after. The 30 duplicate deletions account for exactly 30 removals, the 9
shadowed deletions account for 0 (confirming at runtime that they were never
collectable), nothing unexplained disappeared, and nothing new appeared. No
other test or module imports any deleted symbol.
Cue grouping previously assumed space-separated Latin-style text. Chinese
and Japanese audio fused entire utterances into one unbreakable word (and
therefore one giant cue) because CJK scripts carry no spaces, and Arabic,
Urdu, Hindi and Armenian sentence terminators never triggered a cue break.
Words now also split at CJK character boundaries with basic kinsoku
handling so punctuation stays attached, the sentence-end set covers
script-specific terminators, and the cue length budget counts East Asian
wide characters as double width so CJK cues match the same two-line
subtitle footprint as Latin text.
The previous cue grouping broke every 15 subword tokens or 5s, which
produced uniform ~3s cues that split mid-word and bridged silence gaps,
so subtitles did not track the actual speech. Cues are now built from
whole words and break on sentence-final punctuation, speaker changes,
silence gaps >= 700ms, a 84-char budget, or a 7s duration cap, with
timestamps taken directly from token timings. Untimestamped translation
tokens are excluded from cues so translated text is never mixed into
original-language subtitles.
Drop the transform overrides that swapped response.usage to the chat
shape, which broke the /v1/responses client contract. Provider extras
like server_side_tool_usage_details already survive validation via
ResponseAPIUsage extra fields, so the shared usage bridge now carries
them onto the bridged chat Usage generically. The web_search_call
output gate also reads dict output items, since items that fail SDK
validation stay plain dicts, and the chat path gains billing tests.
Record supports_tool_search on the Bedrock Claude entries in both cost
map files and have _supports_tool_search_on_bedrock read it first via
the provider-resolved capability lookup, keeping the name patterns as a
fallback for ARNs and ids the map cannot resolve. Threads the flag
through ModelInfoBase and drops a dated remark from the pattern list
Claude Code drives Opus 4.7 with thinking {"type": "adaptive"} plus
output_config {"effort": "max"}. The anthropic-to-openai adapter
forwarded thinking verbatim for Claude models but dropped output_config,
and Bedrock Converse streams zero reasoningContent blocks for adaptive
thinking without an effort tier. Forward the effort subset of
output_config for Bedrock targets, accept it in the converse supported
params, and map it with the model's effort ceiling applied. Re-enable
the skipped e2e compat cell that catches this
Bedrock InvokeModel rejects tool_search_tool_* tool types unless the
request body carries the tool-search-tool-2025-10-19 beta. The model
allowlist gating that beta omitted Haiku 4.5 (and Opus 4.7, supported
since launch per live verification), so every tool-search request on
those models got a Bedrock 400. Add both to the allowlist and re-enable
the e2e compat cell that caught it.
The iterations branch in AnthropicConfig.calculate_usage summed
cache_creation_input_tokens but never aggregated the per-iteration
cache_creation 5m/1h breakdown, leaving cache_creation_token_details
as None. As a result all cache-creation tokens fell back to the flat
5m write rate, underbilling 1h cache writes by up to 2x.
Aggregate the ephemeral_5m/ephemeral_1h split across iterations so 1h
writes are priced at the 1h rate.
Fixes LIT-4868
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): reject Anthropic server-side web_search tool with actionable error
Bedrock's Anthropic Messages endpoints cannot execute Anthropic's
server-side web_search tool, so forwarding it returns an opaque
"The provided request is not valid" 400 from Bedrock. Fail fast in the
invoke transform with an error that names the unsupported tool, the
model, and links the web search interception docs as the fix.
* refactor(bedrock): address review nits on web_search guard typing
Pass static AWS credentials through optional_params so the real
credential-resolution path runs locally instead of patching
BedrockRerankHandler._get_boto_credentials_from_optional_params.
BedrockRerankHandler._prepare_request duplicated ad-hoc SigV4 signing
instead of using BaseAWSLLM.get_request_headers, the helper every other
Bedrock handler (embeddings, converse, invoke, image) already uses.
The duplicate skipped header filtering before signing, so any forwarded
header (e.g. x-forwarded-for) got included in the signed set and could
invalidate the signature if rewritten downstream between signing and
delivery, the same class of bug fixed for the invoke path in #19111.
Adds 'agentcore' as a search provider backed by an AgentCore Gateway MCP web-search target, usable from litellm.search()/`/search` and as a websearch_interception backend. Supports SigV4 (AWS_IAM gateways) and bearer tokens (CUSTOM_JWT gateways) via a new BaseSearchConfig.sign_request hook.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
xAI already converts Responses usage to chat Usage so web_search_calls survive
cost tracking. The chat completions bridge then re-ran the Responses usage
transform and crashed on missing input_tokens. Pass through already-chat Usage
and chat-shaped dumps instead
