Access is confirmed on the Anthropic account, and all 11 cells pass live, so the
xfail marker would only hide real regressions.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds claude-fable-5-1 cost map entries on the Anthropic API, Bedrock converse
(base, global, and us/eu geo inference profiles at the 10% regional premium),
Vertex AI, and Azure AI. Specs match Fable 5 (1M context, 128K output, $10/$50
per MTok, adaptive thinking always on, xhigh and max effort), except cache reads
land at $0.25 per MTok, a quarter of Fable 5's price and 0.025x base input
instead of the usual 0.1x.
Registers anthropic.claude-fable-5-1 in BEDROCK_CONVERSE_MODELS, lists the model
in the setup wizard, and extends the reasoning effort e2e grid. The partner cells
carry fail_reason markers until access on the CI accounts is confirmed.
Partner entries deliberately carry no deprecation_date: Anthropic publishes
retirement no sooner than 2027-09-01 for the first-party model, and the Foundry
and Vertex dates are not published yet.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Resolves budget-ratchet conflicts by taking staging's tighter limits and reworks the embedding raw-response helpers so the branch stays net-negative on the LIT001/LIT002 ceilings staging lowered: the request methods now return the LegacyAPIResponse and each caller keeps a single dict(headers) conversion.
The omitted-format path deliberately no longer dispatches through
embeddings.create, so four legacy tests now intercept at the transport or
client.post instead. Also adds a bypass error-path unit test, rewords a stale
comment and a README scope note, and ratchets the lint budgets down.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
A test that asserts on the error inside its own except block passes when the
call stops raising, because nothing runs the handler. That is the exact case
the test exists to catch, so the regression lands green.
Rewrites all 111 such blocks into pytest.raises, which fails when the call
succeeds, and selects PT017 in ruff-tests.toml so no new one lands.
A name bound twice keeps only the second binding. In `tests/` that is nearly
always a repeated import, harmless but misleading, and the same rule is what
catches the cases that are not harmless: a local that shadows an import the
module still calls, and a second `def test_x` that quietly replaces the first.
311 of the 344 sites were repeated imports and came out with ruff's own fix.
The remaining 33 needed a decision. Four modules imported a name they never
used because a local definition below already shadowed it. Two comprehensions
bound `call` over `unittest.mock.call`, which those modules import and use.
One test rebound the two module handles its nested reload closure had captured.
One class attribute shadowed an unused `status` import.
The load-test fixtures move to a conftest, which is how pytest is meant to share
them, so the test module no longer imports three fixture names it never calls.
The nine `prisma_client` parameters keep a narrow `noqa`: pytest resolves that
fixture by name before the body runs, so the parameter never shadows anything.
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
Add sync and async regression tests for the BaseLLMHTTPHandler streaming
path, which forwards provider response headers for the ~30 providers that
ride the generic handler and had no coverage. Also drop redundant setup
prose from the moonshot invoke test docstring.
* test: enforce PT012 so a pytest.raises block cannot hide dead assertions
`with pytest.raises(...)` stops at the first statement that raises. Anything
sequenced after it inside the block never runs, so an assertion written there is
never checked and the test still reports green.
Two sites were doing exactly that, and both assertions turned out to be wrong
once they started running. tests/llm_translation/test_prompt_factory.py asserted
the bedrock rejection names "requires at least one non-system message", which
holds. tests/proxy_unit_tests/test_proxy_server.py asserted the prisma startup
failure mentions "httpx.ConnectError", which never appears: the failure is an
httpx.ConnectError whose message is "All connection attempts failed", so that
test now asserts the type. Its DATABASE_URL override moves to monkeypatch, since
the old restore sat below the assertion and leaked the invalid URL into every
later DB test the moment the assertion started being able to fail.
The remaining 72 sites are rewritten without changing what they exercise: setup
that cannot raise moves above the block, a nested `patch` moves outside it, and
bodies with real control flow (a stream drain, an if/else on sync_mode, a
retry loop) move into a local closure the block calls.
Fixing PT012 unmasked two B017s, since ruff only reports a blind
pytest.raises(Exception) once the block holds a single statement.
tests/proxy_unit_tests/test_auth_checks.py narrows to the ProxyException
can_key_call_model actually raises. tests/local_testing/test_completion_cost.py
was asserting vertex_ai/medlm-medium has no cost entry, which stopped being true
at some point; that dead first half is gone and the rest of the test, which
checks medlm pricing resolves above zero, now runs instead of being skipped.
* chore(ci): ratchet TQ004 to 768 after the prisma test moved to monkeypatch
Every live together_ai call in CI has answered 503 Service unavailable since
2026-08-20, across two runs 2.5 hours apart, while Together's status page
reported no incident in either window. These are real calls, not replayed
cassettes: the VCR layer runs filter_non_2xx_response, so a 503 is never
written to a cassette and cannot be replayed back.
Qwen/Qwen2.5-7B-Instruct-Turbo does not appear anywhere on Together's monitored
component list, whose Qwen entries are all Qwen3.x, so a model-level outage
there would never surface as an incident. The same 503 already forced
test_basic_rerank_together_ai to be skipped on a different together_ai model,
so per-model 503s are an established failure mode here rather than a platform
outage.
openai/gpt-oss-20b is the cheapest together_ai entry that carries real pricing
and the capabilities these suites exercise, at $0.05/$0.20 per 1M tokens with
function calling, response schema and tool choice. Together monitors it as a
served component. The retired model also carries null pricing in the cost map,
which is its own liability now that unpriced models are blocked.
test_multiple_deployments.py keeps the old id: it is a router fallback list
that is green today, and busting its cassette to prove a point would trade a
passing test for a live call this change cannot vouch for.
a369cb0da7 made completion() hand the prefixed model back to responses(), so
that responses() running get_llm_provider() a second time becomes a no-op
instead of stripping a prefix the model id owns. That was deliberate, and it
shipped with its own unit test, but it left two older assertions behind still
expecting the bare id.
#37744 corrected the openai one in test_openai.py. This is its azure sibling,
which llm_translation_testing has been failing on ever since.
Only the expected value moves. The neighbouring custom_llm_provider assertion
already passes and stays as it is.
* test(lint): ban blind pytest.raises(Exception) with ruff B017
A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.
All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.
Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.
* fix(test): widen the openai batch-dispatch assertion to OpenAIError
The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
a369cb0da7 made the chat-to-responses bridge return the routing prefix on the
model it passes to responses(), so responses() re-resolving the provider is a
no-op instead of stripping a second prefix. It updated the bridge's own unit
tests but not this one, which still asserted the stripped id and has been
failing llm_translation_testing since that change landed.
The provider still receives gpt-5.4: responses() strips the openai/ prefix on
its own resolve, one layer later than this assertion used to sit. The stale
comment claiming the prefix is stripped before routing goes with it.
* test: point the live web search, groq and vertex image suites at models that still exist
Three CircleCI jobs on the staging-to-main promotion are red because the models
their live suites call have been retired by the providers, not because anything
in litellm changed.
openai/gpt-4o-search-preview now answers "has been deprecated" (its dated id
gpt-4o-search-preview-2025-03-11 carries deprecation_date 2026-07-23), so the
two web search conformance tests and the web search cost tracking test move to
gpt-5-search-api, the current search model. It keeps mode chat,
supports_web_search and a search_context_cost_per_query map, so the cost
assertion still resolves.
groq/llama-3.1-8b-instant reached its deprecation_date of 2026-08-16 and Groq
answers "does not exist or you do not have access to it". It follows
groq/llama-3.3-70b-versatile to groq/openai/gpt-oss-120b, the same replacement
PR #37422 already picked. The proxy config that job boots routes on a */*
wildcard, so no config change is needed.
vertex_ai/imagen-3.0-fast-generate-001 404s with "was not found or your project
does not have access to it". Google retired the whole Imagen family across
Vertex and the Gemini API, so there is no Imagen id left to point at. The class
is removed rather than repointed: Vertex image generation is already covered
live by TestVertexAIGeminiImageGeneration on vertex_ai/gemini-2.5-flash-image,
and the Imagen request and response transformations keep their offline coverage
in tests/test_litellm/llms/vertex_ai/image_generation/.
Only live call sites move. Remaining references to the old ids sit in offline
cost-map and transformation tests, where the string is a lookup key and no
request leaves the process.
* chore(lint): ratchet the TQ005 ceiling down to the count this branch reached
Removing the retired TestVertexImageGeneration class cleared one TQ005
violation, so the gate demands the limit come down with it.
make lint-budget-update only lowers a limit by the delta a branch cleared, and
this ceiling already sat 2 above the base count, so the tool landed on 2834
while the gate wants the limit at or below the 2832 this branch reached. The
remaining 2 are that stale headroom, which is exactly what the gate is asking
to reclaim.
`assert False` inside a `try:` raises AssertionError, which the `except
Exception` right below it catches, so several tests reported green no matter
what the code did. `pytest.fail` raises Failed, a BaseException, and escapes.
A bare `a == b` statement is evaluated and discarded. Nine of those sat in
tests, and one was comparing against a model name the router never produces.
Selects B011, B015, B018, PT015, PLR0133 and PLW0127 in ruff-tests.toml
alongside F821, with all 50 existing violations fixed, so no budget file or
ratchet is needed. CI already runs this config over tests/.
Python binds a name once per scope, so when a module or class defines the same
test twice only the last one exists. The earlier definitions are unreachable:
pytest never collects them, and nothing that references them can fail.
A sweep in August cleared nine of these. Five have appeared since, which is the
argument for a rule rather than another sweep.
Each survivor is the better version, so nothing is lost. The two SQS logger
twins additionally stub `asyncio.create_task`, which the shadowed copies did
not. The cost-calculator duplicate is a two-line stub that also takes a
`model_item` parameter no fixture supplies, so it could not have run even
unshadowed. The two `test_prompt_caching` bodies are both `pass`.
Collecting the four files reports 416 tests before and after.
`tests/proxy_unit_tests/conftest copy.py` goes with them. pytest only loads a
file named exactly `conftest.py`, nothing imports this one, and the space in the
name says what it was.
gemini/imagen-4.0-generate-001 now 404s with "no longer available, please
update your code to use models/gemini-3.1-flash-image", and
groq/llama-3.3-70b-versatile reached its deprecation_date of 2026-08-16, which
is the day the llm_translation and image_gen jobs went red.
Only the two live call sites move. The remaining references to the old ids sit
in offline transformation tests, where the string is just a routing key and no
request leaves the process.
Python keeps only the last binding for a name, so when a file defines the same
test twice the earlier one is unreachable. pytest cannot collect a function that
no longer exists, so nothing reports it and the file still looks like it covers
the scenario.
These ten are cases where the two definitions have different bodies, meaning a
real test was replaced rather than duplicated. Each is renamed to say what it
actually covers, which makes it reachable again:
- test_gemini_frequency_penalty: the dead copy checks the parameter is listed in
get_supported_openai_params for vertex_ai; the survivor checks get_optional_params
maps a value for gemini. Different function and different provider.
- test_async_log_success_event_adds_to_queue and the failure variant: the dead
copies run without mocking asyncio.create_task, so they exercise the real task
path the survivors mock out.
- test_async_send_batch_triggers_tasks: the dead copy asserts send is not awaited
directly; the survivor asserts create_task was called.
- test_model_id_in_required_metrics: the dead copy checks the model_id label on
twelve further metrics the survivor dropped.
- test_anthropic_messages_pt_file_block_preserves_cache_control: the dead copy
passes model and llm_provider explicitly and uses real base64 PDF content.
- test_translate_streaming_openai_chunk_to_anthropic_with_thinking: the dead copy
covers thinking_delta; the survivor covers signature_delta.
- test_client_initialization and test_client_without_api_key: the dead copies
assert the resource clients are wired with the right base URL and key; the
survivors only construct the object.
- test_client_initialization_strips_trailing_slash: the dead copy constructs
ModelsManagementClient directly rather than going through Client.
Verification: collecting the seven touched files gives 401 node IDs before and
411 after, the ten new names and nothing else, with nothing lost. All ten pass.
Running the touched files in full gives 299 passed, and test_optional_params.py
goes from 111 passed to 112.
Two further shadowed definitions were left alone rather than renamed: the dead
copies of test_prompt_caching and test_cost_calculator_with_base_model_with_router
have no assertions at all, one being a bare pass and the other a lone import, so
restoring them would add tests that cannot fail.
Three groups, all verified by running the suite rather than by inspection.
18 files whose every test function carries an unconditional @pytest.mark.skip,
39 test functions in total. They are collected on every CI run and always skip,
so they advertise coverage the suite does not have. Reasons on the marks include
"AWS Suspended Account", "lakera deprecated their v1 endpoint" and "moved to
using 'otel' for logging"; 26 of the marks predate 2025.
30 test functions with a byte-identical body and identical decorators to a
sibling in the same file and class, differing only in name. Deleting one of each
pair removes no coverage. Four further candidates were excluded because they
override an inherited test, where deleting the override un-shadows the base
class implementation instead of removing a duplicate.
9 test functions that a later definition of the same name shadows, so Python
never binds them and pytest cannot collect them.
One file that is a demo script rather than a test; its own docstring says to run
it with python.
Verification: collecting the 26 edited files gives 2,492 node IDs before and
2,462 after. The 30 duplicate deletions account for exactly 30 removals, the 9
shadowed deletions account for 0 (confirming at runtime that they were never
collectable), nothing unexplained disappeared, and nothing new appeared. No
other test or module imports any deleted symbol.
AWS no longer serves `anthropic.claude-3-sonnet-20240229-v1:0`. The streaming
path returns a plain 404, "Model with the provided id
anthropic.claude-3-sonnet-20240229-v1:0 is not found", and the non-streaming
path answers 500 for the same reason. Our own cost map has carried a
2026-07-30 deprecation date for it since #36538
That accounts for 20 failures across local_testing_part1, local_testing_part2
and llm_translation_testing. litellm maps both statuses correctly, so the
tests are what went stale, not the client
Replacement is `us.anthropic.claude-sonnet-4-5-20250929-v1:0`: a like-for-like
Sonnet, and the newest Bedrock Sonnet this repo exercises against the real API
in tests/e2e. Newer ids exist in the cost map, but nothing in the repo calls
them live, so picking one would be an unverified guess about model access on
the CI account
Scope is limited to the tests that actually issue a request. The occurrences
that assert on the model string itself, or that feed mocked transformations,
keep the old id so their assertions stay meaningful
* test(reasoning-effort-grid): bump cell-count assertion for claude-opus-5
The claude-opus-5 grid entry added in ae81625ee6 raised the Anthropic direct
route to 31 model combos, but test_grid_cell_count still expected 30, so the
suite went red on the tripwire rather than on any behavior change.
* test(openai): swap the retired deep-research model out of the bridge test
OpenAI shut down o3-deep-research and o4-mini-deep-research on 2026-07-23, so
the live call in this test now comes back as a 400 'Model not found'. The test
was never about deep research specifically; the bridge fires on any model whose
cost-map mode is "responses", so it now uses gpt-5.5-pro, the newest
responses-only OpenAI model, and is renamed to say that.
gpt-5.5-pro was confirmed present on the CI account with an authenticated
GET /v1/models before being picked.
Registers claude-opus-5 across the cost maps and provider lists so the model
prices, reports its real 1M/128K limits, and advertises its capabilities instead
of falling through the generalization patterns at zero cost.
Adds the first-party entry plus the Bedrock (base, global, us, eu, au, jp),
Vertex AI, and Azure AI variants. Pricing matches Opus 4.8 at $5/$25 per MTok
with the usual 1.1x regional premium on the cross-region inference profiles, and
fast mode is priced at 2x through provider_specific_entry on the first-party
entry only.
Two fields deliberately differ from Opus 4.8: prompt_cache_min_tokens drops to
512, and bedrock_output_config_effort_ceiling is omitted because Bedrock accepts
output_config.effort="max" for Opus 5.
* fix(proxy): restore atomic user upsert when adding team members
Parallel /team/new calls naming the same not-yet-existing member were
returning 500 "Unique constraint failed on the fields: (`user_id`)".
The upsert in add_new_member passed an empty update branch. Prisma only
compiles an upsert down to a single INSERT ... ON CONFLICT when that branch
writes something; with an empty one it emits SELECT-then-INSERT instead, so
concurrent requests all read "no such user" and all insert. Postgres
statement logs confirm it: the empty form logs BEGIN/SELECT/INSERT/COMMIT,
the non-empty form logs INSERT ... ON CONFLICT ("user_id") DO UPDATE SET.
Re-state user_id in the update branch as a no-op so the native upsert path
comes back. The teams append stays in the filtered update below it, so an
already-existing member still cannot pick up a duplicate team id.
tests/test_team.py::test_team_new failed 9 of 15 runs against a live proxy
before this and 0 of 15 after. The existing unit test asserted only that
upsert had been called on a mock, so it passed either way; it now pins the
shape of both branches and fails when the update branch goes back to empty.
* test: point the live codex tests at gpt-5.3-codex
OpenAI deprecated gpt-5.2-codex, so test_openai_codex and
test_openai_codex_stream started failing against the live API with
model_not_found. gpt-5.3-codex is the current codex model; both tests pass
on it. The remaining gpt-5.2-codex references in the suite are mocked
transformation tests and are unaffected.
* test(e2e): update models page specs for the shared DataTable
The DataTable migration in #34363 changed three things the models page
specs were pinned to, and five tests went red.
Row click no longer opens the detail view; the Model ID cell owns that
now, so both specs click its `model-id-<id>` test id instead of the row.
The search box placeholder switched from an ASCII "..." to a real
ellipsis, so the specs use getByPlaceholder with a substring instead of
an exact attribute match that punctuation can break again. The results
count moved from `models-results-count` ("Showing 1 - 50 of 137 results")
to the shared pagination's `pagination-range` ("Showing 1-50 of 137").
The Team-BYOK test also filtered rows on the team alias, which the Team
ID column has never rendered in either the old or the new table; it
filters on the team id now, which is what the column actually shows and
what the assertion's own comment intends.
Verified against a local proxy serving a fresh build with the seeded
e2e postgres and mock upstream: all five failing tests pass, and the
full suite is 82 passed / 4 skipped at CI parity (workers=1).
* test(realtime): record and replay websocket traffic in redis vcr cassettes
* style(realtime): ruff-format ws-vcr harness
* fix(realtime): warn instead of silently disabling ws-vcr when the redis client cannot be built
test_text_message_blocked_by_guardrail_no_ai_response classified the
model's reply against a safe_markers keyword list to decide whether the
guardrail had blocked the message. gpt-realtime words its refusal of the
guardrail's "say exactly" voice prompt nondeterministically, so any new
phrasing outside the list turned CI red on unrelated PRs; the list had
already been extended in #28191, #28200 and #29477, and drifted again to
"Sorry, I can't comply with that request" (11 of the 13 failed
realtime_translation_testing runs since 2026-06-24, e.g. CircleCI job
2009316 on #32380).
Record every frame the proxy sends to the backend through a
RecordingBackendWebSocket wrapper and assert the invariant the product
actually guarantees: the blocked phrase never reaches OpenAI, only the
guardrail's own conversation.item.create and response.create are
forwarded (the client's reflexive response.create is dropped), and the
blocked phrase never appears in AI output. Replace the fixed
0.3s/3.0s sleeps with an event-driven wait for response.done; client
frames are processed sequentially so no inter-message sleep is needed.
Verified by mutation: disabling the response.create drop fails the
response.create count assertion, and disabling the guardrail fails the
guardrail_violation assertion.
* feat(guardrails/headroom): add CCR (compress-cache-retrieve) support via agentic loop
When Headroom's /v1/compress returns messages containing hash markers
(hash=[a-f0-9]{24}), inject a headroom_retrieve tool into the request.
When the LLM calls that tool, intercept via async_should_run_agentic_loop
and async_build_agentic_loop_plan, call GET /v1/retrieve/{hash} on the
Headroom sidecar, and replay the LLM with the original content as a tool
result -- all transparent to the caller.
* style: run ruff format on headroom guardrail and tests
* fix(guardrails/headroom): detect headroom_retrieve calls in both OpenAI and Anthropic response formats
* test(guardrails/headroom): add test for Anthropic content block format detection in CCR loop
* ci: trigger CI checks
* fix(guardrails/headroom): replace List/Dict with list/dict to fix UP006 ruff violations
* fix(guardrails/headroom): replace except Exception with except ValueError to fix BLE001
* fix(guardrails/headroom): add Responses API output format detection for CCR tool calls
* refactor(guardrails/headroom): extract format-specific helpers to fix C901 complexity
* fix(guardrails/headroom): scope CCR retrieval to hashes produced by current request
Previously any LLM-supplied hash in a headroom_retrieve tool call was
forwarded to the Headroom retrieve API, letting a crafted tool call
fetch arbitrary cached content. Validate the hash against the set
produced by compressing the current request's messages before calling
retrieve.
* fix(guardrails/headroom): track issued hashes server-side, fix Responses API replay shape
Hash validation now also checks an in-memory cache of hashes actually
returned by /v1/compress, not just whether the hash text appears
somewhere in the request's messages. The message-text check alone is
forgeable: an attacker can plant a hash-shaped string in their own
prompt and have it treated as valid.
Responses API follow-up now emits function_call/function_call_output
items keyed by call_id instead of chat-style assistant/tool messages,
since the Responses API does not accept the latter as input. Also
fixes call_id/id field priority when extracting tool calls from
Responses API output, since call_id (not id) is what must match
between the function_call and its output.
* fix(guardrails/headroom): drop redundant quoted type annotations
UP037 flags quotes on annotations that are already lazily evaluated
via `from __future__ import annotations`.
* test(guardrails/headroom): add missing pytest.mark.asyncio decorators
Functional under asyncio_mode=auto, but every other async test in the
file has the decorator for consistency.
* fix(guardrails/headroom): scope CCR hashes per call_id, fix Anthropic replay shape
Two real gaps found in review:
1. The instance-wide issued-hash cache combined with a message-text
check did not actually scope retrieval to the request that produced
the hash. A hash issued for request A stays in the shared cache
until TTL expiry, and the message-text check is satisfied by any
request whose own messages happen to echo that hash string. Request
B could plant A's hash in its own prompt and retrieve A's content.
Fixed by keying the issued-hash cache by litellm_call_id, matching
the pattern already used in compression_interception: a hash is
only honored when it was issued under the exact call_id resolving
for the current request.
2. The Anthropic Messages replay path fell through to the chat-style
assistant/tool-message builder, which Anthropic does not accept.
Anthropic requires the tool_use block echoed in an assistant message
paired with a tool_result block in a user message, keyed by
tool_use_id. Added a dedicated branch for this shape.
* docs: note proactive API-fragmentation helper convention
Add a bullet to the coding-conventions list: look for or add a shared
helper when logic branches on API surface (chat completions vs
Anthropic Messages vs Responses API), instead of duplicating
format-detection per module.
* fix(guardrails/headroom): fix Anthropic tool-shape detection, extract shared cross-API tool util
Live e2e testing against the real Anthropic API surfaced two bugs the
mocked unit tests couldn't catch because they used MagicMock responses
instead of realistic response shapes:
1. has_headroom_retrieve_tool only recognized OpenAI-shaped function
tools. By the time an Anthropic Messages response reaches the
agentic-loop gate, the tool this guardrail injected has already been
transformed into Anthropic's native shape (type: "custom", top-level
"name"), so the gate never fired for real Anthropic requests.
2. AnthropicMessagesResponse is a TypedDict, so real responses are
plain dicts at runtime, not objects with attribute access. The
extractors and format detectors used bare getattr(), which silently
returns nothing for dict responses instead of reading the actual
key.
Extracted the cross-API-surface tool-call extraction and tool-presence
check into litellm/litellm_core_utils/prompt_templates/factory.py
(get_tool_calls_from_response, has_tool_with_name) so this format
fragmentation is handled in one place instead of being duplicated
per-guardrail, and reused the existing repair-aware
parse_tool_call_arguments from common_utils instead of a naive
json.loads. headroom.py now delegates to these shared helpers.
Confirmed live against the real Anthropic API: the retrieve loop now
fires and successfully retrieves the correct hash's content through
the full compress -> tool-call -> retrieve -> replay round-trip.
* fix(guardrails/headroom): fix ruff-strict UP006/I001 budget violations
Use lowercase list/dict generics in the new factory.py tool-call
helpers instead of typing.List/Dict, drop the now-unused Tuple import
in headroom.py, and reorder the new factory import ahead of the
llms.custom_httpx import to satisfy import sorting.
* fix(guardrails/headroom): match Anthropic tools without a type field
Anthropic's documented client tool format is just name + input_schema;
type: "custom" is only one possible value, not a requirement. Match
any non-OpenAI-shaped tool on its top-level name instead of requiring
type == "custom".
Register claude-sonnet-5 across the Anthropic, Bedrock (base + global/us/eu/au/jp
cross-region inference profiles), Vertex AI, and Azure AI cost-map entries in both
the root and bundled-backup model maps, plus BEDROCK_CONVERSE_MODELS and the
setup-wizard provider list.
Sonnet 5 ships with the gen-5 adaptive-thinking profile (adaptive thinking always
on, no extended thinking, effort defaults to high), so the entries mirror the
Fable 5 / Opus 4.8 sampling-param and prefill restrictions rather than the older
Sonnet 4.6 behavior: supports_sampling_params and supports_assistant_prefill are
false while supports_adaptive_thinking, supports_xhigh_reasoning_effort, and
supports_max_reasoning_effort are true. Pricing follows standard Sonnet rates
($3 / $15 per MTok) with the 10% regional premium on the us/eu/au/jp profiles.
Add a reasoning-effort grid entry for the Anthropic direct route and a regression
test pinning pricing, capabilities, regional premiums, backup parity, and bare-name
provider resolution.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke
* style(bedrock): use builtin generics in new Invoke helpers to clear UP006 gate
* fix(bedrock): honor explicit thinking budget_tokens=0 in clear_thinking conversion
The clear_thinking_20251015 -> adaptive conversion resolved the thinking
budget with `thinking.get("budget_tokens") or BEDROCK_MIN_THINKING_BUDGET_TOKENS`,
which treats a caller-supplied `budget_tokens=0` as missing and silently
substitutes the Bedrock minimum. Resolve the budget with an explicit
`is not None` check so an explicit 0 is honored.
* fix(bedrock): gate Fable 5 into clear_thinking adaptive injection on Invoke
_ensure_thinking_for_clear_thinking_context_management returns early when
_supports_extended_thinking_on_bedrock(model) is False, so the adaptive-thinking
injection never runs for models absent from that gate. Opus 4.8 slips through on
the incidental "opus-4" substring, but Fable 5 had no matching pattern, so a
clear_thinking_20251015 request on Fable 5 reached Bedrock with an unsupported
context-management edit and no thinking field; the exact 400 this path exists to
prevent. Add the fable-5 patterns to the gate so Fable 5 (mapped ids and unmapped
aliases) gets thinking.type=adaptive + output_config.effort like the other
adaptive models.
Extend the adaptive-injection regression test to cover Fable 5 (a mapped id and
an unmapped alias) so it fails without the gate entry, and add focused coverage
for the budget->effort tiers, the disabled/enabled/adaptive thinking branches,
output_config.effort preservation, and list/dict system-role normalization.
Also normalize the Invoke transformation module and its test to line-length 88
so ruff format --check (CI format-check) passes.
* refactor(anthropic): make supports_adaptive_thinking flag authoritative for thinking detection
Replace the per-version name helpers (_is_claude_4_6/4_7/4_8_model,
_is_claude_fable_5_model) with cost-map-flag-first detection. _is_adaptive_thinking_model
now reads supports_adaptive_thinking from the model cost map and falls back to a single
generalized family-version regex (_claude_version_at_least(model, 4, 6)) only when a model
is unmapped, instead of hard-coding each new Claude release.
Wire supports_adaptive_thinking through ProviderSpecificModelInfo and ModelInfo so the cost
map flag actually surfaces at lookup time. Reroute the Bedrock Invoke extended-thinking gate
and the two anthropic/chat/transformation.py call sites through _is_adaptive_thinking_model.
Known gap left to the fallback_generalizations work (#29718): unmapped Fable 5 aliases have
no parseable minor version, so they defer to the cost map and are not detected until a mapped
entry or a generalization rule exists. Covered by an explicit regression test.
* refactor(anthropic): drop name-based version fallback; resolve adaptive thinking from cost map only
The prior commit kept a regex (_claude_version_at_least) as a fallback when an id
resolved to no cost-map entry. Remove it: _is_adaptive_thinking_model now reads
supports_adaptive_thinking and nothing else, so "which Claude versions think
adaptively" lives entirely in the model cost map, and a new adaptive release is a
JSON edit rather than a Python edit.
To keep the flag authoritative across the id forms the Bedrock Invoke and anthropic
paths actually see, backfill supports_adaptive_thinking=true on every adaptive Claude
entry that was missing it (Opus 4.6/4.7 and Sonnet 4.6 across region/provider aliases)
in both the root and bundled cost maps, and generalize _model_map_lookup_candidates to
normalize an id to its base cost-map key: strip a Bedrock version suffix (-v1:0 fully,
or just the :0 inference-profile minor so the -v1-keyed 4.6 entries resolve), strip a
dated-release suffix (-20260219), and rewrite a dotted family version (4.6 -> 4-6).
This is id normalization feeding the lookup, not capability-by-name.
Tests load the PR-local cost map (the flags are not on main until merge) and cover each
normalization path plus the unmapped-alias deferral to fallback_generalizations (#29718).
* refactor(reasoning_effort): single-source effort<->thinking-budget mappings
Route every reasoning_effort <-> thinking-budget conversion through the DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET constants so the numbers stay in sync across providers. The five constants are now 2000/5000/10000/20000/40000
Add reasoning_effort_from_thinking_budget() in litellm_core_utils/reasoning_effort_utils.py and route the three OpenAI-style forward maps (anthropic adapters, responses adapters, hosted_vllm) through it. The bedrock invoke and experimental messages adaptive maps now reference the constants directly; the only behavior change is the xhigh threshold moving from 24000 to 20000. Reverse maps and the cross-provider test grid read the same constants
* test(reasoning_effort): lift budget-mode max_tokens above the new high budget
The single-sourced DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET thresholds moved
high from 4096 to 10000. The live reasoning_effort grid sends budget-mode
requests with max_tokens=8192, so reasoning_effort=high now produces
budget_tokens=10000 > max_tokens and every provider returns 'max_tokens must be
greater than thinking.budget_tokens'. Derive a shared BUDGET_MODE_MAX_TOKENS
(2x the high budget) for the spec and the request builder so the ceiling always
clears the largest 200-expected tier. Also resolve the inherited base
test_reasoning_effort assertion off the same high-budget constant instead of the
stale 4096 literal so it tracks the source of truth.
* fix(reasoning_effort): keep effort<->budget thresholds at pre-PR values
The single-sourcing refactor moved the shared effort<->budget thresholds up
(low 1024->2000, medium 2048->5000, high 4096->10000, xhigh 8192->20000,
max 16384->40000). That silently changes the effort->budget direction: a caller
who sets reasoning_effort together with a max_tokens that used to sit above the
old per-tier budget but below the new one now trips the provider's
"max_tokens must be greater than thinking.budget_tokens" 400. It spans every
backend that derives a budget from an effort (Anthropic, Gemini/Vertex,
hosted vLLM), not just Bedrock.
Restore the constants to their pre-PR values while keeping every backend reading
from the shared DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET constants, so the
mapping stays single-sourced without the behavior change. Tests that pinned the
raised thresholds now derive their boundaries from the same constants.
* test(reasoning_effort): derive high effort->budget assertions from the shared constant
The cross-provider translation tests pinned reasoning_effort="high" to a literal
budget_tokens=10000, the raised value. Point them at
DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET so they track the single source
instead of a magic number.
* fix(anthropic): resolve adaptive flag for combined dated+versioned Bedrock ids
The model-map candidate normalization applied each suffix strip independently to
the original id, so the real Bedrock shape "<base>-<YYYYMMDD>-v1:0" never reduced
to its base cost-map key: stripping the version left the date, and the
dated-suffix regex is anchored to the end so it could not fire while the version
was still present. An adaptive Claude model invoked by its full dated+versioned
id (e.g. us.anthropic.claude-sonnet-4-6-20251101-v1:0) therefore resolved to
supports_adaptive_thinking=null and was treated as non-adaptive, reaching Bedrock
with the rejected thinking.type=enabled shape, the exact 400 this path prevents.
Add a composed normalization that rewrites the dotted family version, then peels
the -vN:rev version suffix, then the -YYYYMMDD dated suffix, so the combined form
resolves to its base key. Regression tests pin the combined suffix on sonnet-4-6
and opus-4-8 across provider/region prefixes.
* fix(reasoning_effort): align budget<->effort tests with reverted constants and format common_utils
The constant revert restored the effort<->budget thresholds to their pre-PR
values (1024/2048/4096/8192/16384) and single-sourced the reverse
budget->effort ladder through reasoning_effort_from_thinking_budget, but
several tests still pinned the briefly-raised values and the old hardcoded
reverse buckets, so the "All Other Providers" shard failed
Derive the anthropic chat effort->budget assertions from the shared
DEFAULT_REASONING_EFFORT_*_THINKING_BUDGET constants, and update the
experimental pass-through and responses adapter expectations to the
single-sourced reverse ladder (budget 1024 -> low, 5000 -> high)
Also run ruff format --line-length 88 over anthropic/common_utils.py so the
CI format-check, which checks the whole changed file, passes
