A bare string or list under off_peak_pricing in YAML passed the truthy
guard and crashed _is_off_peak with AttributeError, breaking cost
calculation for that deployment. Malformed pieces of the block are
documented to not match rather than error, so guard the block itself the
same way and bill standard rates.
Since the Rust core handoff in #37241, BedrockConverseLLM.completion read
access_key, secret_key and token off the boto3 credentials before asking
the Rust gate whether it wanted the call. On a deployment that only sets
AWS_BEARER_TOKEN_BEDROCK boto3 resolves no credentials, so every Converse
call through /v1/chat/completions and /v1/responses failed with
"'NoneType' object has no attribute 'access_key'", with or without the
Rust opt-in
Bearer auth resolves no SigV4 principal at all, and both the Python and
the Rust path read the bearer token themselves, so only hand the
principal keys down when boto3 actually resolved one
get_request_headers now accepts credentials=None and raises botocore's
NoCredentialsError when neither a bearer token nor a principal exists
instead of handing SigV4Auth a None
Fixes#38579
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFDpYC45u9p8eKd4aBATnS
The router_code_coverage gate only counts calls made from test files with
router in the filename, so the helper needs direct unit tests beside the
other inheritance helpers in test_router_model_cost_isolation.py: fills
missing base rates from the builtin entry, leaves explicit rates alone,
and no-ops without a block or for an unmapped backend model.
Cost lookup selects the deployment-scoped cost map entry only when custom
pricing is detected and the entry carries a base pricing field. A deployment
whose model_info held nothing but off_peak_pricing failed both conditions, so
its schedule was silently ignored and every request billed at the shared
backend rate.
use_custom_pricing_for_model now also treats deployment-scoped pricing fields
in the metadata model_info as custom pricing, and the router inherits the
backend model's built-in base token rates onto such an entry at registration,
which also lets cache pricing inheritance apply. Regression tests cover the
registration, the detection, and the costed request end to end.
register_model inserted the first deployment's off_peak_pricing dict by
reference into the shared backend cost-map entry, and later deployments
sharing that backend merged their schedules into the same object,
corrupting the first deployment's schedule and polluting the built-in
entry. Nested dicts now merge copy-on-write, and off_peak_pricing stays
off the shared backend keys.
This reverts commit 65a46a5f32 (#38113)
That change made two edits that combine into a hard failure for SDK
users. It dropped the null-router guard in
_select_search_tool_from_router, so a missing router now yields an empty
search_tools list instead of returning early, and it turned the no-match
case in _select_search_tool_from_list from a debug-logged fallback into
a raised ValueError. It also added a call site in
async_pre_call_deployment_hook that invokes the selection purely for the
side effect of raising, discarding the return value
Used together, any SDK caller that sets search_tool_name and sends a web
search tool now raises "Configured search tool '<name>' was not found"
on every request. There is no way to satisfy the check off the proxy,
because search_tools is only ever populated from the proxy router, so
the SDK path cannot register one
tests/pass_through_unit_tests/test_websearch_interception_e2e.py caught
this, but #38113 only updated the handler unit tests, so the break
landed on staging
Reverting restores the previous behavior while we work out a fix that
keeps the stricter validation on the proxy path, where a silently
substituted search provider is the real problem worth rejecting, without
turning the SDK path into an unconditional error
The chat-completions bridge emitted Responses output items whose item ids
were raw Anthropic tool ids (toolu_/srvtoolu_), which OpenAI rejects on
replay with "Expected an ID that begins with 'fc'", breaking router
fallback conversations from gpt-5 to claude models.
Four fixes, composable and independently useful:
- emission: bridge output items get fc_/ctc_-prefixed item ids while
call_id stays raw so tool_result pairing keeps working (streaming and
non-streaming share the same helpers)
- openai replay: request transformation drops tool call item ids that do
not match OpenAI's own shapes instead of forwarding them, gated to
OpenAI and Azure, since the API accepts the items with no id at all
- anthropic replay: a replayed srvtoolu_ call whose paired server tool
result is unavailable degrades to a plain client tool_use instead of a
dangling server_tool_use that 400s the client's tool_result
- tool-only turns no longer emit a message output item with output_text
text null, matching native OpenAI output
Adds claude-fable-5-1 cost map entries on the Anthropic API, Bedrock converse
(base, global, and us/eu geo inference profiles at the 10% regional premium),
Vertex AI, and Azure AI. Specs match Fable 5 (1M context, 128K output, $10/$50
per MTok, adaptive thinking always on, xhigh and max effort), except cache reads
land at $0.25 per MTok, a quarter of Fable 5's price and 0.025x base input
instead of the usual 0.1x.
Registers anthropic.claude-fable-5-1 in BEDROCK_CONVERSE_MODELS, lists the model
in the setup wizard, and extends the reasoning effort e2e grid. The partner cells
carry fail_reason markers until access on the CI accounts is confirmed.
Partner entries deliberately carry no deprecation_date: Anthropic publishes
retirement no sooner than 2027-09-01 for the first-party model, and the Foundry
and Vertex dates are not published yet.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The pure helpers express the same regression: the save-side drop must
leave the alias in place, and two registries must expand it to their
own ids. The full validate path is already covered by the
persists-verbatim test and the live e2e test.
Since PR #29128, key create/update/regenerate resolved every
object_permission.mcp_servers entry against the saving instance's
DB + config registry and persisted the resolved server ids. For
config-loaded servers the id is derived from a hash of the regional
URL, so in a shared-database multi-region deployment the rewrite baked
one region's ids into the row and every other region denied the key.
Grants written before v1.88.0 kept the raw alias and kept working,
which is why only newly provisioned keys broke.
Keep the validation and the stale-entry drop (the LIT-3278 fix), but
persist the caller's original identifiers for everything that resolves.
Read-time expand_permission_list already maps a name to each region's
local server id.
* fix(proxy): include litellm_model_table in GET /v2/team/list
GET /v2/team/list built its find_many queries without joining the
LiteLLM_ModelTable relation, so litellm_model_table (and the
model_aliases it carries) always read back as null there, same bug
class as GH #26312 which PR #33047 fixed on /team/info and /team/list
but never touched this endpoint.
* fix(test): assert observable output, not mock calls, in v2 team list test
The test-quality gate flagged the regression test for asserting on
find_many's call args instead of what the caller gets back. Rewritten
so the fake find_many only attaches litellm_model_table when its own
include kwarg asks for it, so the assertions are on the response.
* fix(proxy): drop invalid litellm_model_table include on deleted-team query
Greptile caught that LiteLLM_DeletedTeamTable has no litellm_model_table
relation in the Prisma schema, so passing that include on the deleted-team
find_many raised UnknownRelationalFieldError against a real database on
every GET /v2/team/list?status=deleted call. Confirmed live against
Postgres. Scope the fix to the active-team branch only, where the relation
exists; update the test to reflect that and assert the deleted branch no
longer requests it.
