PR #38586 changed the fallback-stamp scrub in async_function_with_fallbacks to
rebind kwargs[sibling] to a scrubbed copy instead of popping in place. Every
other router bucket write mutates the caller's dict in place, and everything
below the router resolves the metadata bucket by key presence, so on a proxy
request that carries litellm_metadata the copy becomes a detached object: the
proxy's post_call guardrail write-backs land in request_data while the spend
row is built from the router's copy. Result: guardrail_information and the
guardrail cost silently drop from the spend row on any request that planted a
reserved key, and an SDK caller aliasing one dict as both buckets loses the
router stamps entirely.
Scrub in place again, and move the anti-spoof to the proxy boundary: strip
attempted_fallbacks and original_model_group from client-supplied metadata and
litellm_metadata in add_litellm_data_to_request, next to the pricing-field
strip, so proxy traffic never carries a reserved key and the in-place pop only
ever fires for an SDK caller that planted one. Keep #38586's hop-stamp ordering
fix (caller keys first, stamps appended) untouched.
* fix(router): drop a tier param the routed target cannot take
A complexity tier's litellm_params are an operator override applied to every request that tier
routes, and they were written into the request kwargs unconditionally. When the tier set a param
the target does not declare, get_optional_params raised UnsupportedParamsError before the request
left the proxy, so the whole tier answered 400. The bundled Lite preset sets reasoning_effort on
its complex tier, and four of the thirteen kimi-k3 map entries reject that param, so a router
built from a first-party template failed on every complex prompt
Filter the tier params at both store sites against what the group's deployments declare. The
candidate set is asked of the module that raises rather than derived from a second list, so
credentials, endpoint and transport controls are never at risk: base_url, timeout,
default_headers, organization and deployment_id are not chat completion params and never reach
that comparison. A param survives if any deployment could take it, since routing has not picked
one yet, and it survives an unresolvable provider or an empty group, since a best-effort filter
must not narrow what the request already did
The skip list _check_valid_arg applies before rejecting a param now has one owner both it and the
router read, so the two cannot drift
* fix(router): honor allowed_openai_params when gating tier params
* test(router): cover _declared_param_allowlist malformed declarations
* fix(router): never ask an authenticating provider whether it takes a tier param
Resolving github_copilot or chatgpt runs their OAuth device flow, so the
capability question _deployment_accepts_param asks would freeze the event
loop for minutes inside async_get_available_deployment. Promote
register_model's local skip set to
constants.PROVIDERS_THAT_AUTHENTICATE_ON_PROVIDER_INFO and fail open on
those providers before any lookup
* fix(utils): adopt a declared authenticating prefix instead of resolving it
The tier-param guard alone was not enough: the savings baseline and the
model-info funnels also resolve deployments during routing, and each
resolution of github_copilot or chatgpt runs their OAuth device flow.
declared_authenticating_provider gives every metadata funnel
(get_supported_openai_params, _get_potential_model_names,
_supports_factory, canonical_model) the resolver's answer by string, so
the whole routing path answers without authenticating. A through-test
drives async_get_available_deployment with a copilot deployment and
records that no copilot resolution happens
The registry key was never copied into ModelInfo, so /v1/model/info reported
null for every model, /model_group/info reported false for every group, and
litellm.supports_parallel_function_calling() returned False for provider-prefixed
entries that declare true. Copy it like every other capability flag and pin the
three surfaces with regression tests.
Resolves LIT-6340
A shadow eval whose judge_model is one of the router's tier models, the router's
default model, or a reverse job's baseline_model was accepted with no warning. An
LLM judge scores its own output higher than a rival's, so that tier's win rate
measures the judge instead of the models, and the job's whole budget buys a result
that has to be thrown away.
start_shadow_eval now rejects it with a 400 naming the colliding arm.
`judge_target` is the single answer to "where does a call to this name go for this
caller, and what answers it", and the resolvability gate, the collision gate and
the judge dispatch all read it. It has three outcomes and no others: the router
serves the name, the SDK serves it, or nothing does. Splitting that question is
what every bug here came from, so `router_resolves_model` and `answering_models`
are gone rather than joined by a third.
Two spellings of one model are one identity. A name is compared by what would
answer it, resolved through every channel `get_model_list` composes and then put
in the provider-qualified form litellm itself uses, so a judge given as `gpt-4o`
collides with a tier deployment serving `openai/gpt-4o`, and a judge given as
`openai/gpt-4o` collides with a deployment configured as bare `gpt-4o`. Both ends
are normalised because an admin writes them at different times.
Answering is also per-caller. The shadow and judge calls carry the shadowed key's
`user_api_key_team_id`, which is what the router selects deployments with, so the
endpoint derives the job's teams once from the keys it already looks up and every
check runs under them, and the judge dispatch picks its arm under the same team.
A team's public model name resolves to nothing for everyone else and a team's own
deployment resolves for nobody else, so a check that omits the team answers for a
caller who does not exist. A collision under any one team fails the job, because
every key's verdicts land in the same win rates.
Three sites were separately re-deriving "the provider models this name resolves
to", with unexplained divergence in whether they fell back to the literal name.
`Router.resolved_litellm_models` is now the one owner; the routing-plugin
candidate list and the stream-options check both delegate to it, and
`_deployment_litellm_model` is gone.
The router's arms come from `strategy_router_dependencies`, the same enumeration
the health check reads. Only the roles that serve are arms: a classifier or
embedding model picks the tier and never produces a response anyone judges. A
semantic auto-router keeps its routes in an opaque config blob, so only its
default model is enumerable and the guard is incomplete there by design, able to
miss a collision but never to invent one
The two regenerated artifacts carry `presidio_analyze_chunk_size_bytes` from
alters the spec; the sync gate runs on any PR touching litellm/proxy, so this one
has to carry the base's drift to go green
A non-ProxyException from the team, project or access-group lookup used to
escape the fallback loop and replace the provider's error. Treat it as a
denial and log it. Also drop the unrelated reformatting of test_router.py
and test_fallback_event_handlers.py so both diffs are additions only.
Router fallbacks configured in router_settings were attempted without
re-checking whether the calling key could use the fallback model, so a key
limited to one access group was served by any model listed as a fallback
for something it could call. Auth only validated the requested model and
fallbacks sent in the request body.
Add a fallback_access_check predicate to Router, consulted before every
cross-model-group fallback attempt; rejected targets are skipped and the
primary's own error is raised when none remain. The proxy injects a check
that runs the same key, team and project model access checks the requested
model goes through.
Reverts #37725. The field existed so SDK callers that cannot read
`x-litellm-model-id` could tell which tier an auto-router picked, and the
framework that motivated it was LangChain. `@langchain/openai` builds
`additional_kwargs` and `response_metadata` from fixed key allowlists and drops
unknown fields at both the chunk top level and inside `delta`, so no
proxy-side placement of a namespaced key can reach a LangChain caller.
The complexity router's existing `return_raw_model_name` already covers that
case: it puts the resolved model in the standard `model` field, which
LangChain does propagate (`model_name` is on its metadata allowlist), and the
proxy honors it on both the streaming and non-streaming paths.
Keeps the unrelated cleanup from #37725 that dropped the redundant
function-local `ProxyBaseLLMRequestProcessing` import shadowing the
module-level one in `async_data_generator`.
`TestModelGroupAliasReachesPreRoutingStrategies` asserted on the marker as a
proof of strategy dispatch; the surviving `response.model == "gemini-flash"`
assertion already proves it.
A model_group_alias whose target is an auto-router shows up in /v1/models and
/model_group/info but 400s on call with "Unmapped LLM provider for this
endpoint. You passed model=complexity_router, custom_llm_provider=auto_router".
async_pre_routing_hook picks the strategy using the name the caller passed,
while the alias is only resolved further down in
_common_checks_available_deployment, so the four strategy registries, all keyed
by the marker deployment's model_name, never match. The hook then declines, and
the auto_router/ marker deployment goes out as if it were a real model
Resolve the alias once at the top of the hook, for lookups only, so the
registries, the tag-filtering escape hatch and the marker's forwardable params
all see the name they are keyed under. The caller-facing name is untouched:
spend metadata is stamped before routing and the response still carries the
alias the client sent
Second half, so the same symptom cannot reach a provider through the entry
points this does not fix (the sync selection path that never runs the hook, a
team-scoped router keyed on its internal name), a group that resolves only to
strategy markers is no longer callable: it raises a BadRequestError naming the
marker instead of handing the auto_router/ pseudo-model to the provider
The router's pre-content ping filter dropped AgenticAnthropicStreamingIterator's
hold-back keepalive, so a held-back turn sent the client nothing until the buffer
settled. A ping that no lifecycle frame precedes is now forwarded live, since a
fallback's message_start can still follow it without overlapping lifecycles
The proxy's cancel-refund guard checked isinstance against the iterator, but the
proxy only ever sees it behind FallbackAwareAnthropicMessagesStream and
AnthropicMessagesStreamingResponse, so a disconnect during hold-back refunded the
budget reservation anyway. Both wrappers now forward a duck-typed
has_buffered_provider_output flag, and the router wrapper follows a fallback
source so the flag tracks the stream actually being consumed
anthropic_messages goes through _ageneric_api_call_with_fallbacks rather
than _acompletion, so its returned streaming iterator was never wrapped
by the chat-completions fallback handler. A retriable SSE event: error
frame (overloaded_error, internal_server_error) from a native
Anthropic/Bedrock passthrough passed through to the client unchanged,
and a MidStreamFallbackError raised by the completion-bridge path's
CustomStreamWrapper propagated unhandled.
Add _aanthropic_messages_streaming_iterator, mirroring
_acompletion_streaming_iterator: it detects a retriable SSE error event
via the new parse_anthropic_error_event helper, raises
MidStreamFallbackError once real generated content (a content_block_delta
frame) has not yet reached the caller, and re-enters the Router's
fallback chain. A MidStreamFallbackError raised directly by the source
iterator (the completion-bridge path) is gated the same way via its own
is_pre_first_chunk flag. The raised MidStreamFallbackError carries a
status-coded original_exception built from the parsed error type, so
status_code/cooldown logic sees the real 429/500/503/etc. instead of a
hardcoded 503.
Lifecycle/bookkeeping frames (message_start, content_block_start, ping,
...) never disqualify a fallback attempt by themselves, since Anthropic
routinely sends message_start before an overload error - but they are
buffered rather than forwarded immediately, since forwarding one and
then appending a fallback attempt's own message_start would produce two
overlapping message lifecycles on one SSE stream. Buffered frames flush,
in order, once real content arrives or the stream ends without error.
Once real content has streamed, or the error is a non-retriable 4xx, the
chunk (or exception) is forwarded as-is rather than starting a second
lifecycle. Content and error coalesced into a single physical read are
handled the same way: once the client has genuinely received the content
(bundled in that same forwarded chunk), no fallback is attempted. A
`ping` keepalive is dropped outright before any real content arrives
(it recurs indefinitely on a slow-starting connection and carries
nothing worth buffering), and the pre-content lifecycle buffer is capped
at MAX_BUFFERED_PRE_CONTENT_ANTHROPIC_CHUNKS, forcing an early commit to
the primary stream so a hostile or pathological upstream can't grow it
without bound. is_anthropic_ping_chunk only matches a chunk whose every
event: line is event: ping, so a ping coalesced with real content or a
retriable error into one physical transport chunk is never dropped.
The fallback request kwargs also deep-copy nested litellm_metadata/metadata
(matching the Responses API path) so the primary attempt's
deployment-specific fields never leak into the fallback request, and the
fallback deployment's own provider headers are merged onto the wrapper's
_hidden_params so they still reach the client/logging pipeline. A
fallback that resolves to a non-streaming response (e.g. an agentic
tool-use interception loop) is synthesized into a real Anthropic SSE
event sequence via the new anthropic_messages_response_as_sse_events
helper, instead of yielding a raw dict into the byte stream - including
a trailing signature_delta for a thinking block, and a message_start
whose stop_reason/stop_sequence/output_tokens stay null/zero the way a
real stream's does instead of leaking the completed response's final
state.
Resolves#24004
Router._add_deployment called get_llm_provider without the deployment's api_base, so a config entry with a bare model plus a known OpenAI-compatible endpoint failed startup validation with LLM Provider NOT provided and the proxy returned 400 no healthy deployments for that model group. acompletion had the same gap at request time: it forwarded only base_url into its get_llm_provider call, dropping the api_base kwarg the router passes. Both now forward api_base so endpoint matching resolves the provider the same way sync completion already does
A reasoning model whose map entry names no effort flag now resolves to None, so
the API omits the field and the dashboard keeps its six-level fallback, and a
deployment counts as catalog-known only when the map supplied its mode, so an
operator writing model_info on an off-map deployment no longer empties the
levels its mapped siblings agree on.
Also drops the ultra level nothing asked for, forwards every level the public
literal names across the chat to Responses bridge, and removes the unreachable
supported_reasoning_efforts validator.
The ModelGroupInfo splat let a supported_reasoning_efforts value left in a deployment's
model_info seed the group, so it narrowed the group from whichever deployment was read
first and was silently ignored on every other one. The field is derived from the group's
deployments, so start it unset and let the intersection fill it in.
Also correct two docstring claims that did not match the code: the anthropic chat path
gates xhigh and max on the output_config path only, and the mode signal separates an
unknown deployment from a known non-reasoning one only while that deployment carries no
model_info of its own.
get_model_info answers supports_reasoning None both for a model absent from
the map, which the router registers under a synthesized entry, and for a
mapped model that simply is not a reasoning model. Reading both as "adds no
levels" let one custom deployment wipe every level its mapped siblings agreed
on.
The synthesized entry carries no mode, which every real map entry for a
routable model does, so an unset flag with no mode now resolves to unknown and
never narrows its group. A group that genuinely shares no level still
advertises none, and the dashboard drops the effort control for it instead of
offering levels routing would refuse.
The chat-completions gate only ever owned xhigh. Widening it to max and ultra
made gpt-5.6 answer 400 on requests litellm itself converts to /v1/responses,
where max is valid, because the gate runs before the bridge decision. No map
entry asserts either flag, so the widened gate could only ever reject.
An empty per-group intersection now falls back to the capability-blind level
list in the dashboard, matching what the picker showed before the field
existed, and ModelGroupInfo tolerates whatever shape an operator writes under
supported_reasoning_efforts instead of failing the whole /model_group/info
response.
Enumerating credential-bearing kwargs in RETRY_BREADCRUMB_EXCLUDED_KWARGS is always one
new kwarg behind: it missed top-level extra_headers and provider token fields, which
log_retry still copied into router.previous_models verbatim. Scrub the breadcrumb with
mask_credentials_in_payload instead, so credential-named values are masked at any depth
(extra_headers.authorization, api_key, aws_secret_access_key, vertex_credentials,
azure_ad_token, and future kwargs), and leave the exclusion set to the request payload and
router walk state only.
This hardens the in-memory breadcrumb; it is not a fix for a reproduced SpendLogs leak. The
SpendLogs metadata allowlist and the universal previous_models stripping already keep this
breadcrumb off every persisted surface.
Parametrize the regression test over provider_specific_header, extra_headers, and api_key,
asserting the raw credential value never survives into previous_models for any shape while
the container key still reaches the breadcrumb
log_retry copied every kwarg into the previous_models breadcrumb, so a client's
forwarded Authorization (provider_specific_header) and the deployment api_key /
headers rode along in an in-memory structure whose comment says it reaches spend
logs and logging callbacks. Those values have no diagnostic use in a breadcrumb.
Add provider_specific_header, headers, and api_key to RETRY_BREADCRUMB_EXCLUDED_KWARGS
so the credential is never placed there in the first place. This is defense in depth:
no persisted leak exists today, since the SpendLogs metadata allowlist and every
logging integration already drop previous_models before serialization. Removing the
credential at the source means a future logging path cannot expose it either
* test: add regression coverage for twelve closed issues
Adds targeted regression tests for behavior that was fixed but left ungated,
so the fixes cannot silently regress:
- #33772 openai cache_write_tokens cost
- #34309 Responses API cache cost_breakdown
- #35363 /v1/responses batch spend
- #36619 auto-router api_base/api_key leak on a shared model name
- #35359 batch fallbacks within the owning model group
- #36523 passthrough streamed Responses spend log
- #36646 passthrough embeddings spend log
- #37147 non-object metadata on create_batch is a 400
- #35362 unscoped list files reads the managed-file store
- #33221 gpt-5.6 bridges to Responses on function tools alone
- #34487 LLM complexity classifier runs for every caller metadata shape
- #35124 streamed /v1/messages emits success logging on both bridges
Cost assertions read rates from litellm.model_cost rather than hardcoding
dollar amounts, so they do not drift on repricing.
* fix: stop the new regression tests polluting and tripping over shared global state
Two shard failures, both from global state the new tests share with their
neighbours rather than from the behaviour under test.
test_main.py's local_cost_map pinned litellm.model_cost but left the
get_model_info lru_cache warm, so completion_cost billed at whatever prices
were cached earlier in the process while the assertions read the pinned map.
Clear the cache on both sides of the fixture, matching the local_model_cost_map
fixture in tests/test_litellm/conftest.py.
The anthropic messages streaming tests called GLOBAL_LOGGING_WORKER.flush()
on whatever queue happened to be around. A queue left non-empty by an earlier
test is still bound to that test's loop, so join() either hangs or raises
"bound to a different event loop". Rebind to the running loop before the call
and wait for the captured payload instead of a fixed sleep.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
* fix(router): don't log 'Could not identify azure model' when the deployment name resolves from the cost map
get_router_model_info already falls back to resolving the azure
deployment's model name against the model cost map when base_model is
unset — and for deployments named after real azure models (e.g.
azure/gpt-4o) that resolution returns correct max tokens and costs. The
unconditional ERROR was therefore spurious for exactly the deployments
that need no operator action, and on busy proxies it logs thousands of
times per day per multi-deployment group.
Log at debug when the fallback entry carries usable limits/costs
(membership alone is not enough: Router init auto-registers every
deployment name as a zeroed stub), keep the ERROR otherwise.
Fixes#33172
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(router): use consistent positive checks in azure base_model fallback gate
Review follow-up: token-limit fields used 'is not None' while the cost
field used '> 0' — a cost-map entry explicitly storing 0 limits could
suppress the error log without carrying usable resolution data. All
three checks now require a positive value.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(router): trim fallback gate comment and reuse the shared local_model_cost_map fixture
---------
Co-authored-by: Mihidum Hettiyahandi <55163074+mihidumh@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The auto-routed model group was only reachable through the
x-litellm-model-id response header. SDK and framework callers that do not
expose response headers had no way to read it, and under streaming there
was no body surface at all.
The response body `model` field is deliberately restamped back to the
client-requested alias on both paths, which is correct OpenAI semantics,
so this adds a separate namespaced `router_model_name` key instead of
redefining `model`. The key is written on non-streaming bodies and on
every SSE chunk, including the streaming fast path, and is emitted only
when an auto-routing strategy actually selected the deployment.
After a mid-stream fallback moves the request off the group the router
picked, the key is omitted rather than continuing to claim the original
tier. The router marker already supports per-chunk fallback signals via
`x-litellm-attempted-fallbacks` headers; this wires that signal into
the gate so no stale tier is claimed after a fallback fires.
Also removes a redundant function-local import in the streaming
generator that shadowed the module-level one for the whole function.
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
* test: enforce PT012 so a pytest.raises block cannot hide dead assertions
`with pytest.raises(...)` stops at the first statement that raises. Anything
sequenced after it inside the block never runs, so an assertion written there is
never checked and the test still reports green.
Two sites were doing exactly that, and both assertions turned out to be wrong
once they started running. tests/llm_translation/test_prompt_factory.py asserted
the bedrock rejection names "requires at least one non-system message", which
holds. tests/proxy_unit_tests/test_proxy_server.py asserted the prisma startup
failure mentions "httpx.ConnectError", which never appears: the failure is an
httpx.ConnectError whose message is "All connection attempts failed", so that
test now asserts the type. Its DATABASE_URL override moves to monkeypatch, since
the old restore sat below the assertion and leaked the invalid URL into every
later DB test the moment the assertion started being able to fail.
The remaining 72 sites are rewritten without changing what they exercise: setup
that cannot raise moves above the block, a nested `patch` moves outside it, and
bodies with real control flow (a stream drain, an if/else on sync_mode, a
retry loop) move into a local closure the block calls.
Fixing PT012 unmasked two B017s, since ruff only reports a blind
pytest.raises(Exception) once the block holds a single statement.
tests/proxy_unit_tests/test_auth_checks.py narrows to the ProxyException
can_key_call_model actually raises. tests/local_testing/test_completion_cost.py
was asserting vertex_ai/medlm-medium has no cost entry, which stopped being true
at some point; that dead first half is gone and the rest of the test, which
checks medlm pricing resolves above zero, now runs instead of being skipped.
* chore(ci): ratchet TQ004 to 768 after the prisma test moved to monkeypatch
* test(lint): ban blind pytest.raises(Exception) with ruff B017
A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.
All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.
Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.
* fix(test): widen the openai batch-dispatch assertion to OpenAIError
The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
tiktoken's BPE merge loop is quadratic in the length of a single regex piece, so a long
run of one repeated character turns a multi-MB payload into minutes of CPU. Encoding in
bounded chunks makes that linear, at a drift of at most ~1 token per chunk boundary.
Chunking alone only makes the stall shorter, so the async paths now count in a worker
thread: tiktoken releases the GIL for its Rust encode, so the loop keeps serving other
requests while a count is in flight. The /utils/token_counter endpoint awaits the new
atoken_counter, and the router's async deployment selection counts off-loop and hands
the result to _pre_call_checks instead of making it count inline.
The chunk size knob is bounded to [1, 4096]: a non-positive value used to raise or
silently report zero tokens, and an arbitrarily large one restored the quadratic cost
this exists to remove. Out-of-range and unparseable values warn and fall back to 1024.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Resync registry misses with single-row DB fetches (guardrail by unique
name, agent by unique id or name, model by name then id) instead of
full-table loads, and bound them with a global budget of 20 resyncs per
5s window per registry that fails closed without negative-caching the
key.
Access group create/update now trust the reconcile outcome snapshot
captured under the reload lock instead of a post-lock router read, so a
concurrent reconcile can no longer surface a false degraded-serving 500.
Router.upsert_deployment restores the previously served deployment when
the replacement add fails under ignore_invalid_deployments, so a bad
update no longer silently drops a healthy deployment from serving.
A file uploaded through Router.acreate_file lands in the account of the
deployment that stored it, so a cross-group fallback silently stores the
file with the wrong provider and every later batch or fine-tuning call
against the returned id permanently fails. Extend the provider-scoped
fallback pin that already covers input_file_id and training_file to file
creation, so the original provider error surfaces instead.
Key and team router_settings set enable_tag_filtering on the request kwargs,
and get_deployments_for_tag already treats that as authoritative, but
_select_pre_routing_strategy only consulted the router-wide flag, so tagged
auto-router markers still captured untagged requests from keys that enabled
filtering. The e2e auto-router module now enables tag filtering through
key-level router_settings instead of flipping /config/update module-wide,
which was denying concurrently running tagged requests from other suites on
the shared per-build CI proxy.
