Controlled base-vs-head run of the full llms test shard exposed two
together_ai anthropic-messages tests broken by the thinking_disabled
contract change in this PR:
- test_anthropic_messages_replays_tool_loop: an unsigned thinking block
in replayed history was being dropped entirely. Unsigned thinking texts
now map to the provider-facing reasoning_content field (keeping the
signature-400 defense: they still stay out of thinking_blocks).
- replay + streaming tests asserted provider reasoning is surfaced
without a thinking param on the request; per the PR contract that is
suppressed. Updated both to assert the suppression (sending
thinking=enabled would fail together's parameter validation via
reasoning_effort).
TQ008: the 16 new handler-level tests patch litellm. internals
(litellm.acompletion, the handler adapter, _prepare_* seams). Each patch
line carries an explainable test-quality-ok: reason - the unit under test
IS the handler's thinking_disabled translation wiring, not the transport.
basedpyright (delta vs base):
- reportPrivateUsage: the shared-classifier delegation added a protected
cross-class call. _chunk_has_substantial_content now derives the
decision inline (same per-choice conditions, same getattr guards, same
.strip()/truthy semantics as the classifier, documented).
- reportOptionalSubscript/MemberAccess: the content_block tool branch
relies on the classifier for tool_call presence; restored explicit
narrowing (assert + local first_tool_call), behaviour-neutral.
- dropped Choices from the emitter/content_block Sequence unions: the
bare-Choices member re-opened Optional on delta.tool_calls[0].function
(the 3 # type: ignore it used to sit next to were dead code anyway).
litellm.types.utils.StreamingChoices imported at top level.
The lint job's 'Check ruff format' gate flagged:
- streaming_iterator.py: 3 line-wraps the branch carried from an 88-column
formatting pass (Literal[...] class attr, 2x applied_edits kwargs) that
do not fit the repo's 120-column config.
- test_handler_thinking_disabled.py: same 88-column wraps on a couple of
def lines / patch() calls.
Base test file's pre-existing format debt left untouched (it fails the
check on base too, so the gate excludes it).
_client_async_logging_helper re-submitted logging_obj.success_handler to the
executor after _dispatch_success_logging had already done so, running the same
success pipeline twice per async request and racing on shared logging state.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
FastAPI walked the multi-megabyte /model/info payload through jsonable_encoder
before json.dumps on every request. Return a prebuilt orjson Response instead,
keeping jsonable_encoder as the fallback for datetimes and other non-native values
Resolves LIT-5724
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Annotate screen_login_password_for_breach's update/where dicts with
prisma input TypedDicts and replace authenticate_user's conditional
dict splat with plain keyword arguments, clearing the LIT002 lines
this branch added in login_utils.py. No behavior change: an unflagged
login now passes allowed_routes=None and metadata={} explicitly, which
are the parameter defaults
A breach found during a login previously only flagged the account for the
NEXT login, handing out one free unrestricted 24h session. The HIBP screen
is now awaited before the session key is minted (worst case one 5s window
per user per 24h, fail-open unchanged), so a fresh hit restricts the
current session and the dashboard routes straight to change-password.
Also repairs two casualties of merge f5e47974db that the layout tests
caught: the lost usePathname import and a call to migratedHref, which
staging renamed to uiHref.
The Terraform endpoint audit wanted POST /user/password/change covered
or allowlisted; it is a caller-scoped one-shot action, so allowlist it
next to /user/bulk_update. leftnav.test.tsx mocked next/navigation
without useRouter, which SidebarAccountMenu now calls, so every render
in that file threw. The two unannotated audit-log patches in
test_password_endpoints.py get their test-quality-ok reasons.
Also removes the LIT002 violations the PR added: prisma input TypedDicts
annotate the where/data dicts, a shared HTTPExceptionErrorDetail
TypedDict covers the HTTPException detail dicts, and the route decorator
takes a tags tuple.
Admin password sets on /user/update and per-user /user/bulk_update stay
supported and policy-enforced. The request model hides the password from
repr so management alerts never format the plaintext, and the all_users
bulk path rejects passwords instead of writing one plaintext value to
every row.
/user/bulk_update awaited a separate HIBP lookup for each user in the
batch, so a degraded-slow HIBP (5s timeout per lookup) could stretch a
500-user batch to ~2500s and time out the request after some updates
had already persisted.
validate_passwords_bulk dedupes the batch's passwords, strength-checks
first, then fires every needed HIBP lookup concurrently, bounding the
worst case at one 5s timeout window. bulk_update_processed_users now
screens the whole batch before the serial update loop, so a rejected
password fails only its own entry and validation failures precede any
persistence.
Reconcile the thinking_disabled gating work with upstream changes to the
experimental pass-through adapters:
- handler.py: keep thinking_disabled alongside the new
litellm_logging_obj param in both streaming transform call sites.
- streaming_iterator.py: AnthropicStreamWrapper accepts both
thinking_disabled and litellm_logging_obj.
- transformation.py: keep both the signature-guard on synthesized
thinking blocks (this PR) and the upstream removal of cache_control
from thinking/redacted_thinking blocks (replay-400 fix); keep the
classifier-based content-block loop (refusal-only deltas still
classify as text via the function fallback, matching upstream).
- tests: append the PR's thinking_disabled gating tests to the file
along with the upstream tests added since the fork diverged.
CredentialLiteLLMParams omitted tenant_id, client_id, client_secret,
azure_scope, azure_username and azure_password, so the strict dump used
by credential reuse and Azure client init dropped them and the reused
credential ended with no auth at all
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The e2e harness exists to prove product features end to end against a live
proxy. The prior Hard Rule carved out an exception for "tests that cover the
harness itself" and pointed at coverage_registry/test_collector.py, which in
practice invited unit tests of harness helpers to be staged alongside e2e
work. That is the wrong tool: harness logic that is worth locking down does
not need a mock-driven unit test living under tests/e2e.
Drop the carve-out. The Hard Rule now reads that no unit tests of any kind
belong under tests/e2e, and the passing mention of unmarked harness coverage
in the transport section is removed so the doc no longer contradicts itself.
coverage_registry/test_collector.py still exists on disk and is left in place
for now; whether to relocate or remove it is a separate decision.
Keeps the base's rule that a non-admin id lookup matching no spend-log row answers 403, so the detail route never consults cold storage without an owner row
* fix(proxy): bound tool and guardrail index create_many by the spend-log statement budgets
One flush drains up to MAX_LOGS_PER_INTERVAL source transactions or logs, but a
transaction fans out to one LiteLLM_SpendLogToolIndex row per tool and a log
to one LiteLLM_SpendLogGuardrailIndex row per guardrail, so the index
create_many payload was unbounded. Both index writes now go through
spend_log_write_batches(SPEND_LOG_WRITE_BATCH_MAX_BYTES, SPEND_LOG_WRITE_BATCH_MAX_ROWS).
The tool index write moves out of the rollup batch_() so the split reduces
the query-engine payload; replayed index rows are no-ops under
skip_duplicates, and the daily rollup upserts stay in one transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): pin the row budget in the index fan-out tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The bridge probe asked `responses_api_bridge_check` with the summary read straight off
the Responses object, but `litellm.completion` reads it from `optional_params` via
`peek_reasoning_summary_aliases`, which the bridged request never populated. So gpt-5,
gpt-5.1 and azure/gpt-5 answered "bridging" to the probe and "not bridging" for real,
and the object still landed on Chat Completions, which only takes a string
`reasoning_effort` is now always the effort string, and `summary` rides the
`reasoning_summary` alias that main.py already reassembles into `{effort, summary}` on
the bridged path. The alias is emitted only when the probe says the model bridges, so
no chat provider ever sees it, and the probe is now asked with the exact params this
transform emits
The Responses API takes reasoning as an object, {effort, summary}. Chat
Completions takes reasoning_effort as a string enum and has no equivalent of
summary, but the completion bridge forwarded the whole object whenever summary
was set, which agentic clients set on every request.
Bedrock Converse guards its mapping with isinstance(value, str) and has no else
branch, so the object fell through, thinking was never enabled, and the caller
was billed for a non-thinking turn with nothing in the response to explain it.
The object is still forwarded for the one caller that can consume it: a model
whose cost-map mode is responses, which litellm.completion bridges back onto the
Responses API and reassembles {effort, summary} there. That decision is delegated
to responses_api_bridge_check, the same check litellm.completion runs, rather
than a second copy of the rule that could drift from it. An object carrying no
effort now yields no reasoning_effort at all.
/model/info fills a deployment's missing pricing in from the model cost map so the
Admin UI has a rate to display. Clients echo that whole model_info blob back on save,
and update_db_model merged it into the row, so editing an unrelated setting turned
that day's catalog price into a real per-deployment override. After that the
deployment ignored the cost map and Reload Price Data could no longer move it,
because the reload replays each deployment's stored pricing over the fresh catalog.
Drop the derived pricing from incoming model_info on the two write paths. The
drop-set is read off the same objects the read path uses, CustomPricingLiteLLMParams
plus the tiered *_above_N_tokens pattern that get_model_info passes through and no
model declares, so it cannot drift as new rates are added. output_vector_size is
exempt: it lives on the pricing model but is an embedding dimension, not a rate.
A deployment's own pricing still rides litellm_params, which is untouched, as is the
explicit-null clear, which reads the incoming model rather than the filtered dict.
The filter sits in the endpoint bodies rather than _add_model_to_db, which master-key
rotation reuses to re-serialize every stored deployment.
