`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
sagemaker_chat never put X-Amzn-SageMaker-Inference-Component on the request, so any endpoint
backed by inference components answered 400 INFERENCE_COMPONENT_NAME_MISSING and the call never
reached the container. The legacy sagemaker provider has built that header from model_id since
#8889, and this brings the chat provider in line. It goes on in validate_environment, which runs
before the request is SigV4-signed, so the signature covers it
The request body also always named the endpoint rather than the served model, which containers
that validate the body's model answer with a 404. hf_model_name now becomes the body's model
when it is set, and endpoints that do not set it keep sending exactly what they send today
Add sync and async regression tests for the BaseLLMHTTPHandler streaming
path, which forwards provider response headers for the ~30 providers that
ride the generic handler and had no coverage. Also drop redundant setup
prose from the moonshot invoke test docstring.
`members_with_roles` is a denormalized JSON snapshot written at add-time.
`_update_team_members_list` backfilled `user_id` from `user_email` but never
the reverse, so a member added by `user_id` alone was stored with
`user_email=None` permanently - and `/team/info` returns that blob verbatim
with no join to `LiteLLM_UserTable`, so the Admin UI's member table renders
"-" for a user that plainly has an email.
Fix both ends:
- write path: `_resolve_member_identity` resolves identity both ways off the
user rows the add just touched, so new roster entries stop being born blank.
- read path: `/team/info` fills blank emails from `LiteLLM_UserTable` in one
indexed `user_id IN (...)` query, repairing rows already in the database.
Members that already carry an email are passed through untouched and cost
no query, so this only ever turns a null into the right value.
The cost map shipped cognition/swe-1.7 at $2.50 in / $12.50 out per million with
$1.00 cache reads. Those are the Lightning numbers. Cognition's own model list at
https://docs.devin.ai/desktop/models has uid swe-1-7 at $0.50 / $2.50 with $0.20
cache reads, and uid swe-1-7-lightning at $2.50 / $12.50 with $1.00 cache reads,
so every swe-1.7 call has been costed at 5x since the entry landed.
swe-1.7 now carries the standard rates and the Lightning tier gets its own entry,
in both cost map copies. The source field on both moves to the desktop models page,
which is the one that lists both tiers.
* test: enforce PT012 so a pytest.raises block cannot hide dead assertions
`with pytest.raises(...)` stops at the first statement that raises. Anything
sequenced after it inside the block never runs, so an assertion written there is
never checked and the test still reports green.
Two sites were doing exactly that, and both assertions turned out to be wrong
once they started running. tests/llm_translation/test_prompt_factory.py asserted
the bedrock rejection names "requires at least one non-system message", which
holds. tests/proxy_unit_tests/test_proxy_server.py asserted the prisma startup
failure mentions "httpx.ConnectError", which never appears: the failure is an
httpx.ConnectError whose message is "All connection attempts failed", so that
test now asserts the type. Its DATABASE_URL override moves to monkeypatch, since
the old restore sat below the assertion and leaked the invalid URL into every
later DB test the moment the assertion started being able to fail.
The remaining 72 sites are rewritten without changing what they exercise: setup
that cannot raise moves above the block, a nested `patch` moves outside it, and
bodies with real control flow (a stream drain, an if/else on sync_mode, a
retry loop) move into a local closure the block calls.
Fixing PT012 unmasked two B017s, since ruff only reports a blind
pytest.raises(Exception) once the block holds a single statement.
tests/proxy_unit_tests/test_auth_checks.py narrows to the ProxyException
can_key_call_model actually raises. tests/local_testing/test_completion_cost.py
was asserting vertex_ai/medlm-medium has no cost entry, which stopped being true
at some point; that dead first half is gone and the rest of the test, which
checks medlm pricing resolves above zero, now runs instead of being skipped.
* chore(ci): ratchet TQ004 to 768 after the prisma test moved to monkeypatch
SCIM group members were matched against litellm user ids only. An identity
provider that lists people by email or by the OIDC subject therefore matched
nothing, and the member fell through to placeholder creation.
Since #37688 made a failed member creation fail the group sync rather than drop
the member, that fallthrough is no longer quiet: the placeholder is created with
user_email set to the member value, the duplicate-email check rejects it, and the
whole group push answers 500. So on current staging a group listing anyone by
their email fails outright, every other member in the payload included.
An unmatched member id is now looked up across sso_user_id and user_email in one
query. Searching either field first would hide a value that names one account by
its SSO identity and another by its email, and hand the group to whichever was
searched first. The two are not compared alike: an email is matched the way
new_user matches one before accepting a new account, case-insensitively, because
matching more strictly than the layer that would reject the placeholder is what
turned an id whose casing differed from the stored email into that same 500. An
SSO identity is matched exactly, since OIDC defines sub as case-sensitive and
nothing folds its case on the way in.
An exact user_id hit is checked the same way rather than trusted outright, since a
value can be one account's id and another's SSO identity or email. That is not a
corner case: the placeholders this bug provisioned are keyed by the very id the
provider keeps pushing, so on a tenant that already has them the placeholder wins
the id lookup and the real account can never be matched. Refusing names the
problem instead of silently landing on the placeholder again. Those rows still
have to be deleted before the real account resolves; making the sync heal itself
needs a trustworthy way to tell a placeholder from an account someone created, and
created_via lives in caller-writable metadata, so it is left to a follow-up.
A value that names more than one account is refused with a 400 naming the id
rather than attributed to one of them.
Removals resolve too, since the roster holds canonical user ids and a directory
removes people by the id it added them with. A removal counts the members one
value names: the id as written when the roster holds it verbatim, which is how an
earlier release recorded a member it could not match, together with the members it
resolves to. Counting only the accounts on the roster keeps someone removable
after a second account takes their email, which resolving table-wide would not,
and counting both ways of naming a member together stops one value revoking two
people when it is one member's canonical id and another's email. A value naming
two of the group's own members is undecidable and fails rather than guessing or
reporting a removal it did not perform.
Resolves LIT-5383
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Pre-call processing rewrites request_data["model"] for aliasing and routing, so
matching either key let a routed model count as the client's own name and put the
wrapper model back on an Azure Model Router row.
A chunk carrying usage is stored as a pre-restamp copy, so an alias-restamped
stream reaches disconnect billing with its first chunk still on the deployment
model and every later chunk on the client's name. That is the same shape Azure
Model Router produces, and the previous guard read it as a routed model and
left the alias on the row, which is the unpriced name this PR set out to stop.
Compare the assembled model against the name the proxy stamps chunks with, so
the alias goes back to the deployment's model and the routed model stays.
Pricing a frame through the request's own logging object is what makes custom
deployment pricing work, but _response_cost_calculator does not only return a
number. It also stamps cost_breakdown onto the live logging object, and on a
pricing failure it writes response_cost_failure_debug_information into
model_call_details.
On an ordinary proxy stream that is harmless, because the success handler
recomputes cost_breakdown at end of stream and overwrites whatever the frames
left behind. The pass-through handlers are the problem: they compute their final
cost with a bare completion_cost call and never touch cost_breakdown again, so a
breakdown derived from one mid-stream frame would survive to the end and land in
the spend log's metadata. response_cost itself is unaffected either way, so this
was a reporting surface bug rather than a billing one, but the spend row would
have gone from null to a populated breakdown for a partial frame.
Snapshot both writes and put them back once the cost is read, so pricing a frame
stays a read as far as the rest of the request is concerned. The returned cost is
unchanged, so nothing about the injected usage.cost moves.
team_allowed_routes and admin_allowed_routes only matched exact strings or named route groups, so a whole prefix of pass-through endpoints had to be listed route by route in config. Match trailing-wildcard patterns with the same helper the key-level allowed_routes check uses, so "/prefix/*" covers endpoints registered later.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The logging-object pricing applies to streamed /v1/chat/completions too, not
just Anthropic message_delta, so a deployment with negotiated per-token prices
now gets that price in the streamed usage.cost there as well. Nothing asserted
that half. Adds the discounted and the sticker-fallback case for the OpenAI
chunk shape, plus the branch where the pricer raises and the frame falls back
to model-name pricing instead of breaking the stream.
The disconnect billing path was stamping the wrapper's model over whatever
stream_chunk_builder assembled. For Azure Model Router that throws away the
routed model: the proxy deliberately leaves those chunks unrestamped so the
builder can pick the real model off a later chunk, and overwriting it prices
the row at the router alias instead.
Only apply the wrapper's model when the builder did not find a model beyond
the first chunk's, which is every case except Model Router.
* test(lint): ban blind pytest.raises(Exception) with ruff B017
A bare pytest.raises(Exception) accepts whatever the body throws. The TypeError
a refactor introduces satisfies it exactly as well as the rejection the test was
written for, so the crash reads as a pass and the test never goes red.
All 111 existing sites are narrowed here. A runtime probe recorded the concrete
exception each one actually catches, and each site now names that type. Where
the code under test genuinely raises a bare Exception, the site pins a stable
slice of the message with match= instead.
Two sites tell on themselves. The shared responses-API cancel test raises
"custom_llm_provider is required but passed as None" rather than talking to a
provider at all, because cancel_responses takes a provider, not a model. And
test_bedrock_guardrails_with_streaming was the only test in its file still
passing without AWS credentials, because the NoCredentialsError boto3 raised
long before the guardrail ran satisfied the blind raises.
* fix(test): widen the openai batch-dispatch assertion to OpenAIError
The narrowed NotFoundError only holds where OPENAI_API_KEY is set. Without one
the SDK raises OpenAIError while building the client, long before any 404, so CI
went red. OpenAIError covers both and still rejects a TypeError from a refactor.
cache_read_input_tokens and cache_creation_input_tokens are pydantic extras
on Usage, not declared fields, so filling them in created keys that were not
there before rather than replacing a None. Readers that test for presence
then took the new zero as authoritative: the spend log writer skipped its
own copy from prompt_tokens_details, turning a real cache read of 500 into
0, and the prometheus provider cache counters stopped incrementing.
Carry the prompt_tokens_details counts up before defaulting to zero, so a
partial row reports the same cache numbers a complete one does. Renamed the
helper to say what it now does.
The swe-1.7 rates were briefly lowered to the standard tier. The docs page
records the API-served swe-1.7 as the Cerebras-served Lightning tier, so put
the matching rates back rather than have the cost map and the docs disagree.
Cognition also answers /v1/responses through the chat-completions bridge, the
same as every other provider in the JSON registry, so the endpoints support
matrix should say so instead of under-declaring it.
The swe-1.7 rates were carried over from the closed prior attempt and
match SWE-1.7 Lightning, 5x the SWE-1.7 Max and Medium rates the vendor
publishes. swe-1.6 was already on the standard tier, so the two entries
disagreed with each other. Both now read 0.5 in, 2.5 out, 0.2 cached per
million tokens.
Also drops the redundant registry comment in constants.py.
Every pod schedules the budget reset job, so a fleet re-read the whole due
population and wrote it back against one Postgres at the same calendar
boundary, multiplying a single sweep by its replica count. The job now takes
the shared PodLockManager lease, so one pod sweeps per tick. A deployment with
no Redis keeps its previous behavior, and a Redis that cannot answer sweeps
unguarded rather than stranding every expired budget at its cap.
The per-window scan read every row carrying budget_limits in one statement, so
its cost grew with the deployment's key count. It is now keyset-paginated and
walks to the end of the table on every sweep. A per-run cap would need a resume
position, and no pod can hold one because the lease rotates between ticks, so
the strictly advancing cursor is what terminates the walk.
Found and updated rows were also JSON-serialized into the service hook's
metadata and into debug lines on every chunk, on the event loop, whether or not
anything consumed them. The hooks now carry counts, and the debug payload is
deferred until a record is actually emitted.
Resolves LIT-4793
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
SCIM roster writes were swallowed, so a group or user push returned 200 while the
team roster never received the membership. Surfacing the failure fixes that, but
aborting on the first failed write leaves the rest of the batch unattempted on top
of unrolled-back, which is worse than what it replaces.
Every roster write in a reconciliation is now attempted, and the ones that did not
land are reported together, naming each failed add and remove. Rollback would be the
other option and it is not safe here: the compensating write can fail too, and it can
strip a membership that pre-dated the push. SCIM reconciliation is idempotent, so a
named partial failure is what the IdP's next push needs to close the gap.
The reported status still follows the failures, so a unanimous 404 stays a 404 and
only a batch whose failures disagree falls back to 500.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The read replica never received the operator's DB pool settings, so its
Prisma pool fell back to `num_physical_cpus * 2 + 1` and the configured cap
was not enforced. Both startup paths now pass the same params to the reader:
the CLI, and the componentized entrypoints that go through
`DatabaseURLSettings.apply_to_env`.
Only pool and timeout params are inherited, through a single allowlist both
paths share. Anything that decides which tables a query resolves against
stays on the writer, including entries smuggled in through
`database_extra_connection_params`, so a writer `search_path` cannot repoint
reader queries. Params the operator pinned on the replica URL still win.
Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Cognition serves an OpenAI-compatible /v1/chat/completions endpoint, so it has been onboarded as
custom_llm_provider: openai. That books its traffic as OpenAI, which means OpenAI-specific cost
discounts and provider-level reporting apply to it.
Registers cognition through the JSON provider registry: a providers.json entry with
COGNITION_API_KEY and COGNITION_API_BASE, LlmProviders.COGNITION, the constants.py provider lists,
cost map entries for swe-1.6 and swe-1.7, the provider endpoints matrix, the dashboard provider
fields, and tests. JSON providers can now also be resolved from their base url alone, so an
api_base pointing at a known provider no longer falls through to an unresolved provider.
The monitors are separate servers with their own password, so the data node's Entra or
IAM token has no standing there. Dropping the provider only when a Sentinel password was
configured left it in place for unauthenticated monitors, where redis-py sends it as an
AUTH the monitor rejects and async Sentinel discovery fails.
