* fix(health): probe test_connection with the credential the request names
/health/test_connection matches the request's model string against the
configured deployments and merges the match's litellm_params underneath the
request. A request that named a stored credential but no key of its own
still satisfied the "request sets no connection fields" test, so it inherited
the matched deployment's api_key and api_base, and load_credentials_from_list
then skipped the named credential because api_key was already set.
A wildcard route covering the model is enough to match, so the Add Model
page's Test Connect probed with an unrelated deployment's key while echoing
back the credential that was selected.
Naming a credential the configuration does not name now withholds the
configuration's credential fields, the same set already withheld from a
request that supplies its own endpoint. Naming no credential still inherits
them, as documented.
* test(health): drop test docstrings that restate their own names
* test(health): assert the credential probe on the wire, not on the call args
The connection-test regressions patched litellm.ahealth_check and read the
params handed to it. Driving the endpoint through the app with respx faking
the upstream instead lets the real credential resolution run, so the tests
assert the key and host that actually go out, which is what the bug was about.
It also drops three of the five patched proxy internals; the two that are left
are proxy-global wiring with no injection seam, the same ones the image_edit
connection test already has to reach for.
* chore(ui): regenerate schema.d.ts for the test_connection docs change
The create form rejected names with slashes or spaces even though the
proxy stores and routes any non-empty string. Drop the client-only
character pattern and trim the name before the required check so a
whitespace-only name is still refused
Claude-Session: https://claude.ai/code/session_01HkaXiD6gssHnx3kqu1rR8C
An open classifier circuit routed through the ordinary heuristic or
classifier_fallback path, and both causes are pin-worthy, so a session
whose turn landed on the cooldown fallback held that model for the whole
session_affinity TTL and never reclassified after the breaker closed.
The circuit-open signal now blocks the pin, and _classifier_failure_outcome
tags its outcomes through one helper instead of reassigning a Final.
The shared DataTable clamps a server-mode page index whenever rowCount no
longer reaches it (#39776), including the empty-dataset case this table's
own clamp skipped because it required total_pages > 0. Remove the local
clamp and cover the empty case through the component so the wiring into
the shared behavior is what the tests prove
* feat(team): report per-user spend within a team for JWT traffic
Add GET /team/spend/by_user, which groups raw spend logs by (team_id, user)
so JWT/SSO requests with no virtual key are attributed to the user inside
each selected team. Team admins see every member, plain members see only
their own row. The Team Usage page gets a Spend Per User Within Team card
with CSV export backed by the same endpoint.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(team): cover /team/spend/by_user in behavior suite, tf audit allowlist and EntityUsage unit test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(team): drop explanatory docstrings from /team/spend/by_user and regen schema.d.ts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Server-mode tables kept whatever page index the user was on after the
server's total dropped below it, for example after deleting the last
rows of the final page or when a refetch came back empty. The footer
then read "Page 2 of 1" and "Showing 26-25 of 25" with Previous and
First enabled over an empty body, and every one of the 13 server-mode
consumers was exposed since none of them clamped
The shared DataTable now snaps the controlled page index to the last
valid page as soon as a non-loading rowCount no longer reaches it, so
the fix applies to every consumer without per-table clamps. Loading
responses are ignored so a pending fetch never bounces the user to
page 1
test_event_loop_stall_timeout_burst_keeps_breaker_closed built its timeout
burst by wrapping a healthy fake call in asyncio.wait_for. Before 3.12,
wait_for returns the inner result when the inner future also completed while
the loop was blocked, so no call timed out, the burst never materialised, and
the test's own liveness guard failed with 0 >= 3.
The fake now checks its own client deadline against the clock, the way a client
library does, so the stall produces a real redis TimeoutError burst on every
interpreter. The breaker itself is unchanged: its duration gate is plain
time.time() bookkeeping and never depended on the version.
Drops `-x` from all 24 pytest invocations in .circleci/config.yml. With
`-x`, a job stops at its first failure, so a second broken test in the
same suite stays invisible until the first is fixed and CI is re-run.
That turns one round trip into N when a job has several broken tests.
This is exactly what happened in #39770: fixing
test_missing_model_parameter_curl in
tests/store_model_in_db_tests/test_openai_error_handling.py immediately
unmasked test_chat_completion_bad_model_with_spend_logs in the same
file, which had been failing for a long time without ever being
reported.
Only `-x` is removed; -v/-vv/-s/-n/--reruns and every other flag are
untouched.
Drop the two new docstrings, annotate the new locals Final, and replace the
mutable call recorder with a rebuild stub that fails the test if it is ever
reached.
test_no_linear_scans_in_router: #39468 added config_deployments() and
heuristic_v2_router_limit_violation(), which both scan the whole model_list
from admin-only paths (model add/upsert), so add them to the allowlist. The
allowlist becomes a mapping so each exemption carries its reason as data.
test_missing_model_parameter_curl: a request with no model is rejected by the
proxy when nothing can serve it and by the router when a wildcard or default
deployment exists, and by the upstream provider when a wildcard forwards it,
so the message text is not a stable contract. Assert the contract that holds
in every case: HTTP 400 with a non-empty error message.
test_model_group_info_e2e: /model_group/info resolves wildcards, so it can
never return "anthropic/*" verbatim. cc3f9cd65b rewrote the assertion to
expect the raw pattern after claude-3-5-haiku-20241022 left the price map,
which made it unsatisfiable. Assert the expansion instead.
test_should_derive_ocr_mapping_status_from_live_tests: the audit needs a
native bridge built with the trace-parity feature, which CI never builds, so
skip with the harness's own diagnostic instead of erroring. Extract that
check out of ensure_trace_bridge as trace_bridge_error so a pytest run
reports the state without kicking off a maturin rebuild.
* feat(ui): page the public model hub table off /public/v1/model_hub
The public Model Hub page loaded every published model group in one call and
did all of its searching, sorting and filtering in the browser, so a proxy with
a few thousand groups sent megabytes to render one screen.
The models table now asks /public/v1/model_hub for one page at a time. Paging,
sorting, search and the provider and mode filters are query parameters on that
route, and the pagination footer counts from the response envelope's
total_count rather than the rows on screen. Column sortability is derived from
the fields the route declares sortable, so a header can no longer ask it for a
sort it answers with a 400.
The feature filter is dropped: supports_* are booleans and the route has no
boolean filter, so it could only ever have filtered the page in view.
* fix(ui): offer the model modes litellm actually prices in the hub filter
The mode filter listed 'moderations', which no model group's mode is ever set
to, so picking it could only ever return nothing; 'anthropic_messages' was
dead the same way. Four real modes the catalogue does use, search, ocr,
guardrail and vector_store, were missing entirely.
The list is now the mode vocabulary in model_prices_and_context_window.json,
and a test reads that file so an option that matches nothing, or a mode with
no option, fails instead of silently filtering to an empty table. A failed
page fetch logs the route's error detail again, as it did before the table
moved to the paginated route.
* test(ui): keep the model hub health rows out of the inline-object budget
frontend-lint's local/no-large-inline-object-arg budget went 555 to 556: the
health check rows became arguments to the row helper. They are plain literals
spreading a shared default again, which is what they were before, and the gate
reports 554 against a max of 555.
* feat: keep every model hub filter when the table pages
Moving the table onto /public/v1/model_hub cost it two controls the route
could not serve: the provider filter fell back to one substring because
providers only declared contains, and the feature filter went away entirely
because supports_* are booleans with no filter at all. Options for the
dropdowns went with them, since a page of rows only knows the values on that
page.
The route now declares providers in, a features field whose value is the
capability names a row has, and providers, rpm and tpm as sortable. Features
is one repeated field rather than a boolean per flag so selecting two of them
matches either, which is what the multi-select has always meant. Three facet
routes serve the distinct providers, modes and features across the published
groups, carrying the parent's filters, per section 12 of the list design.
All of it is additive: the route rejects unknown parameters, so no request
that worked before changes, and the design's stability policy calls new
filters and parameters safe within a version.
Health status stays unsortable. Health is read for the rows on the page, and
ordering the match set by it would mean reading it for every published group,
which is the cost the paging exists to avoid.
* fix(ui): put the model hub facet types where the generator emits them
The generated file lists paths in sorted order and operations in path order.
Both new blocks were spliced in one entry too late, after
/queue/chat/completions rather than before it, so the schema.d.ts sync check
regenerated the file and found them misplaced. Same blocks, byte for byte,
moved to the position the generator gives them.
* fix(proxy): type a facet payload as the sequence the framework hands it
The lint job's basedpyright gate flagged one new reportArgumentType: handle_facet
passes a tuple, and FacetListResponse declared data as list[str]. A list would
have traded that error for an LIT002 mutable construction, and both budgets are
already at their ceiling on the base.
Sequence[str] is what the framework actually produces and what the model always
accepted: pydantic emits the same array schema either way, verified against
model_json_schema, so the OpenAPI spec and schema.d.ts are unchanged, and the
existing list-passing caller in spend_logs still type checks.
* test(proxy): pin the facet route's rejection contract
handle_facet answers six ways before it ever reaches the executor, and
none of them was covered: a denied scope, a filter operator the spec does
not offer, a repeated parameter, a non-positive page or page_size, and the
where clause those last two feed. Every one is a 400 or 403 an
unauthenticated caller can reach, so each gets a test that fails when the
branch stops firing.
