A client polling GET /v1/batches/{id} can stamp the provider's terminal
status (cancelled/failed) before the batch cost job runs. The pickup
query excluded cancelled/failed from the not_in list, so the job never
revisited that row and the spend for the completed requests was
permanently lost.
Remove cancelled/failed from the exclusion so the cost job picks them up
like complete/completed. The staleness sweep is extended to also retire
old failed/cancelled rows with batch_processed=False, serving as the
upgrade guard so pre-existing terminal rows do not spike on deploy.
Fixes#37217
The Noma guardrail sends the conversation to the scanner in `inputs`. It
also forwarded `request_data` whole, which repeats that same conversation
under `messages` (or `input` on the responses API), and attached
`logging_obj.model_call_details`, which repeats it a third time.
For image-heavy calls that duplication is most of the request. A
production scan of a request carrying base64 images measured 100MB total,
of which 94.8MB was `request_data` against 5.1MB of `inputs` - the proxy
was uploading ~95% redundant bytes, and paying to serialize them.
Drop `messages` and `input` from `request_data` and from
`model_call_details`. This is a denylist rather than an allowlist on
purpose: every other key is still forwarded untouched, so a scanner-side
change that starts reading a new `request_data` key needs no matching
release of this hook. The removed keys are ones the scanner never reads -
it takes context only from metadata, litellm_metadata,
provider_specific_header, litellm_session_id/trace_id/call_id, stream,
response/responses ids, and litellm_logging_obj.complete_streaming_response,
all of which still pass through.
The conversation still reaches the scanner in full via `inputs`, so no
detection coverage changes.
Trimming happens before serialization, so the duplicate is never encoded.
Existing payload tests asserted the duplication; they now assert the trim
while keeping what they originally guarded - deep-copy semantics and the
unpicklable-object (uvloop.Loop) regression.
A bare toHaveBeenCalled() passes no matter what the caller passed, so the CSV
export could serialize the wrong rows, write the wrong content type, and name
the file wrong while its test stayed green.
Strengthens the load-bearing cases in three files onto the arguments that carry
the behaviour: the rows handed to the CSV serializer, the blob content type, the
anchor that gets attached and cleaned up, and the specific message each
validation failure shows the user. Two of the discount and margin tests
previously asserted the same bare call for different validation failures, so
neither could tell the two apart.
Each rewrite was proven by mutating the source it covers and confirming the test
goes red where the bare assertion stayed green.
* fix(mcp): scope authorization server issuer
Generated with AI
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(mcp): keep the bare-origin issuer when no server was named
The scoped issuer must key off whether the request actually carried a server
name. _build_oauth_authorization_server_response rebinds mcp_server_name when
root discovery resolves the single configured OAuth2 server, so gating on the
rebound value also scoped /.well-known/openid-configuration, whose document is
served from the bare origin and whose issuer must stay the bare origin
Adds the named-server regression test for the reported mismatch, restores the
bare-origin assertion, and covers the OIDC document
* test(mcp): type the delegate_auth_to_upstream helper parameter
* refactor(mcp): bind the discovery issuer to a local before building the response
---------
Co-authored-by: Irosh <15094153+irosh-colombage-ZocDoc2@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
The User Usage view handed EntityUsage a static entityList holding only
the first /user/list page, so its filter could only find the 50 most
recently created users and anyone beyond that page, including users
with spend in the selected period, was unreachable
Add a self-contained UserDropdown that owns useInfiniteUsers (server-side
search plus load-more, mirroring TeamDropdown) and use it both as the
User Usage filterSlot and for the Global Usage user filter. Resolve a
selected user that is outside the loaded page by id so its label
survives view round-trips. Drop the now dead single-select branch from
UsageExportHeader
The field itself landed on staging via 0c5c9c79d7; these are the regression tests from PR #31435 for the retrieval-facing half.
(cherry picked from commit a9a322d63f6d4658b1f28d1622335775e94736a4)
Bedrock batch jobs write their results to s3_output_bucket_name when it differs
from the input bucket, but the file-content retrieval path validated the file id
only against the input bucket (s3_bucket_name). A deployment that configures a
separate output bucket therefore could not retrieve its own batch outputs: the
id validated against the input bucket and was rejected as a foreign bucket.
Resolve the trusted output bucket alongside the input bucket from the immutable
credential snapshot (or AWS_S3_OUTPUT_BUCKET_NAME), and try the file id against
each configured bucket, returning the first that validates. The SSRF guard is
preserved: only server-configured buckets are tried, never a request param, and
an id outside both is still rejected.
(cherry picked from commit 1d407c2f26)
Enables 14 rules at error with zero violations, so the vacuous-assertion class
the previous commit fixed cannot come back. No budget file and no suppressions
baseline: a rule is on only if it is already at zero.
prefer-to-have-value stays off. It matches any attribute whose name contains
"value", so it rewrites toHaveAttribute("aria-valuenow", n) into toHaveValue(n),
and jest-dom's toHaveValue supports only form controls, which breaks every
role="meter" element the dashboard renders.
Records the enabled set, the rules left off with their measured counts, and the
seven ways these plugins' autofixers produce broken output.
One test could not fail: it asserted on the Promise returned by an unawaited
findAllByText, and a pending Promise is always truthy, so the organization
detail view had no working coverage at all.
Rewrites 316 assertions across 128 files onto matchers that describe what the
user perceives, and awaits the queries that were being compared as Promises.
Most of this was mechanical, but the fixers behind these rules are not
trustworthy, so every site they damaged was repaired by hand. The quiet one
worth naming: prefer-to-have-text-content wraps strings in new RegExp() without
escaping, turning toContain("100K+ requests") into a pattern meaning "100
followed by one-or-more K". That compiles, lints clean, and keeps passing while
no longer asserting what it claims.
The pin tests picked the pinned model by position in the option list. antd
virtualizes that list and jsdom reports every row as zero height, so only the
first window ever renders, and adding the Lite preset to the bundled JSON pushed
the pin out of it. Both cases have been red on litellm_internal_staging since
that preset landed.
Typing the model name filters the list to the pin, so the click no longer
depends on where the pin sits among the presets.
Filling each cost field independently let a published batch rate outrank a
standard rate the deployment configured itself: a deployment declaring only
input_cost_per_token had its batches billed at the model's published batch price
rather than half its own rate. Measured on a model that publishes both, that
billed $0.001500 where the deployment's own rate meant $0.000500.
Declaring either rate for a direction now claims that whole direction, so nothing
published can displace it, and a direction the deployment is silent on still
inherits both published rates.
Codecov's remaining uncovered patch line was the early return taken when no model
is available to look a published entry up by, which leaves a deployment's own
declared rates standing alone. Measuring the patch lines against the coverage
report now leaves none uncovered.
batch_cost_calculator gated the batch rate fields on truthiness, so a deployment
that configures input_cost_per_token_batches or its output twin as 0.0 was read
as having configured nothing and that token direction fell through to half the
standard rate. Layering declared rates over published ones made this reachable:
a deployment declaring only a zero batch rate previously kept a fabricated zero
on the standard field, which happened to bill nothing.
The two batch fields are now gated on presence. Verified no cost-map entry
changes behavior: the only three carrying a zero batch rate are embeddings, whose
standard output rate is also 0.0, so both paths yield the same zero.
Adds a parametrized regression over an explicit zero, an explicit non-zero, and
unset, plus coverage for the deployment id get_model_info cannot resolve, which
were the lines Codecov flagged.
Substituting a deployment's pricing wholesale billed the token direction it did
not configure at zero: get_model_info fills an absent cost with 0, and any
non-None pricing field suppressed the global fallback. A deployment declaring
only input_cost_per_token therefore billed output at nothing.
Each of the four batch cost fields now falls back to the model's published rate
when the deployment leaves it unset, so a one-sided override applies to the side
it configures and only that side.
Adds a parametrized regression over input-only, output-only, and both-zero, plus
coverage for a deployment whose model has no published entry. Annotates the new
test helpers per the repo's type-coverage rule and drops the narrative banner
comment from the batch tests.
On a batch retrieve both self.model and litellm_params[model] come back None, so
the cost model fell through to the provider's own response model (an Anthropic id
like claude-opus-4-5-20251101) which does not resolve under a bedrock provider,
leaving bedrock batches at $0 with correct usage.
model_call_details carries the deployment's provider-qualified model
(bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0), confirmed by
instrumenting a live retrieve, so it is preferred with the previous two sources
kept as fallbacks.
self.model can carry the router's model_group alias, which no cost map resolves,
so a bedrock batch still priced at $0 after the model name started being passed.
The deployment's own litellm_params model is used when present.
Verified against the local (image-bound) cost map that dev and prod both force:
alias 'claude-opus-4-5' prices $0.000000 while
'bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0' prices $0.017000
The router registers a model_info entry for every deployment, priced or not, and
get_model_info fills absent costs with 0. Resolving deployment pricing through it
therefore reported a free deployment for any ordinary one, which priced its batches
at $0 while usage stayed correct: the same silent under-count this branch set out
to remove, widened from bedrock to every provider.
Caught by a live batch run, where four vertex batches that price correctly today
came back at $0. The raw registration is now what decides: pricing is used only
when the deployment actually declares one of the batch cost fields, so ordinary
deployments fall back to the global cost map exactly as before.
The earlier test missed this by using a deployment id that was never registered,
where get_model_info does raise; a real deployment is always registered.
Retrieving a completed batch computed its cost with no model identity: neither the
deployment's model nor its configured pricing reached the batch cost calculation.
For bedrock that left the cost model falling back to the provider's own response
model (e.g. "claude-sonnet-4-6"), which does not resolve under a bedrock provider,
so the lookup missed and cost silently became $0 while usage stayed correct.
Dropping the deployment's model info separately discarded any rates configured on
that deployment, billing a zero-cost deployment at the public rate instead.
Both are the same omission at the call site, so both are fixed by passing the
logging object's own model and the pricing the router registered for the
deployment.
The auto-router tests moved onto accessible queries here while staging added a
"Lite" preset and a default-model pin, so the option-label expectations take
staging's list read through this branch's visibleOptions helper.
Staging's new pin tests reached for antd's internal classes, which the lint rule
this branch enables rejects. The edit-modal cases now read the rendered
selection through one selectedValueIn helper, and the clear-affordance click in
ComplexityRouterConfig keeps a reasoned suppression since antd marks that icon
aria-hidden.
