A shadow eval whose judge_model is one of the router's tier models, the router's
default model, or a reverse job's baseline_model was accepted with no warning. An
LLM judge scores its own output higher than a rival's, so that tier's win rate
measures the judge instead of the models, and the job's whole budget buys a result
that has to be thrown away.
start_shadow_eval now rejects it with a 400 naming the colliding arm.
`judge_target` is the single answer to "where does a call to this name go for this
caller, and what answers it", and the resolvability gate, the collision gate and
the judge dispatch all read it. It has three outcomes and no others: the router
serves the name, the SDK serves it, or nothing does. Splitting that question is
what every bug here came from, so `router_resolves_model` and `answering_models`
are gone rather than joined by a third.
Two spellings of one model are one identity. A name is compared by what would
answer it, resolved through every channel `get_model_list` composes and then put
in the provider-qualified form litellm itself uses, so a judge given as `gpt-4o`
collides with a tier deployment serving `openai/gpt-4o`, and a judge given as
`openai/gpt-4o` collides with a deployment configured as bare `gpt-4o`. Both ends
are normalised because an admin writes them at different times.
Answering is also per-caller. The shadow and judge calls carry the shadowed key's
`user_api_key_team_id`, which is what the router selects deployments with, so the
endpoint derives the job's teams once from the keys it already looks up and every
check runs under them, and the judge dispatch picks its arm under the same team.
A team's public model name resolves to nothing for everyone else and a team's own
deployment resolves for nobody else, so a check that omits the team answers for a
caller who does not exist. A collision under any one team fails the job, because
every key's verdicts land in the same win rates.
Three sites were separately re-deriving "the provider models this name resolves
to", with unexplained divergence in whether they fell back to the literal name.
`Router.resolved_litellm_models` is now the one owner; the routing-plugin
candidate list and the stream-options check both delegate to it, and
`_deployment_litellm_model` is gone.
The router's arms come from `strategy_router_dependencies`, the same enumeration
the health check reads. Only the roles that serve are arms: a classifier or
embedding model picks the tier and never produces a response anyone judges. A
semantic auto-router keeps its routes in an opaque config blob, so only its
default model is enumerable and the guard is incomplete there by design, able to
miss a collision but never to invent one
The two regenerated artifacts carry `presidio_analyze_chunk_size_bytes` from
alters the spec; the sync gate runs on any PR touching litellm/proxy, so this one
has to carry the base's drift to go green
* fix(langfuse): warn and drop invalid LANGFUSE_TRACING_ENVIRONMENT instead of failing requests
* fix(langfuse): treat a dynamic environment equal to the raw deployment value as redundant
* feat(alerting): add native Microsoft Teams alerting destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(alerting): preserve active destinations on MS Teams save and confirm health test delivery
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): read persisted alerting destinations at MS Teams save time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Under otel_v2, a client that propagates W3C trace context in params._meta
(SEP-414) pulled the tools/call span out of the gateway's trace:
resolve_mcp_span_context parented the MCP span to the client's remote
context and demoted the gateway's own transport span to a span link. The
gateway's tracing backend only ever receives the gateway's half of such a
trace, so the span was unreachable from the trace view and the POST
transaction showed a dangling link.
Invert the anchoring: the MCP tool-call and tools/list spans now always
nest under the transport span of the request carrying the message, and the
client's propagated context is recorded as the span link instead, so the
correlation survives while every trace stays renderable. With no transport
at all the span roots its own trace and still carries the link, keeping a
single shape for the event. Both returned contexts are built on an
explicitly empty base so ambient session state can never leak in, and the
span inherits the transport's sampling decision like every other
request-level span.
* feat(newrelic): per-team cost and usage metrics via team callbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(newrelic): retry transient 429/408 metric posts instead of dropping
* fix(newrelic): drop only records queued when the drain began, not mid-drain arrivals
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): stop billing and logging response reads as LLM calls
Retrieving, deleting or cancelling a stored response, and vector store management calls, run through the same logging lifecycle as inference. A retrieved response replays the usage of the call that created it, so every read priced it again and wrote a second spend log row for the same tokens. Non-inference calls now cost 0, report no usage, log no placeholder chat message, and get a litellm.responses_management operation name instead of reading as chat.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): keep billing background response jobs after the poll
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): use an empty list for read-call messages
A tuple matches no branch in the loggers that walk this value, so lunary's
parse_messages falls through to clean_message and raises AttributeError on the
success hook. An empty list reads as no messages everywhere: it satisfies the
isinstance(list) checks in newrelic, mlflow and datadog, iterates zero times in
traceloop and helicone, and is what StandardLoggingPayload.messages is typed to
hold. None would be type-legal too but is not iterable, so it trades one crash
for another in mlflow and traceloop.
* fix(otel): stop the legacy emitter reporting replayed tokens on response reads
The zeroing so far lands in the standard logging payload, which the legacy
OpenTelemetry emitter does not read for usage: it takes prompt, completion and
total tokens straight off the response object, so a retrieval span still carried
the token counts of the call that produced the response, and the token usage
histogram still recorded them. That emitter is the default, so the spend row said
zero while the trace said otherwise. The background cost poller keeps its counts,
the same exemption the pricing path already makes.
* fix(logging): keep billing a background response when its retrieval is read
A response created with background=true comes back queued and carries no usage, so
its create bills nothing. The retrieval that first sees the finished job is the only
place that job's tokens are ever visible, and pricing every read at zero therefore
loses the spend outright rather than deduplicating it. On a proxy without the
enterprise cost poller a background job ended up costing $0 end to end.
is_unbilled_non_inference_call now takes the response it is deciding about and treats
a background response the same way it already treats the poller's own read, which is
the same exemption seen from the other side. The legacy OpenTelemetry emitter's time
per output token metric picks up the read gate it was missing, so it stops dividing a
read's latency by the replayed completion token count.
* test(proxy): pass the read response to the non-inference predicate
The poller test called is_unbilled_non_inference_call with the pre-background signature, so it broke when the predicate gained the response it classifies. It now hands the predicate a foreground read, and asserts that the same read is free without the origin stamp, so the stamp is what the test proves.
* fix(otel): stop the v2 metrics recorder reporting replayed tokens on response reads
The v2 span builder sources usage from the standard logging payload, so the
earlier fix already zeroes it there. The metrics recorder reads response_obj
directly, so a responses-management read still recorded the original
generation's tokens into gen_ai.client.token.usage and divided generation time
by them for gen_ai.server.time_per_output_token.
The read still records operation and response duration, under the
litellm.responses_management operation, so it stays observable.
* fix(proxy): keep the response-cost headers on calls priced at zero
Pricing responses reads and vector-store management routes at zero dropped the whole
x-litellm-response-cost family off those replies. The header build reads a falsy zero as
a cost this response never recorded and filters it out, and a call that returns before
pricing stores no cost breakdown for the component headers to read, so a client parsing
the cost off a read got a KeyError where it had previously been handed a number.
Those calls now advertise the family at zero. Retrieving a background response, and the
cost poller's read of one, still report their real cost.
The params-taking form of the predicate moves from opentelemetry into
internal_call_metadata so the proxy header build and the OTEL recorders share one copy.
* fix(proxy): report a zero cost split only under a zero cost total
The component headers were filled from call-type membership alone, while the
total they sit beside keeps its real value when the read priced normally, so a
breakdown that had not landed by the time headers were built could advertise a
real total next to an all-zero split. The split is now reported as zero only
when the total agrees with it, and is otherwise left absent.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
* feat(langfuse): support langfuse_environment as a per-key dynamic callback param
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(langfuse): type the langfuse_environment constructor param
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(langfuse): only pass environment when the SDK client supports it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(langfuse): drop the request-body metadata test for langfuse_environment
The proxy bans request-body callback params by default (derived from
_supported_callback_params in auth_utils), so the metadata channel this
test asserted is rejected with a 401 on the proxy. The supported channel
is admin-set key/team callback_vars, with LANGFUSE_TRACING_ENVIRONMENT
as the deployment-wide fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(langfuse): validate langfuse_environment, avoid redundant clients, honor it in langfuse_otel
Closes the review gaps on the langfuse_environment param:
- Validate values against Langfuse's environment pattern at save time
(/key/generate, /key/update, /team callback all 400 on e.g. 'Production'
instead of 200-then-silently-dropping every trace server-side) and at
logger init; non-string values are str()-coerced instead of crashing
the SDK's regex check per event.
- Treat empty/whitespace values and values equal to the deployment-wide
LANGFUSE_TRACING_ENVIRONMENT as non-dynamic so an environment-only
override that changes nothing no longer mints a duplicate SDK client
against MAX_LANGFUSE_INITIALIZED_CLIENTS.
- langfuse_otel now reads the per-key/team langfuse_environment from
standard_callback_dynamic_params instead of only the env var.
- Advertise the param on the discovery surfaces: callback_configs.json
(langfuse + langfuse_otel), the dashboard callback registry, and the
/team/{team_id}/callback docstring (schema.d.ts regenerated).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style: ruff format langfuse files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(lint): remove duplicate test import, LIT002 dict literal, and mock-echo otel test
- drop redundant in-function import of callback_config_error (F811)
- avoid the `or {}` mutable literal in _set_langfuse_specific_attributes (LIT002)
- rewrite the dynamic-env otel test to observe span.set_attribute output
instead of patching litellm internals (TQ002/TQ008)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
On /v1/responses the prompt template ran inside litellm.aresponses, after the
router had already resolved a deployment and injected its api_key/api_base, so a
prompt whose metadata.model pointed at another provider sent the old
deployment's credentials cross-provider (401). The proxy now runs the prompt
template for aresponses in the pre-call hook, before routing, so the router
picks the deployment that matches the swapped model. As a backstop, the SDK
refuses a cross-provider swap when explicit credentials are already present
instead of forwarding them.
ignore_prompt_manager_model and ignore_prompt_manager_optional_params saved on
a prompt were only read by the generic manager, so dotprompt prompts ignored
them on every endpoint. PromptManagementBase now merges the prompt spec's flags
with the per-request ones for every manager, and the generic manager no longer
drops caller flags when no spec is present.
POST /prompts silently stored an empty template when litellm_params.prompt_id
was combined with prompt_data keyed by template name, because the loader
wrapped the already-keyed dict under prompt_id a second time. The loader now
wraps only a flat template (a dict carrying a content key), and create,
update, and patch reject the ambiguous keyed+prompt_id combination with a 400
that names both valid shapes. The API also returned version null on every
create and lost version, environment, and created_by on registry reload; both
now carry through. Versioned ids like my-prompt.v1, which the create API
itself returns, now resolve to their base template on the SDK prompt hooks,
and a flat DB prompt with no litellm_params.prompt_id registers under its base
API id instead of garbage.
* feat(prometheus): configure deployment caller identity
* test(prometheus): satisfy strict caller identity lint
* fix(prometheus): align caller identity on latency metrics
* fix(prometheus): validate caller identity mode before collectors register
Fail config load on an invalid prometheus_deployment_and_latency_caller_identity
value (including null) and on include_labels entries the selected mode removes
from a target metric, instead of booting green with an empty /metrics.
Validate the mode at the top of PrometheusLogger.__init__ so an invalid value
raises before any collector lands in the process-global registry, keeping
retries free of duplicated-timeseries errors. Label-validation errors now name
the mode setting alongside the rejected label.
---------
Co-authored-by: Mark Philipp <mphilipp622@gmail.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
The /v1/messages route logs the provider's raw BaseLLMException, which carries
no llm_provider, so its 4xx still counted as an expected client error and lost
its traceback. Treat BaseLLMException as provider-originated as well.
is_expected_client_error treated every HTTP 4xx as a rejection the proxy
issued itself, so a 401 or 429 the provider returned lost its traceback in
the standard logging payload and the OTel error span dropped
litellm.provider.error.stack_trace. An exception carrying llm_provider is
an upstream or deployment problem and keeps its traceback; the proxy's own
pre-call rejections still skip it
A request carrying a session/trace header fans the header value into
litellm metadata as both trace_id and session_id. LangSmith then rejected
the whole ingest batch: a root run's trace_id must equal the run id
embedded in dotted_order (400), and a run-body session_id must reference
an existing tracer session (404/422). Override caller trace_id on runs
that post as roots and drop session_id only when it mirrors trace_id,
so deliberate child-run and valid tracer-session fields still pass through.
The new supports_prompt_caching assertion reads a capability this branch
adds to the registry, so it only holds against the bundled map. CI leaves
LITELLM_LOCAL_MODEL_COST_MAP unset and fetches main's copy, which lags the
branch until merge, so the test failed there while passing locally.
Use the local_model_cost_map fixture the repo already provides, matching
what the other two test files in this change do.
The shared cost calculator treats a missing cache rate as free, so routing
Databricks through it billed cached tokens at zero on the 14 entries that
publish no cache pricing. On a 10,000 token prompt with 8,000 cache reads
that is $0.0010000 against the correct $0.0050001, a fivefold undercharge.
Those entries now declare cache rates equal to their input rate, which is
what a model with no caching discount should charge, and a test pins every
priced Databricks entry to declaring cache rates so no future entry can
regress into it.
Also repoints the provider-neutral generalization test off an id the new
Opus 5 entry now shadows, adds backup-to-main parity tests for the five new
entries, pins that Databricks Claude is never auto-injected with cache
control despite reporting caching support, and trims the Sonnet 5 pricing
note, which is served on an unauthenticated route.
litellm_request_total_latency_metric's start_time is set inside
common_processing_pre_call_logic, which only runs after user_api_key_auth
has already succeeded, so the metric silently excluded authentication and
pre-call setup time despite being documented as total request latency. The
sibling litellm_request_queue_time_seconds metric had the same problem:
its arrival_time was captured after auth too, despite its own comment
claiming to track when the request arrived at the proxy.
request.state.litellm_received_at is now stamped unconditionally at the
very first line of user_api_key_auth (previously only when OTEL was
configured), giving a timestamp that precedes all auth work. Both metrics
now derive from it: queue_time_seconds genuinely spans arrival through the
start of pre-call processing, and the total-latency metric adds that
queue time on top of its existing start/end window so it becomes true
end-to-end latency.
queue_time_seconds ends exactly at start_time rather than a separately
captured timestamp, so its window and the total-latency window share a
boundary instead of overlapping and double-counting a few lines of setup
work on every request.
The ceiling used to go through `int(... or 3)`, so anything `int()` accepted
worked. Tightening the new shared validator to `isinstance(int)` turned a
config that boots today into a proxy that refuses to start, because
`max_agentic_loops: os.environ/MAX_AGENTIC_LOOPS` is resolved to a string
before it reaches either check, and a YAML-quoted "5" is a string too.
Accept ints, integral floats, and strings that parse to a whole number. Keep
refusing bools, fractional floats, words, and anything below 1.
* fix(otel): emit LLM Call spans for speech, image, moderation, ocr and transcription
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): log the image request before caller headers are merged in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): map non-chat routes to standard genai operations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop caller image headers aliasing the logged request body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep resolved api_base in async moderation pre_call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): log resolved client endpoint for speech pre_call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(otel): justify mutable request payloads in speech and image pre_call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep caller headers out of the logged speech request body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The ceiling was only checked at the feature level, on
litellm_settings.websearch_interception_params. The per-deployment
litellm_params.max_agentic_loops, which wins over it, went straight into
int(kwargs.get("max_agentic_loops", 3) or 3), so a 0 was swallowed by the
falsy fallback and read as the default 3. Asking for the tightest ceiling
handed you the loosest one. A non-integer booted the proxy and then failed
every request to that model with "invalid literal for int() with base 10".
Both settings now share one validator, which names the field it rejected,
and the per-deployment value is checked while the model list is read at
startup so a bad value stops the proxy rather than surfacing per request.
The check sits in load_config rather than on LiteLLM_Params because the
proxy builds its router with ignore_invalid_deployments=True, where a
validation error drops the deployment silently instead of refusing to
start. This is the same placement the complexity_router_config plugin
check already uses.
Chat completions read the same key through a separate path that turned 0
into 1 and true into a ceiling of 1, so it now shares the validator too
and the key means one thing on both surfaces.
A capped turn on a streaming request is rebuilt into SSE by
FakeAnthropicMessagesStreamIterator. It emitted content_block_stop for
every block but content_block_start only for text, thinking,
redacted_thinking and tool_use, so a web search turn's server_tool_use
and web_search_tool_result blocks produced stops with no matching start.
Anthropic's SDK accumulator appends on content_block_start and then
indexes content[event.index] on content_block_delta, so the orphan stops
shifted every later index and client.messages.stream() raised IndexError
on the text block. Unknown block types now pass through with a start of
their own, which keeps position equal to index.
Also corrects two claims that said no current caller reaches the loop
with stream=True. AgenticStreamingIterator does, and it keeps raising,
because its events are already on the wire.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
Fifteen tests assigned litellm.callbacks directly and left the conftest global
snapshot to clean up after them. monkeypatch.setattr restores it as part of the
test, so the file no longer depends on that safety net to stay isolated.
Ten tests set litellm.s3_callback_params by hand. Four of them reset it to None
on the last line of the test body, which only runs when the test passes; the
other six wrap the body in try/finally to put the old value back. Raising inside
test_s3_verify_false_handling on the current file leaves the whole callback
config, bucket, endpoint and keys, set in the process for whatever runs next.
monkeypatch.setattr covers both shapes and restores on failure, so the 28 TQ005
violations and the try/finally scaffolding come out together.
51 tests pass, and the wider tests/test_litellm/integrations tree is unchanged.
The five TQ002 mock-echo tests in this file are left alone; those need a
judgement about what S3 logging should assert, not a mechanical sweep.
Both datadog test files hand-roll what monkeypatch.setenv already does: read the
old value, write the test value, put the old one back on the way out. The cost
management fixture checks the old value for truthiness rather than for None, so
an operator running the suite with DD_API_KEY set to the empty string gets it
deleted rather than restored. Starting from DD_API_KEY="" and running test_init
leaves it None on the current file, and "" after this.
13 raw os.environ writes become monkeypatch.setenv, the two fixtures stop being
yield fixtures because there is nothing left to do on the way out, and the now
unused os import goes with them.
27 tests pass across the two files, 88 across tests/test_litellm/integrations/datadog.
* test: use monkeypatch.setenv for env writes in tests/test_litellm
`os.environ["X"] = v` inside a test leaks the value into every test that runs
after it in the same worker, so ordering decides the result. 262 of those
writes across 40 files now go through pytest's `monkeypatch` fixture, which
restores the previous value at teardown.
The rewrite skips any test that a mock.patch-family decorator wraps, any test
with defaulted positional parameters, any test whose own name is called
directly elsewhere, and rebinds nothing inside nested defs, because in each of
those cases appending a fixture parameter changes what pytest or mock binds.
Ratchets the TQ004 ceiling from 768 to 506.
* fix(test): delete the key through monkeypatch instead of popping it first
Five tests popped a key straight out of `os.environ`, ran, then restored it with
`monkeypatch.setenv`. By the time monkeypatch saw the name it was already gone,
so it recorded "absent" as the value to go back to and deleted the key at
teardown. On a worker that inherited a real `RESEND_API_KEY`, `SENDGRID_API_KEY`,
`UI_PASSWORD`, `LITELLM_SALT_KEY` or `OPENAI_API_KEY`, every test after the first
one ran without it.
`monkeypatch.delenv(..., raising=False)` removes the key and restores whatever
was there, so the try/finally the manual restore needed goes with it.
* chore(test): leave the two cost-calc files to the PR that rewrites them fully
Both files are also in #37815, which converts the module-global writes as well
as the env writes and folds them into one fixture. Two PRs rewriting the same
lines differently is a conflict nobody benefits from resolving, so this one
drops back to staging on those two and keeps the other 39.
TQ004 clears 200 here instead of 275; the rest moves with #37815.
When the bounded loop cap or the repeated tool-call fingerprint guard refused a
rerun, the raise escaped the parent agentic frame and the client got the raw model
turn back: HTTP 200 carrying an unresolved tool_use block for the internal
litellm_web_search tool and stop_reason "tool_use". The client never declared that
tool, so it had no way to answer it and the conversation could not continue
The safety check now raises AgenticLoopSafetyError, a ValueError subclass, and
_call_agentic_completion_hooks catches it and returns a finalized response: the
blocks belonging to the refused tool calls are dropped, and stop_reason is closed
out to end_turn when nothing the client declared is still waiting. Refused blocks
are matched by the ids and names of the tool calls the rail refused rather than by
hardcoding the web search tool name
Only the non-streaming anthropic messages path ends the turn this way. A streaming
caller has already sent the original message by the time the hooks run, so a
finalized turn would arrive as a second message rather than replace the first, and
the responses surface carries a pydantic model this finalizer does not rewrite.
Both keep raising, exactly as they did before
Also adds max_agentic_loops to websearch_interception_params so the ceiling can be
set once for the whole feature. A per deployment litellm_params.max_agentic_loops
still wins over it, and the field stays on the proxy's untrusted root list so a
client cannot raise its own ceiling
A name bound twice keeps only the second binding. In `tests/` that is nearly
always a repeated import, harmless but misleading, and the same rule is what
catches the cases that are not harmless: a local that shadows an import the
module still calls, and a second `def test_x` that quietly replaces the first.
311 of the 344 sites were repeated imports and came out with ruff's own fix.
The remaining 33 needed a decision. Four modules imported a name they never
used because a local definition below already shadowed it. Two comprehensions
bound `call` over `unittest.mock.call`, which those modules import and use.
One test rebound the two module handles its nested reload closure had captured.
One class attribute shadowed an unused `status` import.
The load-test fixtures move to a conftest, which is how pytest is meant to share
them, so the test module no longer imports three fixture names it never calls.
The nine `prisma_client` parameters keep a narrow `noqa`: pytest resolves that
fixture by name before the body runs, so the parameter never shadows anything.
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
* test: enforce PT012 so a pytest.raises block cannot hide dead assertions
`with pytest.raises(...)` stops at the first statement that raises. Anything
sequenced after it inside the block never runs, so an assertion written there is
never checked and the test still reports green.
Two sites were doing exactly that, and both assertions turned out to be wrong
once they started running. tests/llm_translation/test_prompt_factory.py asserted
the bedrock rejection names "requires at least one non-system message", which
holds. tests/proxy_unit_tests/test_proxy_server.py asserted the prisma startup
failure mentions "httpx.ConnectError", which never appears: the failure is an
httpx.ConnectError whose message is "All connection attempts failed", so that
test now asserts the type. Its DATABASE_URL override moves to monkeypatch, since
the old restore sat below the assertion and leaked the invalid URL into every
later DB test the moment the assertion started being able to fail.
The remaining 72 sites are rewritten without changing what they exercise: setup
that cannot raise moves above the block, a nested `patch` moves outside it, and
bodies with real control flow (a stream drain, an if/else on sync_mode, a
retry loop) move into a local closure the block calls.
Fixing PT012 unmasked two B017s, since ruff only reports a blind
pytest.raises(Exception) once the block holds a single statement.
tests/proxy_unit_tests/test_auth_checks.py narrows to the ProxyException
can_key_call_model actually raises. tests/local_testing/test_completion_cost.py
was asserting vertex_ai/medlm-medium has no cost entry, which stopped being true
at some point; that dead first half is gone and the rest of the test, which
checks medlm pricing resolves above zero, now runs instead of being skipped.
* chore(ci): ratchet TQ004 to 768 after the prisma test moved to monkeypatch
`assert False` inside a `try:` raises AssertionError, which the `except
Exception` right below it catches, so several tests reported green no matter
what the code did. `pytest.fail` raises Failed, a BaseException, and escapes.
A bare `a == b` statement is evaluated and discarded. Nine of those sat in
tests, and one was comparing against a model name the router never produces.
Selects B011, B015, B018, PT015, PLR0133 and PLW0127 in ruff-tests.toml
alongside F821, with all 50 existing violations fixed, so no budget file or
ratchet is needed. CI already runs this config over tests/.
* feat(otel): route Phoenix traces to per-key/team projects under otel v2
The v2 arize_phoenix preset read PHOENIX_PROJECT_NAME once at startup into a
static resource attribute, silently dropping the per-key/team project routing
v1 supported. Route it via Phoenix's x-project-name OTLP/HTTP header instead:
the env var stays the global default, and a phoenix_project_name (or
phoenix_project_name_override) in key/team metadata sends that key's traces
to the named project.
The project comes only from user_api_key_auth_metadata (server-set at auth),
never from client request metadata or StandardCallbackDynamicParams, since
choosing the telemetry destination is a data-exfiltration primitive. The
header is appended to the exporter's static headers rather than replacing
them, so the preset's Authorization survives, and it is gated to OTLP/HTTP
exporters because Phoenix only reads it on /v1/traces.
Also unban the bare phoenix_project_name fields from the request-body gate:
the proxy integrations ignore them (only user_api_key_auth_metadata routes,
and that stays banned), so rejecting them just broke SDK-style callers.
* fix(otel): root project-routed Phoenix spans in their own trace
Phoenix assigns a whole trace to one project by whichever span arrives
first. The request's auth/db/root spans always export through the default
provider without the project header, so a project-routed LLM span parented
into that trace got dragged back into the default project and the header
did nothing (verified against a live Phoenix instance). Detach the routed
span into its own trace with a link back to the request trace, mirroring
how the v1 Phoenix logger exported each request under its own local parent.
* fix(otel): drain in-flight spans before shutting down evicted providers
LRU eviction shut a routed provider down immediately, but an LLM span
opened at pre_call stays open until the later success or failure callback;
with more than 256 overlapping credential/project routes that in-flight
span was silently dropped instead of exported. Refcount open spans per
provider (hold at span open, release when the carrier is removed on close,
carrier-map eviction, or MCP stray-carrier cleanup) and defer a retired
provider's shutdown until its last open span closes.
* fix(otel): take the provider hold inside route_for to close the eviction race
pre_call can run on thread-pool workers, so between route_for returning a
provider and the caller recording its open span, a concurrent request could
overflow the LRU and shut that provider down with a zero span count, dropping
the routed trace. route_for now increments the open-span count in the same
locked critical section as the cache update and hands back an already-held
provider; every caller releases it once its span has landed. The lock also
makes the cache mutations safe under that same thread-pool concurrency.
* fix(otel): skip tenant routing on deferred pre_call
route_for ran before the recordable-parent check, so a thread-pool
pre_call still built or LRU-touched a tenant provider and could evict
an idle one even though the hold was released immediately and close
re-routed. Only route when the span actually opens
* add somethign
* Revert "add somethign"
This reverts commit 2f2cf84c5a.
* fix(otel): cap retired tenant providers draining open spans
* docs(otel): justify the retired-provider cap
