mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-02 07:13:02 +00:00
da71a03f6e
* fix(proxy): trace model list requests with the caller's traceparent
The proxy creates an OTEL SERVER span at the auth boundary whenever a request
carries traceparent, but /v1/models never closed it. Unended spans are not
exported, so callers saw nothing for the model list while inference requests
showed up in the same trace
Route the endpoint through the management span wrapper so the SERVER span is
closed and parented to the caller's trace. The wrapper gains a variant that
skips request and response payloads, since attaching the full model list would
bloat every poll
* refactor(proxy): type the span wrapper with ParamSpec
The strict ruff rules ban new typing.cast imports, so the wrapper keeps the
wrapped endpoint's exact signature via ParamSpec instead of casting the
wrapped callable back to its type. No behavior change
* fix(proxy): keep request bodies off the model list span and tidy its tests
The quiet span wrapper now skips the request body as well as the
response, so no payload reaches the span on success or failure. The
wrapper passes the endpoint's kwargs through instead of copying them,
and reads the caller's key with an isinstance check. The model list
test drives the real endpoint without swapping helpers, and the new
payload test checks both wrappers against a real request body
* fix(proxy): keep the 5xx status on the model list span and trace model retrieve
The span wrapper ended the server span on every failure, so a 500 or 503
from /v1/models lost its status code. It now leaves that span to the
proxy's exception handler, which knows the answered status, and only ends
it itself for an HTTPException, which no handler closes on these routes.
GET /v1/models/{model_id} (the OpenAI SDK's models.retrieve) gets the same
wrapper, so it joins the caller's trace like the listing does.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| data | ||
| __init__.py | ||
| _helpers.py | ||
| conftest.py | ||
| test_otel_admin_endpoints.py | ||
| test_otel_exception_handler.py | ||
| test_otel_passthrough_endpoints.py | ||
| test_otel_unified_endpoints.py | ||
| test_passthrough_parent_span.py | ||