mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-02 07:13:02 +00:00
a084d13635
* fix(passthrough): log the URL model and declared provider on Gemini and Vertex failures Pass-through failures on Gemini and Vertex routes carry the model only in the URL, so the failure spend row and the Prometheus failure counter logged an empty model. The failure payload now falls back to the model parsed from the URL on those routes, and the Prometheus provider label honors the provider the request declares before inferring one from the model name * fix(passthrough): keep model and provider on mid-stream and Vertex failure logs Build the failure payload from the URL at both streamed relay sites so a 200 stream that dies mid-body still logs the model, derive vertex_ai for Vertex routes that never declare a provider, and have the Vertex logging handler reuse get_vertex_model_id_from_url instead of its own regex * fix(passthrough): bound the Prometheus provider label and name Vertex failures like the success handler Declared providers only reach the api_provider label when they name a registered provider, otherwise the label falls back to inference. Vertex failure rows now use the success handler's model naming (vertex_ai/ prefix on rawPredict and streamRawPredict, vertex_ai/search_api on :search) and its host-based provider detection, and the detected-stream failure sites are covered by tests. * fix(passthrough): name Google failures from the URL on untracked methods too A Gemini or Vertex pass-through failure on a method the success handler does not track (countTokens, for one) still logged model "" and no provider, although the model sits in the URL. The failure payload now falls back to the Google AI host for the provider and reads /models/<name> from the URL, so countTokens 400s name their model and provider like generateContent ones do. Non-Google hosts keep the tracked-route rule and still log nothing extra. * fix(passthrough): trust the declared Google provider and Vertex-shaped paths on custom bases The failure path only named the Google provider from the URL when the method was one the success handler tracks or the host was a Google API host, so a Gemini countTokens failure behind a custom GEMINI_API_BASE (or a Vertex one behind a custom base) still logged an empty model. A route that declares gemini or vertex_ai now wins outright, and a URL whose path carries /projects/<p>/locations/<l>/ counts as Vertex on any host, while a tracked method on a non-Google host no longer passes as Vertex. * fix(pass_through): stop the URL model parser at the query string so a configured key never lands in a spend row's model * test(pass_through): run the URL-model failure tests through the real ProxyLogging with a recording callback * fix(passthrough): scope declared provider to pass-through failures and match Google hosts only Prometheus trusts the resolved provider as before and reads the request-declared provider only on pass-through failures. Vertex attribution matches Google hosts, strips operation poll suffixes, prefixes vertex_ai/ only for rawPredict, reads the URL path only, covers Vertex Live WebSocket failures, and labels a router alias failure with the alias the client requested * fix(passthrough): attribute failures from the URL only on built-in Google routes and keep other routes' metric labels A user-defined pass-through route to a Google host keeps its empty model and provider, since its URL names an upstream model rather than a LiteLLM-managed one. Prometheus failure labels read the route's provider and model_group only for gemini and vertex_ai, so Azure, OpenRouter, Bedrock and other pass-through routes keep their earlier labels. The Gemini URL parser drops the query string, Google host matching needs a dot or dash boundary, and the Vertex Claude prefix follows alt=sse like the success handler. * fix(passthrough): log the URL model on guardrail blocks and the named model on Vertex Live failures A pre-call guardrail block on a pass-through route now builds its failure payload the same way upstream failures do, so the model and provider read from the URL reach spend logs and metrics. Vertex Live websocket failures (connect errors, upstream rejections, and the credential failure) now carry the model from the ?model= query parameter under the vertex_ai provider * fix(passthrough): mark guardrail-block and Vertex Live credential failures as pass-through calls Prometheus reads a failure's declared provider only on pass-through calls, so without call_type it fell back to guessing from the model name. That labeled a guardrail block on the Gemini route as vertex_ai and the Live credential failure as None * fix(passthrough): let the route name a failure's provider and give guardrail blocks their own spend row * fix(pass-through): keep URL model attribution off routes that name another provider A built-in route that names a non-Google provider, like /openai pointed at Gemini's OpenAI-compatible host, no longer takes its failure model from the Google URL. A guardrail block response now returns the x-litellm-call-id of its spend row. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| llm_provider_handlers | ||
| __init__.py | ||
| test_carry_guardrail_logging_info.py | ||
| test_codex_passthrough_endpoints.py | ||
| test_deepgram_ws_passthrough_routes.py | ||
| test_injected_stream_usage.py | ||
| test_llm_pass_through_endpoints.py | ||
| test_llm_passthrough_endpoints.py | ||
| test_managed_id_rewriter.py | ||
| test_method_specific_routing.py | ||
| test_pass_through_endpoints.py | ||
| test_passthrough_auth_default.py | ||
| test_passthrough_endpoint_router.py | ||
| test_passthrough_endpoints_common_utils.py | ||
| test_passthrough_guardrail_block_otel_span.py | ||
| test_passthrough_guardrails.py | ||
| test_passthrough_guardrails_field_targeting.py | ||
| test_passthrough_post_call_guardrails.py | ||
| test_streaming_handler.py | ||
| test_streaming_handler_interrupt.py | ||
| test_success_handler.py | ||
| test_upstream_usage_headers.py | ||
| test_vertex_ai_batch_passthrough.py | ||
| test_vertex_ai_live_passthrough.py | ||
| test_vertex_passthrough_load_balancing.py | ||
| test_watsonx_proxy_route.py | ||