Replace list(set(...)) dedupe with dict.fromkeys so callback insertion
order is preserved deterministically instead of being randomized by set
iteration order (influenced by PYTHONHASHSEED). Applies to both the
Logging and ProxyLogging implementations.
Fixes#33003
Co-authored-by: Yassin Kortam <yassin@berri.ai>
When a guardrail blocked a request through a flow-builder policy pipeline, the
proxy discarded the guardrail's own exception and synthesized a generic
guardrail_pipeline_error response, so the same guardrail produced a different
HTTP response and trace span depending on whether it was attached directly or
via a policy. The pipeline now carries the guardrail's original exception and
re-raises it verbatim on block, enriching it with the blocking guardrail's name
and mode exactly as the direct path does, so the two attachment methods are
indistinguishable to clients and tracing. The generic pipeline error remains
only as a fallback for blocks with no underlying exception (e.g. a guardrail
that could not be found).
Resolves LIT-4041
litellm_guardrail_latency_seconds was only emitted for pre-call guardrails.
during_call_hook and post_call_success_hook ran guardrails without recording
any latency, so during-call and post-call guardrail time was invisible in the
metric and leaked into litellm_overhead_latency_metric, making the documented
"subtract guardrail latency from overhead" workaround under-report total
guardrail time.
Extract the find-the-PrometheusLogger-and-record step into _emit_guardrail_metrics
and add _run_guardrail_with_metrics, a single wrapper that times a guardrail
coroutine, classifies its outcome (success / intervened / error), enriches any
raised HTTPException, and records the latency under the given hook_type. Route
the pre-call emit, during_call_hook, and post_call_success_hook through it so
every guardrail phase contributes to the metric the same way.
Resolves LIT-3999
* test(proxy/utils): pin ProxyLogging behavior
Add behavior-pinning tests for the ProxyLogging cluster in
litellm/proxy/utils.py under tests/test_litellm/proxy/utils/proxy_logging/.
Covers InternalUsageCache, _CallbackCapabilities, top-of-file helpers
(print_verbose, _get_email_logger_class, _accepts_litellm_call_info,
_enrich_http_exception_with_guardrail_context), the full ProxyLogging
class (lifecycle, MCP-LLM bridging, capability probes, guardrail
pipeline, pre/during/post/streaming hooks, alerting), plus the
bottom-of-region helpers (on_backoff, jsonify_object, _lookup_deprecated_key).
Each pinned symbol has happy-path and error-path coverage; happy paths
use direct dict-equality with three or more keys (or HiddenParams /
Pydantic model_validate where the surface is a Pydantic shape). The
subdirectory carries a local _pin_check.py and _coverage_check.py that
enforce the gate without surfacing numeric thresholds in CI logs.
Wires tests/test_litellm/proxy/utils into the existing test-path block
in .github/workflows/test-unit-proxy-endpoints.yml.
* test(proxy/utils): drop unused mock_httpx_client fixture
Declared in conftest.py but never referenced by any test. Removing
the dead fixture per Greptile P2 feedback.
* test(proxy/utils): drop local-only gate scripts from PR
_pin_check.py and _coverage_check.py are local stopping signals (not
wired into CI, consume a gitignored .pin_list.txt). They served their
purpose telling the engineer when to stop writing tests; the pytest
suite is the artifact that belongs in the repo.
---------
Co-authored-by: Claude <noreply@anthropic.com>
