scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-28 01:32:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 43
litellm/tests/unit/integrations
History
devin-ai-integration[bot] 26bf575f15
feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 
* feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails

Vector store retrieval runs inside acompletion after the proxy's pre-call
guardrails have already seen the request, so a retrieved chunk carrying an
injection reached the prompt unscanned. Each retrieved context message now
goes through every pre-call guardrail the request is subject to before it
is injected: a block raises the same 400 the guardrail gives for user text,
a masking guardrail rewrites the context, and a guardrail that fails while
scanning fails the request instead of injecting the chunk unscanned

* fix(guardrails): return a guardrail block unmapped from exception_type so the Responses API surfaces the guardrail's own 400

* fix(guardrails): build the deployment hooks' identity from stamped metadata only

Top-level user_api_key_* fields in a request body are client controlled, so the pre-call, chunk scan, and post-call deployment hooks now take UserAPIKeyAuth from the metadata the proxy stamped, and the chunk scanner returns or raises on every branch.

* fix(guardrails): block route verdicts on retrieved chunks, keep guardrail verdicts out of router retries and fallbacks, and scan chunks against the client's request

* fix(guardrails): keep the merged guardrail list when scanning chunks against the client's request

The scan request laid the client's kept body over the deployment kwargs, so a client
that sent its own top-level guardrails list shadowed the merged metadata.guardrails
list and a key or team guardrail skipped the chunk scan. The kwargs now win and the
keys the proxy relocates into metadata are dropped from the body's contribution.

* fix(guardrails): strip the deployment's guardrail keys from the chunk scan request so merged team guardrails still run

* test(guardrails): type the vector store scan test doubles

* test(guardrails): type the scan double's request data

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-26 14:18:38 -07:00
..
arize test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
azure_storage test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
bitbucket test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
cloudzero test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
code_interpreter_interception test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
compression_interception test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
datadog test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
dotprompt test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
focus test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
gcs_bucket test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gcs_pubsub test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gitlab test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
helicone test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
langfuse test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
levo test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
litellm_agent test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
mavvrik_focus test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
newrelic test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
open_telemetry test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
opik test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
otel fix(otel): detach post-response service spans by request phase, name redis spans by operation (#43237) 2026-09-26 10:10:12 -07:00
pointfive test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
SlackAlerting feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
vector_store_integrations feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
websearch_interception refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
zerobus test(zerobus): move tests into active CI selection (#43235) 2026-09-25 23:35:15 +00:00
__init__.py test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
conftest.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
rubrik_test_helpers.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_agentops.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_anthropic_cache_control_hook.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_athina.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_azure_sentinel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_logging.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_span_name.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_guardrail.py feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
test_custom_guardrail_recursion.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_prompt_management.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_deepeval.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_galileo.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_guardrail_logging_sync.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_helicone.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langfuse.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langfuse_otel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langsmith_init.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_lunary.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_mlflow.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_openmeter.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opentelemetry.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opentelemetry_dynamic_imports.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opik_utils.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_guardrail_violation_spans.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_team_attributes_matrix.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_api_promql_escape.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metric_guard.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_db_lookups.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_timeout.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_cache_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_caller_identity.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_carried_budget_state.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_client_ip_user_agent.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_custom_metadata_label_counts.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_deployment_state_proxy_rejects.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_end_user_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_input_sequence_length_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_invalid_key_filtering.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_labels.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_mcp_tool_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_media_generation_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metric_name_consistency.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metrics_endpoint.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_missing_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_none_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_overhead_with_guardrails.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_queue_guardrail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_rate_limit_labels.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_remaining_tokens_router_fallback.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_requested_model_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_service_tier_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_services.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_capture_rate.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_logs_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_stream_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_token_detail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_user_team_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_zero_cost_metric.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prompt_manager_ssti.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_responses_background_cost.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_rubrik.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_s3.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_s3_v2.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_shadow_eval_logger.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_weave_otel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00