mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
e99151bb95
* fix(guardrails): stop Generic Guardrail API 500 on built-in tools
Requests carrying built-in tools (code_interpreter, file_search, ...) crashed
the Generic Guardrail with a 500. GenericGuardrailAPIRequest.tools validated
each tool against ChatCompletionToolParam, whose base TypedDict requires a
function block, so a tool like {"type": "code_interpreter"} raised a Pydantic
ValidationError before the request was ever sent.
Type the field with a permissive GuardrailToolParam model (type required,
extra=allow) so built-in tools validate and their config is forwarded to the
guardrail intact instead of being stripped.
* feat(guardrails): add complete fail-open (fail_on_error) to Generic Guardrail
The Generic Guardrail already honored unreachable_fallback, which fails open
only on network-unreachable errors. This wires up the existing generic
fail_on_error config (so far implemented only by Model Armor) so that
fail_on_error=false degrades any guardrail error to a critical-log warning and
lets the request proceed as if the guardrail were absent.
Only a valid guardrail response can act: a parsed BLOCKED decision still raises,
while endpoint errors, malformed responses, and internal serialization or
validation errors all fall through when fail_on_error=false. To cover that last
class, the request construction now runs inside the protected block, so the kind
of validation error that previously surfaced as a 500 is caught here too.
Defaults to true (fail closed), matching today's behavior; turning it off is an
explicit availability-over-security choice and is logged at critical level on
every bypass.
* test(guardrails): cover fail_on_error on the response path
The existing fail_on_error tests all drive the request path. Add response-path
(input_type=response) coverage: an endpoint error proceeds unchanged under
fail_on_error=false, and a valid BLOCKED decision still raises. Guards against a
future regression that special-cases input_type in the error handling.
* style(guardrails): black-format the fail-open guard expression
CI runs black (line-length 88) over litellm/; the unreachable_fail_open
assignment exceeded it. Wrap it to satisfy the formatter.
* fix(guardrails): validate tools into GuardrailToolParam at the call site
Changing the request field to List[GuardrailToolParam] left the construction
passing List[ChatCompletionToolParam] (list is invariant), which tripped the
basedpyright reportArgumentType budget gate. Validate each tool explicitly,
which is what Pydantic did implicitly, so the types line up with no Any or
suppression and the serialized payload is unchanged.
* fix(guardrails): make fail-open log message accurate for non-network errors
The fail-open path is now shared by fail_on_error, so it fires for any guardrail
error, not just unreachability. The log said 'unreachable' even for an HTTP 400
or a malformed response; reword to 'error' (the status code and exception are
already logged). Addresses the Greptile review's only finding.
* fix(guardrails): align GenericGuardrailAPIResponse.tools with GuardrailToolParam
Greptile flagged that the request side moved to GuardrailToolParam but the
response side still annotated tools as List[ChatCompletionToolParam], which
mandates a function block and contradicts the new built-in-tools support.
Update the response annotation (and the now-unused import) so the two sides
agree. Runtime is unchanged; from_dict stores the raw dicts and the only
consumer assigns through to GenericGuardrailAPIInputs without inspecting
the elements.
---------
Co-authored-by: Itay Ovadia <itay@sun.security>
|
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_config.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_entrypoint.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_passthrough_endpoints.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.