mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-20 00:11:50 +00:00
342d8897b6
* fix(guardrails): walk Responses-API text taxonomy in shared content helpers
Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently
drops all text on the /v1/responses path. AIM turns it into a loud 422 (
{"error":"No messages in the request"}); every other guardrail (Lakera v2,
Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret
detection) scans an empty payload and lets the request through unscanned.
Three defects, all in _content_utils.py:
1. _iter_text_parts_in_content recognised only part.type == "text", but the
Responses API uses input_text (request) and output_text (assistant).
2. _coerce_input_to_messages gated on "every item has a role key"; any
Responses input list containing a function_call or function_call_output
item failed the check and was wrapped as one opaque blob.
3. build_inspection_messages forwarded any role through, including a bare
tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze
reject with a schema error.
Fix walks the actual Responses item taxonomy (message, function_call,
function_call_output, bare content parts and strings), recognises
{text, input_text, output_text} everywhere, and coerces any role outside
{system, user, assistant} to user in the outbound inspection payload.
* style: ruff-format changed guardrail files
* test(guardrails): cover function_call_output string form; drop em-dash in new docstring
* fix(guardrails): map function_call_output straight to user role
Avoids ever materialising a schema-invalid bare tool message. The
downstream role-safety coercion in build_inspection_messages still
guards genuinely caller-supplied non-standard roles (developer,
function, custom values); add a regression test covering that path
so the coercion has real coverage after this simplification.
* test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages
* docs(test): soften AIM-specific claims in LIT-4294 test docstrings
Ryan's review flagged that several test docstrings assert AIM's
/fw/v1/analyze validates + rejects specific schema violations. That
behavior is customer-reported in the LIT-4294 writeup, not directly
verified by us. Rephrase to attribute the AIM 422 to the customer's
writeup and describe the underlying constraint as the OpenAI chat
schema; any downstream API that validates against that schema rejects
the same shape.
* refactor(guardrails): move unsupported-role coercion into AIM only
The generic coercion in build_inspection_messages collapsed any role
outside {system, user, assistant} to user for every caller of the
helper. Combined with the pre-existing apply_redacted_messages_back
write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400
on chat-completions tool-message masking into a silent semantic
corruption of the outbound request (role tool with tool_call_id got
rewritten to bare role user, dropping the assistant + tool_calls
sibling).
AIM specifically requires the coercion because its /fw/v1/analyze
validates the payload against the OpenAI chat schema; other guardrails
either do not validate roles or do their own reconstruction. Move the
coercion to AimGuardrail._build_aim_inspection_messages so the shared
helper keeps caller roles intact and no new cross-guardrail role
corruption is introduced. The pre-existing apply_redacted_messages_back
structural flatten remains as separate follow-up work.
function_call_output items still synthesise role user in the shared
helper because they have no natural role field, which is a different
concern from coercing a caller-supplied role.
* refactor(guardrails): preserve role fidelity in shared _content_utils
Shared inspection helpers should extract text and preserve semantic
role signals; role coercion for third-party schema safety stays inside
the guardrail that needs it (AIM).
Three shared-helper changes:
- Bare content-part dicts (input_text/output_text) with an explicit role
keep it; only role-less parts default to user.
- Responses message items already had their role preserved; the
behavior is now covered by an explicit test.
- function_call_output items default to role tool (semantic equivalent
of the chat-completions tool message shape) instead of role user, so
Responses and chat completions produce symmetric inspection payloads.
A caller-supplied role on the item is still preserved.
AIM's schema-safe coercion in _build_aim_inspection_messages already
handles the resulting role tool: it collapses to user before the POST
to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the
bare tool message (no tool_call_id can survive the flatten). Added a
regression test in test_aim.py covering that path.
(cherry picked from commit
|
||
|---|---|---|
| .. | ||
| a2a_protocol | ||
| anthropic_interface | ||
| assistants | ||
| batch_completion | ||
| batches | ||
| caching | ||
| completion_extras | ||
| compression | ||
| containers | ||
| endpoints/speech/speech_to_completion_bridge | ||
| evals | ||
| experimental_mcp_client | ||
| files | ||
| fine_tuning | ||
| google_genai | ||
| images | ||
| integrations | ||
| interactions | ||
| litellm_core_utils | ||
| llms | ||
| models | ||
| ocr | ||
| passthrough | ||
| proxy | ||
| proxy_auth | ||
| rag | ||
| realtime_api | ||
| repositories | ||
| rerank_api | ||
| responses | ||
| router_strategy | ||
| router_utils | ||
| sandbox | ||
| search | ||
| secret_managers | ||
| skills | ||
| types | ||
| vector_store_files | ||
| vector_stores | ||
| videos | ||
| __init__.py | ||
| _internal_context.py | ||
| _lazy_imports.py | ||
| _lazy_imports_registry.py | ||
| _logging.py | ||
| _redis.py | ||
| _redis_credential_provider.py | ||
| _service_logger.py | ||
| _uuid.py | ||
| _version.py | ||
| anthropic_beta_headers_config.json | ||
| anthropic_beta_headers_manager.py | ||
| blog_posts.json | ||
| budget_manager.py | ||
| constants.py | ||
| cost.json | ||
| cost_calculator.py | ||
| exceptions.py | ||
| main.py | ||
| model_prices_and_context_window_backup.json | ||
| policy_templates_backup.json | ||
| provider_endpoints_support_backup.json | ||
| py.typed | ||
| router.py | ||
| scheduler.py | ||
| setup_wizard.py | ||
| timeout.py | ||
| utils.py | ||