mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-08 03:08:45 +00:00
761ab19209
* fix(bedrock): sanitize batch metadata to prevent Pydantic ValidationError Proxy guardrail hooks (Model Armor, OpenAI Moderations) and internal processing inject non-string values (dicts, floats) into the request metadata. When the Bedrock batch handler passes this metadata directly to LiteLLMBatch (which inherits OpenAI's Batch Pydantic model with metadata: Dict[str, str]), Pydantic raises a ValidationError. This causes the router retry loop to re-submit the same Bedrock job multiple times before ultimately failing. Add _get_openai_compatible_batch_metadata() that serializes non-string values to JSON strings via safe_dumps, skips None values and internal logging keys, ensuring the response object always validates. * test(bedrock): add tests for batch metadata sanitization Covers _get_openai_compatible_batch_metadata: string passthrough, dict/float serialization, None/internal key exclusion, and LiteLLMBatch compatibility. --------- Co-authored-by: Noah Nistler <60981020+noahnistler@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| test_batch_metadata_sanitization.py | ||
| test_handler.py | ||