Commit graph

4 commits

Author SHA1 Message Date
Shivam Rawat
fbff60e9d9
fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425)
* fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata

Batch create was failing with `Invalid type for 'metadata.applied_policies':
expected a string, but got an array instead` whenever a policy attachment
matched the request. The policy engine helpers wrote `applied_policies`,
`applied_guardrails`, and `policy_sources` into `data["metadata"]`
unconditionally, and `/v1/batches` forwarded that dict straight to OpenAI,
which only accepts string values.

- Route proxy-internal tracking into `litellm_metadata` for batch/file
  routes via a shared `_get_or_create_proxy_metadata_bucket` helper.
- Sanitize `data["metadata"]` in `create_batch` to drop known internal
  keys and non-string values before building the OpenAI request.
- Cover both behaviors with unit + endpoint tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): merge metadata buckets for batch policy response headers

Ensure get_logging_caching_headers reads both metadata and litellm_metadata so policy/guardrail headers are emitted on batch routes with user metadata, and log dropped non-string OpenAI metadata at debug level.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 11:35:42 -07:00
Ryan Crabbe
52ec73c07d fix(proxy): improve team expiry enforcement validation
- Change status codes from 400 to 500 for team metadata misconfig errors
  (callers can't fix admin-set config, 400 is misleading)
- Add anchor value validation to batch endpoint (matching files endpoint)
- Coerce seconds to int to handle string values from metadata
- Add error-path tests: missing keys, invalid anchor, status code assertions
- Add happy-path test: team injects expiry when caller sends nothing
2026-03-03 17:29:39 -08:00
Ryan Crabbe
3d15bcdb11 feat(proxy): add team-level batch output expiry enforcement 2026-03-03 10:58:31 -08:00
Ryan Crabbe
43cec8c980 feat(batches): support output_expires_after passthrough 2026-03-03 10:46:45 -08:00