mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
9451a72e89
* fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425) * fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata Batch create was failing with `Invalid type for 'metadata.applied_policies': expected a string, but got an array instead` whenever a policy attachment matched the request. The policy engine helpers wrote `applied_policies`, `applied_guardrails`, and `policy_sources` into `data["metadata"]` unconditionally, and `/v1/batches` forwarded that dict straight to OpenAI, which only accepts string values. - Route proxy-internal tracking into `litellm_metadata` for batch/file routes via a shared `_get_or_create_proxy_metadata_bucket` helper. - Sanitize `data["metadata"]` in `create_batch` to drop known internal keys and non-string values before building the OpenAI request. - Cover both behaviors with unit + endpoint tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(proxy): merge metadata buckets for batch policy response headers Ensure get_logging_caching_headers reads both metadata and litellm_metadata so policy/guardrail headers are emitted on batch routes with user metadata, and log dropped non-string OpenAI metadata at debug level. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> * fix(model-edit): allow clearing custom pricing on wildcard models (#28719) * fix(model-edit): allow clearing custom input/output cost on wildcard deployments A user-set pricing override on a `/model/*` wildcard deployment could not be removed: clearing the Input/Output Cost fields in the UI succeeded visually, but the next read still showed the old values because both `litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`) retained the original rates. UI: when the pricing field is touched but left empty, send `null` instead of dropping it from the payload so the backend sees the clear intent. The cache-read-cost fallback now guards against `null` as well as `undefined` so a cleared input cost cannot silently wipe the cache-read override. Backend: `update_db_model` honors explicit-null clears, but ONLY for `SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the null-clear path prevents a team-scoped caller from using this codepath to null out privileged fields like `team_id` or access groups. Tests cover both clear paths (`litellm_params` and `model_info`), the SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and the security guard that non-pricing nulls don't reach the merged dict. Resolves LIT-3250 * fix(model-edit): run null-clears after both merges, not interleaved The previous version cleared `model_info` from inside the litellm_params merge block, but the subsequent `model_info.update(...)` re-injected the old pricing because the UI's PATCH carries the full model_info blob with the stale values still in it. Move the explicit-null clear pass to after both merges so a model_info passthrough cannot resurrect cleared fields. Adds a regression test for the realistic UI submit shape (both blobs in the patch, model_info still holding the old pricing). * test(e2e): clear-custom-pricing flow with create/delete cleanup Covers the dashboard model edit form's pricing-clear flow end-to-end: seeds a deployment with custom input/output pricing, drives the UI to clear both fields, asserts the outgoing PATCH sends explicit nulls, and confirms via /v2/model/info that the override is gone from both litellm_params and model_info. The dashboard DB persists across this suite, so beforeEach creates a uniquely-named deployment and afterEach POSTs /model/delete to leave the DB clean regardless of test outcome. * fix(model-edit): extend pricing clear to cache_read and cache_write costs Pre-existing parallel of the wildcard input/output cost bug: cleared cache_read_input_token_cost and cache_creation_input_token_cost overrides silently persisted because the UI omitted the key (delete or fallback) and the backend null-clear allowlist did not cover them. - types/router.py: add cache_read_input_token_cost and cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are mirrored between litellm_params and model_info by Deployment.__init__ and honoured by the null-clear loop in update_db_model. - model_info_view.tsx: emit explicit null for touched-but-empty cache_read and cache_write fields. Preserve the input_cost->cache_read mirror only when cache_read itself was not touched. - model_management_endpoints.py: update the allowlist comment. - Tests: three new unit tests for cache clear paths and a preserve check; the e2e spec now seeds, clears, and asserts null PATCH + key-absence for all four pricing fields. --------- Co-authored-by: Shivam Rawat <shivam@berri.ai> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|---|---|---|
| .. | ||
| e2e_tests | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| .env.development | ||
| .env.production | ||
| .eslintrc.json | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| build_release_ui.sh | ||
| build_ui.sh | ||
| build_ui_custom_path.sh | ||
| knip.json | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.js | ||
| README.md | ||
| tailwind.config.js | ||
| tailwind.config.ts | ||
| tsconfig.json | ||
| tsconfig.tsbuildinfo | ||
| ui_colors.json | ||
| vitest.config.ts | ||
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.