mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-10 03:28:53 +00:00
9540f19e38
* feat(proxy): add fail_closed_rate_limit_enforcement to reject requests with 503 while Redis rate limit counters are unreachable * fix(proxy): reject fail-closed rate limit checks before logging the in-memory fallback and pin the boot warning in the lifespan * fix(proxy): coerce the fail-closed flag, fail closed on read-only checks, and refund partial cluster increments * fix(proxy): window-guard rate limit refunds and catch the fail-closed rejection by type * fix(proxy): read the compaction rate-limit gate's limiter from the proxy hook registry * fix(proxy): count the pending request in read-only rate-limit checks and keep the compaction gate off the caller's parallel slot The compaction polyfill's summary-model gate, once it ran against the real v3 limiter, showed two behaviors nobody had chosen. The read-only check compared the stored counter with the same `>` the increment path uses, but a read-only check decides a request that has not been counted yet, so a summary model exactly at its rpm limit still went out. The read-only path now adds the pending increment of 1 before comparing; the increment path is unchanged. The gate also passed the key's max_parallel_requests gauge through, and the read-only gauge count includes the caller's own in-flight slot, so a key with max_parallel_requests: 1 never compacted. The gate now drops that gauge from its descriptors, since the summary call runs inside a request the limiter already admitted. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| .env.development | ||
| .env.production | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| AGENTS.md | ||
| build_release_ui.sh | ||
| build_ui.sh | ||
| build_ui_custom_path.sh | ||
| components.json | ||
| eslint-budgets.json | ||
| eslint-suppressions.json | ||
| eslint.config.mjs | ||
| knip.json | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.js | ||
| README.md | ||
| tsconfig.json | ||
| tsconfig.tsbuildinfo | ||
| vitest.config.ts | ||
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.