mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-03 07:23:33 +00:00
02ef44c388
* fix(streaming): normalize provider logprobs to prevent MockValSer serialization crash * fix(streaming): convert upstream chunk logprobs to litellm ChoiceLogprobs so the finish chunk serializes A streaming chat completion with logprobs on the OpenAI SDK path attached the SDK's own ChoiceLogprobs object to the outgoing chunk when the chunk had no content to rebuild from, which is the provider's finish chunk (empty delta, logprobs for the end token, finish_reason). The SDK parses stream chunks without building that nested class, so its pydantic serializer is still a MockValSer and model_dump_json fails. The proxy then sent an error frame in place of the finish chunk, and when the same chunk also carried usage the SDK wrapper itself raised MidStreamFallbackError. The attach site now validates upstream logprobs through a TypeAdapter that dumps any non-litellm pydantic model before validating into litellm's ChoiceLogprobs, and passes litellm instances through unchanged. * fix(streaming): keep upstream logprobs that don't fit ChoiceLogprobs as plain JSON so the stream still finishes * fix(streaming): build ChoiceLogprobs' serializer up front and keep nonconforming logprobs as JSON on chunk rebuilds ChoiceLogprobs inherits the openai SDK's deferred build, so a fresh worker whose first logprobs arrive on the finish chunk still had no serializer. Foreign logprobs models now dump in full, so a missing top_logprobs becomes [] and SDK callers keep attribute access, and chunk rebuilds for content and usage keep logprobs that don't fit as plain JSON instead of raising. * fix(streaming): keep only the logprobs keys upstream sent and type refusal logprobs * fix(types): pass nonconforming non-streaming logprobs through instead of failing the response --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| .agents/skills/url-state | ||
| public | ||
| scripts | ||
| src | ||
| tests | ||
| vendor | ||
| .env.development | ||
| .env.production | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| AGENTS.md | ||
| build_release_ui.sh | ||
| build_ui.sh | ||
| build_ui_custom_path.sh | ||
| components.json | ||
| eslint-budgets.json | ||
| eslint-suppressions.json | ||
| eslint.config.mjs | ||
| knip.json | ||
| next.config.mjs | ||
| package-lock.json | ||
| package.json | ||
| postcss.config.js | ||
| README.md | ||
| tsconfig.json | ||
| tsconfig.production.json | ||
| vitest.config.ts | ||
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.