* feat(hosted_vllm): keep vLLM's stop_reason and token_ids on chat choices
Non-streamed hosted_vllm chat completions moved vLLM's choice-level
stop_reason and token_ids into provider_specific_fields, so clients
written against vLLM could not find them where vLLM puts them. They are
now also set on the choice itself, and the copy in
provider_specific_fields stays for existing readers.
Fixes#47945
* feat(hosted_vllm): keep stop_reason and token_ids on streamed chunks and proxy stream output
Streamed chat finish chunks and streamed /v1/completions finish chunks now carry vLLM's stop_reason. Each stream chunk keeps the token_ids vLLM sent with it, including held-back empty chunks, and the proxy no longer strips those chunks as empty usage chunks or drops the fields in its fast serializer.
* fix(hosted_vllm): keep the upstream choice index on token-id-only stream chunks
* fix(hosted_vllm): keep vLLM choice fields for null, cache hits, n>1, and streamed completions
Response conversion keeps stop_reason and token_ids on the choice for any OpenAI compatible response, cache replays carry them, finish fields are recorded per choice index, text completion streams keep token_ids, and the proxy passes only these two keys through, nulls included.
* fix(hosted_vllm): send empty text on token-id chunks, keep fields on their own choice, and keep null stop_reason on slow-path streams
Token-id-only chunks now carry an empty content or text string, a finish chunk only takes stop_reason and token_ids from its own upstream choice, content chunks no longer repeat stop_reason, the slow stream serializer keeps a null stop_reason or token_ids, and fields cached under provider_specific_fields come back on the choice
* fix(hosted_vllm): keep stop_reason on the last content chunk, keep both fields in built stream responses, and key the cache on return_token_ids
* fix(hosted_vllm): keep another choice's final token_ids and stop_reason on their own stream chunk
With n above 1, a finish-only vLLM chunk for a choice other than the one the single finish chunk
carries was dropped or taken as choice 0's finish, losing that choice's last token_ids. It now goes
out as an id-only chunk on its own choice index.
* fix(hosted_vllm): send the first choice's finish chunk before another choice's later token-id chunks
* fix(streaming): keep each streamed text completion choice on its own index
* fix(streaming): finish each streamed text completion choice on its own index and null token_ids in redacted logs
* fix(proxy): drop token_ids from guarded streams, redact choice-level provider fields, and keep one message_delta per Anthropic stream
* fix(proxy): skip token-id-only chunks on guarded streams and put stop_reason on the final text chunk
* fix(proxy): drop token_ids from the logged and cached response of a guarded stream
* fix(proxy): drop token_ids under per-chunk stream checks and redact text-completion stream choices
* fix(caching): key return_token_ids and continuation_token sent in extra_body
* fix(streaming): drop token ids from streamed chunks while post_call_rules run
vLLM holds text back behind a possible stop string but sends its token ids at once, so an SDK stream sent ids
before litellm.post_call_rules checked their text. While those rules are set, streamed choices drop token_ids
and chunks that carry only ids are skipped, as the proxy already does ahead of a per-chunk check
* fix(streaming): keep vLLM fields when a stream's only choice index is not 0
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
