docs: add include_cost_in_streaming_usage docs snippet

This commit is contained in:
Saumyaranjan Nayak 2026-07-10 21:52:09 +05:30
parent abe3289398
commit 8a24c9b2fa

View file

@ -0,0 +1,54 @@
## include_cost_in_streaming_usage
Enable this setting to include the final request cost in streaming responses.
Summary
- Purpose: when `litellm_settings.include_cost_in_streaming_usage` is `true`, the proxy will inject the final usage/cost object into the last streaming chunk when the client requests `stream_options.include_usage: true`.
How to enable
In your proxy config (example):
```yaml
litellm_settings:
include_cost_in_streaming_usage: true
```
How to request usage in streaming calls
When making a streaming request to a provider-compatible endpoint, set:
```json
{"stream_options": {"include_usage": true}}
```
Behavior notes
- This setting only affects streaming responses when both the global flag (`litellm_settings.include_cost_in_streaming_usage`) and the per-request `stream_options.include_usage` are enabled.
- For non-streaming requests the `x-litellm-response-cost` header remains available as before.
- The injected usage object contains the same `usage.cost` breakdown used for non-streaming responses (input/output/cache/discounts) and will appear in the final stream chunk's `usage` field.
Example final stream chunk (simplified):
```json
{
"id": "chunk-final",
"object": "chat.completion.chunk",
"choices": [],
"usage": {
"prompt_tokens": 12,
"completion_tokens": 6,
"cost": 0.000095,
"cost_breakdown": {
"input": 0.000060,
"output": 0.000035,
"discount": 0.000000
}
}
}
```
Where to contribute
- The public docs site (`litellm-docs`) hosts user-facing documentation. Add this snippet to the cost-tracking or streaming usage page in `litellm-docs` and link back to the `x-litellm-response-cost` docs if relevant.
Notes for PR authors
- Keep the entry short and add a link to the streaming usage examples/tests if needed. No code changes required in the core repo.