mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-25 05:54:29 +00:00
c98e8decd4
Adversarial review of the previous commit surfaced three defects:
- Cached tokens were double counted: OpenAI's prompt_tokens includes
cached tokens while Anthropic's input_tokens excludes them (total
input = input_tokens + cache_creation_input_tokens +
cache_read_input_tokens). The converter passed the inclusive count
through as input_tokens while also emitting the cache fields, so a
{prompt_tokens: 1000, cached_tokens: 800} response claimed 1800 input
tokens to Anthropic clients. Cache counts are now subtracted whenever
the input count is derived from prompt_tokens; an Anthropic-native
input_tokens key is taken as already exclusive.
- A model with stream_response=Off flips the stream flag after the
payload conversion injected stream_options, and upstreams like OpenAI
and vLLM reject stream:false requests carrying stream_options with
HTTP 400. The OpenAI router now drops stream_options from any
non-streaming payload before sending; this also covers the
pre-existing variant where the web UI sends stream_options for models
with the usage capability enabled.
- The streaming usage capture lacked the null/zero hardening the
non-streaming path already had: explicit null token values leaked
JSON null into message_delta usage (breaking schema-validating
Anthropic SDK clients), and a reported 0 (e.g. Ollama omitting
prompt_eval_count for KV-cached prompts) clobbered the pre-counted
input_tokens. Both paths now share one hardened extraction helper
that treats missing/null/0 counts as unreported.
Ref: https://github.com/open-webui/docs/issues/1328
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0119K4sbiCAFKq9rFaS6i5ij
|
||
|---|---|---|
| .. | ||
| data | ||
| open_webui | ||
| .dockerignore | ||
| .gitignore | ||
| dev.sh | ||
| requirements-min.txt | ||
| requirements.txt | ||
| start.sh | ||
| start_windows.bat | ||