mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-25 05:54:29 +00:00
c98e8decd4
Adversarial review of the previous commit surfaced three defects:
- Cached tokens were double counted: OpenAI's prompt_tokens includes
cached tokens while Anthropic's input_tokens excludes them (total
input = input_tokens + cache_creation_input_tokens +
cache_read_input_tokens). The converter passed the inclusive count
through as input_tokens while also emitting the cache fields, so a
{prompt_tokens: 1000, cached_tokens: 800} response claimed 1800 input
tokens to Anthropic clients. Cache counts are now subtracted whenever
the input count is derived from prompt_tokens; an Anthropic-native
input_tokens key is taken as already exclusive.
- A model with stream_response=Off flips the stream flag after the
payload conversion injected stream_options, and upstreams like OpenAI
and vLLM reject stream:false requests carrying stream_options with
HTTP 400. The OpenAI router now drops stream_options from any
non-streaming payload before sending; this also covers the
pre-existing variant where the web UI sends stream_options for models
with the usage capability enabled.
- The streaming usage capture lacked the null/zero hardening the
non-streaming path already had: explicit null token values leaked
JSON null into message_delta usage (breaking schema-validating
Anthropic SDK clients), and a reported 0 (e.g. Ollama omitting
prompt_eval_count for KV-cached prompts) clobbered the pre-counted
input_tokens. Both paths now share one hardened extraction helper
that treats missing/null/0 counts as unreported.
Ref: https://github.com/open-webui/docs/issues/1328
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0119K4sbiCAFKq9rFaS6i5ij
|
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| events.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||