mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-25 05:54:29 +00:00
c98e8decd4
Adversarial review of the previous commit surfaced three defects:
- Cached tokens were double counted: OpenAI's prompt_tokens includes
cached tokens while Anthropic's input_tokens excludes them (total
input = input_tokens + cache_creation_input_tokens +
cache_read_input_tokens). The converter passed the inclusive count
through as input_tokens while also emitting the cache fields, so a
{prompt_tokens: 1000, cached_tokens: 800} response claimed 1800 input
tokens to Anthropic clients. Cache counts are now subtracted whenever
the input count is derived from prompt_tokens; an Anthropic-native
input_tokens key is taken as already exclusive.
- A model with stream_response=Off flips the stream flag after the
payload conversion injected stream_options, and upstreams like OpenAI
and vLLM reject stream:false requests carrying stream_options with
HTTP 400. The OpenAI router now drops stream_options from any
non-streaming payload before sending; this also covers the
pre-existing variant where the web UI sends stream_options for models
with the usage capability enabled.
- The streaming usage capture lacked the null/zero hardening the
non-streaming path already had: explicit null token values leaked
JSON null into message_delta usage (breaking schema-validating
Anthropic SDK clients), and a reported 0 (e.g. Ollama omitting
prompt_eval_count for KV-cached prompts) clobbered the pre-counted
input_tokens. Both paths now share one hardened extraction helper
that treats missing/null/0 counts as unreported.
Ref: https://github.com/open-webui/docs/issues/1328
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0119K4sbiCAFKq9rFaS6i5ij
|
||
|---|---|---|
| .. | ||
| access_control | ||
| images | ||
| mcp | ||
| telemetry | ||
| actions.py | ||
| anthropic.py | ||
| asgi_middleware.py | ||
| audit.py | ||
| auth.py | ||
| automations.py | ||
| calendar.py | ||
| channels.py | ||
| chat.py | ||
| code_interpreter.py | ||
| context_compaction.py | ||
| embeddings.py | ||
| files.py | ||
| filter.py | ||
| groups.py | ||
| headers.py | ||
| logger.py | ||
| memory.py | ||
| middleware.py | ||
| misc.py | ||
| models.py | ||
| notifications.py | ||
| oauth.py | ||
| payload.py | ||
| pdf_generator.py | ||
| plugin.py | ||
| rate_limit.py | ||
| redis.py | ||
| response.py | ||
| sanitize.py | ||
| security_headers.py | ||
| session_pool.py | ||
| subagents.py | ||
| task.py | ||
| terminals.py | ||
| timers.py | ||
| tools.py | ||
| validate.py | ||
| valves.py | ||
| webhook.py | ||