mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-25 05:54:29 +00:00
9cda762af8
Context compaction failed to trigger for llama.cpp backends with prompt caching enabled, letting chats grow until the server rejected the request with a context overflow error. Two issues combined to cause this: 1. load_messages_from_db() dropped the usage field when reconstructing messages from the DB, so _exceeds_token_threshold() never saw any reported token counts and always fell back to rough char-based estimation. 2. The threshold check summed only input_tokens + output_tokens. llama.cpp reports cached prefix tokens separately (timings.cache_n, merged into the stored usage) and excludes them from prompt_n, so a mostly cached context of ~30k tokens was counted as only a few thousand. Keep usage on DB-loaded messages so the threshold check can use it, count cache_n towards the total, and strip usage in strip_compaction_fields() after compaction runs so the non-standard field never reaches the LLM payload. Reported and verified in https://github.com/open-webui/open-webui/discussions/24410 |
||
|---|---|---|
| .. | ||
| data | ||
| open_webui | ||
| .dockerignore | ||
| .gitignore | ||
| dev.sh | ||
| requirements-min.txt | ||
| requirements.txt | ||
| start.sh | ||
| start_windows.bat | ||