mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-25 05:54:29 +00:00
9cda762af8
Context compaction failed to trigger for llama.cpp backends with prompt caching enabled, letting chats grow until the server rejected the request with a context overflow error. Two issues combined to cause this: 1. load_messages_from_db() dropped the usage field when reconstructing messages from the DB, so _exceeds_token_threshold() never saw any reported token counts and always fell back to rough char-based estimation. 2. The threshold check summed only input_tokens + output_tokens. llama.cpp reports cached prefix tokens separately (timings.cache_n, merged into the stored usage) and excludes them from prompt_n, so a mostly cached context of ~30k tokens was counted as only a few thousand. Keep usage on DB-loaded messages so the threshold check can use it, count cache_n towards the total, and strip usage in strip_compaction_fields() after compaction runs so the non-standard field never reaches the LLM payload. Reported and verified in https://github.com/open-webui/open-webui/discussions/24410 |
||
|---|---|---|
| .. | ||
| data | ||
| internal | ||
| migrations | ||
| models | ||
| retrieval | ||
| routers | ||
| socket | ||
| static | ||
| storage | ||
| tools | ||
| utils | ||
| __init__.py | ||
| alembic.ini | ||
| config.py | ||
| constants.py | ||
| env.py | ||
| events.py | ||
| functions.py | ||
| main.py | ||
| tasks.py | ||