mirror of
https://github.com/open-webui/open-webui.git
synced 2026-10-25 05:54:29 +00:00
9cda762af8
Context compaction failed to trigger for llama.cpp backends with prompt caching enabled, letting chats grow until the server rejected the request with a context overflow error. Two issues combined to cause this: 1. load_messages_from_db() dropped the usage field when reconstructing messages from the DB, so _exceeds_token_threshold() never saw any reported token counts and always fell back to rough char-based estimation. 2. The threshold check summed only input_tokens + output_tokens. llama.cpp reports cached prefix tokens separately (timings.cache_n, merged into the stored usage) and excludes them from prompt_n, so a mostly cached context of ~30k tokens was counted as only a few thousand. Keep usage on DB-loaded messages so the threshold check can use it, count cache_n towards the total, and strip usage in strip_compaction_fields() after compaction runs so the non-standard field never reaches the LLM payload. Reported and verified in https://github.com/open-webui/open-webui/discussions/24410 |
||
|---|---|---|
| .. | ||
| access_control | ||
| images | ||
| mcp | ||
| telemetry | ||
| actions.py | ||
| anthropic.py | ||
| asgi_middleware.py | ||
| audit.py | ||
| auth.py | ||
| automations.py | ||
| calendar.py | ||
| channels.py | ||
| chat.py | ||
| code_interpreter.py | ||
| context_compaction.py | ||
| embeddings.py | ||
| files.py | ||
| filter.py | ||
| groups.py | ||
| headers.py | ||
| logger.py | ||
| memory.py | ||
| middleware.py | ||
| misc.py | ||
| models.py | ||
| oauth.py | ||
| payload.py | ||
| pdf_generator.py | ||
| plugin.py | ||
| rate_limit.py | ||
| redis.py | ||
| response.py | ||
| sanitize.py | ||
| security_headers.py | ||
| session_pool.py | ||
| task.py | ||
| tools.py | ||
| validate.py | ||
| valves.py | ||
| webhook.py | ||