ReMe/reme/components/embedding_store
北沐城歌 1ea932c80a
fix(embedding): recover over-limit items with a bounded input-shortening retry (#655)
A per-item context-length rejection survives the per-item split: the item is resent unchanged and the provider keeps rejecting it, so the chunk never gets a vector and a scoped reindex can never finalize. Re-prepare that single item under a smaller character budget and retry it once, bounded by max_retries.

Only the request shrinks. The chunk keeps its full source text, the vector space is unchanged, and the shortening is logged so the reduced coverage is visible.

Shortening needs an attempt left to send. On the final attempt a shorter request cannot be delivered, so that branch now falls through to the existing input-dependent handling: the rejection stays specific to its item instead of being recorded as a terminal batch failure that drops the healthy items after it. The retry lives inside the input-dependent branch, so the same error is classified once, and the single-use budget helper is inlined.

Follow-up to the merged per-item isolation change (#579), rebased onto the retry-result refactor (#656).
2026-10-23 11:50:16 +08:00
..
__init__.py init: reme version 0.4.0 (#284) 2026-06-22 15:41:19 +08:00
_embedding_types.py refactor(embedding): simplify retry results and error classification (#656) 2026-10-21 21:15:27 +08:00
base_embedding_store.py fix(embedding): recover over-limit items with a bounded input-shortening retry (#655) 2026-10-23 11:50:16 +08:00
local_embedding_store.py fix(embedding): recover over-limit items with a bounded input-shortening retry (#655) 2026-10-23 11:50:16 +08:00