mirror of
https://github.com/fabro-sh/fabro.git
synced 2026-08-28 05:27:41 +00:00
e5f0290229
Anthropic prompt caching is opt-in per request: without explicit ephemeral cache_control breakpoints in the body, no cache writes or reads ever happen. The OpenAI-compatible codec never emitted them, so every run on openrouter Claude models billed the full conversation at the uncached input rate on every turn (0 cache tokens on the billing page, confirmed by OpenRouter's activity portal). - Add a `cache_control_breakpoints` model feature declaring that a route only caches when the request marks the cacheable prefix; set it on the builtin OpenRouter Claude rows. Catalog build rejects the flag without `prompt_cache`. - Teach the Chat Completions wire shape a parts-form content variant so a message can carry the annotation; unmarked messages keep the plain-string form for compatibility with strict servers. - Mark the last system message (covers tools + system upstream) and the second-to-last user turn, counting tool results as user turns — mirroring the anthropic codec's placement so agent loops get incremental cache hits. - Extract the shared placement/opt-out policy into codec::cache and refactor the anthropic codec onto it; anthropic wire snapshots are unchanged. - Honor `provider_options.<name>.auto_cache = false` as an opt-out and consume the control key instead of merging it into the body. - Mirror the new feature through settings (fabro-config), the OpenAPI schema, and the generated TypeScript client. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| brainstorms | ||
| ideation | ||
| internal | ||
| plans | ||
| public | ||
| superpowers | ||