scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 115

docs(ui): state that Anthropic prompt caches are shared per upstream credential

Browse source 
The provider caches a prefix against the credentials that sent it, not per end user, so
turning the flag on makes every caller's prompts cacheable on that shared account. Surface
that where the toggle is, since it is the operator's call to make.
This commit is contained in:
Tin Chi Lo 2026-07-16 15:54:53 -07:00
parent 9f7f53a82a
commit 16e39542a0
1 changed files with 5 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
litellm/proxy/proxy_server.py
View file

@ -14826,7 +14826,11 @@ _GENERAL_SETTINGS_UI_LITELLM_FIELDS: dict[str, GeneralSettingsUILiteLLMFieldSpec
"Automatically add Anthropic cache_control breakpoints to the system prompt and the "
"trailing turn, for Anthropic and Bedrock Claude models that support prompt caching. "
"Lets clients that never set cache_control themselves still get cached prompts. "
"Requests that already carry their own cache_control are left untouched."
"Requests that already carry their own cache_control are left untouched. "
"The provider caches a prefix against the upstream credentials that sent it, not per "
"end user, so this makes every caller's prompts cacheable on that shared account. "
"Leave this off if callers sharing a set of credentials must not learn whether "
"another caller recently sent a given prompt."
),
},
"anthropic_prompt_caching_ttl": {