scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-13 03:59:12 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 66
litellm/litellm
History
yuneng-jiang 10e77ce406
fix(caching): stand default cache points down when extra_body hides a direct client mark (#43342) 
* fix(caching): stand default cache points down when extra_body hides a direct client mark

On native /v1/messages the extra_body envelope is dropped, so a client tool mark or root cache_control reaches Anthropic even when extra_body overrides it. The stand-down check only counted the envelope-merged view and injected two default marks on top of the client's.

(cherry picked from commit 01ef7c4725)

* fix(caching): keep chat completions on the envelope-merged mark count for the default stand-down

Chat completions merge extra_body over the request, so a direct tool mark that extra_body replaces never reaches the provider there. Only /v1/messages, where the native transforms drop the envelope, needs to count marks on both sides.

(cherry picked from commit a9c5fa760e)
2026-09-26 16:45:26 -07:00
..
a2a_protocol fix(a2a): narrow discovery status codes through a structural protocol 2026-09-17 17:17:21 -07:00
anthropic_interface route stuff through dispatch no direct main 2026-09-17 11:06:46 -07:00
assistants
batch_completion
batches Merge remote-tracking branch 'origin/main' into litellm_mistral_ocr_batches 2026-09-18 21:51:33 -07:00
caching refactor(proxy): trim the invalidation docstrings and inject the page read failure 2026-09-16 16:41:13 -07:00
chat_completions route stuff through dispatch no direct main 2026-09-17 11:06:46 -07:00
completion_extras style(responses_bridge): apply ruff format 2026-09-16 23:52:53 +00:00
compression merge: main into litellm_headroom_protect_cached_prefix 2026-09-15 04:12:28 +00:00
containers
endpoints/speech/speech_to_completion_bridge
evals
experimental_mcp_client docs(mcp): clarify method-preserving redirect scope 2026-09-19 15:52:07 -07:00
files Merge remote-tracking branch 'origin/main' into litellm_mistral_ocr_batches 2026-09-18 21:51:33 -07:00
fine_tuning
google_genai
images feat(azure_ai): support FLUX.2 flex images 2026-09-15 12:25:19 -05:00
integrations fix(caching): stand default cache points down when extra_body hides a direct client mark (#43342) 2026-09-26 16:45:26 -07:00
interactions refactor(interactions): remove expired use_legacy_interactions_schema shim 2026-09-17 20:14:07 +00:00
litellm_core_utils fix: backport five regression fixes to rc/1.103.0 (#43331) 2026-09-26 11:47:22 -07:00
llms fix(bedrock): stream /v1/messages Invoke bytes through instead of holding them in a 1024-byte chunker (#42607) 2026-09-22 20:03:14 -07:00
messages route stuff through dispatch no direct main 2026-09-17 11:06:46 -07:00
models fix(proxy): estimate auto-router baseline costs from durable cache history 2026-09-19 12:44:47 -07:00
ocr feat(rust): add Amazon Textract to litellm.ocr and sign provider requests after host hooks 2026-09-19 09:11:52 -07:00
passthrough Merge pull request #41557 from BerriAI/litellm_azure_speech_passthrough 2026-09-18 14:05:55 -07:00
proxy fix(jwt,otel): backport session conversation id and JWT team header selection to rc/1.103.0 (#43343) 2026-09-26 15:03:25 -07:00
proxy_auth
rag fix(s3_vectors): embed registered-store ingests with the store's embedding model 2026-09-19 04:31:15 -07:00
realtime_api fix(vertex_ai): apply finals before interims and refresh the token per stream 2026-09-18 16:53:06 -07:00
repositories refactor(batches): annotate the stored file row so its model import is a real use 2026-09-19 12:39:44 -07:00
rerank_api
responses fix: backport five regression fixes to rc/1.103.0 (#43331) 2026-09-26 11:47:22 -07:00
router_strategy fix: backport five regression fixes to rc/1.103.0 (#43331) 2026-09-26 11:47:22 -07:00
router_utils fix(router): enforce model tpm limits against shared redis usage across replicas 2026-09-19 00:41:55 +00:00
rust_bridge fix: backport five regression fixes to rc/1.103.0 (#43331) 2026-09-26 11:47:22 -07:00
sandbox
search
secret_managers refactor(vault): type the KV read body walk as Mapping[str, object] 2026-09-18 09:28:20 +00:00
skills feat(skills): semantic search over the LiteLLM-hosted skill registry (#39401) 2026-09-07 12:28:38 -07:00
types fix: backport five regression fixes to rc/1.103.0 (#43331) 2026-09-26 11:47:22 -07:00
vector_store_files
vector_stores
videos
__init__.py fix(proxy): register transcribe as a known provider for model grants 2026-09-18 21:04:25 -07:00
_internal_context.py fix(proxy): price one cost estimate at one moment 2026-09-07 18:38:12 -07:00
_lazy_imports.py fix(lazy_imports): type import_map as Mapping to stay under the LIT001 budget 2026-09-05 16:12:45 -07:00
_lazy_imports_registry.py fix(vertex_ai): keep reasoning_effort unsupported on Vertex AI Mistral partner models 2026-09-18 13:16:56 -07:00
_logging.py Merge pull request #40934 from BerriAI/litellm_fix_ocr_native_multipage_pdf 2026-09-16 18:06:11 -07:00
_redis.py fix(redis): accept every truthy flag and sign serverless ElastiCache caches 2026-09-10 10:13:03 -04:00
_redis_credential_provider.py fix(redis): accept every truthy flag and sign serverless ElastiCache caches 2026-09-10 10:13:03 -04:00
_service_logger.py chore: merge origin/litellm_internal_staging into litellm_decrease_anys_opus5_r4 2026-09-04 18:59:35 -07:00
_uuid.py
_version.py
anthropic_beta_headers_config.json Merge pull request #41203 from clonylu/fix/anthropic-thinking-binding-beta-header 2026-09-18 12:50:55 -07:00
anthropic_beta_headers_manager.py
blog_posts.json
budget_manager.py
constants.py revert(usage): remove the top-N aggregated key cap and daily global spend rollup code from rc/1.103.0 (#43326) 2026-09-26 11:14:53 -07:00
cost.json
cost_calculator.py fix(cost): keep the deployment's OCR page rate when the model has no published price 2026-09-19 01:59:39 -07:00
exceptions.py fix(exceptions): keep internal_server_error as the public type of an upstream 500 2026-09-18 22:28:58 -07:00
main.py Merge remote-tracking branch 'origin/main' into litellm_invalid_tool_choice_400 2026-09-19 02:35:29 -07:00
model_prices_and_context_window_backup.json chore(prices): sync OpenRouter prices: 2 models 2026-09-20 01:00:52 +00:00
policy_templates_backup.json
provider_endpoints_support_backup.json fix(batches): honor deployment OCR page pricing in batch cost and answer 400 for unsupported Mistral file purposes 2026-09-18 23:57:41 -07:00
py.typed
router.py fix: backport five regression fixes to rc/1.103.0 (#43331) 2026-09-26 11:47:22 -07:00
scheduler.py
setup_wizard.py
timeout.py
utils.py fix(proxy): estimate auto-router baseline costs from durable cache history 2026-09-19 12:44:47 -07:00