mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-11 08:38:24 +00:00
9ebbcc7710
* feat(anthropic): forward message threads on native /v1/messages Keep the thread body field on native Anthropic legs, add the message-threads-2026-08-12 beta for it, reject it with a 400 on legs that cannot continue a stored thread, and relay error.details of an upstream Anthropic error so thread_not_found reaches SDK thread helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(anthropic): keep a stale thread 404 from cooling the deployment and type the thread beta ignores Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: sort the cooldown handler imports Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic): dedupe the message-threads beta map rows and annotate test finals Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(anthropic): bind message threads to the key that created them and answer unsupported continues with thread_not_found Every virtual key on a proxy shares one Anthropic credential, so forwarding thread alone let any key continue another key's thread by its message id. A new default proxy hook binds each thread reply id to its virtual key, or to the signed-in identity when there is no key, and answers a continue naming an id it did not bind with the 404 thread_not_found the SDK replays on. Legs without message threads now serve a create without the thread and answer a continue with that 404 before any upstream call, instead of answering it with no history. * fix(proxy): keep a cached message thread reply with the key that got it first A response cache hit replays the original reply id, and binding it again handed the thread to the second key, so the creator's continue got thread_not_found. * fix(anthropic): keep message threads bound on every route and stable across turns Check the thread binding on every route that carries thread except pass-through, keep bindings in their own cache so they never evict rate-limit counters, add no tool or message derived betas on thread turns, answer a create on a leg without threads with a 400 thread_unsupported_request unless drop_params is set, and forward threads on Claude Platform on AWS * fix(anthropic): reject message threads on /v1/messages passthrough deployments A deployment that forwards /v1/messages untranslated to an OpenAI-compatible server inherited the native config's thread support, so `thread` reached a server that keeps no thread state. It now answers the same 400 thread_unsupported_request on create and 404 thread_not_found on continue as a translated leg * fix(anthropic): resend a thread's create betas on continues and check bound ids on pass-through A thread create now gets the betas LiteLLM derives from its tools and messages again, so an advisor tool create that answered 400 on the previous commit answers 200. Anthropic fingerprints the beta set, so the proxy keeps the anthropic-beta header the create sent upstream in the thread binding and adds it to every continue on /v1/messages. It reads that header from the upstream request, since success logging overwrites the logged request details before the success hook runs A continue on the /anthropic pass-through route of a thread the proxy bound to another key now gets the 404 thread_not_found. Ids the proxy never bound still pass through, as before * fix(anthropic): continue unbound threads off /v1/messages and keep every anthropic-beta on chat and Responses * fix(anthropic): skip non-string anthropic-beta list entries instead of raising * test(anthropic): keep the header merge regression test clear of main's appended tests * fix(anthropic): add a thread's create betas to the client's own on continues and keep base header merging on chat and Responses * fix(anthropic): answer an unbound thread id on chat and Responses with the 404 and drop the stale beta-join tests * fix(proxy): check a thread continue sent inside extra_body against its owner Azure AI promotes extra_body into the request body, so another key could continue a thread by putting thread there, past the ownership hook that read only the top-level field * fix(proxy): write a thread's first owner atomically and bind creates sent inside extra_body Two workers that both miss a reply's binding now agree on the first key stored in Redis, since the claim uses SET NX and each worker caches the stored winner. A create sent inside extra_body now binds its reply like a top-level create. The redundant Claude Platform supports_message_threads override is removed, since the provider already resolves to the native Anthropic beta list * fix(proxy): include agent_id in the key-less message thread owner * fix(proxy): tag message thread ownership Redis calls with their key family --------- Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| pre_call_checks | ||
| __init__.py | ||
| test_access_windows.py | ||
| test_add_retry_fallback_headers.py | ||
| test_client_initalization_utils.py | ||
| test_cooldown_cache.py | ||
| test_cooldown_handlers.py | ||
| test_deployment_pricing.py | ||
| test_embedding_batch_size_check.py | ||
| test_endpoint_eligibility.py | ||
| test_fallback_event_handlers.py | ||
| test_get_retry_from_policy.py | ||
| test_handle_error.py | ||
| test_health_check_allowed_fails_integration.py | ||
| test_health_state_cache.py | ||
| test_kubernetes_pod_discovery.py | ||
| test_pattern_match_deployments.py | ||
| test_prompt_caching_cache.py | ||
| test_ptu_shares.py | ||
| test_reasoning_effort_capability.py | ||
| test_request_priority.py | ||
| test_router_health_check_routing.py | ||
| test_router_interactions_endpoints.py | ||
| test_router_utils_common_utils.py | ||
| test_routing_read_batch.py | ||