mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-11 22:51:28 +00:00
docs(router): name both affinity TTL knobs in the _claim_pin docstring (#40663)
The docstring cited session_affinity_ttl_seconds as the keepalive bound, but the Router-level knob feeding ttl_seconds is deployment_affinity_ttl_seconds; session_affinity_ttl_seconds is the separate per-request PreRoutingHookResponse override. Anyone grepping the docstring's name to shrink the Router default finds only the override. Name both, scoped correctly.
This commit is contained in:
parent
9316b4194a
commit
dfab4794ec
1 changed files with 3 additions and 1 deletions
|
|
@ -349,7 +349,9 @@ class DeploymentAffinityCheck(CustomLogger):
|
|||
first write instead of the last. Re-claiming with the stored value refreshes its
|
||||
TTL, the same keepalive the complexity router's model pin documents: an active
|
||||
session must not lose its pin mid-conversation just because it outlives the
|
||||
original write, so `session_affinity_ttl_seconds` bounds idle time, not total
|
||||
original write, so the affinity TTL (the Router's
|
||||
`deployment_affinity_ttl_seconds`, or a pre-routing hook's per-request
|
||||
`session_affinity_ttl_seconds` override) bounds idle time, not total
|
||||
session length. On Redis one Lua script does the get-or-set-or-refresh
|
||||
atomically (same registration seam the rate limiters use) and the in-memory
|
||||
tier is synchronized to the winner; without Redis, and whenever Redis is
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue