The stream chunk builder started its per-chunk accumulators at 0 and adopted only nonzero counts, then fell back to litellm's tokenizer whenever the accumulated value was falsy, so a provider that reported an explicit 0 for prompt or completion tokens was billed the estimate instead. The accumulators now start at None, a usage chunk that reports a count marks it reported (a later chunk's 0 never replaces a reported nonzero), and the estimate only runs when no chunk reported the count. The Anthropic message_start cursor reset now yields None so the estimate still covers a cancelled stream, and Ollama chat streaming only attaches usage on the done chunk when both counts are present instead of inventing 0/0 on every chunk
Optimize the aws-sdk dependency tree and cache-s3 for size in the release profile and switch to fat LTO so the native extension stays under the wheel verification gate (27.87 -> 21.88 MB)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When the config file declares alerting_args at all, the resolver hands the
file's dict to every reader and the stored row never reaches one. Reporting
a nested field as "db" because the row happens to carry it told the admin a
value was in effect that the proxy does not serve: a live proxy answered
source "db" for outage_alert_ttl while serving the default.
A config-owned parent now reports the field's own default, and the DB is
consulted only when the file leaves the parent alone.
Extend cache-redis so RedisTopology::Cluster routes single-key commands by
hash slot, groups pipelines by slot while keeping reply order, scans and
scoped-flushes every primary, and fans admin commands out to all nodes.
The bridge projects RedisClusterCache startup_nodes into the typed topology,
accepts the exact RedisClusterCache identity, guards nodes_manager state,
and falls back to Python for anything it cannot project.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
_nested_setting_source returned "db" whenever the stored row held a value,
without first asking whether the config file declares the same key. For a
config-owned alerting_args field that disagrees with the database, the
endpoint reported source "db" while the proxy actually serves the file's
value and rejects any write to it.
Config ownership is now checked first, matching SettingsStore.source and
the precedence the rest of the resolver applies. The source test set grows
a field that only the database sets, a field only the file sets, and a
stored empty list, so each reported source is discriminating.
A caller-supplied extra_body model overrode the authorized model in the request the shared HTTP handler sends upstream. Strip it before dispatch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Register s3_access_key_id, s3_secret_access_key and s3_encryption_key_id as
LiteLLM-owned batch params so they are no longer forwarded to Bedrock as
additionalModelRequestFields (which 400s ordinary chat on a batch-configured
deployment), keep them on CredentialLiteLLMParams so the batch/file paths
still receive them, and redact the S3 credential key names in debug logs.
Resolves LIT-8290
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Keep the exact eventstream pins but move them to =0.61.4 so aws-runtime 1.9.4's eventstream requirement resolves, downgrade aws-sdk-s3 to 1.146.1, and echo the context ttl from S3Cache::get_ttl to match Python
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
