The docs quickstart pipes a compose file hosted on the docs site straight
into `docker compose -f -`. That puts the content users execute in the docs
repo rather than here, and nothing lands on disk for them to read first.
Move the two-service stack (gateway + Postgres) into docker/ so it ships and
is reviewed alongside the code it starts, and pin the image to main-stable
instead of latest. The docs change to download-then-run follows separately.
Verified: `docker compose up -d` brings the stack healthy, /health/liveliness
returns 200, /v1/models returns 200 with the placeholder key and 401 without,
and /ui/ serves.
The stream chunk builder started its per-chunk accumulators at 0 and adopted only nonzero counts, then fell back to litellm's tokenizer whenever the accumulated value was falsy, so a provider that reported an explicit 0 for prompt or completion tokens was billed the estimate instead. The accumulators now start at None, a usage chunk that reports a count marks it reported (a later chunk's 0 never replaces a reported nonzero), and the estimate only runs when no chunk reported the count. The Anthropic message_start cursor reset now yields None so the estimate still covers a cancelled stream, and Ollama chat streaming only attaches usage on the done chunk when both counts are present instead of inventing 0/0 on every chunk
Optimize the aws-sdk dependency tree and cache-s3 for size in the release profile and switch to fat LTO so the native extension stays under the wheel verification gate (27.87 -> 21.88 MB)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Extend cache-redis so RedisTopology::Cluster routes single-key commands by
hash slot, groups pipelines by slot while keeping reply order, scans and
scoped-flushes every primary, and fans admin commands out to all nodes.
The bridge projects RedisClusterCache startup_nodes into the typed topology,
accepts the exact RedisClusterCache identity, guards nodes_manager state,
and falls back to Python for anything it cannot project.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A caller-supplied extra_body model overrode the authorized model in the request the shared HTTP handler sends upstream. Strip it before dispatch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Register s3_access_key_id, s3_secret_access_key and s3_encryption_key_id as
LiteLLM-owned batch params so they are no longer forwarded to Bedrock as
additionalModelRequestFields (which 400s ordinary chat on a batch-configured
deployment), keep them on CredentialLiteLLMParams so the batch/file paths
still receive them, and redact the S3 credential key names in debug logs.
Resolves LIT-8290
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Keep the exact eventstream pins but move them to =0.61.4 so aws-runtime 1.9.4's eventstream requirement resolves, downgrade aws-sdk-s3 to 1.146.1, and echo the context ttl from S3Cache::get_ttl to match Python
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
