The new Lyria passthrough branch runs before the image-generation branch
and keys on the same `predictions[0].bytesBase64Encoded` shape imagen
returns, so only the cost-map lookup separates them. Cover an imagen
predict response end to end so a future change that drops that lookup
fails here instead of misbilling images as audio.
Google prices Lyria per generated clip, so every Vertex Lyria entry in the
price map now carries a single output_cost_per_image and both the speech
and the passthrough cost paths read that one field. The old
output_cost_per_second and audio_seconds_per_prediction pair assumed a
30 second clip, which does not match the 32.768 second WAV Vertex returns,
and no other model in the map priced audio that way
Drops max_audio_length_hours and max_audio_per_prompt from the price map,
its schema, the generator, and ModelInfo, since nothing reads them, and
drops the audio_mime_type hidden param for the same reason: the response
already carries the resolved content type on its own header
Folds the per-model bundled catalog lookups into one cached parse of the
local cost map, validated with a TypeAdapter over a ReadOnly TypedDict
Percent-encode project, location, and model as single path segments. Lyria 3 speech builds the interactions URL through staging's minter with the already resolved project
* fix(ollama_chat): stamp finish_reason tool_calls when tool calls streamed before the done chunk
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ollama_chat): trim finish_reason override comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
* fix(proxy): mark session/SSO/SAML cookies Secure behind a TLS-terminating reverse proxy
litellm only sees a plain-HTTP hop when TLS terminates at a reverse proxy,
so cookie Secure attributes previously derived from (or defaulted without
regard to) the literal request scheme could be dropped in production. The
token session cookie set by every login path never carried Secure/HttpOnly/
SameSite at all.
Adds IPAddressUtils.is_request_https, a single trust-aware resolver used by
every cookie-setting call site: PROXY_BASE_URL, then X-Forwarded-Proto only
from a configured trusted proxy (general_settings.use_x_forwarded_for +
mcp_trusted_proxy_ranges), then the literal scheme. An unconfigured or
untrusted caller cannot spoof the header to force Secure on.
Resolves LIT-6748
* fix(proxy): make the shared session-cookie helper public, type new test helpers
set_session_token_cookie is imported across modules (ui_sso.py ->
proxy_server.py), so the leading underscore was misleading and breached
basedpyright's reportPrivateUsage budget with zero headroom. Also adds
missing parameter/return type annotations to the new test helper functions
per repo convention.
* refactor(python-bridge): split non-streaming bridge modules
* refactor(python-bridge): bring shared function tracing into route layer
* feat(dev): list Python route functions and call sites
* feat(dev): list Rust route functions and call sites
* docs(dev): record OCR parity gaps across Python and Rust
* feat(dev): list executed SDK calls with runtime tracing
* feat(dev): report Python vs Rust SDK pipeline steps in one CLI
* feat(dev): side-by-side pipeline step report in compare CLI
* fix(dev): drop invalid Final annotations in compare cell loop
* feat(dev): blue python-only and yellow rust-only steps in compare CLI
* feat(dev): vertical layout with section spacing in compare CLI
* fix(dev): validate SDK trace stages across sync and async routes
* refactor(rust): align SDK route call structure with Python
* refactor(python-bridge): share sync and async route call wrappers
* refactor(dev): split compare CLI into fixtures, runtime, and report modules
* fix(ci): run SDK trace tests and satisfy test lint
* test(proxy): verify NO_DOCS/NO_REDOC/NO_OPENAPI gate every doc surface
Confirms /docs, /redoc and /openapi.json are reachable by default and
each 404 with no schema in the body when NO_DOCS/NO_REDOC/NO_OPENAPI are
set, and that disabling them does not affect other routes.
* test(proxy): pin the production docs_url construction to the real helpers
Addresses Greptile 4/5 feedback: the disabled-path tests built a fresh
FastAPI() app, which could miss drift in the actual
FastAPI(docs_url=..., ...) call in proxy_server.py. This pins that
exact call site against _get_docs_url/_get_redoc_url/_get_openapi_url.
Coordination Redis (spend counters, budget-window enforcement, and the
reset_spend cache-eviction broadcast) previously only attached when a
deployment set general_settings.coordination_redis or
litellm_settings.cache. Bare REDIS_HOST/REDIS_PORT env vars alone did
nothing, so a multi-replica proxy with no cache block got no cross-pod
coordination at all: a key reset on one pod never cleared another
pod's stale budget enforcement.
The inferred Redis is pinged before being adopted, and a malformed
REDIS_CLUSTER_NODES/REDIS_SENTINEL_NODES value is tolerated too: env
vars can be set for an unrelated reason with nothing reachable there,
and guessing wrong must not turn a previously harmless in-memory-only
proxy into one that fails to boot or raises on its next cache write.
