A guardrail modify_response verdict on a streaming request only produced a
proper replacement on /v1/messages: the chat completions and Responses API
translations had no build_block_sse_chunks, so the ModifyResponseException
re-raised and surfaced as an in-stream 500 error frame (or a whole-request
500 in buffered mode) instead of the documented 200 replacement.
Implement build_block_sse_chunks for both OpenAI translations: chat emits a
content delta plus a finish_reason content_filter chunk with real usage;
Responses emits the typed event sequence (standalone via
build_synthetic_response_events pre-stream, or an output-item continuation
under the in-progress response id mid-stream) ending in response.completed.
A non-admin switching an existing key's type between the safe preset
buckets (llm_api_routes, info_routes, and empty = full access) got a 403
from the allowed_routes admin gate, because /key/update, unlike
/key/generate and /key/regenerate, had no carve-out for preset-derived
values. Skip the gate only when both the incoming and the stored
allowed_routes consist entirely of safe presets, so clearing an
admin-set custom route restriction still requires proxy admin.
The batch start told the seed which LiteLLM_SpendLogs rows were its own, but
using it as a hard cutoff also dropped rows another pod had already persisted.
Those rows are only repaid by that pod's own increment, so if it died first the
window row stayed permanently under the recorded spend.
The seed now reads both sums in one scan and takes off this batch's own spend,
flooring at the pre-batch total for the case where its log rows have not landed
yet. Redis payloads keep an empty request_ids so a leader from before the field
was dropped can still merge what it pops during a rolling deploy.
Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
precached_prompt_tokens is a subset of prompt_tokens (OpenAI cached_tokens
semantics), so map it to prompt_tokens_details.cached_tokens instead of
adding it on top of prompt/total. Emit stream usage from any final chunk
carrying it rather than only finish_reason stop, which dropped tokens for
function_call and length streams. Merge auth metadata into a new dict in
the gigachat router handler instead of mutating the shared parsed-body
cache in place.
- sync llm_passthrough_route: read and close an error-status streaming
response before mapping it, so upstream 4xx/5xx surface as the provider
error instead of httpx.ResponseNotRead
- AsyncPassthroughStreamingResponse: expose aiter_bytes() and carry
_hidden_params so the router attaches headers in place instead of
wrapping the stream in HiddenParamsAsyncIteratorWrapper, which 500'd
every streaming azure router-model passthrough request
- logging: swap the passthrough httpx result for the transformed
ModelResponse/EmbeddingResponse when firing success callbacks
- get_llm_provider: resolve gigachat from its api base and drop the dead
gigachat_models elif branch
- constants: register the gigachat api base in openai_compatible_endpoints
Adapts streaming_model_restamp.py to the LIT001/LIT010 gates that landed
on staging since this branch was cut (Final annotations, Mapping in
annotations instead of dict).
* feat(spend_tracking): persist router metadata in spend logs for internal router models
* test(spend_tracking): expect router_metadata key in exact-payload tests, type the routed-kwargs helper
The chat-to-Responses reverse transform kept only text blocks, so an image
input was dropped before the count went to OpenAI. A 256x256 image request
counted 13 tokens instead of 268.
/v1/responses/input_tokens returned 200 with a count for an empty
"input" ("" or []), while OpenAI returns a 400 missing_required_parameter.
The route also went through optimistic budget reservation, which is only
released by LLM success/failure callbacks that a token count never
reaches, so every call leaked a reservation until TTL expiry and could
429 real traffic. Both routes plus the /openai alias now join
/utils/token_counter in the reservation exemption set.
Resolving github_copilot/chatgpt names through get_llm_provider runs the
provider's OAuth device flow synchronously on the event loop. Adopt the
declared provider in PatternMatchRouter.get_pattern, which the auth
layer's zero-cost budget check walks on every request against wildcard
routers, and in /utils/supported_openai_params.
* fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL
ProxyLogging.budget_alerts forwarded to the alerting pipeline only when
'slack' was in general_settings.alerting, so alerting: ['webhook'] plus
WEBHOOK_URL silently never delivered a budget alert (the config
/health/services?service=webhook exists to test). Forward when 'webhook'
is present too; SlackAlerting.send_alert already fans out per channel.
Also accept a provider-neutral ALERTING_WEBHOOK_URL env fallback for the
Slack-format channel (any Slack-compatible receiver works), mark it as a
sensitive var, and de-brand the admin UI alerting copy.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): format settings.tsx with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for updated alerting description
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger checks after ALERTING_WEBHOOK_URL docs merged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Default remains true so lite claude turns tool search back on through
a proxy. An explicit false or auto in the env or settings is left alone
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Claude Code turns tool search off when ANTHROPIC_BASE_URL is a proxy.
lite claude, lite up, login --config-claude, and autoroute now force
ENABLE_TOOL_SEARCH=true so MCP tools stay deferred through the proxy
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
- forward unrouted /gigachat/* requests with env credentials like other passthrough providers (the old fallback returned 400 on any request without a routed model, /gigachat/models included)
- fix basedpyright budget breaches across the gigachat provider, common_request_processing, and llm_passthrough_endpoints with real narrowing, no new suppressions
- add regression tests for the fallback target, auth header, and model-less endpoints
The route-level regression test returns a real prisma row from a mocked
update and asserts both routes serialize it to a 200 with the toggled
blocked flag, which is exactly the path that raised AttributeError before
the validator guard. Also binds the loop variable in the e2e poll lambda
(ruff B023).
