- sync llm_passthrough_route: read and close an error-status streaming
response before mapping it, so upstream 4xx/5xx surface as the provider
error instead of httpx.ResponseNotRead
- AsyncPassthroughStreamingResponse: expose aiter_bytes() and carry
_hidden_params so the router attaches headers in place instead of
wrapping the stream in HiddenParamsAsyncIteratorWrapper, which 500'd
every streaming azure router-model passthrough request
- logging: swap the passthrough httpx result for the transformed
ModelResponse/EmbeddingResponse when firing success callbacks
- get_llm_provider: resolve gigachat from its api base and drop the dead
gigachat_models elif branch
- constants: register the gigachat api base in openai_compatible_endpoints
Adapts streaming_model_restamp.py to the LIT001/LIT010 gates that landed
on staging since this branch was cut (Final annotations, Mapping in
annotations instead of dict).
* feat(spend_tracking): persist router metadata in spend logs for internal router models
* test(spend_tracking): expect router_metadata key in exact-payload tests, type the routed-kwargs helper
The chat-to-Responses reverse transform kept only text blocks, so an image
input was dropped before the count went to OpenAI. A 256x256 image request
counted 13 tokens instead of 268.
/v1/responses/input_tokens returned 200 with a count for an empty
"input" ("" or []), while OpenAI returns a 400 missing_required_parameter.
The route also went through optimistic budget reservation, which is only
released by LLM success/failure callbacks that a token count never
reaches, so every call leaked a reservation until TTL expiry and could
429 real traffic. Both routes plus the /openai alias now join
/utils/token_counter in the reservation exemption set.
Resolving github_copilot/chatgpt names through get_llm_provider runs the
provider's OAuth device flow synchronously on the event loop. Adopt the
declared provider in PatternMatchRouter.get_pattern, which the auth
layer's zero-cost budget check walks on every request against wildcard
routers, and in /utils/supported_openai_params.
* fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL
ProxyLogging.budget_alerts forwarded to the alerting pipeline only when
'slack' was in general_settings.alerting, so alerting: ['webhook'] plus
WEBHOOK_URL silently never delivered a budget alert (the config
/health/services?service=webhook exists to test). Forward when 'webhook'
is present too; SlackAlerting.send_alert already fans out per channel.
Also accept a provider-neutral ALERTING_WEBHOOK_URL env fallback for the
Slack-format channel (any Slack-compatible receiver works), mark it as a
sensitive var, and de-brand the admin UI alerting copy.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): format settings.tsx with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for updated alerting description
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger checks after ALERTING_WEBHOOK_URL docs merged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Default remains true so lite claude turns tool search back on through
a proxy. An explicit false or auto in the env or settings is left alone
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Claude Code turns tool search off when ANTHROPIC_BASE_URL is a proxy.
lite claude, lite up, login --config-claude, and autoroute now force
ENABLE_TOOL_SEARCH=true so MCP tools stay deferred through the proxy
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
- forward unrouted /gigachat/* requests with env credentials like other passthrough providers (the old fallback returned 400 on any request without a routed model, /gigachat/models included)
- fix basedpyright budget breaches across the gigachat provider, common_request_processing, and llm_passthrough_endpoints with real narrowing, no new suppressions
- add regression tests for the fallback target, auth header, and model-less endpoints
The route-level regression test returns a real prisma row from a mocked
update and asserts both routes serialize it to a 200 with the toggled
blocked flag, which is exactly the path that raised AttributeError before
the validator guard. Also binds the loop variable in the e2e poll lambda
(ruff B023).
The one-time seed for a budget window row subtracted the batch's own
LiteLLM_SpendLogs rows by request_id, and request_id is the client's
x-litellm-call-id whenever the response carries no id of its own. Carrying
that set through the queue meant an unbounded, client-controlled aggregate
that the commit-failure requeue kept alive across retries.
Every log row at or after a batch's earliest start is owed by an increment
that still reaches the row, so summing only rows before it needs nothing
from the request. That drops request_ids end to end and closes the
cross-pod double count the id list could not see.
#34940 widened the mask-in-place safety guard so a Responses-API
`instructions` field (and a combined messages+input body) skips the
PII masking branch. With `on_flagged: "monitor"` that fell straight
through to "allow", so PII that used to be masked now reaches the
model unredacted.
Monitor means "don't block", not "don't redact". Recover the one shape
whose payload is still fully writable: mask it and write the redacted
instructions back into `data["instructions"]` directly, since
apply_redacted_messages_back has no path for that field and would
otherwise fold the instructions text into `data["input"]`.
The combined messages+input and multimodal shapes stay unmasked - both
are unsafe to write back, not merely unwritable - and now log an error
naming the reason instead of passing silently.
No block/allow decision changes: block and inject_system_message keep
the exact outcomes #34940 shipped.
Direct access on /model/info was read from the user record alone, so an
unrestricted user calling with a key limited to a few models saw every non-team
deployment, including ones the key gets a 403 on. Resolve the key's grant the
same way and intersect the two.
Resolving a grant now also expands access groups, which the key path needs and
the user path was missing.
Claude-Session: https://claude.ai/code/session_01XL7LBFEew4wi8gphVCDq6n
Model access group budgets shipped API-only, so the only way to give a group a
budget was a curl. Adds an Access Group Budgets tab under Models & Endpoints
that lists every group with the spend drawn against its shared pool, and a
modal to set, edit or clear the budget.
/access_group/list now carries each group's budget and spend inline, so the
table renders from one read instead of one follow-up request per row.
