227 commits

Author	SHA1	Message	Date
joshua-berri	04f0ea8c33	feat(mcp): translate input requests and bind continuations (#45464) ... * feat(mcp): translate input requests and bind resumable continuations * fix(mcp): keep continuations bound to their original upstream * fix(mcp): sync advertised protocol API schema * test(mcp): run interaction regressions in GitHub CI * test(mcp): set source paths for interaction proxy processes * test(mcp): verify salt guidance across interaction carriers * fix(mcp): preserve interaction authentication and elicitation policy * fix(mcp): bound preflight bodies and preserve challenge routes * test(mcp): type interaction regression fixtures --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-10-08 20:10:10 -07:00
devin-ai-integration[bot]	0ae62c5d02	feat(ui): add the evaluation mode to the Add Model form (#45481) ... Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-08 19:53:27 -07:00
devin-ai-integration[bot]	390bea6a53	fix(proxy): save file details for every batch output file so they list and retrieve (#41761) ... * fix(proxy): register bedrock batch output files with a file object so they list Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): guard output file size when content is missing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover managed batch output file listings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): persist metadata for batch output files Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover batch output file listing and retrieval end to end Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): preserve provider metadata for managed batch files Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): update managed batch output registration mocks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): inject fake prisma client in managed batch output file tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): retry provider file details and refresh fallback batch output entries Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover fallback batch output file details refresh after provider recovers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover model-name managed batch file retrieval Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(managed_files): bound the batch output file lookup so a slow provider cannot stall GET /v1/batches * fix(managed_files): skip the provider lookup for a fallback entry written moments ago * fix(proxy): safely refresh managed batch file details Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep batch listing provider-free Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): return metadata for empty S3 objects Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): strengthen batch fallback regression assertions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(proxy): describe provider reads and DB writes for batch output files Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): sanitize caller-derived ids in managed file logs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(proxy): simplify batch and file endpoint docstrings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ui): regenerate API types from proxy OpenAPI spec Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(test): resolve managed files lint violations Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(proxy): save refreshed managed file details through ManagedFileRepository Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): classify retrieved file purpose by its own bucket prefix Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(proxy): move batch and file endpoint behavior notes to litellm-docs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): resolve strict lint violations in file retrieval Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): mark retrieval request metadata handoffs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): resolve managed batch type-check diagnostics Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): avoid duplicate final response bindings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: Yucheng He <yucheng@berri.ai>	2026-10-08 18:51:45 -07:00
devin-ai-integration[bot]	c5c5154c4b	fix(cli): pin pi compat flags so lite pi stops sending store to Anthropic models (#40739) ... * fix(cli): pin pi compat flags for gateway models Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): annotate pi compatibility field Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): clarify pi compatibility suppression Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cli): drop the stale mutable-ok suppression on the pi compat block --------- Co-authored-by: jesus <jesus@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-08 18:30:27 -07:00
devin-ai-integration[bot]	8a176c0f0a	fix(router): drop the encrypted reasoning a fallback hop's target cannot decrypt (#45393) ... * fix(router): drop the encrypted reasoning a fallback hop's target cannot decrypt An order-based or configured fallback hop replayed the failed provider's encrypted reasoning items to the next deployment, which answered 400 (Bedrock Mantle: invalid encrypted reasoning; OpenAI: invalid_encrypted_content), so every multi-turn Responses fallback for Codex-style clients failed. The hop now drops the encrypted reasoning its target cannot decrypt and keeps each item's readable summary. With encrypted_content_affinity on, the pin narrows to the hop's target order instead of emptying it, so the hop reaches the next order instead of failing with no deployments available. * fix(router): keep encrypted reasoning a same-boundary fallback hop can decrypt Unmarked encrypted reasoning on a hop is attributed to the deployment that just failed, read from the retry breadcrumb, so a hop to a deployment on the same api_base and api_key keeps it and a cross-provider hop still drops it. The hop tests script the upstream at the httpx boundary instead of doubling the handler, and the router coverage script lists the two hop helpers with their tests * fix(router): read the hop's failed deployment from its own metadata bucket and carry it into the Responses mid-stream snapshot * test(router): use a real Router without the origin deployment in the hop strip test * test(router): cover the hop strip when no failed deployment is known * fix(proxy): drop the router's fallback hop state keys from the client body * fix(proxy): keep a request's max_fallbacks cap, drop only the hop state keys * test(integration): audit cells for the fallback hop encrypted reasoning strip Forty-five checked-in cells under tests/integration/routing prove the hop strips the previous deployment's encrypted reasoning on /v1/responses, /v1/chat/completions and /v1/messages (httpx, OpenAI and Anthropic SDKs, sync and async, streaming and not), that the affinity pin yields to the hop, that a client-sent fallback_depth, _target_order and attempted_targets never move or strip a request, and that a concurrent burst, an order-1 outage and a killed worker keep every request stripped and logged once. Every call goes through a lane pinned to one worker that already lists the deployments it needs, because the peer worker learns a /model/new row through the config-sync resync up to sixteen seconds later --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-08 17:30:01 -07:00
yucheng-berri	51fea825cc	feat(guardrails): support logging_only mode for the Akto guardrail (#45461) ... * feat(guardrails): support logging_only mode for the Akto guardrail * test(guardrails): wait for the spend row before reading Akto calls and cover mixed modes * test(guardrails): cover Akto logging_only on MCP tool calls * test(guardrails): type the Akto logging_only unit tests and inject the HTTP handler * test(guardrails): cover failing Akto replies, provider failures and a mixed outage burst under logging_only * test(guardrails): cover an unreachable Akto with fail_open under logging_only	2026-10-08 17:28:08 -07:00
devin-ai-integration[bot]	f09408dd37	test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) ... * test: replace 61 live logging and otel tests with offline unit and integration coverage * test: restore datadog formatting, deliver datadog logs and redis failures over the wire, assert full router hook payloads, tighten stream usage and otel checks Restores the pre-existing test_datadog.py lines the branch had reflowed. Datadog success, failure and redis-failure replacements now assert the gzip body posted to the intake, with redis failing through a real cache on a closed local port. Router hook sequence recorder checks the legacy field types and asserts concrete payloads, exact streaming and fallback sequences. Stream usage asserts the default include_usage request body and the redacted messages value. Otel asserts response id and token counts. * test: assert budget envelope figures, guardrail inspection and exact prometheus samples per request * test: isolate the prometheus latency test on its own deployment so counts do not depend on order * test: cover timed slack delivery, keep the redis failure test off the network, and make new payload types read-only * test: drain the redis test's logging and scope otel span checks to the test's own trace * test: drive the periodic slack flush without a wall-clock interval * test: scope router hook events to the test, script the db clock, split a nested comprehension --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-08 15:55:57 -07:00
devin-ai-integration[bot]	5d207d85ee	fix(proxy): accept team-scoped models by their public name on POST /fallback (#45455) ... * fix(proxy): accept team-scoped models by their public name on POST /fallback create_fallback validated the primary and fallback models against the router's stored model names only, so a model created through POST /model/new with model_info.team_id was rejected with a 404 unless the caller used the generated model_name_<team_id>_<uuid> name. The endpoint now also accepts the team public model names, which request time fallback matching already keys on, and lists both kinds of names in the 404's available_models. * fix(proxy): read fallback rules fresh and clear the config cache after a fallback write A second POST or DELETE /fallback within the 60 s config cache TTL started from a cached copy of router_settings and dropped every rule stored since that copy was taken, by any instance. Both endpoints now evict the cached row before the read and invalidate it after the upsert * test(proxy): type the fallback endpoint tests and prove a team request fails over by public name * fix(proxy): keep fallback writes working when the Redis config cache is down and type the stored settings read * fix(proxy): keep the non-standard fallback shapes the router accepts on writes and resolve the config cache at call time * test(proxy): mark the config cache outage test's result as Final * fix(proxy): replace a same-key fallback rule in place and read a null rule list as empty * test(proxy): let the stored router settings fixture carry a null rule list * test(proxy): audit cells for fallback rules by team public name * test(proxy): delete the fallback rules the audit cells save --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-08 22:38:38 +00:00
devin-ai-integration[bot]	577d74c1ce	fix(bedrock): count tokens on bedrock-mantle when bedrock-runtime cannot count a Claude model (#45317)	2026-10-08 15:26:49 -07:00
devin-ai-integration[bot]	406514fcaf	feat(spend_logs): configure which metadata fields are stored in LiteLLM_SpendLogs (#44659) ... * feat(spend_logs): configure which metadata fields are stored in LiteLLM_SpendLogs Adds general_settings.spend_logs_metadata_fields with mutually exclusive include and exclude lists. The filter runs on a copy of the row right before it is queued for Postgres, so daily spend rollups, budgets and callbacks still see every key. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(spend_logs): keep excluded auto-router savings keys out of published spend log metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend_logs): cover metadata retention across endpoints, failures, cache hits, batches, runtime updates and auto-router publication Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ui): regenerate schema.d.ts for spend_logs_metadata_fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(spend_logs): filter metadata at DB write so guardrail usage sees full rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend_logs): poll guardrail daily metrics instead of reading once Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend_logs): drop timeout comment Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(spend_logs): read spend_logs_metadata_fields through typed general settings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-08 14:51:44 -07:00
moe-berri	4ed0267b5a	fix(lens): prevent progress updates from starving analysis budget reservations (#45432) ... * fix(lens): serialize analysis budget reservations with progress updates * test(lens): verify budget reservations block competing progress writes	2026-10-08 21:08:55 +00:00
devin-ai-integration[bot]	85a3869dfd	feat(guardrails): per-mode stream_scope with bedrock stream and pass-through fixes (#43801) ... * feat(guardrails): run each mode only on streaming, non-streaming, or both Add stream_scope so a rail can target streaming inference, non-streaming inference, or both per pre, during, and post mode Co-authored-by: Cursor <cursoragent@cursor.com> * fix(guardrails): honor stream_scope in pipelines and dashboard types Pipeline steps skipped the stream_scope filter, direct construction ignored mixed-case maps, and schema.d.ts was stale. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(guardrails): skip unmatched stream_scope steps instead of allowing Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ui): keep stored stream_scope keys for modes not on screen Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): format stream_scope helpers and update fork MCP unit tests Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): keep hang cancellation tests from timing out during setup Co-authored-by: Cursor <cursoragent@cursor.com> * fix(guardrails): honor path-defined streaming for stream_scope Passthrough routes like Gemini streamGenerateContent decide streaming from the URL, so stamp that onto hook data before guardrails run. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(rust): copy AnthropicModelCapabilities instead of cloning Clippy treats clone-on-Copy as an error, which failed rust-lint on the messages request tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(guardrails): trust only server stream classification Co-authored-by: Cursor <cursoragent@cursor.com> * fix(guardrails): keep streaming marker through deepcopy scan_raw_request snapshots copy each field, so a plain object() marker would lose identity and skip streaming-only rails. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cost-map): drop duplicate perceptron-mk1.5 row Two main cost-map PRs both added the OpenRouter model, so the merge left a second key that CI rejects. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(rust): expect native transcription 429 as RustUpstreamError HTTP status errors from native routes map through route_error_to_pyerr, so the wheel SIGINT child was dying on an outdated RuntimeError check and never reached the hang probe. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tests): follow Google Interactions OpenAPI without hardcoded names The live spec dropped CreateModelInteractionParams and renamed the item path to {interactionsId}. Misc CI failed because the compliance tests still looked those names up as literals. * test(guardrails): reproduce stream_scope bedrock and passthrough field gaps Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): cover invalid stored stream_scope reads Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): classify bedrock stream actions and keep caller is_streaming_request Co-authored-by: Shivi Jain <mobile.350017@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(guardrails): make stream_scope_allows public and drop mutable builds Co-authored-by: Shivi Jain <mobile.350017@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): classify pass-through and Bedrock stream scopes Co-authored-by: Shivi Jain <mobile.350017@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): harden stream classification and validation Co-authored-by: Shivi Jain <mobile.350017@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): add stream scope integration audit Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): avoid mutating passthrough custom body Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): run stream scope audit without enterprise license Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): keep stored scope restart cell on one worker Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): scope stream scope audit sink assertions to the rail under test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): assert logging_only scope absence behind an ordered barrier rail Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): assert one logging_only scan per phase after the barrier rail Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): set request scope on pass-through stream fixtures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): tolerate invalid YAML stream_scope in v1 guardrails list Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: merge main into litellm_guardrail_stream_scope_fixes Update pass-through pre-call test callbacks for main's endpoint_type argument Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): add request paths to pass-through fixtures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): cover websocket pass-through stream scope and LIT-9050 outage spend rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(lint): remove unused type discipline suppressions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): script the vertex live upstream in the websocket stream scope test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep stream marker json-serializable and restore pass-through helper names Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(lint): allow required Bedrock action re-export Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): strip the stream marker from pass-through payloads Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): drop stream marker by value in scans and snapshots, plain-tuple stream scope state Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): render tag-scoped guardrail modes read-only in the custom code editor A guardrail whose litellm_params.mode is the tag-scoped dict {tags, default} crashed the Custom Code editor on open: normalizeMode wrapped the dict into the mode array and StreamScopeFields rendered it as a React child (error #31, whole dashboard unmounted). Treat a non-string non-array mode as no editable modes, show formatGuardrailMode(mode) in a disabled input (read-only, matching the guardrail info view), and keep mode/stream_scope out of the update payload for such guardrails. * fix(ui): resolve merge fallout in guardrails components Deduplicate toModeArray import after the merge, and move the read-only guardrail details block back into GuardrailReadOnlyDetails (now rendering the shared mode/logging-only rows plus the stream-scope detail) so guardrail_info.tsx stays under the 800-line lint budget. Guardrails UI suite: 298 passed. * fix(ui): drop duplicate toModeArray import reintroduced by merge * chore(pass-through): document the deliberate in-place marker strip as mutable-ok The clear/update on _parsed_body is load-bearing: rebinding to a fresh mapping instead breaks 75 pass-through tests because the marker-free body must propagate through the caller's request dict so downstream guardrail scans and snapshots never observe the server streaming marker. * fix(guardrails): move stream_scope after timeout in CustomGuardrail init Inserting stream_scope before the existing timeout parameter shifted the positional slot of timeout, so positional callers constructed with their timeout bound to stream_scope (ValueError) and timeout silently None. Restores the base parameter order; keyword callers are unaffected. * fix(guardrails): typing pass for the lint gates stream_scope leaves the declared constructor parameters (restoring the base positional surface; keyword construction unchanged), unknown config values crossing the new stream-scope code paths get typed locals or cast-ok boundaries, and the passthrough payload literals are annotated. All three lint gates pass against current main; the guardrail suites are unchanged (236+5019 passing; one known anyio-driver failure pre-existing on base). * style: sort cast imports for the ruff gate * chore(pass-through): drop unused BEDROCK_STREAMING_ACTIONS re-export The streaming check now uses is_bedrock_streaming_endpoint; nothing in the repo imports the name from this module. --------- Co-authored-by: Shivi Jain <mobile.350017@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: gabriele <gabriele@berri.ai>	2026-10-08 13:46:53 -07:00
yucheng-berri	29f48e62bf	feat(fireworks_ai): forward the LiteLLM user id as user behind fireworks_forward_user_id (#45265) ... * feat(fireworks_ai): forward the LiteLLM user id as user behind fireworks_forward_user_id * fix(fireworks_ai): keep the forwarded user id when the caller sends extra_body.user	2026-10-08 12:48:16 -07:00
Mateo Wang	b9ff2d66a4	fix(proxy): retry lock-timed-out daily spend batches in place so the shutdown flush keeps them (#45273) ... * fix(proxy): retry lock-timed-out daily spend batches in place so the shutdown flush keeps them Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): annotate the lock-timeout retry test and carry its context in assert messages Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): mark the remaining lock-timeout test local Final Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-08 12:05:47 -07:00
Solomon Mithra	0e455ae7a9	fix(proxy): use budget reset window for projected spend alerts (#31942) ... * fix(proxy): use budget reset window for projected spend alerts * style(proxy): use PEP 585 tuple annotations in projection helpers * fix(proxy): derive projection date from budget reset timezone * fix(proxy): project spend in real time within the budget reset window * fix(proxy): derive the projection window start from the reset schedule 30d and monthly keys reset on the 1st of the month and unrecognized spellings like 1hr reset at the next midnight, but the window start stepped back the literal duration, so now could land before the window and the elapsed floor inflated the projection into a false alert. The window start now mirrors get_next_standardized_reset_time, and a reset more than one window away no longer projects at all --------- Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>	2026-10-08 12:02:40 -07:00
devin-ai-integration[bot]	b7ecb05340	fix(proxy): mark background responses stale_expired when provider returns 404 (#41724) ... * fix(proxy): mark background responses stale_expired when provider returns 404 Responses deleted upstream (store=false / ZDR rows dropped after provider retention) now move to stale_expired on the poll instead of being retried every cycle until the 7 day stale cleanup. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(proxy): drop inline comment in check_responses_cost Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): drive _get_response seam in 404 stale_expired tests Assigning AsyncMock on the instance keeps the new tests inside the TQ002/TQ008 test-quality ceilings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): only expire 404 responses polled through a resolved deployment A NotFoundError from the bare SDK fallback can mean a missing or misconfigured deployment, which a config fix inside the staleness window can still recover. Rows whose poll went through their router deployment are the ones the provider actually dropped, so only those move to stale_expired. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): resolve the deployment before fetching so via_router binds once Drops the reinitialised loop flag flagged by review and modernizes the moved annotation. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): drive the real response fetch in the poll 404 regression tests * fix(enterprise): expire background response rows on any provider 404 status The GET path maps the provider's 404 body to litellm.BadRequestError that still carries status_code 404, so an except on NotFoundError never fired and the poll job kept retrying the row every cycle. Key the terminal decision on the status code instead of the class * fix(enterprise): expire a background response only on a 404 naming it and skip a bad deployment entry per job * chore(enterprise): record the poll-cycle bound on the managed-object IN lists * test(integration): cover background response poll retirement on the real proxy --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-08 10:35:52 -07:00
devin-ai-integration[bot]	d5173e3d70	test: move 76 live top-level tests to offline unit and integration coverage (#45352) ... * test: move 76 live top-level tests to offline unit and integration coverage * test: keep the mock-param opt-in guard valid once the top-level senders are gone * test: address review on offline replacements * docs(test): drop the deleted harness test file from the harness check command * test: make the key rebind, team member delete and routes integration tests exercise the legacy paths * test: make fallback, rpm and spend integration contracts deterministic and clean up their rows, assert the rpm limit in usage-based routing * test: expect the no-deployments error at the rpm limit and cover the strategy check without pre-call checks * test: hand member cleanups to the scenario instead of growing a budget list, flatten callback kinds * test: scope the admin health check to the test's own deployment --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-08 10:33:35 -07:00
joshua-berri	49266e3b17	fix(mcp): preserve elicitation context and report relay failures (#45255) ... * fix(mcp): correlate elicitation relays and report failures * fix(mcp): preserve elicitation timeout errors on Python 3.10 * fix(mcp): keep request ID alias in type-checking imports * fix(mcp): preserve elicitation type narrowing and clarify HTTP limits * ci(mcp): publish elicitation coverage from GitHub Actions --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-10-08 10:18:25 -07:00
devin-ai-integration[bot]	82a5d9842a	fix(ui): list only catalog models in the Add Model picker (#44877) ... The Add Model selector read the merged runtime cost map, so deployment ids and provider-prefixed backend keys registered for proxy deployments showed up as duplicate model choices. The public cost map endpoint now accepts catalog_only=true to return the catalog as loaded, and the Add Model panel requests that view. The default response is unchanged. Resolves LIT-9263 Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-08 11:18:29 -05:00
ishaan-berri	f824d11a22	fix(lens): block teamless feedback writes and fix retention test (#45266) ... * fix(lens): reject feedback writes from callers with no team or key * test(lens): teamless callers cannot overwrite feedback * test(lens): expect lens_feedback retention during schema setup * refactor(lens): flatten feedback summaries without a stacked comprehension * chore(lens-ui): drop routine comment on the feedback panel * chore(lens-ui): drop routine comment on the feedback view * chore(lens-ui): drop routine comment on the low feedback check * chore(lens-ui): drop routine comment on the post stub	2026-10-08 07:54:11 -07:00
joshua-berri	d0202ac364	feat(mcp): support upstream OAuth client metadata identities (#45231) ... * feat(mcp): support upstream OAuth client metadata identities * fix(mcp): load client metadata on cold OAuth requests * fix(mcp): discover client metadata with configured OAuth endpoints * fix(mcp): preserve configured endpoints when discovery fails * fix(mcp): retain CIMD identities across refresh and catalog reloads * fix(mcp): reuse saved CIMD identity for token endpoint refresh * fix(mcp): keep dynamic client registration ahead of CIMD unless the deployment opts in A provider that advertises both a registration endpoint and client ID metadata documents now gets the registration flow the gateway used before, so a gateway on a private network keeps working against it; the metadata document identity applies when the provider offers no registration, or when general_settings.mcp_prefer_client_id_metadata_document is true. The saved CIMD refresh identity compares tokens as bytes so a non-ASCII refresh token cannot crash the token route. * chore(ui): regenerate dashboard API types for the new MCP general setting --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-08 04:10:34 -07:00
devin-ai-integration[bot]	3a3b8cad80	feat(ui): add TypeSafe and Strands Decider to the Add Model provider list (#45335) ... * feat(ui): add TypeSafe and Strands Decider to the Add Model provider list * test(public_endpoints): type the provider fields helper against the route's model --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-08 04:04:07 -07:00
devin-ai-integration[bot]	0e26edfdb9	test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) ... * test: make legacy live tests in spend, batches, openai endpoints and audio dirs offline (partial) * test: migrate wave-1b live tests offline (guardrails, images, ocr, search, openai endpoints) * test: fix wave-1b review items, add responses/ocr integration tests and firecrawl unit test * test: anthropic messages router/bedrock/openai-bridge unit tests for wave-1b nodes * test: anthropic messages logging, prompt-caching and tool-search unit tests; drop migrated base nodes * test: finish pass_through_unit_tests nodes, logging drain fix and mutations * test: migrate anthropic passthrough tests to integration wire tests * test: fix passthrough migration wire spend row lookup and wildcard config * test: migrate hosted vllm and openai file passthrough tests offline * test: move assemblyai and vertex passthrough nodes to in-process unit tests * test: restore unlisted router node and fix logging worker drain in passthrough unit tests * test: drop spend-row BUG skip and sharpen non-streaming skip reason for anthropic messages * test: use public presidio alias after merge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: restore the batch and file unit tests the migration rewrote away * test: restore full legacy intent in anthropic messages router unit tests Drop the false BUG skip on non-streaming aanthropic_messages logging (success callbacks do fire; the skipped body filtered on the wrong model), assert the logged model_group, messages, cost and usage, cover streaming logging for both Anthropic and Bedrock invoke, assert dict content blocks for Anthropic, Bedrock invoke and the OpenAI bridge, add the Bedrock invoke leg of the router test, fall back from a real 401, test system-prompt caching and streaming message_start cache fields on converse and invoke, send the legacy tool-search tools and beta header, remove the type: ignore and bare dict helper, and add in-process native /anthropic passthrough spend logging tests * test: fix passthrough migration wire tests and drop false native spend BUG skip The native spend row test read rows by the shared master-key digest, so it matched other tests' rows; read each row by its own message id instead and assert tokens, total, spend, tags, provider, api_base and end_user on both the non-streaming and streaming native routes. Merge the streaming test that never checked spend, assert exact tags from litellm_metadata, stop rebinding a Final in a loop, require cost > 0, and add the chat-completions bridge cost case the legacy test covered * test: harden the spend, OCR, image, search and guardrail migration replacements The OCR spend tests now build fresh kwargs per case instead of mutating a shared fixture, and every payload case asserts the exact logged spend. The OCR wire test reads its spend row by request id and checks exact page pricing. Image edit, Nova Canvas, DuckDuckGo, Firecrawl, Bedrock guardrail and Presidio replacements now fake only the provider HTTP boundary (respx or an in-process aiohttp server) and assert the outbound request, so the DuckDuckGo limit, Azure base_model pricing and guardrail masking are proven rather than assumed * test: cover Exa and Perplexity search structure and max_results offline and retire the two base search methods * test: drive batch and file replacements through the provider HTTP boundary and real logging callback Replace monkeypatched litellm.afile_content and AsyncHTTPHandler doubles with respx routes, read batch logging metadata from a registered success callback instead of get_logging_payload, use the real managed-files hook for the GEN-2166 regression, assert outbound request bodies, pin poller ownership explicitly in the migrated DB-sync tests, and require the scripted upstream to be hit in the responses error-status wire tests * test: assert file content download headers pass through the proxy * test: point spend coverage references at the tests that replaced the retired spend job * test: assert passthrough identity, spend and route dispatch from the code under test The AssemblyAI non-admin test asserted metadata it wrote itself and leaked a background poll to the real AssemblyAI host. It now drives assemblyai_proxy_route with a real Request and waits for the success callback for its own transcript id. The Vertex spend test matches its log by call id instead of taking the first event. The OpenAI files wire test hit the native /{provider}/v1/files route; it now calls /openai/files so the passthrough is what forwards the upload and delete. * test: mock only the HTTP boundary in the migrated audio tests Vertex TTS tests no longer replace _ensure_access_token or AsyncHTTPHandler.post; the token comes from a mocked Google OAuth endpoint and the synthesize call from respx. Speech tests assert the outbound body, the transcription cache test polls for the cache write instead of relying on test ordering, and the model pass-through test checks the multipart model field per model. * test: wait on a logger event instead of polling the clock in anthropic messages unit tests Recorders keep payloads in a rebound tuple and set an asyncio.Event; tests await it with asyncio.wait_for instead of a sleep-and-deadline poll loop * test: freeze module-level batch and file response fixtures as Final MappingProxyType * test: fake the presidio analyzer with an in-memory aiohttp connector The blocked-entity tests started an aiohttp TestServer, which binds a local socket. They now hand the guardrail a ClientSession whose connector answers /analyze and /anonymize in process, so no socket is opened and the outbound analyze text and entities are still asserted * test: type anthropic messages router test helpers with LiteLLM's Anthropic TypedDicts Messages, cached system blocks and tool-search tools now use AnthropicMessagesUserMessageParam, AnthropicMessagesTextParam, AnthropicToolSearchToolRegex and AnthropicMessagesTool instead of bare dict shapes; tools are converted to plain dicts only at the acreate call, whose tools parameter is list[dict] * test: type batch limiter helpers with TypedDicts and wait on the logging callback event instead of polling * test: give the migrated OCR, image and presidio helpers precise types OCR spend helpers take ReadOnly TypedDicts for kwargs and responses and use LiteLLM's OCRResponse/OCRUsageInfo instead of local pydantic stand-ins; spend metadata is validated with a TypeAdapter. The presidio fake uses LiteLLM's PresidioAnalyzeRequest/ResponseItem types, and the image-edit logger validates the logged payload instead of storing an untyped dict * test: signal callback and cache events instead of polling Recorders keep tuples and set an asyncio.Event, thread-safely, when the payload for this test's transcript id or upstream URL arrives. The transcription cache test waits on a Cache subclass that signals after async_add_cache. No clock polling or sleeps remain in these tests. * test: assert the batch limiter hook updates the caller's request in place * test: tolerate model-list probes and read native passthrough rows by owned key The router's OpenAI-compatible model-info refresh (litellm/router.py:10710) sends GET /v1/models to configured openai api_bases, so the wire answers it with an empty list and excludes it from the provider-call assertions. Native /anthropic spend rows are now read by a per-request virtual key digest and call_type, then the row's request_id is checked against the message id * test: expect the OCR alias in the proxy response model The proxy restamps every OpenAI-compatible response model to the name the client requested (_override_openai_response_model), so /v1/ocr returns the scenario alias. The upstream model is now checked on the drained request body instead of inside the peer, where a failed assert never reached the test --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-08 02:38:12 -07:00
devin-ai-integration[bot]	04d97abffb	test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) ... * test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs * test: restore lost coverage in litellm_utils offline replacements Bound live health-check tasks to the concurrency limit so eager task creation behind a semaphore fails, route the Langfuse trace-id check through completion -> Logging.get_trace_id with all four metadata combinations, run the callback dedup checks through acompletion, pass the dynamic key via litellm_params, cover the env-inferred default model list, and add the per-provider audio transcription config lookup * test: restore router coverage lost in the offline move Add the missing non-stream LIT-3058 header/count-once test, pin the UTC minute on every usage-counter test, assert router-level client reuse for transcription, and replace the private selector-attribute checks with routing outcomes per strategy. Assert outbound bodies for speech, rerank, image, assistants and moderation, add timeouts to event waits, and drop the router_unit_tests husk files that no longer hold tests * test: restore sync stream, router sync stream, error event and field-type coverage for migrated responses tests Add the sync streaming logging and sync Router.responses streaming cases the offline replacements dropped, port the legacy per-event and response field-type validation, assert raw headers, the float created_at conversion, MCP tool headers, the search_context_size input, and add an offline replacement for the in-stream context-window error event. Remove helpers left dead in the legacy file. * test: run numpydoc-backed unit tests in GHA and drop a test that pinned a crash * test: restore sequence number, item id and content part checks in the responses stream validator * test: match router logging events to the test's own deployment so late events from other tests are ignored * test: restore the s3 cold storage history test to its original assertions --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-08 02:37:55 -07:00
Mateo Wang	fbbb54c841	fix(proxy): answer a rejected websocket handshake without crashing the HTTP exception handler (#45251) ... * fix(proxy): answer a rejected websocket handshake without crashing the HTTP exception handler * fix(proxy): read the OTLP route check's scope keys with .get and drive the websocket auth tests through the real auth path * test(proxy): build the websocket auth test keys without reading the clock	2026-10-07 22:27:46 -07:00
Mateo Wang	328f5a720c	fix(router): honor a per-request fallbacks list on a mid-stream fallback (#45227) ... * fix(router): honor a per-request fallbacks list on a mid-stream fallback The completion entrypoints record the per-request fallbacks, context_window_fallbacks, and content_policy_fallbacks in the mid-stream controls carrier the Responses and Messages paths already use, and each completion attempt restores them into the kwargs its stream re-enters the fallback chain with, so a streaming request that dies before its first chunk fails over to the list the request named instead of the router-level list only. The sync vs async parity cell now asserts the backup's text on both twins. * test(router): annotate the new fallback test locals as Final and flatten the leak check * fix(proxy): skip null key and team router settings when merging per-request overrides * test(proxy): annotate the null router settings test locals and drop a redundant comment	2026-10-07 22:25:22 -07:00
Mateo Wang	33d908e0ae	feat(decisions): add OpenAI as a Decisions provider behind a shared decisions format (#45214) ... * feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions The System One request format moves to /v1/systemone and /systemone. /v1/decisions and /decisions now accept the OpenAI Decisions API format, translate it into a System One request, route it through the same pipeline, and translate the answers back. * fix(decisions): import assert_never from typing_extensions for Python 3.10 * fix(decisions): cap OpenAI-format questions at the System One limit /v1/decisions accepted up to 200 questions, but the System One request it translates to takes at most 128, so 129 to 200 questions failed with an internal validation error. Both now share MAX_DECISION_QUESTIONS. * fix(decisions): return 400 for bodies that are not JSON * test(decisions): send System One bodies to /v1/systemone in integration tests * feat(decisions): add OpenAI as a Decisions provider System One requests to openai/ models are translated to OpenAI's Decisions shape on the way out and OpenAI's answers are translated back, so both /v1/systemone and /v1/decisions can route to gpt-6-luna. * refactor(decisions): move the OpenAI wire translation into llms/openai * feat(decisions): route both request formats through a shared decisions IR System One and OpenAI-format bodies now convert to one internal representation, and each provider translates it to its own wire format. OpenAI deployments receive the caller's messages, images, names, typed choice values, level descriptions and safety_identifier unchanged. System One providers return a 400 for image input. Cached and cache-write tokens are priced on both response shapes, and provider response extras survive the round trip. * fix(decisions): keep provider fields inside System One answers * fix(decisions): load on Python 3.10 and 3.11 and bill cached tokens once under custom pricing Decisions IR dataclasses used a mappingproxy default, which is only hashable from Python 3.12, so import litellm failed on 3.10 and 3.11. Decisions usage now reaches cost_per_token as a Usage with prompt_tokens_details, so custom pricing no longer adds cached and cache-write tokens on top of input_tokens that already include them * fix(decisions): honor litellm OpenAI key and base settings for the openai provider OpenAI decisions now resolve the key from litellm.api_key, litellm.openai_key, then OPENAI_API_KEY, and the base from litellm.api_base, OPENAI_BASE_URL, then OPENAI_API_BASE, matching other OpenAI calls. Providers own their configured key and base lookup through the endpoint config. * test(decisions): clear every OpenAI base setting in the proxy OpenAI deployment test * fix(decisions): send OpenAI instructions for System One questions without them and reject single-option questions * fix(decisions): return guardrail_information on OpenAI-format decisions when requested * refactor(decisions): validate proxy request data before reading guardrail settings	2026-10-08 04:57:41 +00:00
Mateo Wang	8fc31b1e4e	feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions (#45184) ... * feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions The System One request format moves to /v1/systemone and /systemone. /v1/decisions and /decisions now accept the OpenAI Decisions API format, translate it into a System One request, route it through the same pipeline, and translate the answers back. * fix(decisions): import assert_never from typing_extensions for Python 3.10 * fix(decisions): cap OpenAI-format questions at the System One limit /v1/decisions accepted up to 200 questions, but the System One request it translates to takes at most 128, so 129 to 200 questions failed with an internal validation error. Both now share MAX_DECISION_QUESTIONS. * fix(decisions): return 400 for bodies that are not JSON * test(decisions): send System One bodies to /v1/systemone in integration tests * fix(decisions): return guardrail_information on OpenAI-format decisions when requested * refactor(decisions): validate proxy request data before reading guardrail settings	2026-10-07 21:41:03 -07:00
Mayuri	9506f9e58b	fix(proxy): admit litellm_proxy/hosted_vllm in provider-endpoint discovery (#38617) ... * fix(proxy): admit litellm_proxy/hosted_vllm in provider-endpoint discovery get_provider_models gated all endpoint discovery on membership in the static litellm.models_by_provider dict, which litellm_proxy and hosted_vllm are intentionally absent from since their model list only exists behind the provider's own endpoint. GET /v1/models returned the literal wildcard string instead of the expanded model list for these providers. Now falls through to ProviderConfigManager, which already knows about them, before giving up. * fix(tests): stop patching an SDK internal in the discovery regression test TQ008 flagged patching litellm.proxy.auth.model_checks.get_valid_models. Assert the gate's own return value instead of mocking past it. * chore: drop redundant comment per repo convention * test(proxy): cover litellm_proxy wildcard model discovery end to end --------- Signed-off-by: mayuriphad <163738104+mayuriphad@users.noreply.github.com>	2026-10-07 21:15:57 -07:00
ishaan-berri	03ba79d261	feat(lens): show end-user feedback on traces, stored in ClickHouse (#45171) ... * feat(lens): add lens_feedback ClickHouse table for human trace scores ReplacingMergeTree(UpdatedAt, IsDeleted) keyed like agent_traces_by_key so feedback joins traces in sort order, with a bloom filter on TraceId, a score CHECK, and the same retention as traces Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens): add scoped ClickHouse reads for trace feedback feedback_target resolves a visible trace's team, key and trace_ref before a write; feedback lists the latest live entry per author; feedback_summary returns count, average and lowest score for a batch of traces Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(lens): pin the Claude session to trace id hash shared with feedback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens): expose feedback queries to the Python trace bridge Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens): add typed feedback models and ClickHouse feedback store Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens): add /lens/feedback API for 0-10 human scores on traces PUT saves or replaces the caller's score and comment, GET lists every entry, DELETE removes the caller's, POST /summary feeds the trace list. Callers can target a trace_id or a Claude Code session_id Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens-ui): add typed feedback calls to the traces API Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens-ui): flag rated traces and filter the list by feedback A Feedback column shows each run's average score and rating count, marked red when any score is 4 or below, and a filter narrows to rated or low-score runs through the URL Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens-ui): show and edit human feedback inside a trace Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens): let app keys post end-user feedback on their own traces Any key can now PUT/DELETE feedback on traces its team or key sent, naming the end user in an optional user field. Reading feedback stays with Lens admins Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(lens-ui): show end-user feedback first in a run and flag low-scored rows Replace the edit popover and feedback filter with a read-only view: the list shows each run's score in a fixed-width badge and tints the row red when a user scored it 4 or below, and opening a run shows what users said with their score right under the header Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 20:14:30 -07:00
devin-ai-integration[bot]	c912236fa5	fix(proxy): declare the Moyai settings write's service target and allowlist its routes (#45220) ... * fix(proxy): declare the config_params service target for the Moyai UI settings write Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): exercise the Moyai settings write against a real DualCache Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): allowlist the Moyai quick-connect routes on the backend component Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): observe the Moyai settings write target through a composed DualCache Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-07 20:03:44 -07:00
devin-ai-integration[bot]	4f49f88172	fix(proxy): keep idle responses websockets open until a configurable session limit (#44433) ... * add config-driven bound to /responses websocket connection * test(proxy): cover responses websocket session limit and idle first frame Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ci): sync API schema and Ruff formatting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): close responses websocket client before upstream cleanup at session limit Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): assert recorded websocket outcomes and simplify session reaping Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): move responses websocket session limit timing coverage to integration Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): run provider-close checks off the event loop Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover handshake auth, subprotocol, DB override and worker kill for the responses websocket session limit Adds the audit cells for the configurable session limit: a rejected bearer fails the handshake with 403, a requested Sec-WebSocket-Protocol is echoed back, a DB override through /config/field/update caps new sessions on every worker without a restart while sessions accepted before it keep their limit, deleting the override restores the default, out-of-range and wrong-type updates answer 400 and leave the stored row alone, and SIGKILLing the worker holding idle sockets drops only its sockets while the proxy keeps serving * test(integration): open override sockets until both proxy workers hold one uvicorn workers share one listening socket and a worker that wakes first accepts a whole simultaneous burst, so eight sockets opened at once can all land on one worker. The DB override cell now opens its sockets one at a time, alternating the two paths, until at least eight are open and every worker holds one, bounded at forty --------- Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-07 19:26:30 -07:00
yucheng-berri	efe0715831	fix(proxy): accept router-wide default_litellm_params in required-param validation (#44483) ... * fix(proxy): accept router-wide default_litellm_params in required-param validation The required-body-param check added in #43787 only consulted request data and deployment-level litellm_params, so a param supplied solely by router_settings.default_litellm_params (e.g. max_tokens for /v1/messages, documents for /rerank) was rejected with a 400 at route entry even though the router would have injected it during dispatch. Consult the router-wide defaults in the same place, treating None-valued defaults as absent to mirror the setdefault merge. * fix(proxy): consult only the defaults the dispatching router will merge Review follow-up: router-wide defaults are now consulted only where dispatch actually applies them. user_config requests dispatch on their own throwaway Router, so they consult that config's defaults instead of the global router's; search, managed-agent and eval routes, and model-less direct dispatch never pass through the router's defaults merge, so they keep the route-entry 400. Also adds rerank router-default coverage (unit + e2e) and drops the redundant source comment.	2026-10-07 19:25:28 -07:00
devin-ai-integration[bot]	02b8b5cc80	fix(proxy): traceparent/baggage fallback must not override caller metadata (#43688) ... * fix(proxy): traceparent/baggage fallback must not override caller metadata The W3C traceparent/baggage fallback in add_litellm_metadata_from_request_headers documents itself as last-resort: Lower priority than everything above - only fires when neither the explicit litellm headers nor the Anthropic-metadata path found anything But it guards on the top-level `litellm_trace_id` / `litellm_session_id` body keys and never checks `metadata`, which is the documented way callers set trace_id / session_id (`metadata: {"trace_id": ...}` on /chat/completions). A caller that explicitly sets metadata.trace_id has it silently replaced by the header value, so the implementation contradicts its own stated precedence. This is not a corner case on managed platforms: GCP's front end injects a traceparent into every inbound request, so the fallback fires on traffic whose caller never sent the header. The request still returns 200 and the trace still reaches the logging backend, just under an id the caller never chose, so any caller correlating by its own id silently fails to find its trace. Guard both fallbacks on the caller's request-body metadata as well. Deliberately narrow: - x-litellm-trace-id still outranks the body (documented priority #1) - a request that steers neither field still adopts traceparent/baggage exactly as before - steering is per-field: setting only trace_id still lets session_id come from baggage - litellm_metadata is checked too, for LITELLM_METADATA_ROUTES (/v1/responses, /v1/messages, batches, files) Also corrects the comment, which understated the guard. 4 new tests; each fails without the source change. The existing traceparent and baggage tests are unchanged and still pass. * fix(proxy): check only the active metadata container and ignore empty values Addresses review. The first version treated any value in either `metadata` or `litellm_metadata` as caller steering. On LITELLM_METADATA_ROUTES the body `metadata` is provider-facing and is only promoted later, so a session_id there suppressed baggage before apply_missing_session_id_policy ran, and a request with a usable baggage session id got a 400 under `missing_session_id: reject`. An empty session_id did the same, since the policy treats "" as absent Now only the active metadata container counts, and only a truthy value, which matches how apply_missing_session_id_policy decides a session id is present. The helper takes a typed `object` rather than a bare dict Adds regression tests for both cases, including the end to end reject path, and drops the test docstrings * fix(proxy): count promoted caller trace ids on litellm_metadata routes Addresses review. On LITELLM_METADATA_ROUTES the proxy promotes the caller's trace control fields (trace_id, session_id, ...) from `metadata` into `litellm_metadata`, but only after the header fallback runs. Checking only `litellm_metadata` let traceparent and baggage fill those fields first, and the promotion then skipped them because they were already set, so the trace was recorded under the header ids The check now also reads `metadata` for fields in LITELLM_TRACE_CONTROL_METADATA_FIELDS on those routes, and apply_missing_session_id_policy uses the same check, so `reject` no longer refuses a request whose session id is about to be promoted The earlier test asserting that a session_id in `metadata` must not block baggage on /v1/responses had the premise wrong, since that value is promoted. It is replaced by tests that assert the promoted caller ids win on /v1/responses and /v1/messages, with no policy, reject, and generate * fix(proxy): an empty trace field in litellm_metadata shadows the promoted one Addresses review. Promotion copies a trace control field from `metadata` into `litellm_metadata` only when the key is absent, so a key that is present but empty in `litellm_metadata` wins and the `metadata` value is never used. The check still looked at `metadata` in that case, which let `missing_session_id: reject` accept a request that ends up with an empty session id, and let an empty trace_id or session_id block the headers The check now follows the same rule as promotion. If the active container has the key, its value decides. Only when the key is absent does the promoted `metadata` value count. This makes the conflicting-bucket cases behave exactly as on main * fix(proxy): stay within the LIT006 cast budget The type-discipline gate failed: this branch added 4 unsuppressed `cast()` calls and LIT006 was already at its limit. Each cast now carries a `# cast-ok` reason. The two in the helper follow an `isinstance` check that proves the Mapping. The two at the call sites stay because the method's `data` parameter is a bare `dict`, and dropping those casts adds basedpyright unknown-type errors instead. No behavior change * test(proxy): cover caller metadata precedence over W3C trace headers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): generated session uses promoted caller trace id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): move test context into assertion messages Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): audit cells for w3c fallback precedence Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): tighten audit chaos cells Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(test): drop stray blank lines Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): ignore body-less model-info probes in langfuse precedence upstreams Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep root trace/session fields on equal ids and usable caller values The _caller_trace_field gating added for caller-metadata precedence was presence-aware, which regressed three pre-call behaviors versus main: - Equal ids in W3C headers and body metadata suppressed the traceparent/baggage branch entirely, leaving the root litellm_trace_id/litellm_session_id unset so router fallbacks minted a fresh uuid4 per request. The header branch now also fires when the caller value equals the header-derived id, so equal ids stamp the root fields exactly like main. - A truthy but non-string metadata value (e.g. session_id 4815162342) counted as caller-supplied and suppressed the baggage fallback, so downstream str-only consumers (code interpreter sandbox reuse) got a new sandbox per turn. _caller_trace_field now counts only non-empty string values, and the generate policy falls through to generation when the caller value is not usable. - On litellm_metadata routes a usable caller session id suppressed the missing_session_id policies while never landing on the root field, so the root session stayed unset. The policy now promotes the caller's usable session id to litellm_session_id instead of leaving it stranded. * test(integration): non-string caller ids fall back to W3C like empty ids The audit cell compared a headers leg against a headerless leg for equality, which only held while a truthy non-string id suppressed the W3C fallback on both legs. With unusable values ignored again, the headers leg resolves to the header ids while the headerless leg cannot, so assert the headers leg's concrete outcome instead, mirroring the empty/null cell. * style(proxy): trim the session promotion comment to the non-obvious why --------- Co-authored-by: Filipe Andujar <filipeandujar@gmail.com> Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-07 19:24:50 -07:00
moe-berri	dc2a14a656	feat(lens): simplify deployment and first trace setup (#45230) ... * feat(lens): simplify deployment and first trace setup * test(lens): keep setup fixtures within lint budgets * fix(lens): preserve setup state and harden bundled storage startup * fix(lens): handle setup recovery and Helm endpoint boundaries	2026-10-07 19:22:56 -07:00
Mateo Wang	58258409c9	refactor(proxy): expose public names for private proxy helpers (#45170) ... * refactor(proxy): expose public names for private proxy helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep original class names behind public aliases Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): preserve internal callback filtering Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep _PROXY_ class names for managed files hooks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep old private names in package exports Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): preserve recursive auth helper name Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): align MCP limiter tests with server enforcement Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): match main's MCP limiter tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep old private names bound in importing modules Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): preserve compatibility imports through strict lint Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): use exact pyright suppression in password helper test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(proxy): add reasons to compatibility import noqa comments Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): update IN-list baseline for renamed helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-07 18:59:27 -07:00
ishaan-berri	5531169710	feat(ui): add Moyai cloud coding agent to the view switcher (#45196) ... * feat(ui): add Moyai cloud coding agent to the view switcher Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(ui): restore hover scale on Moyai GitHub CTA Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-08 00:41:13 +00:00
joshua-berri	6498bef8db	fix(mcp): honor scoped cache freshness (#45165) ... * fix(mcp): honor upstream freshness and caller scope in discovery caches * test(mcp): type scoped freshness regression helpers * fix(mcp): age discovery freshness through cleanup * fix(mcp): preserve keyless discovery caller isolation --------- Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-10-07 17:19:24 -07:00
moe-berri	e9cfba2c17	feat(lens): isolate ingestion and investigations in a Rust service (#45148) ... * feat(lens): isolate trace storage and investigation in a Rust service * fix(lens): include Rust sources in the image build context * feat(lens): wire service setup, scoped delivery receipts and lease attempts * fix(lens): complete service routing and reject stale investigation results * fix(lens): retry key propagation and validate isolated Compose setup * fix(lens): seed through isolated ingestion and preserve upstream queue fixes * chore: sync schema.prisma copies from root * fix(lens): bind nullable due timestamps as text for Prisma * chore(ui): remove stale lint suppressions * fix(lens): address CI failures and review findings * refactor(lens): remove retired Python worker and run evaluations in Rust * fix(lens): reuse control connections and satisfy review checks * test(lens): install and upgrade both Helm charts on Kubernetes * test(lens): run connection reuse coverage as an integration test * fix(ui): upgrade Next.js to 16.3.8 security release * fix(lens): fence stale attempts and preserve reviewed evidence * Revert "fix(ui): upgrade Next.js to 16.3.8 security release" This reverts commit 2f79a51b25. * fix(lens): stop failed investigations and stream history excerpts * test(lens): cover model tool and result contracts * test(lens): fix retired routes and reuse installation build artifacts * test(lens): use portable grep in Helm installation smoke * test(lens): wait for migrations before forwarding Helm services * fix(lens): keep failed evidence reads retryable * fix(lens): preserve sandbox output during process exit --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-10-07 17:18:58 -07:00
ishaan-berri	bf9bd35469	feat(lens): scope traces to one agent with a header picker (#45202) ... * feat(lens): add trace_agents rollup query * feat(lens): register the trace_agents read query * feat(lens): add trace_agents params and row types * feat(lens): dispatch the trace_agents query * feat(lens): export trace_agents wire schemas * test(lens): pin the trace_agents query name * test(lens): cover trace_agents scope, window and failure counts * feat(lens): cap the agent list size * feat(lens): declare the trace_agents bridge query * feat(lens): read trace agents from clickhouse storage * feat(lens): add trace agent response models * feat(lens): list agents within the reader's trace scope * feat(lens): add GET /v1/traces/agents * chore(lens): regenerate trace models with trace_agents * chore(lens): regenerate trace types with trace_agents * chore(lens): regenerate read query name schema * chore(lens): add trace agent row schema * chore(lens): add trace agents params schema * test(lens): cover the trace agents route * test(lens): cover agent listing scope and timestamps * chore(ui): regenerate api types with trace agents route * feat(lens): derive trace agent types from the schema * feat(lens): fetch the agent list from the traces api * feat(lens): roll up demo runs into agents * feat(lens): serve the agent list in demo data * feat(lens): load agents seen in the last two weeks * feat(lens): remember the selected agent per browser * feat(lens): add the agent picker * feat(lens): wire agent selection into the lens header * feat(lens): show the agent picker next to the lens title * refactor(lens): drop the toolbar agent filter in favor of the header picker * test(lens): remove tests for the toolbar agent filter * test(lens): cover agent resolution and rollup * test(lens): cover scoping, switching and remembering the agent * test(lens): read only the create request in guided setup * test(lens): assert the toolbar agent filter is gone * test(lens): read stubbed requests without their abort signal	2026-10-07 23:48:04 +00:00
devin-ai-integration[bot]	e48f8d928d	feat(mcp): rate limit all MCP operations and add server-level rpm (#44600) ... * feat(mcp): rate limit all MCP operations and add server-level rpm Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(mcp): drop comments copied onto list fallbacks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): count discovery once per server and rate limit REST tools listing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): cover rate-limited catalog error propagation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): avoid fastapi import in mcp operations test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): apply server rate limits to paginated catalog listings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(mcp): restore main's unused prompt and resource listing helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): add server rate limit coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): harden rate-limit integration setup Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): prevent rejected calls from consuming shared limits Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: joshua <joshua@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-07 16:30:52 -07:00
joshua-berri	e66dbfc366	fix(mcp): preserve client application type during registration (#45159) ... Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>	2026-10-07 16:30:10 -07:00
devin-ai-integration[bot]	225fd3bcdf	feat(proxy): add denied_passthrough_routes deny list for custom pass-through endpoints (#44924) ... * feat(proxy): add denied_passthrough_routes deny list for custom pass-through endpoints Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): harden denied_passthrough_routes against non-admin clears and dot-segment paths Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ui): regenerate schema.d.ts for typed denied_passthrough_routes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): match trailing-slash deny entries, block null metadata from dropping denies Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): close bulk update, encoded ?/# and ordering gaps in passthrough deny list Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): check denied pass-through routes against the path the forwarder sends upstream Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): type matching_denied_passthrough_route metadata mappings * fix(proxy): treat a `/` deny entry as denying every pass-through route Also types the new deny-list tests and drops their get_server_root_path mock in favour of unsetting SERVER_ROOT_PATH. * test(proxy): type the deny-list tests and drop unrelated test reformatting --------- Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Yucheng He <yucheng@berri.ai>	2026-10-07 16:19:53 -07:00
ishaan-berri	9e20264607	perf(lens): classify signals within seconds of a trace finishing (#45186) ... * perf(lens): add a 2 second live signal sweep over recently finished traces Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(lens): run the live and backlog signal sweeps side by side Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(lens): cover the live sweep window and backlog draining Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(lens): keep known signal results on screen and poll every 2s while runs wait Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(lens): cover signal polling speed and results surviving list changes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(lens): show Checking instead of Queued and leave clean runs blank Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(lens): expect Checking for runs waiting on signals Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 16:09:49 -07:00
devin-ai-integration[bot]	ec0af8d5f8	fix(proxy): evict cached user on every proxy for tpm/rpm updates and edit limits in the users UI (#44130) ... * test(proxy): cover cross-worker cache eviction for user tpm/rpm limit updates Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): broadcast user cache eviction when tpm_limit or rpm_limit changes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(proxy): return tpm_limit and rpm_limit from /v2/user/info Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): edit user tpm and rpm limits from the user edit form Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): clear user tpm_limit and rpm_limit when sent as null Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover user tpm/rpm limit updates across proxies Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover user rate-limit routes and bulk updates Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): reject unsafe rate limit integers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): cover user rate limit seed and saved state Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): type user endpoint test doubles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * revert: drop rate limit upper bound Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ui): validate user rate limits with zod and clear user edit lint warnings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ui): rename user edit schema and name its input and output types Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): drop redundant event loop yields in rate limit eviction test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-07 16:02:32 -07:00
Rohan G	4417bf08ae	feat(guardrails): extend Akto guardrail to responses, MCP tools, attachments and masking (#44343) ... * feat(guardrails): extend Akto guardrail to responses, MCP tools, attachments and masking - post_call now waits for Akto and blocks or masks the reply instead of only logging it - pre_mcp_call and post_mcp_call check MCP tool arguments and tool results - attached images, audio and files are sent to Akto's file check - streamed replies are checked every streaming_sampling_rate chunks - context_source routes traffic to Akto's endpoint or agentic policies - tags carry user email, team alias and key alias for attribution * fix(guardrails): harden Akto MCP detection, keep AGENTIC default, block unmappable masking - MCP handling trusts the logger's call type, so request body keys can't skip the prompt check - context_source defaults to AGENTIC, as before - masking that also hits text we can't write back now blocks - move tests to tests/unit and rename attachments.py to akto_attachments.py - regenerate the OpenAPI snapshot and dashboard types * fix(guardrails): ignore a client-sent response in Akto output checks - a "response" field sent in the request body is no longer scanned in place of the model's reply - MCP tool calls in the reply are still checked in that case - split match or-patterns so CodeQL can follow the bound names - cover a masked payload that is not JSON and drop unused test imports * refactor(guardrails): read Akto attachment fields directly instead of pattern captures * fix(guardrails): check Akto attachments in both messages and input * fix(guardrails): check every Akto attachment source and keep more prompt text in scope - check all of a file or image block's sources (file_data, file_url, file_id), since providers pick different ones - send Anthropic search_result blocks to the file check as text - keep document title and context, and legacy functions, in the checked request - take the client IP from the proxy's requester_ip_address before client forwarding headers - read litellm_params identity only from server-side call details * fix(guardrails): never drop an Akto attachment the text check removed - optional metadata (filename, title, format, media type) that isn't a string is ignored instead of failing the block - an attachment block that still can't be read blocks the request - search_result text is checked once, as a file, instead of also in the text check * fix(guardrails): strip only what the Akto file check sends from the text check - the text check keeps every attachment field except the ones the file check sends - document title/context and search_result source/title go to the file check as text, since the /v1/messages text check drops them - accept every image shape LiteLLM forwards (image_url or url, string or object) and check each source - ignore blocks whose type is not a string instead of failing the file check * fix(guardrails): keep model-visible text in the Akto text check - document title/context, text documents and search_result stay in the text check, so no Akto backend skips them - on /v1/messages the text check reads the messages Anthropic receives, with the guardrail's skip/scan scoping applied - a client-sent "response" key can only add reply checks, never skip recording or MCP tool-call checks - a "messages" key on the Responses API can't replace its input in the text check - the recorded IP comes only from the proxy's requester_ip_address * fix(guardrails): keep AktoGuardrail positional args backward compatible * fix(guardrails): keep zero Akto timeouts working and record every stream check A guardrail_timeout of 0 used to fall back to the default; the new ge=1 made the config invalid, so the proxy dropped the guardrail. Zero settings now fall back to the defaults again. The end-of-stream check can be skipped when the last sampled check covered the reply, so mid-stream checks now record, like base. * fix(guardrails): use defaults for non-positive Akto timeouts and sampling rate A zero or negative guardrail_timeout, file_guardrail_timeout or streaming_sampling_rate used to reach the HTTP call or the stream cadence. They now fall back to the defaults, like unset values.	2026-10-07 15:50:38 -07:00
tin-berri	2803a16b36	feat(ui): add auto-router usage and savings table (#45152)	2026-10-07 15:04:07 -07:00
devin-ai-integration[bot]	fa2c8984ba	test: move the unit half of 126 mixed legacy files into tests/unit (#45090) ... * test: move the unit half of 126 mixed legacy files into tests/unit * test: restore litellm globals that moved tests set * test: finalize migration test cleanup * test: restore original bodies of moved legacy tests The move into tests/unit had rewritten 612 test bodies, and some of the rewrites dropped assertions. Each moved test now carries its original body from the legacy file, with only the imports, helpers, fake provider credentials and monkeypatched env it needs to run under tests/unit test_timeout_streaming goes back to tests/local_testing because it needs the fake OpenAI endpoint server. The image payload fixture moves with its only user, and two tests that leaked global state (a registered model cost entry and queued logging tasks) are now isolated * test: drop module imports shadowed by restored local imports * test: assert on LiteLLM output in no-assertion moved tests and isolate leaks Twenty no-assertion candidates get one assertion on the value LiteLLM returns, with the original lines unchanged. Four tests go back to their legacy files because they only check types or imports, write into the working directory, or cannot assert without a body change Two moved tests leaked globals into later tests in the same worker, so monkeypatch fixtures now restore the retry-after header parser and the end user cost tracking flags * test: drain queued logging tasks before the Phoenix span test The moved Phoenix test counted spans from logging tasks that earlier tests had queued, so the drain fixture moves to tests/unit/conftest.py and both it and the Datadog batch test use it. test_factory_function goes back to its legacy file because its returned wrapper calls the real Assistants API and cannot be asserted on without a body change --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-07 14:07:43 -07:00
devin-ai-integration[bot]	127278f951	feat(guardrails): add logging_only_scope to observe input, output, or both (#43695) ... * feat(guardrails): add logging_only_scope to observe one direction Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(guardrails): remove unrelated test churn Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): allow native lifecycle logging-only scope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ui): regenerate api types for logging_only_scope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): remove callbacks when scope validation fails Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep output-only scans when request copy fails Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): configure logging_only_scope on guardrails Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep guardrails enforcing when logging_only_scope is invalid at load Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(ui): avoid inline guardrail test fixtures Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): support directional scope in PATCH and provider UI Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): format guardrail files for frontend lint Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): reset unsupported directional scope selections Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): normalize logging-only scope and sanitize warning logs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): normalize scope in shared guardrail field Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(api): sync guardrail schema artifacts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): tolerate invalid stored logging-only scopes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): add logging_only_scope integration audit cells Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): fix scope test lint issues Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(guardrails): extend K4 chaos test timeout Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): restore stored guardrail row verbatim on rejected patch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): satisfy collection lint budgets Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): validate masked params through TypeAdapter Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): tolerate invalid stored params on reads Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep constructor coercion in tolerant params parser Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(ui): edit logging_only_scope in the custom code guardrail modal Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(ui): extract custom code logging-only scope control Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep no-op PUT and logging_only scan-failure semantics stable Three regressions from the logging_only_scope feature, fixed while keeping the input/output/both selection working: 1. PUT with byte-identical litellm_params no longer forces a teardown + re-init of the live callback. reject_invalid_logging_only_scope now only validates; re-init still happens exactly when params/name change. Previously a description-only PUT re-appended the callback at the END of litellm.callbacks, reordering guardrails: with a BLOCK guardrail created before a MASK one, the mask started winning and blocked requests started succeeding (400 -> PUT 200 -> 200). Invalid unchanged scopes are still rejected with 422 without touching the live instance. 2. CustomGuardrail._scan_logged_call no longer swallows input-scan exceptions per branch: a raising or BLOCKED input scan aborts the logging_only hook again (one policy call, one verdict) for guardrails that never selected a logging_only_scope. Explicit input/output/both selection keeps selecting which scans run. 3. A failed PATCH now rolls the DB row back to exactly the stored raw litellm_params (a legacy 4-key row stays 4 keys) instead of expanding it to a full LitellmParams dump; pinned with a test. This matches the PUT rollback shape and is the faithful rollback. * refactor(guardrails): type directional-scope provider list as a tuple * fix(lint): stay within the basedpyright budget - drop a Final annotation assigned inside the validation loop (reportGeneralTypeIssues over ceiling by one) - rename the PR-introduced _configured_event_hooks to public configured_event_hooks; its cross-module import added the two reportPrivateUsage errors that pushed the rule over its ceiling * fix(guardrails): keep the response verdict for explicit logging_only_scope=both on input-scan failure Greptile P1: an explicitly configured both-direction observer asked for a verdict on each direction, so a failed request scan must not silently drop the response verdict. The implicit default (logging_only_scope None) keeps the abort semantics of a logging_only hook whose scan raised, which is the base behavior the earlier fix restored. Also drops two test comments that restated their assertions (Greptile P2). * test(guardrails): align logging_only_scope integration rows with abort-on-input-failure and encrypted params Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(guardrails): keep the response scan when the request copy fails for explicit both scope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-07 14:03:33 -07:00
devin-ai-integration[bot]	8caf271e6c	feat(lens): flag traces with global System 1 signals (#45094) ... * feat(lens): add global trace signals Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * feat(lens): show global trace signals in Settings and Traces Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(proxy): resolve lens signal router at call time Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * feat(lens): signal library, all signal pills and an always-on Signals column Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(lens): tidy signal setup and stop trace ids crowding signal pills Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(lens): address review on signal scans, claims and settings drafts Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(lens): resume signal scan at a partially consumed page Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(lens): keep custom signals separate from library signals Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(lens): give toggled library signals their own row keys Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(lens): restore signal toggle-off checks Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(lens): map signal spans and remove recursive helpers Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>	2026-10-07 13:27:22 -07:00