* feat(proxy): add denied_passthrough_routes deny list for custom pass-through endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): harden denied_passthrough_routes against non-admin clears and dot-segment paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for typed denied_passthrough_routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): match trailing-slash deny entries, block null metadata from dropping denies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): close bulk update, encoded ?/# and ordering gaps in passthrough deny list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): check denied pass-through routes against the path the forwarder sends upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type matching_denied_passthrough_route metadata mappings
* fix(proxy): treat a `/` deny entry as denying every pass-through route
Also types the new deny-list tests and drops their get_server_root_path mock in favour of unsetting SERVER_ROOT_PATH.
* test(proxy): type the deny-list tests and drop unrelated test reformatting
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yucheng He <yucheng@berri.ai>
* perf(lens): add a 2 second live signal sweep over recently finished traces
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* perf(lens): run the live and backlog signal sweeps side by side
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens): cover the live sweep window and backlog draining
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(lens): keep known signal results on screen and poll every 2s while runs wait
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens): cover signal polling speed and results surviving list changes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(lens): show Checking instead of Queued and leave clean runs blank
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens): expect Checking for runs waiting on signals
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): cover cross-worker cache eviction for user tpm/rpm limit updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast user cache eviction when tpm_limit or rpm_limit changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): return tpm_limit and rpm_limit from /v2/user/info
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): edit user tpm and rpm limits from the user edit form
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): clear user tpm_limit and rpm_limit when sent as null
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover user tpm/rpm limit updates across proxies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover user rate-limit routes and bulk updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): reject unsafe rate limit integers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover user rate limit seed and saved state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): type user endpoint test doubles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert: drop rate limit upper bound
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): validate user rate limits with zod and clear user edit lint warnings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): rename user edit schema and name its input and output types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drop redundant event loop yields in rate limit eviction test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(streaming): stop re-wrapping a bridged stream's MidStreamFallbackError
A chat completion served over the Responses API already gets its mid-stream
error wrapped by the Responses iterator. The chat stream wrapper wrapped it a
second time, and the router unwraps one layer, so the client saw the inner
sentinel (message prefixed litellm.MidStreamFallbackError, type null) instead
of the provider's RateLimitError. The chat wrapper now re-raises an already
wrapped error untouched.
* test(streaming): type the bridged stream regression test's locals
* fix(streaming): rebuild a bridged mid-stream error with the outer wrapper's bookkeeping
* test(integration): cover the bridged stream error typing on every surface
Checked-in audit cells for the Responses bridge: chat completions through the OpenAI SDK and httpx, /v1/messages through httpx and the Anthropic SDK, native /v1/responses, the litellm and Router SDK stream paths, and a chaos file with a mixed burst, a worker SIGKILL and a proxy SIGTERM against an owned two-worker proxy. Every cell scripts the provider through a wire server and asserts the caller's body, the upstream's received requests and the spend row by id.
* test(integration): read bridged chat content through one helper
* test(integration): build the bridged fallback config without mutating the loaded yaml
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(guardrails): extend Akto guardrail to responses, MCP tools, attachments and masking
- post_call now waits for Akto and blocks or masks the reply instead of only logging it
- pre_mcp_call and post_mcp_call check MCP tool arguments and tool results
- attached images, audio and files are sent to Akto's file check
- streamed replies are checked every streaming_sampling_rate chunks
- context_source routes traffic to Akto's endpoint or agentic policies
- tags carry user email, team alias and key alias for attribution
* fix(guardrails): harden Akto MCP detection, keep AGENTIC default, block unmappable masking
- MCP handling trusts the logger's call type, so request body keys can't skip the prompt check
- context_source defaults to AGENTIC, as before
- masking that also hits text we can't write back now blocks
- move tests to tests/unit and rename attachments.py to akto_attachments.py
- regenerate the OpenAPI snapshot and dashboard types
* fix(guardrails): ignore a client-sent response in Akto output checks
- a "response" field sent in the request body is no longer scanned in place of the model's reply
- MCP tool calls in the reply are still checked in that case
- split match or-patterns so CodeQL can follow the bound names
- cover a masked payload that is not JSON and drop unused test imports
* refactor(guardrails): read Akto attachment fields directly instead of pattern captures
* fix(guardrails): check Akto attachments in both messages and input
* fix(guardrails): check every Akto attachment source and keep more prompt text in scope
- check all of a file or image block's sources (file_data, file_url, file_id), since providers pick different ones
- send Anthropic search_result blocks to the file check as text
- keep document title and context, and legacy functions, in the checked request
- take the client IP from the proxy's requester_ip_address before client forwarding headers
- read litellm_params identity only from server-side call details
* fix(guardrails): never drop an Akto attachment the text check removed
- optional metadata (filename, title, format, media type) that isn't a string is ignored instead of failing the block
- an attachment block that still can't be read blocks the request
- search_result text is checked once, as a file, instead of also in the text check
* fix(guardrails): strip only what the Akto file check sends from the text check
- the text check keeps every attachment field except the ones the file check sends
- document title/context and search_result source/title go to the file check as text, since the /v1/messages text check drops them
- accept every image shape LiteLLM forwards (image_url or url, string or object) and check each source
- ignore blocks whose type is not a string instead of failing the file check
* fix(guardrails): keep model-visible text in the Akto text check
- document title/context, text documents and search_result stay in the text check, so no Akto backend skips them
- on /v1/messages the text check reads the messages Anthropic receives, with the guardrail's skip/scan scoping applied
- a client-sent "response" key can only add reply checks, never skip recording or MCP tool-call checks
- a "messages" key on the Responses API can't replace its input in the text check
- the recorded IP comes only from the proxy's requester_ip_address
* fix(guardrails): keep AktoGuardrail positional args backward compatible
* fix(guardrails): keep zero Akto timeouts working and record every stream check
A guardrail_timeout of 0 used to fall back to the default; the new ge=1 made the config invalid, so the proxy dropped the guardrail. Zero settings now fall back to the defaults again.
The end-of-stream check can be skipped when the last sampled check covered the reply, so mid-stream checks now record, like base.
* fix(guardrails): use defaults for non-positive Akto timeouts and sampling rate
A zero or negative guardrail_timeout, file_guardrail_timeout or streaming_sampling_rate used to reach the HTTP call or the stream cadence. They now fall back to the defaults, like unset values.
* test(e2e): move the harness self-tests out of tests/e2e
The nightly Buildkite run copies tests/e2e into the runner image and runs
bare pytest, so the 672 tests of the harness itself (fixture parsing, JUnit
properties, the stack lock, the load aggregators, the Claude Code driver)
counted as e2e tests on the status page even though none of them reaches a
proxy. They now live in tests/e2e_harness, mirroring the tests/e2e layout,
and run in the GitHub Actions lint job and the CircleCI
provider_replay_harness job instead
* fix(ci): point the providers replay controls at tests/e2e_harness
The providers integration job still selected the four replay-control
tests under tests/e2e/test_provider_edge.py, so pytest exited before
they ran. The raw-HTTP check's file walk also drops to one loop per
comprehension
* style(tests): mark the raw-HTTP check's bindings Final
* refactor(rust): separate Messages settings from capability inputs
* refactor(rust-bridge): unify Messages OCR and Responses call inputs
* refactor(rust-bridge): share NativeCall across inference entrypoints
* fix(rust-bridge): preserve Responses URL aliases and public test inputs
* feat(types): declare above_100k_tokens price fields on ModelInfoBase
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 186f6c81a0)
* fix(router): mirror above_100k pricing fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 75c9c3ff1e)
* chore(ui): regenerate schema.d.ts for above_100k pricing fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 53163d2184)
* refactor(types): mark above_100k ModelInfoBase fields ReadOnly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit febed3199a)
* fix(model_prices): bill claude-haiku-5-5 long prompts on every provider and in batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 218c00cf5a)
* fix(cost): pass *_above_Nk_tokens_batches rates through get_model_info
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 37b6f2fba0)
* fix(model_prices): allow disabling thinking and forced tool use on claude-haiku-5-5
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 6e91e0ed36)
* test(model_prices): cite the vendor source for claude-haiku-5-5 capability flags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 1f97bb27b4)
* test(model_prices): type and tidy the claude-haiku-5-5 config tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 3d2526035f)
* feat(bedrock): add claude haiku 5.5 over 100k token tier
Price-Sync: litellm-providers
(cherry picked from commit ee3822c29c)
* feat(model_prices): add openrouter claude-haiku-5.5
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model_prices): dedupe above_100k pricing keys from text merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model_prices): bill vertex claude-haiku-5-5 prompts over 100k tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model_prices): add adaptive thinking and cache minimum to openrouter claude-haiku-5.5
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com>
* test: move the unit half of 126 mixed legacy files into tests/unit
* test: restore litellm globals that moved tests set
* test: finalize migration test cleanup
* test: restore original bodies of moved legacy tests
The move into tests/unit had rewritten 612 test bodies, and some of the rewrites dropped assertions. Each moved test now carries its original body from the legacy file, with only the imports, helpers, fake provider credentials and monkeypatched env it needs to run under tests/unit
test_timeout_streaming goes back to tests/local_testing because it needs the fake OpenAI endpoint server. The image payload fixture moves with its only user, and two tests that leaked global state (a registered model cost entry and queued logging tasks) are now isolated
* test: drop module imports shadowed by restored local imports
* test: assert on LiteLLM output in no-assertion moved tests and isolate leaks
Twenty no-assertion candidates get one assertion on the value LiteLLM returns, with the original lines unchanged. Four tests go back to their legacy files because they only check types or imports, write into the working directory, or cannot assert without a body change
Two moved tests leaked globals into later tests in the same worker, so monkeypatch fixtures now restore the retry-after header parser and the end user cost tracking flags
* test: drain queued logging tasks before the Phoenix span test
The moved Phoenix test counted spans from logging tasks that earlier tests had queued, so the drain fixture moves to tests/unit/conftest.py and both it and the Datadog batch test use it. test_factory_function goes back to its legacy file because its returned wrapper calls the real Assistants API and cannot be asserted on without a body change
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* feat(guardrails): add logging_only_scope to observe one direction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(guardrails): remove unrelated test churn
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): allow native lifecycle logging-only scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate api types for logging_only_scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): remove callbacks when scope validation fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep output-only scans when request copy fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): configure logging_only_scope on guardrails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep guardrails enforcing when logging_only_scope is invalid at load
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): avoid inline guardrail test fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): support directional scope in PATCH and provider UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): format guardrail files for frontend lint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): reset unsupported directional scope selections
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): normalize logging-only scope and sanitize warning logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): normalize scope in shared guardrail field
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(api): sync guardrail schema artifacts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): tolerate invalid stored logging-only scopes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): add logging_only_scope integration audit cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): fix scope test lint issues
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): extend K4 chaos test timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): restore stored guardrail row verbatim on rejected patch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): satisfy collection lint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): validate masked params through TypeAdapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): tolerate invalid stored params on reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep constructor coercion in tolerant params parser
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): edit logging_only_scope in the custom code guardrail modal
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): extract custom code logging-only scope control
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep no-op PUT and logging_only scan-failure semantics stable
Three regressions from the logging_only_scope feature, fixed while keeping
the input/output/both selection working:
1. PUT with byte-identical litellm_params no longer forces a teardown +
re-init of the live callback. reject_invalid_logging_only_scope now
only validates; re-init still happens exactly when params/name change.
Previously a description-only PUT re-appended the callback at the END
of litellm.callbacks, reordering guardrails: with a BLOCK guardrail
created before a MASK one, the mask started winning and blocked
requests started succeeding (400 -> PUT 200 -> 200). Invalid unchanged
scopes are still rejected with 422 without touching the live instance.
2. CustomGuardrail._scan_logged_call no longer swallows input-scan
exceptions per branch: a raising or BLOCKED input scan aborts the
logging_only hook again (one policy call, one verdict) for guardrails
that never selected a logging_only_scope. Explicit input/output/both
selection keeps selecting which scans run.
3. A failed PATCH now rolls the DB row back to exactly the stored raw
litellm_params (a legacy 4-key row stays 4 keys) instead of expanding
it to a full LitellmParams dump; pinned with a test. This matches the
PUT rollback shape and is the faithful rollback.
* refactor(guardrails): type directional-scope provider list as a tuple
* fix(lint): stay within the basedpyright budget
- drop a Final annotation assigned inside the validation loop
(reportGeneralTypeIssues over ceiling by one)
- rename the PR-introduced _configured_event_hooks to public
configured_event_hooks; its cross-module import added the two
reportPrivateUsage errors that pushed the rule over its ceiling
* fix(guardrails): keep the response verdict for explicit logging_only_scope=both on input-scan failure
Greptile P1: an explicitly configured both-direction observer asked for a
verdict on each direction, so a failed request scan must not silently drop
the response verdict. The implicit default (logging_only_scope None) keeps
the abort semantics of a logging_only hook whose scan raised, which is the
base behavior the earlier fix restored.
Also drops two test comments that restated their assertions (Greptile P2).
* test(guardrails): align logging_only_scope integration rows with abort-on-input-failure and encrypted params
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep the response scan when the request copy fails for explicit both scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(openai-compat): send provider attribution headers on the default SDK path
Provider-specific headers set in validate_environment are never sent for
OpenAI-compatible providers on the default OpenAI SDK path, which doesn't
call it. Novita's X-Novita-Source has been silently missing as a result.
Add BaseConfig.get_attribution_headers() and merge it into the outbound
headers in _complete_custom_openai, which feeds both the SDK and the
experimental http-handler paths. Caller headers win, case-insensitively.
* feat(perplexity): send X-Pplx-Integration attribution header
Ports the change from #38565 onto the attribution-header hook so it is
sent on the default SDK path too.
Co-authored-by: Saleh Alghusson <1331721+qirh@users.noreply.github.com>
* test: capture attribution headers in-process instead of over a socket
Address review: unit tests now drive litellm.completion into an httpx
transport rather than a local HTTP server; type the header helper as
dict[str, str]; bind the merged headers to a Final instead of
rebinding headers in _complete_custom_openai.
* style: drop explanatory comments flagged by review
---------
Co-authored-by: Saleh Alghusson <1331721+qirh@users.noreply.github.com>
* test(e2e): cover ollama and ollama_chat on chat completions, responses and messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): declare Subject metadata and check streamed tool call ids in the ollama suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(lens): bound single trace reads by the sampled start time
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(lens): allow unused query fixture field in load tests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(lens): pass start_time in every lens content and evidence test
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(logging): bound data URI regex so base64 truncation stays linear
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(logging): cover whitespace-free data: prefixes in data URI regex regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(lens): claim worker jobs from an indexed due queue instead of scanning every lens
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): apply the due_at index concurrently on its own and default legacy rows to due
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(lens): pass repository to claim lifecycle tests
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): page past unsupported due lenses and declare the full due_at index
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(lens): claim due lenses in a loop instead of recursion
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(realtime): skip guardrail VAD session.update injection for transcription sessions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): run transcript guardrails on raw-path transcription sessions with a transcription-safe block
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): assert transcription session keeps transcribing after a guardrail block
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): only expect a follow-up transcript when the block keeps the session open
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): type the transcription block regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): flag transcription sessions from the route intent and backend events only
A client session.update declaring session.type transcription on a voice
session no longer sets the transcription flag, so it cannot switch off the
guardrail's create_response gate or skip the transcript guardrail
* fix(realtime): flag transcription sessions from provider-transformed session events
* test(realtime): cover transcript guardrail blocks on transcription sessions
---------
Co-authored-by: gabriele <gabriele@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
litellm_llm_api_latency_metric, litellm_llm_api_time_to_first_token_metric,
litellm_request_total_latency_metric, and litellm_deployment_latency_per_output_token
previously carried requested_model/litellm_model_name/model_id but not model_group,
so pooled-deployment latency couldn't be grouped by model pool on dashboards --
only the proxy-overhead-only metrics (litellm_overhead_latency_metric and friends)
had model_group. All four metrics read enum_values.model_group through the existing
prometheus_label_factory plumbing, so no new parameter threading was needed, just
the label-list addition.
Co-authored-by: ahamedshaik16 <24526479+ahamedshaik16@users.noreply.github.com>
* fix(router): preserve native baseline identity and accounting
* fix(router): preserve injected system caches in native baselines
* fix(router): prepare native baselines through shared request owners
* fix(router): capture native baseline fields from the provider schema
* refactor(router): reuse native provider parameter discovery
* fix(router): keep long native baselines and abstain after compaction
Message history no longer counts against the settings snapshot budget, so long
and non-ASCII native sessions keep modeled baselines. Selected-tier compaction
now abstains because the baseline would otherwise inherit the compacted history.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(router): judge implicit caching against the selected request
The implicit-cache guard compared the selected response's cache usage with the
projected baseline's breakpoints, so selected-tier cache markers made a usable
unmarked baseline plan look like unexplained caching. The guard now checks the
selected wire request. Also removes a stamp-reuse branch that could never run
because routing clears the stamp first; every pass already captures caller
settings from fresh kwargs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(caching): keep tool calls and tool results in semantic cache prompts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): keep semantic tool prompt helpers within lint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): keep structured function_call_output text in semantic prompts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): split Responses text-field collection to stay within complexity budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): tag each tool result with the position of the call it answers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): encode tool result position and output together so tool text cannot forge result tags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(caching): expect encoded tool result record in qdrant semantic prompt parity case
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(caching): cover tool result arrangements, SDK clients, concurrency and qdrant outage for semantic cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(caching): embed every semantic cache prompt field except volatile ones
Replace the per-shape allowlist in the Python and Rust semantic cache prompt
walkers with one include-by-default walker. Plain text keeps its old
concatenation; any other block or message is embedded as compact JSON with
call ids mapped to ordinals, cache_control dropped, and signatures, encrypted
content and base64 data replaced with a short sha256 digest.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(code-quality): allow the bounded semantic cache prompt walkers in the recursion check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): expect structured JSON for unknown fields in redis and valkey semantic prompts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(rust): expect structured JSON for unknown fields in redis and valkey semantic prompts"
This reverts commit 86c82b949b.
* Revert "test(code-quality): allow the bounded semantic cache prompt walkers in the recursion check"
This reverts commit 39efb5d9da.
* Revert "feat(caching): embed every semantic cache prompt field except volatile ones"
This reverts commit 5aed3ab3de.
* refactor(caching): rename get_str_from_messages_with_tools to get_semantic_cache_prompt_from_messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): split semantic cache prompt extraction by API format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): drop TypeIs guard and register Responses prompt walker with the recursion check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): pick the Responses text field without a Final inside a loop
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): walk semantic cache prompts as plain dicts, dumping pydantic items once up front
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): write the semantic cache prompt builders as plain loops
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): skip the cache past max_messages and keep tool_result text in semantic prompts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(caching): drop formatting-only churn from the redis semantic cache tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): drop the caching group wiring that main already carries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): recurse into tool_result content in the semantic cache prompt helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): check the max_messages cap on the shared exact-cache proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): read list-form function_call_output text in semantic cache prompts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): extract nested Responses input lookup to keep walker under complexity limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "refactor(caching): extract nested Responses input lookup to keep walker under complexity limit"
This reverts commit 0665296bf1.
* style(caching): suppress C901 on the Responses input walker instead of splitting it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add litellm_settings.force_redis_hash_tag_grouping so Redis endpoints that enforce cluster slot rules behind a standalone protocol (Redis Enterprise clustering policy) group multi-key scripts by slot like RedisClusterCache, and return grouped batch values in the caller's key order.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ollama): turn streamed prompt-based JSON tool calls into real tool calls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ollama): separate replayed tool calls from text and tighten parser typing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): attribute background health check results to their own deployment
Co-authored-by: Dennis Pfisterer <302635+pfisterer@users.noreply.github.com>
Co-authored-by: Suhas Hanamannavar <hanamannavarsuhas17@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(health): annotate locals with Final and split nested comprehension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Dennis Pfisterer <302635+pfisterer@users.noreply.github.com>
Co-authored-by: Suhas Hanamannavar <hanamannavarsuhas17@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(auth): clear the recent-miss user memo when /user/new creates the user
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(auth): pin the miss memo window and drop the class patch in the new_user regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin a first-time SSO user's first message and second sign-in on one worker
* test(integration): cover /user/new clearing the user-miss memo on the creating worker
The fake IdP now signs in the subject a login_hint names, so cells on the shared one-worker proxy can each use a fresh user. New cells: a plain-key miss followed by /user/new is budgeted at once (same on both legs, the auth prefetch loads the row) and the admin-created user's first SSO sign-in inside the window completes (500 at the callback before the fix). The first-sign-in cell moved onto the shared one-worker proxy fixture
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): inject the cleanup job stagger offset so the runtime sync test no longer depends on the runner's host and pid
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(policy_engine): count resolver calls instead of timing them so the linear dedup guard is deterministic under CI load
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(policy_engine): annotate the line-count guard's locals as Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ci): stop deferred pydantic builds leaking caller locals and add missing Lens FK migration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(types): drop narrating comment from caller-locals regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): guard the Lens review FK migration with DO blocks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(types): run the caller-locals regression in-process
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy-extras): scope the Lens review FK guards to their table
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): skip guardrail VAD session.update injection for transcription sessions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): flag transcription sessions from the route intent and backend events only
A client session.update declaring session.type transcription on a voice
session no longer sets the transcription flag, so it cannot switch off the
guardrail's create_response gate or skip the transcript guardrail
* fix(realtime): flag transcription sessions from provider-transformed session events
* test(integration): cover transcription sessions skipping the VAD auto-response injection
Adds the realtime transcript guardrail audit cells: transcription sessions on all three
realtime routes, the OpenAI SDK, the beta protocol, the whisper default deployment, the Azure
GA path over a TLS scripted upstream, and Meta Muse push-to-talk sessions keep the client's
session.update verbatim and get their transcript, while voice sessions keep the injected
create_response gate and a client-declared transcription type no longer bypasses it. Sad,
edge, and chaos cells cover malformed session fields, duplicate and older backend session
events, unauthenticated upgrades, repeated sessions, an upstream outage under open sessions,
and a worker kill with a proxy restart.
The scripted upstream now answers session.update the way the vendor does (session.updated,
or the missing turn_detection.type and session-type errors), records every websocket frame,
serves the Azure and Muse realtime paths, and can run over TLS from an owned upstream.
---------
Co-authored-by: gabriele <gabriele@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(mcp): run the static root path issuer discovery test in-process
The test spawned a fresh interpreter with a 60 s deadline to import litellm
and the MCP discovery router cold, so on a loaded box it died with
subprocess.TimeoutExpired before any assertion ran. The discovery routes
bake SERVER_ROOT_PATH into their paths when the module executes, so the
test now reloads that one module under the gateway env, restores its
namespace afterwards, and asserts on the same four discovery documents.
PROXY_BASE_URL now names an origin distinct from the test client's, so the
assertions fail when it stops being honored.
* test(mcp): type the gateway discovery fixture and its test
* test(mcp): mark the registry fill the fixture hands the test
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(bedrock): live e2e asserting nova sonic realtime delivers each assistant sentence once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): drop ticket reference from nova sonic e2e docstring
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): forward each Nova Sonic assistant sentence once over the realtime API
Nova 2 Sonic sends every assistant text block twice, a SPECULATIVE preview
next to the audio and a FINAL transcript once the audio turn has ended. The
realtime bridge forwarded both, so voice clients rendered each sentence twice
and the FINAL copies opened extra responses after response.done, the last of
which never closed. FINAL assistant text blocks are now dropped whole, so a
turn carries each sentence once inside the one response with its audio
* test(bedrock): tag the live Nova Sonic test and type its helpers
* test(bedrock): pin that the Nova Sonic barge-in marker is dropped with its FINAL block
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: delete unconditionally skipped legacy tests
* test: move whole-unit legacy test files into tests/unit
* test: keep moved legacy tests free of import-time global state
* test: keep the module-level invocation scan pointed at tests/local_testing
* ci: drop the agent_testing CircleCI job emptied by the move
* test: fix moved-test isolation and router coverage
* test: add Tinyfish search package marker
* test: isolate moved tests from logger state leaks
* test: isolate Helicone logging fixture state
* test: isolate Vertex pass-through credentials between moved tests
* test: cancel S3 periodic flush tasks started by moved tests
* ci: restore CircleCI assistant test selection after move
* test: prevent Datadog datetime import shadowing
* ci: drop the litellm_assistants_api_testing CircleCI job emptied by the move
* test: deduplicate imports in rebased unit tests
* test: remove duplicate passthrough router patch import
* test: remove moved legacy source files after rebase
* test: align moved tests with rebased main
* test: carry main's legacy-file edits into moved destinations
* test: make the moved cost map fallback tests assert the fetch and the backup
The four fallback cases only checked the result was non-empty, so they still
passed with integrity validation disabled. They now inject a mock client, assert
one fetch happened, and assert the result is exactly the local backup with the
fallback reason recorded.
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* feat(claude_code_gateway): issue rotating refresh tokens and a revocation endpoint
The Claude Code gateway's device-code grant now returns a refresh token
alongside the session JWT, so a Claude Code session renews itself before
the JWT expires instead of forcing the user back through the browser sign-in.
grant_type=refresh_token re-mints the JWT from the live user row, rotates the
refresh token, and refuses a replayed, foreign, or identity-only token with
invalid_grant. The discovery document now advertises an RFC 7009 revocation
endpoint, which Claude Code's /logout calls with both tokens, so sign-out
burns the refresh token. The refresh token is the MCP gateway's sealed
session refresh token bound to the fixed client id claude_code, so both
front doors share one single-use record.
* fix(claude_code_gateway): mint the whole credential before claiming the device code
A refresh token that failed to mint answered 500 after the device code was
already claimed and the login deleted, so the client could not redeem the
completed sign-in again. The response is now built first, and the code is
claimed only when it is a 200.
* fix(claude-code-gateway): keep device sign-in working when session signing is unusable
* feat(claude-code-gateway): end the whole refresh chain on a replay or a revocation
* fix(mcp-gateway): fail the single-use peek closed on a Redis fault
* fix(mcp-gateway): read the single-use marker under the cache namespace
* fix(mcp-gateway): refuse a replayed refresh token without ending its chain
A refresh token presented a second time is refused as already used and
nothing else happens to the chain it was rotated from. Claude Code renews
from its in-memory copy of the credential, so a second terminal on the
same machine presents the token the first terminal already rotated and
then recovers from the shared credential file; ending the chain there
would sign both terminals out at every expiry. Only a revocation ends a
chain, so the 503-on-unrecorded-chain-ending path and its tests go away.
* fix(mcp-gateway): end the refresh chain before burning the revoked token
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(exceptions): keep upstream 402 status and cool down 402 deployments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(exceptions): map 402 to PaymentRequiredError subclass of BadRequestError
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(exceptions): annotate PaymentRequiredError methods
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): skip 402 cooldown on single-deployment model groups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(exceptions): single prefix and 402 fallback response for PaymentRequiredError
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): map billing_error to PaymentRequiredError regardless of status
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): honor explicit allowed-fails policy for single-deployment 402s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(anthropic): drop billing_error body mapping to PaymentRequiredError
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(exceptions): type the PaymentRequiredError constructor parameters
* test(integration): cover 402 PaymentRequiredError mapping and cooldown
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(mcp): refresh server catalog for each gateway operation
* fix(mcp): reject configuration changes during scoped dispatch
* fix(mcp): refresh shared catalog state for each operation
* fix(mcp): refresh catalog before native alias routing
* docs(mcp): clarify native route resolution order
* test(mcp): align streaming fixture and generated API documentation
* fix(mcp): preserve discovery published during catalog refresh
* fix(mcp): coalesce queued catalog reads without a stale window
* fix(mcp): reconcile concurrent route changes when publishing catalog
* test(mcp): provide catalog scope in post-call hook fixtures
* fix(mcp): retain valid live routes and handlers during refresh
* fix(mcp): keep refreshed OpenAPI operation membership authoritative
* test(mcp): preserve logging fixtures after catalog integration
* fix(mcp): isolate transport test admission state and exhaust OAuth outcomes
* refactor(mcp): narrow catalog consistency change to ticket scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(mcp): gate catalog refresh on a database revision marker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): refresh waiters that observed a newer catalog revision
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): awaitable catalog revision doubles in prisma mocks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): satisfy catalog refresh lint and type gates
* test(mcp): provide catalog scope in toolset fixtures
* test(mcp): exercise temporary OAuth through catalog operations
* fix(mcp): share active catalog snapshots across discovery tasks
* fix(mcp): retain discovered routes across cached catalog operations
* fix(mcp): preserve local tool ownership across discovery
* refactor(mcp): satisfy tightened immutability lint ceiling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): authorize local handlers by registered server ownership
* fix(mcp): check registered handler freshness and isolate fixtures
* fix(mcp): read catalog revisions and snapshots from the writer
* fix(mcp): resolve access groups from the operation catalog
* fix(mcp): preserve empty access group restrictions
* fix(mcp): retain rediscovered routes across concurrent updates
* fix(mcp): enforce registered ownership for local tool dispatch
* fix(mcp): preserve issuer discovery across catalog snapshots
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(lens): add dataset case and size limits
* feat(lens): add dataset, case and build models
* feat(lens): build dataset cases from traces, findings and text
* feat(lens): store dataset revisions insert-only
* feat(lens): add dataset routes for build, save, export and eval cases
* feat(lens): mount the dataset router before lens routes
* feat(lens): add LiteLLM_LensDataset table
* feat(lens): add LiteLLM_LensDataset table to proxy schema
* feat(lens): add LiteLLM_LensDataset table to extras schema
* feat(lens): add migration that creates the dataset table
* test(lens): cover dataset case building, dedupe and limits
* test(lens): cover dataset revisions, conflicts and eval cases
* chore(ui): regenerate API types for lens datasets
* feat(lens): add dataset UI types
* feat(lens): add datasets API client
* feat(lens): add dataset query and mutation hooks
* feat(lens): add case selection and expected edit logic
* test(lens): cover case selection and expected edits
* feat(lens): add the add to dataset dialog
* test(lens): cover saving picked cases from the dialog
* feat(lens): add datasets list
* feat(lens): add dataset detail with revisions and export
* test(lens): cover editing, revisions and export in datasets tab
* feat(lens): expose datasets on the lens API
* feat(lens): add in-memory datasets for demo mode
* feat(lens): wire demo datasets into the demo lens API
* feat(lens): add optional lens API hook
* feat(lens): add optional onboarding hook
* feat(lens): add datasets tab and dataset routing
* feat(lens): show the datasets tab
* feat(lens): add to dataset from the trace header
* feat(lens): add a single turn to a dataset from a step
* feat(lens): add finding evidence to a dataset
* docs(lens): add datasets screenshots for the PR
* test(lens): cover dataset revision storage against Postgres
* test(lens): cover dataset trace paging, findings and route errors
* test(lens): cover dataset build fallbacks and no_content skips
* fix(lens): register the dataset table for postgres span names
* feat(lens): add invalid skip reason for unparseable case lines
* fix(lens): keep valid JSONL cases, provenance and size limits; stop reading past the case cap
* test(lens): cover malformed JSONL, re-import provenance, size fields and early cap
* chore(ui): regenerate API types for the invalid skip reason
* feat(lens): show text for the invalid skip reason
* feat(lens): render datasets in the same inspector table as investigations
* test(lens): open a dataset by clicking its table row
* docs(lens): update the datasets list screenshot
* style(lens): format the datasets table
* fix(lens): normalize JSON text span input and output into messages when building cases
* test(lens): cover JSON text span normalization for dataset cases
* Revert "fix(lens): normalize JSON text span input and output into messages when building cases"
This reverts commit 29c3e34962fa12427dbd516c08f2599db997a905.
* fix(traces): normalize agent assistant summaries into UI messages
* feat(lens): add dataset case view helpers built on the trace parsers
* test(lens): cover dataset case view helpers
* feat(lens): show dataset cases in an inspector table
* feat(lens): open a dataset case in a side panel with trace message cards
* feat(lens): rebuild the dataset page header and layout
* feat(lens): keep the open dataset case in the URL
* test(lens): drive dataset edits through the case table and panel
* docs(lens): update dataset view screenshots
* Revert "test(lens): cover JSON text span normalization for dataset cases"
This reverts commit 30229ae8ae90265c09d755c43c1f1eeaa66050c7.
* fix(lens): import StateMessage from shared in dataset detail
* fix(lens): import StateMessage from shared in datasets list
* fix(lens): give the datasets migration a unique timestamp after review checkpoints
* fix(proxy): resolve oidc/ pass-through credentials on every request
A pass-through credential (a use_in_pass_through deployment's api_key or
the provider's env var, e.g. TYPESAFE_API_KEY) was sent as a literal
string, so an `oidc/...` reference such as
`oidc/file//var/run/secrets/<name>/token` ended up on the wire as
`Bearer oidc/file/...`, and a rotating projected Kubernetes service
account token could not be used for pass-through auth.
Resolve credentials that start with `oidc/` through get_secret_str() on
every get_credentials() call, so the file is re-read and rotations apply
without a restart. oidc/file/ keeps its credential-directory allowlist;
other credentials are returned unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): only resolve local oidc/ pass-through credentials inline
get_credentials() runs synchronously inside async pass-through routes, so
resolving oidc/google/, oidc/github/ etc. on a cache miss would make a
blocking HTTP call to the identity provider (timeout up to 600s) on the
event loop. Limit per-request resolution to oidc/file/, oidc/env/ and
oidc/env_path/, which need no network I/O; network-backed references are
left unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): surface unreadable oidc/ pass-through credentials as 401
A missing or disallowed token file raised FileNotFoundError or
OidcPathNotAllowedError out of the route, which the proxy turned into a
generic 500 on every request. The resolver now raises an HTTPException
401 that names the credential reference and the reason, matching how the
pass-through routes already report an unset key.
oidc/env_path/ is no longer resolved inline because it reads any file
named by an environment variable without the oidc/file/ credential
directory allowlist; it is forwarded unchanged like the network-backed
references.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
