* fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics
Requests rejected before dispatch (bad key, blocked key, budget, rate limit, malformed body) were spend-logged with an empty call_type because the synthesized logging object never reached the failure lifter. The caching dashboard rolled all of them, plus failed calls on info routes such as /model/info, into one Unknown group.
Resolve call_type from the matched route first, falling back to body shape, and keep the synthesized logging object on request_data so the lifter sees it. Log bare auth exceptions with the 401 ProxyException the client gets so error_code is never empty. Exclude info routes from the cache analytics groups and error breakdown. The dashboard explains the Unknown group when older rows still produce one.
Resolves LIT-5884
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the raw auth exception for failure callbacks
Record the client-facing status in the spend log through a separate client_exception argument so custom failure callbacks still receive the exception auth raised.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the route for multi-operation endpoints and exclude info routes from cache filter options
Routes such as /v1/files map to several operations (create, list) and the
method is not available in the failure hook, so a rejected request there is
filed under its route instead of the first mapped call type. The key alias and
model filter-option queries now apply the same info-route exclusion as the
groups and error breakdown, so every offered filter value returns data. The
info-route exclusion and Unknown grouping are now covered against a real
Postgres in tests/proxy_behavior/spend/test_cache_activity.py
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop client_exception, the spend log row never used it
The DB spend row for a gate rejection is written by _ProxyDBLogger from the
original exception, so the status-bearing copy only reached the in-memory
logging payload. Live runs at the tip still recorded bare auth exceptions as
Unknown/Exception, the same as the base branch. Removing the plumbing keeps
this PR to endpoint attribution and the info-route exclusion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Codex 0.130 and 0.145 require supports_reasoning_summaries and
supports_parallel_tool_calls on every catalog entry, so a catalog written
for 0.154 made those releases exit at startup with a parse error. Every
field some release since 0.105.0 deserializes without a default is now
written, with Codex's own fallback values, and the catalog is read back
once through the installed binary (`codex debug models`) before launch.
A Codex that rejects it, or one older than 0.130 with no such command,
gets the skip notice and launches on its built-in catalog instead.
The provider_config path skipped run_realtime_guardrails for transcription
sessions to avoid sending response.create, which also dropped every
realtime_input_transcription guardrail: no violation error reached the
client and on_violation / end_session_after_n_fails never fired. Run the
guardrail for every completed transcript and only suppress response.create
when the session has no assistant turn.
Every proxy and router cost lookup goes through get_model_info, which copies
cost map keys explicitly, so the new audio cache-read branch always fell back
to the text cache-read rate there. Copy the key so models whose audio
cache-read rate differs from the text one bill cached audio correctly.
When a slow backend times out many logging callbacks at once, each timeout
hit verbose_logger.exception in _process_log_task and produced a full ERROR
traceback, clustered milliseconds apart. Count timeouts instead and arm a
debounced flush that logs one WARNING with the burst count, callback name,
timeout and cumulative total. Real programming errors keep their traceback.
Muse partials carry no turnId and belong to the most recent speechStart,
and the docs say the model may keep post processing a turn after speechEnd
until speechComplete. Releasing the active turn on speechEnd made any
partial arriving in that window raise and get dropped in ENDPOINTING mode.
The turn now stays active until its speechComplete or final transcript.
* fix(guardrails): log mask when a guardrail adds request keys
_inputs_were_modified only compared keys present in the pre-hook baseline, so a
guardrail that injected a new key such as tools was logged as allow. Compare over
the union of both key sets, and narrow the pre_call return value to the same
prompt-bearing keys the baseline holds so passthrough stays allow.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): snapshot apply_guardrail inputs before the hook mutates them
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_attached_policies_with_reasons rescanned the sorted matches with next() once
per distinct policy, which is quadratic and misses the one second budget past a
few thousand global attachments. Build a policy to broadest attachment map in one
pass instead, keeping the specificity sort and result order.
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): accept both deferred stream logging arg shapes on native routes
_arm_deferred_stream_dispatch armed a one-argument closure on every
anthropic_messages/aresponses stream that was not a CustomStreamWrapper or a
LiteLLMCompletionStreamingIterator. The bridged /v1/messages path returns a
plain SSE generator that shares its inner CustomStreamWrapper logging_obj, so
it stores (assembled_response, cache_hit) and _fire_deferred_stream_logging
raised TypeError, dropping spend logs and callbacks and ending the stream with
an error. The closure now dispatches on the stored args shape: a single
coroutine is enqueued, a two-tuple runs success handlers, anything else is
logged and dropped
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert dropped deferred payload via caplog instead of patching the logger
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Gate the reasoning_effort translation on the cost-map flag or the model name containing
claude, so unmapped Claude serving endpoints keep translating. Flag the newer Claude
entries that were missing it. Expose supports_anthropic_thinking_payload as a public
helper next to the other supports_* wrappers instead of importing the private factory.
Drop the adaptive-only guard, since the adaptive flags only ever match Claude ids, and
add regression tests for an unmapped Claude endpoint and an adaptive Claude model
A duplicate speechStart for a turn that already stopped used to make that
closed turn active again, so the next turnless PUSH_TO_TALK transcript was
routed to the finished item and dropped.
Turns no longer wait behind each other in a FIFO queue, so an empty
server_vad turn (speechStart then speechEnd with no transcript) cannot
stall every later turn, and a PUSH_TO_TALK speechComplete now closes its
turn without waiting for a speechEnd that never arrives. Each turn keeps
its own idempotent emit state, so late or duplicate speechEnd,
speechComplete and transcript frames are no-ops, and finished turns are
remembered in a bounded map instead of a separate tombstone deque.
The session.created ack and the sanitized error frame are now typed as
members of OpenAIRealtimeEvents, which removes the typing.cast calls
that the strict ruff budget flagged.
Servers whose upstream catalog depends on the caller (oauth2 per-user, token exchange, id_jag, per-user env vars, delegated auth) now keep one listed-tool mapping per (user_id, api_key) hash under the server id, so one caller's tools/list cannot supply another caller's description or inputSchema to pre-call guardrails. Shared servers and OpenAPI-backed servers keep a single server-wide entry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
