Removes the Singulr async_logging_hook and logging_hook overrides so logging_only runs through CustomGuardrail.async_logging_hook: the response scope reaches Singulr as an assistant message instead of a raw ModelResponse dump, a vendor timeout is recorded as guardrail_failed_to_respond, a request-scope block ends the scan, and the sync success callback thread makes no Singulr call.
Decides MCP versus LLM by the proxy logging object's call_type (then the call_type or server-only markers in request_data), never by name, arguments or mcp_tool_name keys a client can put in a chat body. REST /mcp-rest/tools/call pre-scans reach Singulr as mcp_request and a non-mapping arguments value is forwarded as tool_arguments instead of raising.
should_block is a strict bool defaulting to false so a null verdict is an invalid response that block_on_error decides; payload fields drop Any for Sequence, Mapping and AssistantMessage types; metadata carries only the keys present; docstrings and section comments removed per the repo comment policy.
Squash of BerriAI/litellm#37464 (head da298ca7) by @aniket-kardile, adopted onto main: v2 gateway payload contract with request, response, mcp_request and mcp_response scopes, typed payload models, proxy user, org and team metadata forwarded to Singulr, and the logging_only, pre_mcp_call and post_mcp_call modes.
On pre_call, during_call and post_call the hook data is the client request body, so a
client-supplied litellm_params.metadata.internal_call_origin must not skip enforcement.
Type the during_call helper's request payload as dict[str, object]
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The request-side prompt told the judge to focus on the most recent user turn but the text under review was every extracted request message joined together, so a multi-turn request with an off-topic earlier turn and an on-topic latest turn scored 50 and was blocked. Request-side judging now evaluates the last extracted request text (after the configured message scoping) and passes the full role-labelled conversation only as context. Response-side judging still evaluates all extracted response text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Judge sub-calls now carry the internal_call_origin metadata stamp and the
guardrail skips any logged call bearing it, so a logging_only judge no longer
recurses into an unbounded chain of judge requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The reconcile now records the database clock of the scan behind the last complete
run and, on the next run, rewrites every closed day with per-key rows updated since
then, however old the day is. Replaying only the marker day and the one before it
missed a delayed flush or retry that landed on an older date, and reads through the
marker come from the global table alone, so that spend was never counted.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The write path no longer dual-writes the global table. The cron rolls up closed UTC days
only, so a pod still flushing the current day can never leave the global table short. The
key-free arm reads days through the marker from the global table and later days from
LiteLLM_DailyUserSpend in one UNION ALL, and the marker comes from the config cache
rather than a per-request database lookup.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds a daily spend table without api_key or user_id, written atomically alongside
LiteLLM_DailyUserSpend from the batched writer, reconciled from history by a
scheduled job that advances a marker in LiteLLM_Config, and read by the key-free
arm of the aggregated usage query once the marker covers the requested range.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Match the pass-through skip to what FastAPI actually dispatched (the user-defined
endpoint marker or a provider handler's {endpoint:path} param) instead of the
mapped route prefixes, which also cover native routes such as /openai/v1/responses
and /cursor/chat/completions. Wrap the added test lines to the 120-column limit.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
