A registered S3 Vectors store usually carries only its "bucket:index" id,
and the previous commit stopped forwarding the caller's bucket and index for
a managed store, so ingesting into one raised KeyError 'vector_bucket_name'.
The ingestion now derives both from vector_store_id with the rule the search
side already uses, explicit keys still winning. The caller's
litellm_credential_name is dropped for a managed store too, since it expands
into api_key and api_base, and max_embedding_requests_per_min joins the
per-upload options a caller may still set.
The cost tracking callback f-stringed chosen_metadata, litellm_metadata,
and old_metadata into the failed_tracking_spend alert on every failure,
at every log level, so one 250-byte request produced a 23 KB alert
carrying the client's metadata, headers, and key-auth reprs four times
over. The alert now carries the exception, the traceback, the model, and
the call type; the metadata keys are logged once at debug level through
lazy formatting, so nothing is built at warning level
With store_prompts_in_spend_logs on, the persisted request body kept the client's model string even when the row's model, model_group, and error text had been replaced by the unknown-model placeholder. The body's model now takes the same placeholder on those rows. Also annotates the new test locals with Final and wraps the four test lines that ran past 120 characters.
A bulk item that carried only tags reached the DB with max_budget, team_id,
and budget_id as explicit nulls, wiping the key's budget and detaching it
from its team. The per-key update is now built from the fields the item
actually set, so a field left out keeps its value and an explicit null still
clears it, the same as /key/update. Items carrying a field the bulk path
cannot apply (object_permission and the like) are rejected with 422 instead
of being silently dropped.
Guardrails created through POST /guardrails on older releases have api_version "v1" saved in the database, because the writer persists every default. Azure Content Safety never accepts that value, so those guardrails kept answering 404 after the default moved to None. The Azure base now resolves "v1" to 2024-09-01 the same way it resolves a missing value. Also restores the OpenAPI snapshot line that a Python 3.14 regeneration had dedented
When a deployment priced one OCR batch family and the other still needed a
published rate, a failed cost-map lookup returned zero for the whole line and
discarded the deployment rate that was already resolved. Those pages were
billed as free. The lookup failure now only logs, and the families the
deployment prices are billed at the configured rate
LitellmParams mixes every provider config model into one class, so the
Javelin api_version default of "v1" reached the Azure Content Safety
guardrails whenever config.yaml omitted api_version and Azure answered 404.
The shared field now defaults to None, Javelin keeps filling in "v1" itself,
and the Azure guardrails fall back to the documented 2024-09-01 at request
time so a DB update that omits api_version stays on the default too.
POST /v1/rag/ingest authorized the managed vector store the request
named but then handed the raw request options to the ingestion
pipeline, which defaults to OpenAI. A request naming only a registered
store id uploaded the document to OpenAI Files, got an OpenAI 400, and
answered HTTP 200 with status "failed"; naming azure_ai explicitly
escaped as a 500.
The store's provider and litellm_params now merge into the request the
way /v1/rag/query already does (store wins, None values dropped), the
merged provider is checked against the ingestion registry before any
upload so unsupported providers get a 400 naming the supported ones, and
persistence keeps reading the caller's original options so registry
credentials never reach the database. A registry store with no database
row is no longer written as a new row.
The router coverage gate in code-quality flags every router.py function
no router test calls by name, and the two helpers get_credential_deployment
gained (the team public-name lookup and the team-aware wildcard lookup)
were only reached through it. Each now has a test of its own: the
public-name lookup resolves only for the owning team, and the wildcard
lookup prefers the team's own pattern over the shared one and never hands
another team's wildcard deployment to a caller outside that team.
The vector-store file routes accept a model hint through the ?model= query
param and the x-litellm-model header. That hint was authorized with a hand
rolled check that covered only the key allowlist and the team allowlist, so
a key restricted by its project's model grant, a team-member restriction, or a
key config still routed through the hinted deployment. Greptile flagged the
gap as a P1 on the replacement PR.
The hint now goes through the same authorize_model_for_key path the batches
and files routes use, which runs can_key_call_resolved_model with every rule
the proxy enforces elsewhere. Keys those extra rules deny now get a 403 on
these routes. The two remaining behavioral differences are edge cases the
old check tolerated: a key whose team_models is set without a team_id no
longer runs the team allowlist, and a key with a config set skips the key
allowlist, both matching the rest of the proxy.
The regression test caches a project whose grant excludes the hinted model
and asserts the request is refused before any deployment lookup. The two
patch() calls on litellm.proxy.proxy_server carry a test-quality-ok reason
because can_key_call_resolved_model reads prisma_client and
user_api_key_cache through a lazy module import with no injection seam.
A batch retrieved by its model-encoded id takes the direct (non-router) path,
which resolved credentials without stamping the deployment's model_info, so a
completed batch on a deployment with its own per-page pricing was billed at
the published rate with an empty model_id on the spend row.
Extract the router's credential lookup into get_credential_deployment and
stamp the resolved deployment's model_info onto the retrieve call the way the
router does for routed calls.
* fix(proxy): dispatch llm_api_check moderation through during_call_hook
ProxyLogging.during_call_hook only ran async_moderation_hook for CustomGuardrail callbacks, so a
CustomLogger such as the prompt injection detector with llm_api_check enabled never called the
configured moderation model. Dispatch any CustomLogger that overrides async_moderation_hook and hand
the proxy router to every registered prompt injection detector at startup so that call can route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(enterprise): resolve openai_moderations model at call time and default to omni-moderation-latest
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep queued moderation running past a V1 pre_call guardrail
A V1 CustomGuardrail with moderation_check pre_call returned out of
during_call_hook before asyncio.gather, abandoning already-queued
CustomLogger moderation coroutines and skipping every later callback.
Skip only that guardrail instead.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(utils): skip null tool_calls when formatting prompts for moderation hooks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Post-call pipeline rewrites on buffered streams failed open on three shapes:
chat streams with n > 1 (the rebuilt response collapsed every choice into
index 0), streams that ended without a finish marker, and Responses streams
whose final event carried no response envelope.
The chat handler now rebuilds the ended stream one choice index at a time and
writes each choice's rewrite back to that choice's buffered deltas. The
Anthropic handler writes an unended stream's rewrite across its text deltas.
The Responses handler spreads an envelope-less rewrite over the buffered
output_text events, still failing open when a scanned event cannot be placed.
Tool-call rewrites on n > 1 chat streams keep failing open.
Applying a database row dropped the runtime layer for every key the row
carried, including keys the config file owns. Those runtime entries hold
the env-resolved config values, so after a reload a key written as
os.environ/<NAME> read back as that literal string. The store now keeps
the runtime entry for a key the config owns and clears only the rest.
Visible as store_model_in_db silently turning itself off: the reload read
the raw reference, coerced it to False, and overwrote the resolved global.
