Two issues from the previous push's review:
1. **Greptile P1**: ``get_vector_store_info`` had the same catch-all
``except Exception`` pattern as ``update_vector_store``, so the
HTTPException(403/404) raised by both the in-memory access check and
the new ``_fetch_and_authorize_vector_store`` helper was rewritten as
500. Mirror the ``except HTTPException: raise`` guard from
``update_vector_store``.
2. **code-quality CI** (``tests/code_coverage_tests/recursive_detector.py``)
flagged ``_redact_sensitive_litellm_params`` as an unallowlisted
recursive function. Match the convention of other allowlisted
helpers ("max depth set"): bound recursion at depth 10 (well above
any plausible nesting level for real ``litellm_params`` payloads),
return the redaction sentinel on overflow, and add the function
name to ``IGNORE_FUNCTIONS``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues surfaced in review of the previous commit:
1. **Veria — Medium**: ``litellm_params`` carries a nested
``litellm_embedding_config`` dict (auto-resolved from the model
registry on create / update) which itself holds ``api_key`` /
``aws_*`` / ``vertex_credentials``. The previous redactor only
inspected top-level keys, so the nested values passed through
unredacted. Recurse into nested dicts.
2. **Greptile — P2**: when ``litellm_params`` is a JSON-serialized
string (the in-memory registry occasionally stores it that way), the
previous redactor silently no-op'd via the ``isinstance(..., dict)``
guard and echoed the raw payload back. Now: parse, redact, re-serialize.
If the string is not valid JSON, replace it with the redaction
sentinel rather than echo it.
3. **mypy** flagged ``_redact_sensitive_litellm_params``'s
``Optional[Dict[str, Any]]`` signature as incompatible with the
``object``-typed call site. Widened to ``Any -> Any`` to reflect the
actual contract (the function now handles dict / str / None / other).
Also fixes a related test regression in
``test_remove_sensitive_info_from_deployment_with_excluded_keys``: the
``"credentials"`` plural addition to ``SensitiveDataMasker`` defaults
caused the first call (without ``excluded_keys``) to mutate the input
dict's ``litellm_credentials_name`` to a masked value. The second call
(with ``excluded_keys``) then saw the already-masked value rather than
the original. Construct fresh input for each call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
/simplify pass:
- ``update_vector_store`` (newly added) and ``get_vector_store_info``'s
DB-fallback path duplicated the same shape: ``find_unique`` →
``model_dump`` → ``LiteLLM_ManagedVectorStore(**)`` →
``_check_vector_store_access`` → raise 404/403. Extract into
``_fetch_and_authorize_vector_store`` so the pattern lives in one
place; future endpoints that need the same gate get it via one call.
- The ``except HTTPException: raise`` guard added in the prior commit is
retained — the helper raises HTTPException(403/404) and the catch-all
``except Exception`` would otherwise rewrite them as 500.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two architectural extensions to the credential-redaction in the previous
commit:
1. ``/vector_store/update`` had two gaps:
- No per-store access control. Any authenticated principal that
passed the premium-feature gate could mutate *any* vector store,
including stores belonging to other teams.
- The response returned the full DB row including ``litellm_params``,
so the caller could read another team's persisted provider
credentials by submitting a no-op metadata change.
Mirror the access-control check ``/vector_store/info`` already
performs (``_check_vector_store_access`` against the existing row),
redact ``litellm_params`` in the response, and add an
``except HTTPException: raise`` guard so the 403/404 responses don't
get rewritten as 500 by the catch-all.
2. ``SensitiveDataMasker``'s default ``sensitive_patterns`` set used
segment-exact matching, so ``credential`` matched ``vertex_credential``
but not ``vertex_credentials`` (the actual Vertex field name). The
previous commit worked around this with a per-call extension; this
commit puts the plural in the upstream defaults so every caller
(Redis config dump, MCP debug headers, cache routes, ...) gets the
correct behavior. The local override in
``vector_store_endpoints/management_endpoints.py`` is removed.
Also updates ``test_excluded_keys_exact_match`` which relied on
``credentials`` *not* being a sensitive pattern to demonstrate
case-sensitive ``excluded_keys`` matching. The intent of the test
(case-sensitive match) is preserved; the assertion now reflects that
when ``excluded_keys`` fails to apply (wrong case), the field falls
through to standard pattern-based masking instead of being passed
through unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
/simplify pass:
- Remove the single-call-site ``_redact_vector_store`` wrapper. Inline
the two-line redaction at its only caller in ``list_vector_stores``;
``get_vector_store_info`` was already calling the inner helper directly.
- Inherit ``SensitiveDataMasker``'s default sensitive-key set instead of
duplicating the 12-element list, then add only the plural
``credentials`` extension. Won't drift if upstream defaults change.
- Trim the over-explained docstring on ``_redact_sensitive_litellm_params``
to a one-paragraph summary; the WHY (credential-leakage class) belongs
in the commit message, not in every consumer's IDE tooltip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``LiteLLM_ManagedVectorStore.litellm_params`` carries the upstream provider
credential — OpenAI ``api_key``, AWS ``aws_access_key_id`` /
``aws_secret_access_key``, GCP ``vertex_credentials``, etc. ``GET
/vector_store/list`` and ``POST /vector_store/info`` return these
verbatim to any authenticated principal. Because both routes are in
``openai_routes``, ``RouteChecks.is_llm_api_route`` short-circuits the
standard role gate, so even read-only users and narrowly-scoped keys can
read every stored credential.
Replace credential-bearing values with the ``REDACTED_BY_LITELM``
sentinel in both responses while preserving non-secret keys
(``api_base``, ``region``, ``model``, ``api_version``) so callers can
still see *which* upstream is configured. Detection reuses
``SensitiveDataMasker.is_sensitive_key`` with the default heuristics
plus the plural ``credentials`` pattern (covers Vertex's
``vertex_credentials`` field, which the singular ``credential`` pattern
misses on segment-exact matching).
Applied at:
- ``list_vector_stores`` (``GET /vector_store/list``,
``GET /v1/vector_store/list``)
- ``get_vector_store_info`` (``POST /vector_store/info``), both the
in-memory-registry path and the prisma-DB fallback
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test was flaking on unrelated asyncio ERROR records (e.g. "Unclosed
client session" from background tasks in other tests). Restrict the
assertion to records emitted by LiteLLM loggers so the test only fails
on errors actually produced by the code under test.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* feat: add AIHubMix provider to providers.json
* fix: add aihubmix to provider_endpoints_support.json for CI check
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
`prerelease: false` was hardcoded, so dispatching create-release with
`1.84.0rc1`, `1.84.0.dev42`, or legacy `v1.83.13-nightly` would publish
them as stable releases on the GitHub Releases page. Derive the flag
from the tag instead.
The detector matches `rc`, `.dev`, `nightly`, `alpha`, `beta`. PEP 440
post-releases (`1.84.0.post1`) and legacy `-stable[.patch.N]` are
stable maintenance releases per PEP 440, so they intentionally do not
match.
The tag validator required a leading `v`, so dispatching create-release
with `1.84.0` (or `1.84.0rc1`, `1.84.0.dev42`, `1.84.0.post1`) failed
even though those are the new naming convention. Make the leading `v`
optional in both create-release.yml and create-release-branch.yml so
both legacy (`v1.83.10-stable`, `v1.83.14.rc.1`, `v1.82.3.dev.9`,
`v1.82.3-stable.patch.4`, `v1.83.13-nightly`) and new PEP 440 forms are
accepted during the transition. Refresh the input descriptions to show
the new examples.
* Unify cost calc in success_handler dict and typed branches
* Trim verbose comments and docstrings
---------
Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain>
Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local>
Claude 3.5 Sonnet v2 reached EOL on Bedrock 2026-03-01, returning the same
404 EOL error as 3.7 Sonnet. Sonnet 4.5 supports both InvokeModel and
Converse APIs on Bedrock, so use the same model for both routes.
AWS Bedrock has reached end-of-life for `claude-3-7-sonnet-20250219-v1:0`,
returning 404s with "This model version has reached the end of its life."
Update test references to `claude-sonnet-4-5-20250929-v1:0` (same capability
surface: thinking, tools, prompt caching, PDF input, vision, computer use).
The bedrock/invoke pass-through tests stay on Sonnet 3.5 since Sonnet 4.5
is converse-only on Bedrock.
Companion to the prior commit. process_items only converted empty
`items: {}` to `{"type": "object"}`. But anyOf branches like
`{"type": "array"}` (no items field at all) were untouched, so after
convert_anyof_null_to_nullable stripped the null branch and added
nullable, the array branch was sent to Vertex as
`{"type": "array", "nullable": true}` — which Vertex rejects with
INVALID_ARGUMENT (`any_of[0].items: missing field`).
Make process_items synthesize `items: {"type": "object"}` for any
`type == "array"` schema where items is missing or empty.
Also:
- Convert test_gemini_tool_calling_working_demo to a hermetic mock
test asserting items is present on the array branch in the sent
body. Was previously a real-network call to Vertex and was the
test the user reported still failing in CI.
- Add unit test test_build_vertex_schema_array_branch_missing_items_in_anyof
covering the missing-items shape directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cache provider config lookups for Vertex Anthropic messages so repeated requests reuse the same config object and preserve credential cache state. Add a regression test to catch any future loss of config reuse.
Made-with: Cursor
* fix(caching): preserve prompt_tokens_details through embedding cache round-trip
The embedding caching layer was dropping prompt_tokens_details (including
image_count) because CachedEmbedding had no field for usage metadata and
the cache retrieval code reconstructed Usage without it. This caused
inconsistent responses where the first call returned image_count but
cached responses did not, breaking cost tracking for multimodal embeddings.
Add prompt_tokens_details to CachedEmbedding, persist per-item details
during cache storage, aggregate them on retrieval, and merge them in
combine_usage() for partial cache hits.
* style: apply Black formatting to caching files
* fix(caching): address Greptile review — cyclic import, guarded construction, nested dict merge
Move PromptTokensDetailsWrapper to inline import to resolve CodeQL cyclic
import warning. Guard PromptTokensDetailsWrapper construction with
try/except to handle unexpected cached keys. Add recursive dict merging
in _merge_prompt_tokens_details for nested fields like
cache_creation_token_details.
convert_anyof_null_to_nullable was stripping the items field from array
branches inside anyOf when a sibling null branch was present, leaving
{"type": "array"} without items. Vertex requires items whenever
type == "array" (even inside anyOf) and rejects the call with
INVALID_ARGUMENT.
Leave the (possibly empty) items in place so the downstream process_items
step can convert {} to {"type": "object"}, which is what Vertex wants.
Also:
- Update test_build_vertex_schema expected output, which was codifying
the broken shape.
- Convert test_gemini_tool_calling_not_working to a hermetic mock test
that asserts the request body sent to Vertex includes items inside
the callbacks anyOf array branch. The previous form made a real
network call and was flaky in CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vi.clearAllMocks does not reset mockImplementation, so the error-notification
test was inadvertently relying on a deleteField stub set up in earlier tests
and would time out when run in isolation.
Previously, useStoreRequestInSpendLogs and useDeleteProxyConfigField
did not refresh the proxyConfig cache on success, so the Logging
Settings form continued to render the pre-save values until React
Query refetched on its own. Wire both hooks to invalidate
proxyConfigKeys on success so any active observer (currently the
Logging Settings page) repulls fresh data.
Export proxyConfigKeys for cross-hook reuse.
Switch the spend-logs save flow from mutateAsync + try/catch to
mutate + callbacks. Errors now surface through a single onError path
(no more double toast on failure), and the delete-then-update sequencing
runs through onSettled instead of awaited promises. handleFormSubmit is
no longer async.
Tighten the corresponding test to assert exactly one error toast fires.
Adds a CLI flag (`--timeout_worker_healthcheck`, env `TIMEOUT_WORKER_HEALTHCHECK`)
that forwards to uvicorn's `timeout_worker_healthcheck` Config kwarg (added in
uvicorn 0.37.0). Lets operators raise the supervisor's worker-ping timeout above
the default 5s when triaging workers being killed and respawned under load.
The helper introspects `uvicorn.Config.__init__` and only sets the kwarg if
supported, otherwise prints a warning - so the existing uvicorn>=0.32.1,<1.0.0
floor pin is unaffected. Gunicorn and Hypercorn paths are unchanged (the uvicorn
supervisor isn't running there); the value is also not passed to the helper at
all on those paths so the "uvicorn too old" warning never fires spuriously.
