The async client cache was keyed on id(loop). CPython recycles those ids so
aggressively that a fresh event loop nearly always lands on the id of one already
collected, measured at 37 of 40 rounds, so the cache handed the new loop an
AsyncMongoClient bound to a closed loop and every operation on it raised
"Event loop is closed".
The entry now carries a weak reference to the loop it was built on and a hit only
counts when that reference still points at the running loop, so a recycled id
misses and builds a fresh client. A stale entry can also be replaced once the
cache is full, which the old size check prevented.
pymongo's own client keeps its loop alive, which is why the sync proxy path never
saw this; a script calling asyncio.run() per search, or a test suite with a loop
per test, does.
The create form now offers MongoDB Atlas with its connection string, database,
collection, embedding model, vector field, text field and candidate count. The
connection string renders as a password input because it carries the database
user's password, and the embedding model is picked from the proxy's own models,
matching how Milvus and Valkey do it.
The vector store id doubles as the Atlas Vector Search index name, so the
placeholder says so.
* fix(helm): scale the classic chart's HPA out at the documented 60 percent CPU
The litellm-helm chart shipped targetCPUUtilizationPercentage: 80, which is
unexamined helm create scaffold rather than a chosen number. It arrived packaged
with the stock minReplicas: 1, maxReplicas: 100, a commented-out
targetMemoryUtilizationPercentage: 80, and the boilerplate "such as Minikube"
comment, the same provenance as the 128Mi resource example this file just
corrected.
60 is the documented recommendation. The mechanism behind it is scale-up lag:
the chart's own startupProbe is failureThreshold: 30 times periodSeconds: 10, so
a replica can take up to 300 seconds to become ready, and a pod added at 80
percent utilization arrives minutes after saturation.
The memory target stays commented out on purpose. The prisma query engine's
resident memory is a high-water mark that ratchets to the pod's worst-ever write
and is never returned, so a memory-target HPA reads the largest write a pod ever
did rather than what it is doing now, and replicas ratchet up without scaling
back in.
hpa_tests.yaml carried its second suite after a YAML document separator, and
helm-unittest loads only the first document per file, so that suite never ran;
an assertion planted in it still passed. Fold it into the one live suite and add
coverage pinning the rendered CPU target, the absence of a memory metric by
default, and that overrides still take effect.
Bump the chart to 1.1.2, since rendered output changes for anyone running with
autoscaling enabled.
* fix(helm): bump litellm-helm to 1.1.3 after rebase onto 1.1.2
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Atlas answers a wrong password with code 8000 "AtlasError" rather than the 18 a
self-hosted deployment returns, so the code-only check never fired and a bad
password came back as a generic "MongoDB rejected the vector search", pointing
the reader at the index instead of at their credentials. Verified live against
Atlas with a tampered password.
litellm.exception_type passes a litellm exception through untouched and wraps
anything else into APIConnectionError, so every bare ValueError this provider
raised reached the caller as HTTP 500 with a Python traceback in the response
body. "max_num_results must be between 1 and 50" is the caller's to fix, not a
connection failure.
Configuration and validation failures now raise BadRequestError (400) and the
two timeout cases raise Timeout (408). ExecutionTimeout subclasses
OperationFailure, so it is matched before it; previously an Atlas query that ran
out of time was reported as "MongoDB rejected the vector search".
A MongoDB vector store's whole credential is its connection string, and
mongodb+srv://<user>:<password>@<cluster> embeds the database password. None of
the masker's default patterns (api_key, secret, token, credential) match a key
named mongodb_connection_string, so /vector_store/list and /vector_store/info
returned it verbatim to every caller that can read a vector store.
SensitiveDataMasker gains extra_sensitive_patterns, which unions onto the
defaults instead of replacing them, and the vector-store redactor adds
"connection" so the URI is masked while mongodb_database, mongodb_collection and
the field names stay readable.
Driving the sad path against a live Atlas cluster showed four cases returning
an empty result set instead of failing: a missing index, a missing database, a
missing collection, and the async path for all three. $vectorSearch reports
none of these as errors, so a misconfigured store looked exactly like a query
that matched nothing, which is the worst shape for this to fail in.
An empty result set is now checked against the index catalogue, which does
report all three correctly, and a store that cannot work says so. The check
costs one extra round trip and only on the empty path, so a search that
returned hits is unaffected.
Atlas also reports a wrong vector path and a dimension mismatch under the same
error code. Both previously surfaced as "index not found", which sent the
reader looking in the wrong place; they are now told apart and each names the
setting that is actually wrong.
65 cases across pipeline construction, response mapping, parameter validation,
client caching, and driver-error translation. The sad-path cases assert on the
message the caller actually sees, since a vector search that fails quietly
returns an empty result set rather than an error.
Atlas Vector Search has no HTTP query API, since the Data API and HTTPS
Endpoints are end-of-life, so this provider extends BaseDirectVectorStoreConfig
and runs the $vectorSearch aggregation through pymongo rather than shaping an
httpx request. That is the same seam Valkey uses for RESP.
vector_store_id names the Atlas Search index, matching Valkey, with the
database and collection supplied through litellm_params.
pymongo lives in a new optional `mongodb` extra and is imported lazily, so the
base install still pulls no MongoDB driver. The floor is 4.17 because that is
where dnspython became a core dependency instead of the `srv` extra, and Atlas
issues mongodb+srv:// URIs that will not resolve without it.
Clients are cached per connection rather than opened per search. Measured
against Atlas, a fresh client costs ~890ms versus ~80ms warm, so copying the
Valkey open-and-close-per-call pattern would have added ~810ms to every query.
* fix(guardrails): run apply_guardrail-only providers in logging_only mode
A CustomGuardrail that implements only apply_guardrail inherited the CustomLogger
no-op async_logging_hook, so mode: logging_only never scanned anything and never
recorded guardrail_information. CustomGuardrail.async_logging_hook now routes the
logged request and response through the call type's guardrail translation on
copies and appends the verdict to standard_logging_object.guardrail_information.
Resolves LIT-4876
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep logging_only scan copies inside the error boundary and return a fresh logging payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): cover embedding scan, native-hook bypass, and unmapped call type in logging_only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Gemini 3.8 Flash launches today with the same promotional pricing, limits,
and thinking settings as Gemini 3.7 Flash, so the gemini/, vertex_ai/, and
bare cost map entries mirror the 3.7 Flash ones. Regression tests lock the
launch prices, the 4096-token cache minimum, and the gemini-3 thought
signature gate in for the new model.
The monolithic images install the saml extra but the split backend image
did not, so /sso/saml/* returned 501 on Helm split-image deployments.
The gateway image is unchanged since /sso/ routes are backend-only.
Tamper tests rewrote the last two base64url characters of the signature,
which on roughly 1 in 250 RS256 tokens (1 in 1000 HS256) only touched
padding bits, so the decoded signature was unchanged and still verified.
Corrupt the decoded signature bytes instead.
The fuzzy picker driver sent keys after fixed sleeps, so a slow worker
could receive the filter text before the widget had highlighted the match.
Wait on the widget's highlighted choice instead.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Typing-only pass over backend modules that carried the most reportAny and
reportExplicitAny errors. Every new annotation is backed by a construction
site, a call site, or an isinstance narrowing that already existed; untyped
JSON boundaries were left alone rather than declared without validation.
Tree-wide basedpyright errors drop 138,481 to 138,007. reportAny drops 8,854
to 8,645 and reportExplicitAny drops 3,119 to 2,814.
initialize_presidio registers up to three callbacks per guardrail but the
registry only kept the first, so deleting or re-syncing the guardrail left
the post_call siblings serving the old config. The initializer now returns
every callback it registered, the registry tracks primary and siblings per
guardrail id, delete purges all of them from every callback list, and
update pushes the new params into each while siblings keep their stage.
* fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping
SearchAPIRouter dropped every parameter configured on a search tool, forwarding
only per-request kwargs. Any tool-level setting (mode, max_results, ...) was
silently lost on the way to the adapter, for every search provider.
Also completes the Parallel AI v1 search surface: after_date, fetch_policy,
location and include_domains now nest under advanced_settings instead of being
sent as unknown top-level fields, responses preserve search_id / session_id /
warnings / raw excerpts, and search cost is derived from the request mode and
the provider's reported usage rather than a single flat rate.
* fix(parallel_ai): stop a caller from pricing its own search request
`_parallel_ai_usage` carries the provider's reported usage into cost
calculation. It was only written when the response contained a usage block, so
a caller could pass `_parallel_ai_usage=[{"name": "sku_search", "count": 0}]`
and, whenever the provider omitted usage, bill $0.00 instead of $0.005 — the
value also reached the upstream request body as an unknown field.
The key is now stripped from inbound params and written unconditionally from
the parsed response, so only the provider can populate it.
* fix(parallel_ai): price fast search mode correctly
* test(parallel_ai): fake search at HTTP boundary
* fix(parallel_ai): tolerate null search result fields
---------
Co-authored-by: khushishelat <shelatkhushi@gmail.com>
