filters was already refused, but ranking_options and rewrite_query were
accepted and then dropped. A caller asking for score_threshold 0.9 got results
scoring 0.5 with a 200 and no indication the threshold never ran, which is the
silent-wrong-answer case the filters check exists to prevent. Both now raise
the same 400 naming the parameter and what to do instead.
The async client cache is keyed per event loop, and pymongo's AsyncMongoClient
holds a reference to the loop it was built on, so an entry for a closed loop
kept that client and its sockets alive for the life of the process. A script
that calls asyncio.run once per search fills the cache to its cap this way and
then stops caching entirely. Measured live against Atlas over 40 loops: 32
pinned clients and 212 open descriptors before, 1 cached client and no
monotonic descriptor growth after.
litellm.exception_type passes only litellm's own exception types through
untouched, so the NotImplementedError the search-only refusal raised reached
the caller as APIConnectionError. The proxy served that as a 500 with a
traceback in the body for what is a plain client mistake. Raising
BadRequestError gives the caller a 400 and the message on its own.
tests/test_litellm/llms/mongodb imports pymongo's exception classes to check the
error translation against the real hierarchy, and the shard that runs it
(tests/test_litellm/llms, per test-unit.yml) synced --extra google, proxy,
semantic-router and saml but not mongodb, so 24 of 109 tests would have errored
with ModuleNotFoundError on the first CI run. CircleCI hid this because it syncs
--all-groups --all-extras.
uv export --frozen ... --extra saml -> no pymongo
uv export --frozen ... --extra saml --extra mongodb -> pymongo==4.17.0
Also close the two gaps a mutation run found in the suite: nothing asserted that
a short request timeout shortens server selection as well as connect, and the
existing code 13 case carried "not authorized", which the message markers match
too, so it could not tell whether the code was still being checked. 28 of 28
mutants now die.
The type-discipline and test-quality gates blamed the branch for 4 LIT001, 12
LIT002 and 5 TQ008 violations. Rather than suppress them:
- the $vectorSearch and $project stages are MappingProxyType and the query
vector a tuple, verified against live Atlas to encode identically. The outer
pipeline stays a list because pymongo's common.validate_list raises
"pipeline must be a list, not <class 'tuple'>", which a unit test now pins.
- the client caches are Final[dict[...]] and _client_kwargs returns a
MappingProxyType.
- _field_value recurses over the dotted path instead of rebinding a local.
- _client_key declared Final locals in one branch and reassigned them in the
others, so it is split into an early-returning _timeout_ms.
- the injected callables carry explicit Final[Callable[...]] annotations, which
stops pyright resolving self.embedding_fn against litellm.embedding's
overloads.
- get_sync_client and get_async_client take an optional client_class, so the
cache tests inject a recording double instead of patching the importer, and
can assert the connection string and timeouts the client was built with.
SensitiveDataMasker is public SDK surface, so extra_sensitive_patterns moves to
the end of the signature: in slot two it silently reinterpreted an existing
caller's positional override set as extra sensitive patterns.
When the headroom_retrieve tool is exposed to a client that runs its own
tool-execution loop (the LiteLLM MCP gateway path), the client executes the
retrieve call and sends the recovered original content back as a tool result
on the next turn. The guardrail then compressed that row again, and because
CCR is content-addressed it collapsed back to the exact same hash it was just
retrieved from. The model never saw the expansion and the agent looped.
Hold tool-result rows that carry headroom_retrieve output back from the
compression service, the same way the live turn and trailing tool exchange are
already protected, so the expansion survives. Retrieve calls are matched by the
direct headroom_retrieve name and the mcp__<server>__headroom_retrieve gateway
name. Because a long gateway name is truncated past 64 chars in the
OpenAI-translated view the guardrail scans, the pairing also falls back to the
tool-call id read from the request's own untranslated messages, which is never
truncated.
Fixes#38558
Building the client parses the URI and, for mongodb+srv://, performs a DNS SRV
lookup, so it fails on exactly the inputs a user is most likely to get wrong. It
sat outside the try that translates driver errors, so a malformed URI or an
unresolvable cluster escaped as a raw pymongo exception and reached the caller as
a 500 with a traceback in the body.
The three DNS-shaped failures are also told apart now: a lookup that ran out of
time is a Timeout, a cluster name that is not in DNS says so and points at the
URI Atlas shows under Connect Drivers, and anything else keeps the generic
"not a usable MongoDB connection string".
Verified live: a tampered scheme, a nonexistent cluster and a 1ms timeout each
come back as their own message instead of a traceback.
Atlas matches on the vector alone, so a mistyped mongodb_text_field still returns
confidently scored results whose content is empty, and the model is handed an
empty context with nothing to explain it. When every matched document lacks the
field the search now says which setting to fix; a sparse document among others
that do have it, and a document whose text is genuinely the empty string, both
still come back normally.
Unrecognised mongodb_* parameters are named too. The params model has to ignore
unrelated keys because litellm_params carries plenty of them, which turned a
mistyped mongodb_collection into "mongodb_collection is required" pointing the
reader at a key they can see they have set.
The async client cache was keyed on id(loop). CPython recycles those ids so
aggressively that a fresh event loop nearly always lands on the id of one already
collected, measured at 37 of 40 rounds, so the cache handed the new loop an
AsyncMongoClient bound to a closed loop and every operation on it raised
"Event loop is closed".
The entry now carries a weak reference to the loop it was built on and a hit only
counts when that reference still points at the running loop, so a recycled id
misses and builds a fresh client. A stale entry can also be replaced once the
cache is full, which the old size check prevented.
pymongo's own client keeps its loop alive, which is why the sync proxy path never
saw this; a script calling asyncio.run() per search, or a test suite with a loop
per test, does.
Atlas answers a wrong password with code 8000 "AtlasError" rather than the 18 a
self-hosted deployment returns, so the code-only check never fired and a bad
password came back as a generic "MongoDB rejected the vector search", pointing
the reader at the index instead of at their credentials. Verified live against
Atlas with a tampered password.
litellm.exception_type passes a litellm exception through untouched and wraps
anything else into APIConnectionError, so every bare ValueError this provider
raised reached the caller as HTTP 500 with a Python traceback in the response
body. "max_num_results must be between 1 and 50" is the caller's to fix, not a
connection failure.
Configuration and validation failures now raise BadRequestError (400) and the
two timeout cases raise Timeout (408). ExecutionTimeout subclasses
OperationFailure, so it is matched before it; previously an Atlas query that ran
out of time was reported as "MongoDB rejected the vector search".
A MongoDB vector store's whole credential is its connection string, and
mongodb+srv://<user>:<password>@<cluster> embeds the database password. None of
the masker's default patterns (api_key, secret, token, credential) match a key
named mongodb_connection_string, so /vector_store/list and /vector_store/info
returned it verbatim to every caller that can read a vector store.
SensitiveDataMasker gains extra_sensitive_patterns, which unions onto the
defaults instead of replacing them, and the vector-store redactor adds
"connection" so the URI is masked while mongodb_database, mongodb_collection and
the field names stay readable.
Driving the sad path against a live Atlas cluster showed four cases returning
an empty result set instead of failing: a missing index, a missing database, a
missing collection, and the async path for all three. $vectorSearch reports
none of these as errors, so a misconfigured store looked exactly like a query
that matched nothing, which is the worst shape for this to fail in.
An empty result set is now checked against the index catalogue, which does
report all three correctly, and a store that cannot work says so. The check
costs one extra round trip and only on the empty path, so a search that
returned hits is unaffected.
Atlas also reports a wrong vector path and a dimension mismatch under the same
error code. Both previously surfaced as "index not found", which sent the
reader looking in the wrong place; they are now told apart and each names the
setting that is actually wrong.
65 cases across pipeline construction, response mapping, parameter validation,
client caching, and driver-error translation. The sad-path cases assert on the
message the caller actually sees, since a vector search that fails quietly
returns an empty result set rather than an error.
* fix(guardrails): run apply_guardrail-only providers in logging_only mode
A CustomGuardrail that implements only apply_guardrail inherited the CustomLogger
no-op async_logging_hook, so mode: logging_only never scanned anything and never
recorded guardrail_information. CustomGuardrail.async_logging_hook now routes the
logged request and response through the call type's guardrail translation on
copies and appends the verdict to standard_logging_object.guardrail_information.
Resolves LIT-4876
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep logging_only scan copies inside the error boundary and return a fresh logging payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): cover embedding scan, native-hook bypass, and unmapped call type in logging_only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Gemini 3.8 Flash launches today with the same promotional pricing, limits,
and thinking settings as Gemini 3.7 Flash, so the gemini/, vertex_ai/, and
bare cost map entries mirror the 3.7 Flash ones. Regression tests lock the
launch prices, the 4096-token cache minimum, and the gemini-3 thought
signature gate in for the new model.
* fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping
SearchAPIRouter dropped every parameter configured on a search tool, forwarding
only per-request kwargs. Any tool-level setting (mode, max_results, ...) was
silently lost on the way to the adapter, for every search provider.
Also completes the Parallel AI v1 search surface: after_date, fetch_policy,
location and include_domains now nest under advanced_settings instead of being
sent as unknown top-level fields, responses preserve search_id / session_id /
warnings / raw excerpts, and search cost is derived from the request mode and
the provider's reported usage rather than a single flat rate.
* fix(parallel_ai): stop a caller from pricing its own search request
`_parallel_ai_usage` carries the provider's reported usage into cost
calculation. It was only written when the response contained a usage block, so
a caller could pass `_parallel_ai_usage=[{"name": "sku_search", "count": 0}]`
and, whenever the provider omitted usage, bill $0.00 instead of $0.005 — the
value also reached the upstream request body as an unknown field.
The key is now stripped from inbound params and written unconditionally from
the parsed response, so only the provider can populate it.
* fix(parallel_ai): price fast search mode correctly
* test(parallel_ai): fake search at HTTP boundary
* fix(parallel_ai): tolerate null search result fields
---------
Co-authored-by: khushishelat <shelatkhushi@gmail.com>
tests/e2e/test_junit_properties.py fed a hand-rolled FakeItem to
result_properties and attach_result_properties, both typed pytest.Item,
so uv run basedpyright tests/e2e reported 3 reportArgumentType errors on
litellm_internal_staging and every make check that scopes a litellm/ or
tests/e2e/ Python file failed.
Each test now looks up its own collected Item in request.session.items
and applies the covers marker at run time through request.applymarker,
so the coverage registry's collect-only pass never sees the test ids and
the production functions keep their pytest.Item signatures. No casts, no
ignores.
Resolves LIT-6669
