* feat(skills): semantic search over the LiteLLM-hosted skill registry
Adds GET /v1/skills?query= (custom_llm_provider=litellm_proxy) and a
skill_search MCP virtual tool, ranking the caller's accessible skills by
semantic similarity, mirroring the A2A agent registry search (LIT-6309).
Also fixes a pre-existing bug where create_skill() dropped description and
instructions for the litellm_proxy provider, which left every LiteLLM-hosted
skill with no searchable text.
* fix(mcp): coerce skill_search top_k instead of raising 500 on malformed input
The MCP-REST skill_search dispatch validated raw tool arguments through a
pydantic model directly, so a non-numeric top_k raised a ValidationError
that the endpoint's catch-all turned into an HTTP 500. Mirrors the
agent_search branch's tolerant coerce_top_k handling instead.
* fix(skills): enforce key limits on search embeddings and bound the semantic index
Semantic search embeddings now run the same pre_call_hook the /embeddings
route runs, so key rate limits, budgets and guardrails apply before the
embedding model is called. The shared SemanticTextIndex caps cached vectors
and evicts the least recently searched entries, and each skill's embedded
text is capped so one skill cannot inflate the embedding batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): surface proxy 429s from search embeddings instead of a 503
ProxyRateLimitError is also an OpenAIError, so the search engine was folding
a key rate limit into skill_search_unavailable. Proxy HTTPExceptions now
propagate so the caller gets the same 429 the /embeddings route returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): import assert_never from typing_extensions for Python 3.10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): embed the request as the pre-call hooks returned it, not the original text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(skills): keep the litellm_proxy provider check for GET /v1/skills?query= inside llms/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(skills): move the GET /v1/skills?query= endpoint tests under tests/test_litellm/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: limit Rust workflows to Rust directory changes
* ci: run Rust checks when their workflow changes
* ci: report Rust wheels only for successful Rust changes
* ci: keep Rust wheel reports in the workflow summary
* ci: group Rust lint and validation jobs
* ci: keep Rust job names distinct from required lint and test checks
* ci: drop the unused Python setup from the Rust lint job
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Its job id was `test`, the same check-run name test-mcp.yml publishes and
guard-internal-staging requires, so a commit touching the sweep carried two
check runs called `test`. Renaming it to `sweep-tests` leaves the required
context with exactly one job behind it.
A ruleset's required status check names a check run and GitHub matches it by
that name alone, so two jobs publishing the same name leave the gate unable to
say which job proved it. The new code-quality check reads every workflow,
expands matrix values and local reusable-workflow calls the way Actions does,
and fails when one name has more than one job behind it.
A harness-only change (proxy_client.py, conftest.py, pytest.ini, the gateway
config, .github/e2e-stack, or the workflow) selected nothing, so the stack was
never exercised by the change that touched it. select_tests.py keeps the
changed-file rule and adds the access_control suite whenever a harness file
changes. The run step now reports the pytest exit code before the evidence
check, prints pytest's summary line per pass so the rerun count is visible,
and assert_tests_ran.py names each failed or errored test as classname::name
The changed-tests workflow overrode the suite's `--reruns 1` with `--reruns 0`, so a
transport blip failed a pass that pytest.ini already scopes to network errors and
5xx responses. Pass 2 of run 33692484803 also went red 15s after a model write with
"no healthy deployments": the barrier only polled /v1/models through nginx, which
proves one gateway converged, and the next request rolled the other. The stack now
exports LITELLM_PROXY_REPLICA_URLS, the barrier polls every replica with the full
budget before settling, and up.sh refuses to boot without DD_API_KEY, since the
gateway config enables the datadog callback on every run
* fix(ci): grant pull_requests write for release wheel reporter
The reporter posts a PR comment via github.rest.issues.createComment.
GitHub requires both issues=write and pull_requests=write to comment on
a PR issue, as returned in x-accepted-github-permissions. The workflow
had pull-requests: read, so the POST failed with 403 'Resource not
accessible by integration'. Bumping to pull-requests: write fixes the
create path; the read-only pulls.get call still works. Same-repo scope
is preserved by the existing head_repository.full_name check.
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
* fix(ci): scope release wheel reporter permissions to pull requests
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
Co-authored-by: Yujong Lee <yujong@berri.ai>
Replace test-model-map.yml with a pull_request_target guard that validates the
cost map, its backup, and its generated schema on every PR, and additionally
enforces the sync bot contract on litellm_cost_map_sync_* branches: only the
three cost map files may change, no model or field is removed, and the special
root keys stay untouched.
Clippy never links, so python-config's pyo3/auto-initialize needs no
libpython and the gateway clippy step can cover every feature at once.
The test step stays on --features server because cargo test does link
and this job installs no Python.
The check list existed in three places that had already drifted apart;
CLAUDE.md is now the only copy and the other two point at it.
The build-ui check compiled the dashboard from a full checkout, so any
import reaching above ui/litellm-dashboard/ resolved there and only broke
inside the images, where the stage copies the dashboard tree alone.
Building the stage itself puts the check on the same file boundary the
shipped images use.
litellm-ai-gateway's server feature is off by default and nothing in the workspace turns it on, so the workspace clippy and test steps never compiled src/auth, src/routes, src/state, src/realtime or the gateway binary. 43 tests ran instead of 57.
Adds the two steps CLAUDE.md already documents as the local gate, and fixes the three collapsible_if violations that had accumulated behind the flag.
Forwarding limit to OpenAI made the ownership filter cut the page down after
the fact, so a key that owned an older container got an empty first page and
its cursor never moved. Non-admin lists now walk upstream pages of 100 until
they have enough owned containers (or five pages), trim to the requested
limit, and report first_id, last_id and has_more off what the caller keeps.
Also assigns tests/test_litellm/proxy/container_endpoints to a CI shard.
* refactor(python-bridge): split non-streaming bridge modules
* refactor(python-bridge): bring shared function tracing into route layer
* feat(dev): list Python route functions and call sites
* feat(dev): list Rust route functions and call sites
* docs(dev): record OCR parity gaps across Python and Rust
* feat(dev): list executed SDK calls with runtime tracing
* feat(dev): report Python vs Rust SDK pipeline steps in one CLI
* feat(dev): side-by-side pipeline step report in compare CLI
* fix(dev): drop invalid Final annotations in compare cell loop
* feat(dev): blue python-only and yellow rust-only steps in compare CLI
* feat(dev): vertical layout with section spacing in compare CLI
* fix(dev): validate SDK trace stages across sync and async routes
* refactor(rust): align SDK route call structure with Python
* refactor(python-bridge): share sync and async route call wrappers
* refactor(dev): split compare CLI into fixtures, runtime, and report modules
* fix(ci): run SDK trace tests and satisfy test lint
Adds a required-check candidate that selects the tests/e2e test files a PR added or
modified, boots a stage-mirror stack on the runner (migrations, backend, two gateway
processes behind nginx, Postgres, Jaeger, TLS cluster Valkey), and runs those files
three times with retries off. The run job sits behind the e2e-changed GitHub
environment, so a reviewer approves each run before the OIDC token that reads the
provider keys from AWS Secrets Manager exists. Supersedes #34981
tests/test_litellm/llms/mongodb imports pymongo's exception classes to check the
error translation against the real hierarchy, and the shard that runs it
(tests/test_litellm/llms, per test-unit.yml) synced --extra google, proxy,
semantic-router and saml but not mongodb, so 24 of 109 tests would have errored
with ModuleNotFoundError on the first CI run. CircleCI hid this because it syncs
--all-groups --all-extras.
uv export --frozen ... --extra saml -> no pymongo
uv export --frozen ... --extra saml --extra mongodb -> pymongo==4.17.0
Also close the two gaps a mutation run found in the suite: nothing asserted that
a short request timeout shortens server selection as well as connect, and the
existing code 13 case carried "not authorized", which the message markers match
too, so it could not tell whether the code was still being checked. 28 of 28
mutants now die.
The Dockerfile, docker/Dockerfile.non_root and docker/Dockerfile.database uv sync stages never passed --extra bedrock-realtime, so aws-sdk-bedrock-runtime was absent from the image venv and Bedrock Nova Sonic /v1/realtime sessions failed with 'Missing aws_sdk_bedrock_runtime'. gateway/Dockerfile already had the extra (PR #34426).
Adds a static check over every uv sync in the proxy Dockerfiles and an image-level import probe that the image-scan workflow runs against the built root, non-root and gateway images.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(redis): coerce env var string types and fix param discovery through decorator wrappers
inspect.getfullargspec doesn't work on redis.Redis/redis.RedisCluster because
their __init__ is wrapped by @deprecated_args, which replaces the explicit
signature with *args/**kwargs internally. getfullargspec returns an empty arg
list, so _get_redis_kwargs and _get_redis_cluster_kwargs silently dropped every
real constructor parameter not in their hand-picked include_args set --
cluster_error_retry_attempts and connection_error_retry_attempts among them, so
an operator's configured retry bound never reached the Redis Cluster client and
it fell back to redis-py's own default instead.
Rebased onto litellm_internal_staging, which had independently added
_init_arg_names (MRO-walking, inspect.unwrap-based) for the same class of bug
in _get_redis_url_kwargs. Reused that pattern (as _unwrapped_init_args, without
the MRO walk: redis.Redis/RedisCluster declare every real parameter directly on
their own __init__, and MRO-walking breaks the tests here that mock the class
with autospec=True, since inspect.getmro needs a real __mro__) rather than
introducing a second, differently-shaped fix for the same problem.
_get_redis_cluster_kwargs now also honors its own client argument instead of
ignoring it, so the async cluster client's own extra constructor kwargs
(cluster_error_retry_attempts, connection_error_retry_attempts,
decode_responses, ...) are no longer filtered out by introspecting the sync
class regardless of which client is actually built.
Also fixes environment variables and Helm --set values always arriving as
strings: redis-py 8.x changed health_check_interval's arithmetic to require a
real number, so a stringified value raised TypeError on every Redis operation
instead of connecting. _coerce_redis_kwargs_types coerces to each parameter's
declared type at the end of _get_redis_client_logic, with an explicit type
table for max_connections/socket_timeout/socket_connect_timeout since redis-py
8.x changed the timeout defaults from None to int 5, which would otherwise
make a fractional value fail int() and get dropped.
Co-authored-by: mangabits <1457532+mangabits@users.noreply.github.com>
* ci: verify redis-py client version compatibility across a version matrix
* test(redis): assert an async-only cluster kwarg every matrix version declares
connection_error_retry_attempts is on the async cluster constructor in redis-py
5.x only; 6.0 removed it in favor of retry. The 6.4.0, 7.4.1 and 8.0.1 legs were
failing on that missing parameter name rather than on the behavior under test,
while the allow-list itself was doing the right thing on all four versions.
decode_responses is async-cluster-only on every version the matrix covers, so it
stands in for the same property: the sync cluster class takes it through **kwargs
and never names it in its signature. Reverting _get_redis_cluster_kwargs to ignore
its client argument still fails both tests on 5.3.1 and 8.0.1.
test_async_cluster_passes_async_only_kwargs now builds the real async cluster
client and reads connection_kwargs off it, so it no longer needs a patched class
factory; the constructor does no I/O. The retry-attempts test keeps its patch,
since redis-py >= 6 stores no cluster_error_retry_attempts attribute on the built
client and the constructor call is the only place the forwarded value shows up.
The _get_redis_cluster_kwargs docstring cited the same two parameters as its
examples of async-only kwargs, which is what made the test look reasonable;
cluster_error_retry_attempts is on both classes and connection_error_retry_attempts
is gone from 6.0 on, so it now names decode_responses instead.
* test(redis): drop internal patches from the kwarg coercion tests
The test-quality gate flagged the new patch() calls on litellm internals these
tests added. Three of them faked litellm._redis.inspect.signature with a MagicMock
to hand _coerce_redis_kwargs_types a synthetic parameter; that function already
takes a client argument, so they pass stub functions instead, matching the
_redis_signature_8x idiom the file uses elsewhere. The fourth patched
_redis_kwargs_from_environment to {} to prove _get_redis_client_logic raises
without a host or url, which clearing the real env keys through
_get_redis_env_kwarg_mapping does without pinning the test to that call.
Both files now sit one TQ008 below the merge base rather than six above it.
* fix(redis): keep the sync client construction inside the basedpyright budget
_get_redis_client_logic now returns dict[str, object] rather than an untyped
dict, which is the honest type for operator-supplied config, but it turns the
33 reportUnknownArgumentType errors at redis.Redis(**redis_kwargs) into 33
reportArgumentType errors plus one reportCallIssue, both over their budget.
No static type fits: redis-py's constructor declares 40-odd differently typed
parameters and the values arrive from config and env, so the allow-list and
coercion above are derived from that same signature and redis-py validates each
value itself at runtime.
The two suppressions name their exact rule and carry that reason. The file ends
up 42 basedpyright errors below the merge base, with reportArgumentType and
reportCallIssue back at the base counts of 3 and 0.
* fix(redis): coerce cluster-only and None-default bool kwargs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mangabits <1457532+mangabits@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
