Router.acompletion() takes messages positionally, so splatting a body that omits it raised a TypeError that the generic handler mapped to a 500. Validate the required body param at the routing boundary and raise the existing 400 contract instead.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Team/key router_settings.enable_tag_filtering was stored and echoed by
/team/info but never applied at request time: the per-request override
whitelist in route_llm_request.py dropped it, tag filtering only read the
router-level flag, and UpdateRouterConfig silently discarded the field on
/key/generate and /config/update. Requests from teams with the toggle on
were load balanced across all deployments instead of tag-matched ones.
- add enable_tag_filtering to the router_settings_override whitelist and
strip any client-supplied copy from the request body first, so only the
key/team value reaches the router
- run tag filtering when the request carries enable_tag_filtering=True; a
request-level False cannot disable a router-level True, so per-request
settings can only scope down, never escape the global policy
- add the field to UpdateRouterConfig so key and config update paths stop
dropping it, and to all_litellm_params so it never leaks into provider
request bodies
- allow it through Router.update_settings/get_settings so the global UI
toggle persists across DB config reloads
Resolves LIT-4390
* fix(router): honor per-request routing_strategy from key/team router_settings
Key and team router_settings.routing_strategy was stored and shown in the
UI but never forwarded to the shared Router, so the global strategy always
won. Forward it through router_settings_override and resolve it in
_get_routing_context: a validated per-request strategy takes precedence
over routing groups and the top-level strategy, with lazily built cached
selectors for strategies that need one. Unknown or unsupported strategy
values are ignored with a warning instead of failing the request, and
routing_strategy is registered in all_litellm_params so it is stripped
before the provider call.
* fix(router): sweep override selectors on strategy re-init and cover coverage-gate helpers
routing_strategy_init now unregisters cached per-request override
selectors so a later update_settings strategy change cannot leave a
zombie selector receiving callback events. Adds direct tests for the
two new helpers so the router code coverage gate passes.
* docs(team): document mcp_rpm_limit in update_team docstring
The documentation CI job walks management_endpoints and requires every
UpdateTeamRequest field to appear in the update_team docstring;
mcp_rpm_limit was added to the model without a docstring line, failing
the job on unrelated PRs depending on walk order. Regenerates
schema.d.ts since the docstring feeds the OpenAPI spec.
_with_resolved_session_model was overwriting the nested
input_audio_transcription.model and audio.input.transcription.model with the
realtime conversation model, silently replacing a caller's transcription model
(e.g. whisper-1) since those are a different model than the realtime deployment.
It now only resolves the top-level session model.
Also restores session.model taking precedence over the top-level model in
acreate_realtime_client_secret, matching the proxy's own
_prepare_client_secret_session ordering and avoiding a backwards-incompatible flip.
Adds routing coverage for arealtime_calls (api_base resolution) and
acreate_realtime_transcription_session (api_key resolution) so all three realtime
HTTP endpoints have router credential-resolution tests, plus regression tests for
the two fixes above.
Co-authored-by: Cursor <cursoragent@cursor.com>
Realtime client_secrets, calls, and transcription_sessions were bypassing
the router and falling back to an empty OPENAI_API_KEY for wildcard, team-scoped,
and credential-name deployments.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(proxy): fix vector store retrieve/list/update/delete routing without model
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(proxy): remove unchecked query-param injection in vector store management endpoints
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(proxy): use subset assertion for vector store route test to allow extra kwargs like shared_session
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
CodeQL flagged the previous ``from litellm.types.router import
MockRouterTestingParams`` at module top-level — ``litellm.types.router``
indirectly imports back into proxy modules, so the dataclass may not
exist yet when ``route_llm_request`` is being imported.
Hardcode the three flag names instead, with a guard test
(``test_mock_testing_kwarg_names_matches_dataclass``) that asserts the
hardcoded list matches ``MockRouterTestingParams.fields`` so drift is
caught at test time rather than missed in production.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes that together prevent a caller from smuggling unauthorized
models past the API key's allowlist via per-request router overrides.
1. ``_enforce_key_and_fallback_model_access``: also walk fallback models
nested inside ``router_settings_override.fallbacks`` /
``context_window_fallbacks`` / ``content_policy_fallbacks``.
``route_llm_request.py`` promotes those to per-request kwargs after
auth, so without this they bypassed the model allowlist entirely.
New ``iter_router_fallback_model_names`` helper extracts leaf names
from both the simple top-level shape (str | {"model": str}) and the
nested router-config shape ({primary: [fallbacks]}). The two fallback
validation loops are unified — every name (top-level + override) is
deduplicated and validated once via ``can_key_call_model`` +
``is_valid_fallback_model``.
2. ``route_request``: strip router-internal ``mock_testing_*`` flags
from user-supplied data. These are testing-only flags that
deterministically force the router into fallback logic by raising a
synthetic ``InternalServerError`` etc. Combined with override
fallbacks they made the smuggling path trivially exploitable. Test
code that calls the router directly bypasses the strip and is
unaffected. The strip list is derived from ``MockRouterTestingParams``
so a new ``mock_testing_*`` flag added to that dataclass is
automatically covered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* (#13284) add avector_store_create to route_type which doesn't require model
* (#13284) exclude hidden params in metadata when create vector store
* (#13284) fix lint error
* (#13284) keep metadata None if metadata is None(not empty dict)
* (#13284) add test code
* (#13284) change test code name
* (#13284) add avector_store_search to route_type which doesn't require model
