mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-05 02:41:56 +00:00
693ad49719
* Litellm ishaan march23 - MCP Toolsets + GCP Caching fix (#25146) * feat(mcp): MCP Toolsets — curated tool subsets from one or more MCP servers (#24335) * feat(mcp): add LiteLLM_MCPToolsetTable and mcp_toolsets to ObjectPermissionTable * feat(mcp): add prisma migration for MCPToolset table * feat(mcp): add MCPToolset Python types * feat(mcp): add toolset_db.py with CRUD helpers for MCPToolset * feat(mcp): add toolset CRUD endpoints to mcp_management_endpoints * fix(mcp): skip allow_all_keys servers when explicit mcp_servers permission is set (toolset scope fix) * feat(mcp): add _apply_toolset_scope and toolset route handling in server.py * fix(mcp): resolve toolset names in responses API before fetching tools * feat(mcp): add mcp_toolsets field to LiteLLM_ObjectPermissionTable type * feat(mcp): register LiteLLM_MCPToolsetTable in prisma client initialization * feat(mcp): validate mcp_toolsets in key-vs-team permission check * feat(mcp): register toolset routes in proxy_server.py * feat(mcp): add MCPToolset and MCPToolsetTool TypeScript types * feat(mcp): add fetchMCPToolsets, createMCPToolset, updateMCPToolset, deleteMCPToolset API functions * feat(mcp): add useMCPToolsets React Query hook * feat(mcp): add toolsets (purple) as third option type in MCPServerSelector * feat(mcp): extract toolsets from combined MCP field in key form * feat(mcp): extract toolsets from combined MCP field in team form * feat(mcp): show toolsets section in MCPServerPermissions read view * feat(mcp): pass mcp_toolsets through object_permissions_view * feat(mcp): add MCPToolsetsTab component for creating and managing toolsets * feat(mcp): add Toolsets tab to mcp_servers.tsx * feat(mcp): pass mcpToolsets to playground chat and responses API calls * feat(mcp): generate correct server_url for toolsets in playground API calls * docs(mcp): add MCP Toolsets documentation * docs(mcp): add mcp_toolsets to sidebar * fix(mcp): replace x-mcp-toolset-id header with ContextVar to prevent client forgery * fix(mcp): use ContextVar + StreamingResponse for toolset MCP routes (fixes SSE streaming) * fix(mcp): cache toolset permission lookups to avoid per-request DB calls * test(mcp): add tests for toolset scope enforcement, ContextVar isolation, and access control * fix(mcp): cache toolset name lookups in MCPServerManager to avoid per-request DB calls * fix(mcp): prevent body_iter deadlock + use cached toolset lookup in responses API - _stream_mcp_asgi_response: add done callback to handler_task that puts the EOF sentinel on body_queue when the task exits, preventing body_iter from hanging forever if the handler raises after headers are sent. - litellm_proxy_mcp_handler: replace raw get_mcp_toolset_by_name() DB call with global_mcp_server_manager.get_toolset_by_name_cached() so toolset resolution uses the 60s TTL cache added for this purpose instead of hitting the DB on every responses-API request. * fix(mcp): toolset access control, asyncio fix, and real unit tests - server.py: _apply_toolset_scope now enforces that non-admin keys must have the requested toolset_id in their mcp_toolsets grant list; admin keys always bypass the check. - mcp_management_endpoints.py: three access-control fixes: * fetch_mcp_toolsets: non-admin keys with mcp_toolsets=None now return [] instead of all toolsets (only admins get 'all' when the field is absent) * fetch_mcp_toolset: non-admin keys that haven't been granted the requested toolset_id now get 403 instead of the full result * add_mcp_toolset: duplicate toolset_name now returns 409 Conflict instead of an opaque 500 - proxy_server.py: use asyncio.get_running_loop() instead of get_event_loop() inside an already-running coroutine (Python 3.10+). - test_mcp_toolset_scope.py: replace four hollow tests that only asserted local variable properties with real tests that call the production fetch_mcp_toolsets() and handle_streamable_http_mcp() functions with mocked dependencies. * fix(mcp): add mcp_toolsets to ObjectPermissionBase, fix multi-toolset overwrite, fix delete 404, allow standalone key toolsets * fix(mcp): add auth check on toolset resolution in responses API; union mcp_servers in _merge_toolset_permissions * fix(mcp): handle RecordNotFoundError in update_mcp_toolset; union direct servers with toolset servers * fix(mcp): use _user_has_admin_view; deny None mcp_toolsets for non-admin; use direct RecordNotFoundError import; fix docstring * fix(mcp): add @default(now()) to MCPToolsetTable.updated_at; fix test for non-admin toolset access * fix: use UniqueViolationError import; guard _ensure_eof for error/cancel only * fix(mcp): preserve mcp_access_groups in toolset scope, use shared Redis cache for toolset perms - Remove mcp_access_groups=[] from _apply_toolset_scope (server.py) and the responses API toolset path (litellm_proxy_mcp_handler.py). A key's access-group grants remain valid even when the request is scoped to a single toolset; clearing them silently revoked legitimate entitlements. - Switch resolve_toolset_tool_permissions and get_toolset_by_name_cached to use user_api_key_cache (Redis-backed DualCache in production) instead of per-instance in-memory dicts. Cache entries are now shared across workers, eliminating the per-worker stale-toolset-permission window flagged as a P1 by Greptile. - Use union merge (set union of tool names per server) when applying toolset permissions in the responses API path so direct-server tool restrictions are not overwritten by toolset permissions. * fix(mcp): return 404 when edit_mcp_toolset target does not exist * fix(mcp): align mcp_toolsets default to None in LiteLLM_ObjectPermissionTable * fix(mcp): admin toolset visibility, in-place tool name mutation, test helper coercion * fix(mcp): treat None/[] team mcp_toolsets as no restriction in key validation * fix(mcp): allow_all_keys backward compat, blocked_tools API write-path, efficient startup query * fix(mcp): use _mcp_active_toolset_id ContextVar to detect toolset scope, avoiding DB-default false-positive * fix(mcp): remove dead toolset cache stubs, log invalidation failures, align schema updated_at defaults * fix(mcp): deserialise MCPToolset from Redis cache hit, replace fastapi import in test * fix(mcp): evict name-cache on toolset mutation, 409 on rename conflict, warning-level list errors * fix(redis): regenerate GCP IAM token per connection for async cluster (#24426) * fix(redis): regenerate GCP IAM token per connection for async cluster clients Async RedisCluster was generating the IAM token once at startup and storing it as a static password. After the 1-hour GCP token TTL, any new connection (including to newly-discovered cluster nodes) would fail to authenticate. Fix: introduce GCPIAMCredentialProvider that implements redis-py's CredentialProvider protocol. It calls _generate_gcp_iam_access_token() on every new connection, matching what the sync redis_connect_func already does. async_redis.RedisCluster accepts a credential_provider kwarg which is invoked per-connection. * refactor(redis): move GCPIAMCredentialProvider to its own file Extract GCPIAMCredentialProvider and _generate_gcp_iam_access_token into litellm/_redis_credential_provider.py. _redis.py imports them from there, keeping the public API unchanged. * fix: address Greptile review issues - GCPIAMCredentialProvider now inherits from redis.credentials.CredentialProvider so redis-py's async path calls get_credentials_async() properly - move _redis_credential_provider import to top of _redis.py (PEP 8) - remove dead else-branch that silently no-oped (gcp_service_account from redis_kwargs.get() was always None since it's popped by _get_redis_client_logic) - remove mid-function 'from litellm import get_secret_str' inline import - remove unused 'call' import from test_redis.py * chore: retrigger CI/review * chore: sync schema.prisma copies from root * chore: sync schema.prisma copies from root * fix(proxy_server): use bounded asyncio.Queue with maxsize to prevent unbounded growth * fix(a2a/pydantic_ai): make api_base Optional to match base class signature * fix(a2a/pydantic_ai): make api_base Optional in handler and guard against None * fix(mcp): remove unused get_all_mcp_servers import * fix(mcp): remove unused MCPToolset import * refactor(mcp): extract toolset permission logic to reduce statement count below PLR0915 limit * fix(tests): update reload_servers_from_database tests to mock prisma directly --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix(toolset_db): lazy-import prisma to avoid ImportError when prisma not installed * fix(tests): update UI tests for toolset tab and updated empty state text * fix(tests): add get_mcp_server_by_name to fake_manager stub --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| 20260108_add_user_email_lower_idx | ||
| 20250326162113_baseline | ||
| 20250326171002_add_daily_user_table | ||
| 20250327180120_add_api_requests_to_daily_user_table | ||
| 20250329084805_new_cron_job_table | ||
| 20250331215456_track_success_and_failed_requests_daily_agg_table | ||
| 20250411215431_add_managed_file_table | ||
| 20250412081753_team_member_permissions | ||
| 20250415151647_add_cache_read_write_tokens_daily_spend_transactions | ||
| 20250415191926_add_daily_team_table | ||
| 20250416115320_add_tag_table_to_db | ||
| 20250416151339_drop_tag_uniqueness_requirement | ||
| 20250416185146_add_allowed_routes_litellm_verification_token | ||
| 20250425182129_add_session_id | ||
| 20250430193429_add_managed_vector_stores | ||
| 20250507161526_add_mcp_table_to_db | ||
| 20250507161527_add_health_check_fields_to_mcp_servers | ||
| 20250507184818_add_mcp_key_team_permission_mgmt | ||
| 20250508072103_add_status_to_spendlogs | ||
| 20250509141545_use_big_int_for_daily_spend_tables | ||
| 20250510142544_add_session_id_index_spend_logs | ||
| 20250514142245_add_guardrails_table | ||
| 20250522223020_managed_object_table | ||
| 20250526154401_allow_null_entity_id | ||
| 20250528185438_add_vector_stores_to_object_permissions | ||
| 20250603210143_cascade_budget_changes | ||
| 20250618225828_add_health_check_table | ||
| 20250625145206_cascade_budget_and_loosen_managed_file_json | ||
| 20250625213625_add_status_to_managed_object_table | ||
| 20250707212517_add_mcp_info_column_mcp_servers | ||
| 20250707230009_add_mcp_namespaced_tool_name | ||
| 20250711220620_add_stdio_mcp | ||
| 20250718125714_add_litellm_params_to_vector_stores | ||
| 20250802162330_prompt_table | ||
| 20250806095134_rename_alias_to_server_name_mcp_table | ||
| 20250918083359_drop_spec_version_column_from_mcp_table | ||
| 20250926194702_unnamed_migration | ||
| 20251003165142_add_allowed_tools_to_mcp | ||
| 20251003190954_extra_headers_to_mcp_table | ||
| 20251006143948_add_mcp_tool_permissions | ||
| 20251011084309_add_tag_table | ||
| 20251023141814_add_search_tool_table | ||
| 20251031181430_add_cache_config_table | ||
| 20251101131415_add_managed_vector_store_index_table | ||
| 20251103072422_add_static_headers | ||
| 20251104220043_add_credentials_to_mcp_servers | ||
| 20251113000000_add_project_table | ||
| 20251113000001_add_project_fields | ||
| 20251114173537_add_request_id_to_daily_tag_spend | ||
| 20251114180624_Add_org_usage_table | ||
| 20251114182247_agents_table | ||
| 20251119131227_add_prompt_versioning | ||
| 20251122125322_Add organization_id to spend logs | ||
| 20251204124859_add_end_user_spend_table | ||
| 20251204142718_add_agent_permissions | ||
| 20251209112246_add_ui_settings_table | ||
| 20251210125210_add_storage_backend_to_managed_files | ||
| 20251210205007_add_daily_agent_spend_table | ||
| 20251211100212_schema_sync | ||
| 20251219110931_add_deleted_keys_and_deleted_teams_tables | ||
| 20251220144550_schema_update | ||
| 20260102131258_add_metadata_urls_to_mcp_servers | ||
| 20260105151539_add_allow_all_keys_to_mcp_servers | ||
| 20260106155622_add_endpoint_to_daily_activity_tables | ||
| 20260107111013_add_router_settings_to_keys_teams | ||
| 20260116142756_update_deleted_keys_teams_table_routing_settings | ||
| 20260123131407_add_policy_tables_and_policies_field | ||
| 20260131150814_add_team_user_to_vector_stores | ||
| 20260203120000_add_deprecated_verification_token_table | ||
| 20260205091235_allow_team_guardrail_config | ||
| 20260205144610_add_soft_budget_to_team_table | ||
| 20260207093506_add_available_on_public_internet_to_mcp_servers | ||
| 20260207110613_add_soft_budget_to_deleted_teams_table | ||
| 20260209085821_add_verificationtoken_indexes | ||
| 20260212103349_adjust_tags_policy_table | ||
| 20260212143306_add_access_group_table | ||
| 20260213105436_add_managed_vector_store_table | ||
| 20260213170952_access_group_change_to_model_name | ||
| 20260214094754_schema_sync | ||
| 20260214163027_add_pipeline_to_policy_table | ||
| 20260214185341_object_permissions_for_end_users | ||
| 20260218231534_add_last_active_to_key_table | ||
| 20260219105005_add_project_id_to_deleted_keys | ||
| 20260219181415_baseline_diff | ||
| 20260220124742_add_spec_path_to_mcp_servers | ||
| 20260220153844_add_composite_index_aggregate_tables | ||
| 20260221000000_ensure_project_id_verification_token | ||
| 20260221183800_add_policy_versioning | ||
| 20260222000000_add_batch_processed_to_managed_object_table | ||
| 20260224201417_spend_logs_request_duration | ||
| 20260224203854_add_agent_object_permissions_table | ||
| 20260226000000_add_blocked_tools_to_object_permission | ||
| 20260226120000_add_spend_log_tool_index | ||
| 20260226202727_add_agent_id_to_delete_keys | ||
| 20260228000000_add_claude_code_plugin_table | ||
| 20260228100000_add_spend_logs_composite_index | ||
| 20260228110000_mcp_default_public_internet_true | ||
| 20260228170127_support_team_based_guardrails | ||
| 20260303000000_update_tool_table_policies | ||
| 20260304175016_add_spend_to_agent_table | ||
| 20260305000000_add_agent_headers | ||
| 20260305000000_add_rate_limits_to_agents | ||
| 20260306175056_add_configs_override_table | ||
| 20260306233848_schema_sync | ||
| 20260309000000_add_mcp_approval_status | ||
| 20260309000001_add_mcp_source_url | ||
| 20260312124619_schema_sync | ||
| 20260318140652_add_index_to_team_table | ||
| 20260319000000_restore_mcp_approval_fields | ||
| 20260321000000_add_mcp_toolsets | ||
| 20260331000000_add_prompt_environment_and_created_by | ||
| migration_lock.toml | ||