scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-17 04:34:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 172
litellm/tests/code_coverage_tests/unbounded_in_baseline.txt
Mateo Wang 622daf3cdf
fix(scim): write a group's roster in one locked transaction instead of one member at a time (#45197) 
* fix(scim): write a group's roster in one locked transaction instead of one member at a time

POST, PUT and PATCH /scim/v2/Groups resolve every member in one query and write the roster, the
membership rows and the users' team tags in one transaction under the team's advisory lock, so a
500-member push runs a fixed number of statements instead of thousands. PUT now answers with the
roster it just wrote

* test(scim): assert the roster the IdP gets back and boot the bulk roster cell with two workers

* fix(scim): skip the team lock for a rename-only PATCH and lock member rows in id order before the roster write

A PATCH naming no members opened the roster transaction and took the team advisory lock for nothing. A SIGKILLed worker then left its orphaned query engine holding that lock until the 60 s transaction timeout, and /team/delete, which takes the same lock, timed out behind it. An empty delta is now answered from a plain team read, with no transaction and no lock

Two groups trading members in parallel locked user rows in opposite orders, and one request died with a Postgres deadlock. Every user row a push touches is now locked with SELECT ... ORDER BY user_id FOR UPDATE before the first write, so the two requests serialize

The bulk roster cell gets a 120 s client through a new owned_proxy_process client_timeout, so an unfixed push fails on the statement count instead of the harness clock, requires positive counts on both pushes, and gains a parallel roster-trade test that answered 500 before the lock

* fix(scim): evict the cached team on a group rename and read a no-op roster from the writer

A rename-only PUT or PATCH wrote the new alias and left the team cached under
its id and old alias on every worker, so keys kept logging spend against the
old team alias until the TTL expired. The group write now evicts the cached
team (and broadcasts the eviction) right after the alias or metadata update.

sync_team_roster reads the team from the writer when the plan names nobody,
so a replica cannot hand back the pre-rename row; a plan already matching the
roster returns inside the lock without writing, evicting, or emitting the
members metric; the response is built from the written row, so lastModified
reflects the roster write; and detaching a removed member bumps the user's
updated_at like every other user write.

* fix(scim): bump detached members' updated_at in UTC

The detach UPDATE wrote updated_at = CURRENT_TIMESTAMP, which Postgres casts
into Prisma's naive timestamp column in the session timezone, so on a database
whose session is not UTC the removed member's row landed hours away from the
UTC instants Prisma itself writes. Write NOW() AT TIME ZONE 'UTC' instead, and
have the bulk roster cell assert the bumped row sits between two reads of the
database's own UTC clock taken around the replace.

* fix(scim): chunk the group write's member lookups under the bind cap

Every Prisma "in" filter the group write issues bound one parameter per
member with nothing bounding the list, which check_unbounded_in_lists
reports as 11 new findings. The single-field lookups now go through
litellm.repositories.chunked_in (caller_transaction inside the roster
transaction, per_chunk_ok for the role recompute), and the one
cross-field OR read slices its values at IN_LIST_CHUNK_SIZE and records
the bound. A push of IN_LIST_CHUNK_SIZE + 1 members now reads in two
slices, and a member named only in the last slice still resolves.

* fix(proxy): send bulk cache evictions in slices so the broadcaster drops none

Veria pointed out that the invalidation publisher caps its pending tasks at
1,024 and drops the excess with a warning, and that a group push now detaches
every removed member's keys in one call instead of one member at a time, so a
removal of more than roughly 340 members (three cache keys each) silently left
stale key objects on the other replicas. The merge base never hit the cap
because each member was a separate request.

Bulk evictions (team, user, organization, and session key deletes, plus the
roster settle) now publish in slices of 256 and wait, bounded at two seconds,
for the backlog already handed to the event loop before each slice. A Redis
that never answers costs a bulk caller at most that wait per slice instead of
wedging the request, and single-key callers are untouched. Regression tests
push 1,500 keys and a 500-member removal through a slow publisher and assert
every eviction reaches Redis; both fail against the previous tip.

* fix(proxy): clear this worker's cache before pacing bulk eviction broadcasts

Greptile pointed out that pacing the broadcasts in slices also paced the
local eviction: a bulk delete waited for the publish backlog before dropping
the next slice from this worker's own memory, so on a Redis that accepts
connections but never answers a key deleted from the database kept
authenticating on the handling worker for up to two seconds per slice. The
merge base cleared every key locally at once.

Bulk evictions now drop every affected entry from this worker's memory first,
through the cache's own partition lookup since key objects live apart from
the other auth objects, and only then publish in paced slices. The roster
settle does the same for the deleted tokens, the JWT mapping keys and every
changed member's user and membership entries before any step that can wait.
Regression tests run both paths against a publisher that never answers and
check that nothing is still cached on this worker before the first bound
elapses; both fail with the local eviction removed.

* fix(proxy): drop the Redis copies of bulk-evicted auth objects before pacing the broadcasts

* docs(proxy): drop the module docstring that repeated the PR description

* fix(scim): bound bulk cache evictions to one Redis round trip and close lock-order gaps

Bulk key evictions now drop entries from memory, delete them from Redis in one
chunked DEL, and broadcast every key in one pipelined publish, so a large roster
removal no longer overflows the publisher's pending cap or costs a Redis round
trip per member. A push waits for in-flight membership loads before evicting.

User rows are locked in byte order to match the spend writer, a bulk user delete
takes every team lock before its first user write, and a POST whose roster write
fails deletes the empty team so the IdP retry creates it again.

* test(team): observe delete broadcasts at the Redis layer

* fix(scim): lock user rows in id order on every multi-row writer and create unprovisioned members concurrently

* fix(team): chunk the email lookup behind the member row lock

* fix(scim): create a member deleted mid-push again instead of failing the group write

* fix(scim): refuse strict-mode mid-push deletes with 400 and check license seats once per batch

A member deleted mid-push is created again only when scim_upsert_user allows it,
under the verb's own created_via. New group members are checked against the
license seat limit as one batch before any is created. Auth cache publish drops
log the key count instead of the key.

* fix(scim): lock member rows while recomputing admin roles after a group push

* fix(users): re-read teams under the row locks in bulk user delete so a racing group push keeps no deleted member

* fix(users): take /user/delete and SCIM DELETE /Users users off their current teams under the row locks so a racing group push keeps no deleted member

* fix(scim): keep retrying a push while deletes race it, lock rows in member bulk delete, evict removed keys from Redis first

A group push gave up after one retry when SCIM user deletes kept landing between attempts, answering 500 in 9 of 10 race rounds. It now retries while each attempt finds newly deleted members, and gives up only when a member it created again goes missing a second time.

The team member bulk delete now locks the removed members' user rows in the shared order and takes the team off their teams arrays in one array_remove statement, so it no longer writes back a teams array read before a concurrent push added another team.

A push that removes members now drops their deleted team keys from memory and Redis together with the members' other cache entries before the team eviction, so a removed key cannot refill from Redis mid-cleanup.

* fix(user-delete): keep the team alias label on the team members gauge after removals

* test(integration): count relay statements over plaintext so an SSL-enabled Postgres does not hide them

* fix(user-delete): skip a team deleted while the user delete waited on its lock instead of answering 404

* fix(scim): answer 404 and roll back when the team is deleted during a roster write

* test(integration): audit cells for SCIM bulk roster writes, user delete locking and the eviction broadcast
2026-10-15 10:57:00 -07:00

134 lines
15 KiB
Text
Raw Blame History

# Grandfathered findings of check_unbounded_in_lists.py: path::scope::kind::subject::occurrence.
# Fix a site and delete its line; regenerate with `check_unbounded_in_lists.py --update-baseline`.
litellm/integrations/shadow_eval_logger.py ShadowEvalLogger._active_jobs prisma job_id.in `[str(record.id) for record in records]` 0
litellm/llms/litellm_proxy/skills/handler.py LiteLLMSkillsHandler.list_skills prisma created_by.in `owner_scopes` 0
litellm/proxy/_experimental/mcp_server/db.py get_mcp_servers prisma server_id.in `server_ids` 0
litellm/proxy/_experimental/mcp_server/db.py get_user_env_vars_bulk prisma server_id.in `ids` 0
litellm/proxy/_experimental/mcp_server/db.py purge_user_oauth_credentials_for_server prisma user_id.in `[row.user_id for row in oauth_rows]` 0
litellm/proxy/_experimental/mcp_server/oauth2_flow_backfill.py backfill_null_oauth2_flows prisma server_id.in `server_ids_for_flow` 0
litellm/proxy/_experimental/mcp_server/oauth2_flow_backfill.py backfill_null_oauth2_flows prisma server_id.in `server_ids` 0
litellm/proxy/_experimental/mcp_server/toolset_db.py list_mcp_toolsets prisma toolset_id.in `toolset_ids` 0
litellm/proxy/agent_endpoints/endpoints.py _attach_keys_to_agents prisma agent_id.in `agent_ids` 0
litellm/proxy/agent_endpoints/endpoints.py get_agents prisma agent_id.in `agent_ids` 0
litellm/proxy/anthropic_endpoints/claude_code_endpoints/claude_code_skill_access.py SkillVisibility.where prisma name.in `sorted(self.granted)` 0
litellm/proxy/auth/auth_checks.py _fetch_uncached_model_access_group_budgets prisma access_group_name.in `list(uncached_groups)` 0
litellm/proxy/auth/auth_checks.py _fetch_uncached_tags prisma tag_name.in `list(tags_to_fetch)` 0
litellm/proxy/auth/auth_checks.py get_jwt_key_mapping_cache_keys_for_tokens prisma token.in `tuple(hashed_tokens)` 0
litellm/proxy/auth/auth_checks.py get_managed_vector_store_rows_by_uuids prisma vector_store_id.in `cache_misses` 0
litellm/proxy/common_utils/reset_budget_job.py _budget_link_where prisma budget_id.in `list(budget_ids)` 0
litellm/proxy/container_endpoints/ownership.py _get_allowed_container_ids prisma created_by.in `owner_scopes` 0
litellm/proxy/db/tool_registry_writer.py get_tools_by_names prisma tool_name.in `tool_names` 0
litellm/proxy/guardrails/guardrail_endpoints.py list_guardrail_submissions prisma team_id.in `visible_team_ids` 0
litellm/proxy/guardrails/usage_endpoints.py _build_usage_logs_where prisma ?.in `guardrail_ids` 0
litellm/proxy/guardrails/usage_endpoints.py guardrails_usage_detail prisma guardrail_id.in `metric_ids` 0
litellm/proxy/guardrails/usage_endpoints.py guardrails_usage_detail prisma guardrail_id.in `metric_ids` 1
litellm/proxy/guardrails/usage_endpoints.py guardrails_usage_detail prisma guardrail_id.in `metric_ids` 2
litellm/proxy/guardrails/usage_endpoints.py guardrails_usage_logs prisma request_id.in `request_ids` 0
litellm/proxy/list_api/list_framework.py _render raw-sql {field}.IN `IN ({placeholders})` 0
litellm/proxy/management_endpoints/access_group_endpoints.py _require_teams_exist prisma team_id.in `team_ids` 0
litellm/proxy/management_endpoints/access_group_endpoints.py _teams_touching prisma team_id.in `stored_team_ids` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py _with_target_labels prisma team_id.in `list(team_ids)` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py _with_target_labels prisma token.in `list(tokens)` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py _with_target_labels prisma user_id.in `list(user_ids)` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py get_shadow_eval_job prisma job_id.in `leg_ids` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py start_shadow_eval prisma target_id.in `list(ids)` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py start_shadow_eval prisma team_id.in `list(data.team_ids)` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py start_shadow_eval prisma token.in `list(data.api_key_ids)` 0
litellm/proxy/management_endpoints/auto_router_endpoints.py start_shadow_eval prisma user_id.in `list(data.user_ids)` 0
litellm/proxy/management_endpoints/budget_management_endpoints.py info_budget prisma budget_id.in `data.budgets` 0
litellm/proxy/management_endpoints/common_utils.py _team_admin_can_invite_user prisma team_id.in `admin_user_obj.teams` 0
litellm/proxy/management_endpoints/common_utils.py user_has_admin_privileges prisma team_id.in `user_obj.teams` 0
litellm/proxy/management_endpoints/customer_endpoints.py delete_end_user prisma user_id.in `data.user_ids` 0
litellm/proxy/management_endpoints/customer_endpoints.py delete_end_user prisma user_id.in `data.user_ids` 1
litellm/proxy/management_endpoints/internal_user_endpoints.py _check_user_info_v2_access prisma team_id.in `caller_user.teams` 0
litellm/proxy/management_endpoints/internal_user_endpoints.py _resolve_user_email_metadata prisma user_id.in `list(user_ids)` 0
litellm/proxy/management_endpoints/internal_user_endpoints.py delete_user prisma user_id.in `data.user_ids` 0
litellm/proxy/management_endpoints/internal_user_endpoints.py delete_user prisma user_id.in `data.user_ids` 1
litellm/proxy/management_endpoints/internal_user_endpoints.py get_users prisma organization_id.in `org_id_list` 0
litellm/proxy/management_endpoints/internal_user_endpoints.py get_users prisma sso_user_id.in `sso_id_list` 0
litellm/proxy/management_endpoints/internal_user_endpoints.py get_users prisma user_id.in `user_id_list` 0
litellm/proxy/management_endpoints/internal_user_endpoints.py ui_view_users prisma organization_id.in `org_filter_ids` 0
litellm/proxy/management_endpoints/key_management_endpoints.py _apply_non_admin_alias_scope raw-sql team_id.IN `IN ({team_placeholders})` 0
litellm/proxy/management_endpoints/key_management_endpoints.py _build_key_filter_conditions prisma team_id.in `admin_team_ids` 0
litellm/proxy/management_endpoints/key_management_endpoints.py _build_key_filter_conditions prisma team_id.in `member_only_team_ids` 0
litellm/proxy/management_endpoints/key_management_endpoints.py _build_key_filter_conditions prisma team_id.in `member_team_ids` 0
litellm/proxy/management_endpoints/key_management_endpoints.py _fetch_user_team_objects prisma team_id.in `complete_user_info.teams` 0
litellm/proxy/management_endpoints/key_management_endpoints.py list_key_helper prisma user_id.in `all_ids` 0
litellm/proxy/management_endpoints/key_management_endpoints.py bulk_update_team_keys prisma token.in `hashed_key_ids` 0
litellm/proxy/management_endpoints/key_management_endpoints.py delete_key_aliases prisma key_alias.in `key_aliases` 0
litellm/proxy/management_endpoints/key_management_endpoints.py delete_verification_tokens prisma token.in `hashed_tokens` 0
litellm/proxy/management_endpoints/key_management_endpoints.py info_key_fn_v2 prisma key_alias.in `data.key_aliases` 0
litellm/proxy/management_endpoints/mcp_management_endpoints.py fetch_all_mcp_servers prisma server_id.in `byok_server_ids` 0
litellm/proxy/management_endpoints/model_access_group_management_endpoints.py update_deployments_with_access_group prisma model_name.in `model_names` 0
litellm/proxy/management_endpoints/model_management_endpoints.py delete_team_models prisma model_id.in `model_ids` 0
litellm/proxy/management_endpoints/organization_endpoints.py deprecated_info_organization prisma organization_id.in `data.organizations` 0
litellm/proxy/management_endpoints/organization_endpoints.py list_organization prisma organization_id.in `membership_org_ids` 0
litellm/proxy/management_endpoints/router_weights.py validate_router_settings_weights prisma model_id.in `list(deployment_ids)` 0
litellm/proxy/management_endpoints/session_endpoints.py revoke_ui_session_keys prisma token.in `revoked_tokens` 0
litellm/proxy/management_endpoints/tag_management_endpoints.py _get_model_names prisma model_id.in `model_ids` 0
litellm/proxy/management_endpoints/tag_management_endpoints.py _get_tag_list_scope prisma api_key.in `scoped_api_keys` 0
litellm/proxy/management_endpoints/tag_management_endpoints.py info_tag prisma tag_name.in `data.names` 0
litellm/proxy/management_endpoints/tag_management_endpoints.py list_tags prisma tag_name.in `used_tag_names` 0
litellm/proxy/management_endpoints/team_endpoints.py _append_permissions_to_specific_teams prisma team_id.in `team_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py _authorize_and_filter_teams prisma organization_id.in `allowed_org_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py _batch_resolve_access_group_resources prisma access_group_id.in `unique_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py _build_team_list_where_conditions prisma organization_id.in `org_admin_org_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py _build_team_list_where_conditions prisma organization_id.in `org_admin_org_ids` 1
litellm/proxy/management_endpoints/team_endpoints.py _build_team_list_where_conditions prisma team_id.in `list(own_team_ids)` 0
litellm/proxy/management_endpoints/team_endpoints.py _build_team_list_where_conditions prisma team_id.in `user_team_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py _get_keys_count_by_team prisma team_id.in `page_team_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py _hydrate_member_user_details prisma user_id.in `sorted(user_ids)` 0
litellm/proxy/management_endpoints/team_endpoints.py _resolve_existing_member_user_ids prisma user_id.in `sorted(requested_user_ids)` 0
litellm/proxy/management_endpoints/team_endpoints.py resolve_team_daily_activity_scope prisma team_id.in `list(team_ids_list)` 0
litellm/proxy/management_endpoints/team_endpoints.py _sweep_deleted_team_references prisma team_id.in `tuple(team_ids)` 0
litellm/proxy/management_endpoints/team_endpoints.py _sweep_deleted_team_references_tx prisma team_id.in `tuple(team_ids)` 0
litellm/proxy/management_endpoints/team_endpoints.py _team_member_delete prisma user_id.in `sorted(addressed_user_ids)` 0
litellm/proxy/management_endpoints/team_endpoints.py _team_member_delete prisma user_id.in `sorted(user_ids_to_delete)` 0
litellm/proxy/management_endpoints/team_endpoints.py _team_member_delete prisma user_id.in `sorted(user_ids_to_delete)` 1
litellm/proxy/management_endpoints/team_endpoints.py _team_user_spend_sql raw-sql sl.team_id.IN `IN ({team_placeholders})` 0
litellm/proxy/management_endpoints/team_endpoints.py delete_team prisma team_id.in `data.team_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py delete_team prisma team_id.in `data.team_ids` 1
litellm/proxy/management_endpoints/team_endpoints.py get_all_team_memberships prisma team_id.in `team_ids` 0
litellm/proxy/management_endpoints/team_endpoints.py list_available_teams prisma team_id.in `available_teams` 0
litellm/proxy/management_endpoints/tool_management_endpoints.py get_tool_spend prisma tool_name.in `[row.tool_name for row in top_tools]` 0
litellm/proxy/management_endpoints/tool_management_endpoints.py get_tool_usage_logs prisma request_id.in `request_ids` 0
litellm/proxy/management_endpoints/ui_sso.py fetch_cli_sso_team_details prisma team_id.in `teams` 0
litellm/proxy/management_endpoints/user_agent_analytics_endpoints.py get_per_user_analytics prisma tag.in `tag_filters` 0
litellm/proxy/management_endpoints/user_agent_analytics_endpoints.py get_per_user_analytics prisma token.in `list(api_keys)` 0
litellm/proxy/management_endpoints/user_agent_analytics_endpoints.py get_per_user_analytics prisma user_id.in `user_ids` 0
litellm/proxy/management_endpoints/workflow_management_endpoints.py list_workflow_runs prisma ?.in `statuses` 0
litellm/proxy/management_helpers/bulk_user_creation.py _existing_user_conflicts prisma user_email.in `emails` 0
litellm/proxy/management_helpers/bulk_user_creation.py _existing_user_conflicts prisma user_id.in `user_ids` 0
litellm/proxy/management_helpers/bulk_user_creation.py _insert_users prisma user_id.in `list(requested)` 0
litellm/proxy/management_helpers/bulk_user_creation.py _load_teams prisma team_id.in `sorted(team_ids)` 0
litellm/proxy/management_helpers/bulk_user_creation.py _write_audit_logs prisma user_id.in `created_ids` 0
litellm/proxy/management_helpers/bulk_user_deletion.py _in_filter prisma [field].in `sorted(values)` 0
litellm/proxy/management_helpers/object_permission_utils.py _get_db_mcp_servers_by_identifiers prisma alias.in `identifier_list` 0
litellm/proxy/management_helpers/object_permission_utils.py _get_db_mcp_servers_by_identifiers prisma server_id.in `identifier_list` 0
litellm/proxy/management_helpers/object_permission_utils.py _get_db_mcp_servers_by_identifiers prisma server_name.in `identifier_list` 0
litellm/proxy/management_helpers/resource_display_names.py agent_display_names prisma agent_id.in `tuple(wanted)` 0
litellm/proxy/management_helpers/resource_display_names.py key_display_names prisma token.in `tuple(frozenset(tokens))` 0
litellm/proxy/management_helpers/resource_display_names.py mcp_server_display_names prisma server_id.in `tuple(wanted)` 0
litellm/proxy/policy_engine/policy_resolve_endpoints.py _build_alias_where prisma [field].in `exact` 0
litellm/proxy/policy_engine/policy_resolve_endpoints.py _find_affected_by_team_patterns prisma team_id.in `matched_team_ids` 0
litellm/proxy/proxy_server.py _add_access_group_models_to_team_models prisma access_group_id.in `list(all_access_group_ids)` 0
litellm/proxy/proxy_server.py _not_in_router_ids prisma not.in `list(ids)` 0
litellm/proxy/proxy_server.py _gather_team_accessible_model_ids prisma model_name.in `_resolved_names` 0
litellm/proxy/proxy_server.py get_all_team_models prisma team_id.in `user_teams` 0
litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py _prune_filter prisma model.in `chunk` 0
litellm/proxy/auth/authorization_dependencies.py load_permitted_log_team_ids prisma team_id.in `user_obj.teams` 0
litellm/proxy/spend_tracking/spend_management_endpoints.py _read_scope_where prisma team_id.in `list(scope.team_ids)` 0
litellm/proxy/ui_crud_endpoints/proxy_setting_endpoints.py _validate_default_teams_exist prisma team_id.in `list(team_ids)` 0
litellm/proxy/utils.py PrismaClient.check_view_exists raw-sql viewname.IN `IN ( {expected_views_str} )` 0
litellm/proxy/utils.py PrismaClient.delete_data prisma team_id.in `team_id_list` 0
litellm/proxy/utils.py PrismaClient.delete_data prisma team_id.in `team_id_list` 1
litellm/proxy/utils.py PrismaClient.delete_data prisma token.in `hashed_tokens` 0
litellm/proxy/utils.py PrismaClient.delete_data prisma token.in `hashed_tokens` 1
litellm/proxy/utils.py PrismaClient.get_data prisma budget_id.in `budget_id_list` 0
litellm/proxy/utils.py PrismaClient.get_data prisma team_id.in `team_id_list` 0
litellm/proxy/utils.py PrismaClient.get_data prisma user_id.in `user_id_list` 0
litellm/proxy/utils.py prefetch_config_params prisma param_name.in `param_names` 0
litellm/repositories/daily_activity_repository.py DailyActivityRepository.daily_rows prisma [scope.entity_id_field].in `list(scope.entity_ids)` 0
litellm/repositories/daily_activity_repository.py DailyActivityRepository.daily_rows prisma api_key.in `list(scope.api_keys)` 0
litellm/repositories/daily_activity_repository.py DailyActivityRepository.daily_rows prisma not.in `list(scope.exclude_entity_ids)` 0
litellm/router_utils/auto_router_model_naming.py <module> raw-sql classifier_type.IN `IN ({_LLM_CLASSIFIER_TYPES_SQL})` 0
Reference in a new issue View git blame Copy permalink