From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9B4C35A958 for ; Thu, 29 Jan 2026 20:57:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769720242; cv=none; b=IsG3oUmlFwnDUtLmrndyElyTZLnoVAUPmfoGi2T9apye35XYBUYZ/yN4RIcOqU57UJl3BSm2xmZ/O6fsMVt96e9JkZ6yb74v9R/bFLAPN63IBzGUJQpFt92goJuA0fE0HqPxGR9/1ZReS1MRWqEIqAo2XY7cGGRZzzGcEmKvKDc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769720242; c=relaxed/simple; bh=p4FTC5ZyH+C3t13NMpOGBz9eFYM/+qAG9LyXQfJNC38=; h=From:Message-ID:Date:MIME-Version:Subject:To:Cc:References: In-Reply-To:Content-Type; b=uSb2kfW1wPaYBcyX9Rn/7DXXbZQU90u1U8CITDmNwPtU7tRQTHypwfWvekOJGw2jezcOVT8tzC7lRHy9tU5PmHHZSA8QRto1hsFBX1JV/0ZL4PuQUztJLGFEpVbYdQFtYYhksF5RtjUu+DMkDKm9bXKgmdEqM0xR/li58BYADr4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VMvXgbMe; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ft5UpGEu; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VMvXgbMe"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ft5UpGEu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1769720240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/QG3wvjCSAd8hZsGbKuvepZVeOKrzgGWIR2Hdb2A8lo=; b=VMvXgbMeqSJy56xmu1yI9DKCUnc9qIZB62Iz4s4YjaCQvAyckjNiLvY+t+sxJwNhPkhJub BJEikH1zkzABHrxBPvQ1ytjogdHgaH2qac+clEC2WylMktzXSKS0LCMVKC9Rh5NJx+scEd 2C+0s6Ik8e5SOrQToU41tU4LBr7KXg4= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-180--uGpKnlZNumkirn6g1s5oQ-1; Thu, 29 Jan 2026 15:57:18 -0500 X-MC-Unique: -uGpKnlZNumkirn6g1s5oQ-1 X-Mimecast-MFC-AGG-ID: -uGpKnlZNumkirn6g1s5oQ_1769720238 Received: by mail-qk1-f197.google.com with SMTP id af79cd13be357-8c70cff1da5so368800385a.2 for ; Thu, 29 Jan 2026 12:57:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1769720238; x=1770325038; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:subject:user-agent:mime-version:date:message-id:from:from:to :cc:subject:date:message-id:reply-to; bh=/QG3wvjCSAd8hZsGbKuvepZVeOKrzgGWIR2Hdb2A8lo=; b=Ft5UpGEuLbwdGSkVIPjEfpzx4urdH220iJox5JZ2B3V2Ib1ISQSgKU50NDD0hz2ABa QD6fY+wi0TWItaZZaNGjTE2Bvhdunqs7upnX4f4iQhvpqFAPHd0XDg1R4RuGHsPdqjyk iYQzGnjlfCOK8ISaduWxj2wOxb3aDZpkWGyD5JCQXwZliO7dZ9ZaEE4SBiZCBujpTX5n EChL+CHc8KIKwxwJA8L2qon6moWSRGO8SzsPXnx78uhZPVynOXkbC3NR3kQK36Id0h1W jhBJ6Zot4HoITChaReT2Djozc1xu3Rh7hSsZz86JyIOzUD/Abt1WOg41HfSmei64TIq1 k2aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769720238; x=1770325038; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:subject:user-agent:mime-version:date:message-id:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/QG3wvjCSAd8hZsGbKuvepZVeOKrzgGWIR2Hdb2A8lo=; b=YVlxzUdeqsyTzuOxDaTyWTBHFo9QMMIYK0UlFEIGVMujcx4mDNRgnr9sUa8dpg38+A vUVr/pg80s6fNZCOVBjM+s7duwQiE4XSRRyVJX9wv6E2lael7SRMnDx2zQ6rFlzI+BNi wesJ6/9tx8/Y4ozSC9lMS/yapR9JacTKSXOL6CdH1q9wUWCSi9aDMClyDNp2DefUxfA8 XQiMX10AIUO05fQZEiq2KZrV+XxIzl3rdgbHfwHxxPibH/XUdaYTG88fhuDdKOQLST7g AhL02a29HcQxIUSo1m8tGDz/fSgd+hP5tnuUL6wt5sxliFlW3GkpC8Qs0hHaF5sFEanG APvg== X-Forwarded-Encrypted: i=1; AJvYcCXjEbQihvsrm4SJWMQ0ZQLoKbg7JB0/WFbsw+NWpWPxZ2sJIX5M+NzJhDRHla1hFQf+6AM5WU96QPVvLJU=@vger.kernel.org X-Gm-Message-State: AOJu0YwEbWFlmVYvVjv1ltCV+4gnHPfrLJNo2Qen70Ykl89vHrUuIEmr 1zw/lO7IvWf1koQgIV4P5VcurYSR1JX9yTQgsRGqrfmfdIEOp+VEK2CPMCwXN/WwTYxaPHSF0Gh e0HciV6m8s+Jt0YkY0GLRddB9emDacF/bB53kD5y/5w212hoJWJ2vj1x44C5USP6gZA== X-Gm-Gg: AZuq6aLaoo2IJ+FWSsBob2Q373i9cEaO98s/rXJXYKYmmWA2ekQnYwowwsQ7BkiwKWL DxzKz2tdCwEROmRv/kJk3ByDE3K6nviUbMXN/O0tiCxzaVy9ukoRgWIFk9FOeNgcngvdKqCiUdI c+Om92Zo7RQicMQ48nCUQ3ZF9BKaTGd4ZEYUU++segtodFPL3TKetl2k98rcUcm3Qiouvy1QD4C kxiiTLv2ZvAEUO5rPvTZP5uhNDxn5Tbj0lWDXceZL11fsEOncPdVoGRo86lwXxJ4rdF7GKPUFZy c4y1YMkThAtOuTjdmGxX7xS/QcX/LysaHSvQhC6ZJXQHnGKbh/dhUjiTDKG4ytvDPVODunutpqz +LiC8O06MtmyQVGokQFX/vFQBbv1y9DYwU0uD01T1254CMik9V6STKxuD X-Received: by 2002:a05:620a:4154:b0:8b2:e1d7:ca6a with SMTP id af79cd13be357-8c9eb31574bmr130734285a.75.1769720237889; Thu, 29 Jan 2026 12:57:17 -0800 (PST) X-Received: by 2002:a05:620a:4154:b0:8b2:e1d7:ca6a with SMTP id af79cd13be357-8c9eb31574bmr130732385a.75.1769720237466; Thu, 29 Jan 2026 12:57:17 -0800 (PST) Received: from ?IPV6:2601:188:c102:b180:1f8b:71d0:77b1:1f6e? ([2601:188:c102:b180:1f8b:71d0:77b1:1f6e]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8c711b95e4esm536422085a.15.2026.01.29.12.57.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 29 Jan 2026 12:57:16 -0800 (PST) From: Waiman Long X-Google-Original-From: Waiman Long Message-ID: Date: Thu, 29 Jan 2026 15:57:15 -0500 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH/for-next 2/2] cgroup/cpuset: Introduce a new top level isolcpus_update_mutex To: Chen Ridong , Tejun Heo , Johannes Weiner , =?UTF-8?Q?Michal_Koutn=C3=BD?= , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Anna-Maria Behnsen , Frederic Weisbecker , Thomas Gleixner , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org References: <20260128044251.1229702-1-longman@redhat.com> <20260128044251.1229702-3-longman@redhat.com> <08c3fad6-b881-4089-b081-bde6efbafbd2@huaweicloud.com> <8fb3aab5-ba0a-4523-a404-5643b8a749c9@huaweicloud.com> Content-Language: en-US In-Reply-To: <8fb3aab5-ba0a-4523-a404-5643b8a749c9@huaweicloud.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 1/29/26 3:20 AM, Chen Ridong wrote: > > On 2026/1/29 16:01, Chen Ridong wrote: >> >> On 2026/1/28 12:42, Waiman Long wrote: >>> The current cpuset partition code is able to dynamically update >>> the sched domains of a running system and the corresponding >>> HK_TYPE_DOMAIN housekeeping cpumask to perform what is essentally the >>> "isolcpus=domain,..." boot command line feature at run time. >>> >>> The housekeeping cpumask update requires flushing a number of different >>> workqueues which may not be safe with cpus_read_lock() held as the >>> workqueue flushing code may acquire cpus_read_lock() or acquiring locks >>> which have locking dependency with cpus_read_lock() down the chain. Below >>> is an example of such circular locking problem. >>> >>> ====================================================== >>> WARNING: possible circular locking dependency detected >>> 6.18.0-test+ #2 Tainted: G S >>> ------------------------------------------------------ >>> test_cpuset_prs/10971 is trying to acquire lock: >>> ffff888112ba4958 ((wq_completion)sync_wq){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x7a/0x180 >>> >>> but task is already holding lock: >>> ffffffffae47f450 (cpuset_mutex){+.+.}-{4:4}, at: cpuset_partition_write+0x85/0x130 >>> >>> which lock already depends on the new lock. >>> >>> the existing dependency chain (in reverse order) is: >>> -> #4 (cpuset_mutex){+.+.}-{4:4}: >>> -> #3 (cpu_hotplug_lock){++++}-{0:0}: >>> -> #2 (rtnl_mutex){+.+.}-{4:4}: >>> -> #1 ((work_completion)(&arg.work)){+.+.}-{0:0}: >>> -> #0 ((wq_completion)sync_wq){+.+.}-{0:0}: >>> >>> Chain exists of: >>> (wq_completion)sync_wq --> cpu_hotplug_lock --> cpuset_mutex >>> >>> 5 locks held by test_cpuset_prs/10971: >>> #0: ffff88816810e440 (sb_writers#7){.+.+}-{0:0}, at: ksys_write+0xf9/0x1d0 >>> #1: ffff8891ab620890 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x260/0x5f0 >>> #2: ffff8890a78b83e8 (kn->active#187){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x2b6/0x5f0 >>> #3: ffffffffadf32900 (cpu_hotplug_lock){++++}-{0:0}, at: cpuset_partition_write+0x77/0x130 >>> #4: ffffffffae47f450 (cpuset_mutex){+.+.}-{4:4}, at: cpuset_partition_write+0x85/0x130 >>> >>> Call Trace: >>> >>> : >>> touch_wq_lockdep_map+0x93/0x180 >>> __flush_workqueue+0x111/0x10b0 >>> housekeeping_update+0x12d/0x2d0 >>> update_parent_effective_cpumask+0x595/0x2440 >>> update_prstate+0x89d/0xce0 >>> cpuset_partition_write+0xc5/0x130 >>> cgroup_file_write+0x1a5/0x680 >>> kernfs_fop_write_iter+0x3df/0x5f0 >>> vfs_write+0x525/0xfd0 >>> ksys_write+0xf9/0x1d0 >>> do_syscall_64+0x95/0x520 >>> entry_SYSCALL_64_after_hwframe+0x76/0x7e >>> >>> To avoid such a circular locking dependency problem, we have to >>> call housekeeping_update() without holding the cpus_read_lock() >>> and cpuset_mutex. One way to do that is to introduce a new top level >>> isolcpus_update_mutex which will be acquired first if the set of isolated >>> CPUs may have to be updated. This new isolcpus_update_mutex will provide >>> the need mutual exclusion without the need to hold cpus_read_lock(). >>> > When I reviewed Frederic's patches, I concerned about this issue. However, I was > not certain whether any flush worker would need to acquire cpu_hotplug_lock or > cpuset_mutex. > > Despite this warning, I do not understand how wq_completion would need to > acquire cpu_hotplug_lock and cpuset_mutex. > > The reason I want to understand how wq_completion acquires cpu_hotplug_lock or > cpuset_mutex is to determine whether isolcpus_update_mutex is truly necessary. > As I mentioned in my previous email, I am concerned about a potential > use-after-free (UAF) issue, which might imply that isolcpus_update_mutex is > required in most places that currently acquire cpuset_mutex, with the possible > exception of the hotplug path? A circular lock dependency can invoke more than 2 tasks/parties. In this case, the task that hold wq_completion does not need to acquire cpu_hotplug_lock. If a worker that flushes a work function required for the completion to finish and it happens to acquire cpu_hotplug_lock with another task trying to acquire cpus_write_lock in the interim, the worker will wait there for the write lock to be released which will not happen until the original task that calls flush_workqueue() release its read lock. In essence, it is a deadlock. Cheers, Longman