mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Daniel Wagner <wagi@kernel.org>
To: "Jens Axboe" <axboe@kernel.dk>, "Keith Busch" <kbusch@kernel.org>,
	"Christoph Hellwig" <hch@lst.de>,
	"Sagi Grimberg" <sagi@grimberg.me>,
	"Kashyap Desai" <kashyap.desai@broadcom.com>,
	"Sumit Saxena" <sumit.saxena@broadcom.com>,
	"Shivasharan S" <shivasharan.srikanteshwara@broadcom.com>,
	"Chandrakanth patil" <chandrakanth.patil@broadcom.com>,
	"Martin K. Petersen" <martin.petersen@oracle.com>,
	"Nilesh Javali" <njavali@marvell.com>,
	GR-QLogic-Storage-Upstream@marvell.com,
	"Don Brace" <don.brace@microchip.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"Jason Wang" <jasowang@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"Eugenio Pérez" <eperezma@redhat.com>,
	"Xuan Zhuo" <xuanzhuo@linux.alibaba.com>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Thomas Gleixner" <tglx@linutronix.de>
Cc: "Costa Shulyupin" <costa.shul@redhat.com>,
	"Juri Lelli" <juri.lelli@redhat.com>,
	"Valentin Schneider" <vschneid@redhat.com>,
	"Waiman Long" <llong@redhat.com>,
	"Ming Lei" <ming.lei@redhat.com>,
	"Michal Koutný" <mkoutny@suse.com>,
	"Frederic Weisbecker" <frederic@kernel.org>,
	"Mel Gorman" <mgorman@suse.de>, "Hannes Reinecke" <hare@suse.de>,
	"Sridhar Balaraman" <sbalaraman@parallelwireless.com>,
	"brookxu.cn" <brookxu.cn@gmail.com>,
	linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
	linux-nvme@lists.infradead.org, megaraidlinux.pdl@broadcom.com,
	linux-scsi@vger.kernel.org, storagedev@microchip.com,
	virtualization@lists.linux.dev, "Daniel Wagner" <wagi@kernel.org>
Subject: [PATCH v4 9/9] blk-mq: issue warning when offlining hctx with online isolcpus
Date: Tue, 17 Dec 2024 19:29:43 +0100	[thread overview]
Message-ID: <20241217-isolcpus-io-queues-v4-9-5d355fbb1e14@kernel.org> (raw)
In-Reply-To: <20241217-isolcpus-io-queues-v4-0-5d355fbb1e14@kernel.org>

When we offlining a hardware context which also serves isolcpus mapped
to it, any IO issued by the isolcpus will stall as there is nothing
which handles the interrupts etc.

This configuration/setup is not supported at this point thus just issue
a warning.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 block/blk-mq.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index de15c0c76f874a2a863b05a23e0f3dba20cb6488..f9af0f5dd6aac8da855777acf2ffc61128f15a74 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3619,6 +3619,45 @@ static bool blk_mq_hctx_has_requests(struct blk_mq_hw_ctx *hctx)
 	return data.has_rq;
 }
 
+static void blk_mq_hctx_check_isolcpus_online(struct blk_mq_hw_ctx *hctx, unsigned int cpu)
+{
+	const struct cpumask *hk_mask;
+	int i;
+
+	if (!housekeeping_enabled(HK_TYPE_MANAGED_IRQ))
+		return;
+
+	hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
+
+	for (i = 0; i < hctx->nr_ctx; i++) {
+		struct blk_mq_ctx *ctx = hctx->ctxs[i];
+
+		if (ctx->cpu == cpu)
+			continue;
+
+		/*
+		 * Check if this context has at least one online
+		 * housekeeping CPU in this case the hardware context is
+		 * usable.
+		 */
+		if (cpumask_test_cpu(ctx->cpu, hk_mask) &&
+		    cpu_online(ctx->cpu))
+			break;
+
+		/*
+		 * The context doesn't have any online housekeeping CPUs
+		 * but there might be an online isolated CPU mapped to
+		 * it.
+		 */
+		if (cpu_is_offline(ctx->cpu))
+			continue;
+
+		pr_warn("%s: offlining hctx%d but there is still an online isolcpu CPU %d mapped to it, IO stalls expected\n",
+			hctx->queue->disk->disk_name,
+			hctx->queue_num, ctx->cpu);
+	}
+}
+
 static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx,
 		unsigned int this_cpu)
 {
@@ -3638,8 +3677,10 @@ static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx,
 			continue;
 
 		/* this hctx has at least one online CPU */
-		if (this_cpu != cpu)
+		if (this_cpu != cpu) {
+			blk_mq_hctx_check_isolcpus_online(hctx, this_cpu);
 			return true;
+		}
 	}
 
 	return false;

-- 
2.47.1


  parent reply	other threads:[~2024-12-17 18:30 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-12-17 18:29 [PATCH v4 0/9] blk: honor isolcpus configuration Daniel Wagner
2024-12-17 18:29 ` [PATCH v4 1/9] lib/group_cpus: let group_cpu_evenly return number of groups Daniel Wagner
2025-01-07  7:51   ` Hannes Reinecke
2025-01-07  8:20     ` Daniel Wagner
2025-01-07 10:35       ` Hannes Reinecke
2025-01-07 10:46         ` Daniel Wagner
2025-01-07 11:35           ` Hannes Reinecke
2024-12-17 18:29 ` [PATCH v4 2/9] sched/isolation: document HK_TYPE housekeeping option Daniel Wagner
2025-01-07 15:39   ` Waiman Long
2024-12-17 18:29 ` [PATCH v4 3/9] blk-mq: add number of queue calc helper Daniel Wagner
2025-01-08  7:04   ` Hannes Reinecke
2024-12-17 18:29 ` [PATCH v4 4/9] nvme-pci: use block layer helpers to calculate num of queues Daniel Wagner
2025-01-08  7:19   ` Hannes Reinecke
2024-12-17 18:29 ` [PATCH v4 5/9] scsi: " Daniel Wagner
2024-12-17 18:29 ` [PATCH v4 6/9] virtio: blk/scsi: " Daniel Wagner
2024-12-19  6:25   ` Christoph Hellwig
2024-12-19  8:31   ` Michael S. Tsirkin
2024-12-17 18:29 ` [PATCH v4 7/9] lib/group_cpus: honor housekeeping config when grouping CPUs Daniel Wagner
2024-12-17 18:29 ` [PATCH v4 8/9] blk-mq: use hk cpus only when isolcpus=managed_irq is enabled Daniel Wagner
2024-12-19  6:26   ` Christoph Hellwig
2024-12-19  9:20   ` Ming Lei
2024-12-19 15:38     ` Daniel Wagner
2024-12-20  8:54       ` Ming Lei
2025-01-10  9:21         ` Daniel Wagner
2025-01-11  3:31           ` Ming Lei
2025-01-13 13:19             ` Daniel Wagner
2024-12-17 18:29 ` Daniel Wagner [this message]
2024-12-19  6:28   ` [PATCH v4 9/9] blk-mq: issue warning when offlining hctx with online isolcpus Christoph Hellwig
2024-12-20  9:04   ` Ming Lei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241217-isolcpus-io-queues-v4-9-5d355fbb1e14@kernel.org \
    --to=wagi@kernel.org \
    --cc=GR-QLogic-Storage-Upstream@marvell.com \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=brookxu.cn@gmail.com \
    --cc=chandrakanth.patil@broadcom.com \
    --cc=costa.shul@redhat.com \
    --cc=don.brace@microchip.com \
    --cc=eperezma@redhat.com \
    --cc=frederic@kernel.org \
    --cc=hare@suse.de \
    --cc=hch@lst.de \
    --cc=jasowang@redhat.com \
    --cc=juri.lelli@redhat.com \
    --cc=kashyap.desai@broadcom.com \
    --cc=kbusch@kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=llong@redhat.com \
    --cc=martin.petersen@oracle.com \
    --cc=megaraidlinux.pdl@broadcom.com \
    --cc=mgorman@suse.de \
    --cc=ming.lei@redhat.com \
    --cc=mkoutny@suse.com \
    --cc=mst@redhat.com \
    --cc=njavali@marvell.com \
    --cc=pbonzini@redhat.com \
    --cc=sagi@grimberg.me \
    --cc=sbalaraman@parallelwireless.com \
    --cc=shivasharan.srikanteshwara@broadcom.com \
    --cc=stefanha@redhat.com \
    --cc=storagedev@microchip.com \
    --cc=sumit.saxena@broadcom.com \
    --cc=tglx@linutronix.de \
    --cc=virtualization@lists.linux.dev \
    --cc=vschneid@redhat.com \
    --cc=xuanzhuo@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®