From: Daniel Wagner <dwagner@suse.de>
To: Ming Lei <ming.lei@redhat.com>
Cc: "Daniel Wagner" <wagi@kernel.org>, "Jens Axboe" <axboe@kernel.dk>,
"Keith Busch" <kbusch@kernel.org>,
"Christoph Hellwig" <hch@lst.de>,
"Sagi Grimberg" <sagi@grimberg.me>,
"Kashyap Desai" <kashyap.desai@broadcom.com>,
"Sumit Saxena" <sumit.saxena@broadcom.com>,
"Shivasharan S" <shivasharan.srikanteshwara@broadcom.com>,
"Chandrakanth patil" <chandrakanth.patil@broadcom.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
"Nilesh Javali" <njavali@marvell.com>,
GR-QLogic-Storage-Upstream@marvell.com,
"Don Brace" <don.brace@microchip.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Jason Wang" <jasowang@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Stefan Hajnoczi" <stefanha@redhat.com>,
"Eugenio Pérez" <eperezma@redhat.com>,
"Xuan Zhuo" <xuanzhuo@linux.alibaba.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Costa Shulyupin" <costa.shul@redhat.com>,
"Juri Lelli" <juri.lelli@redhat.com>,
"Valentin Schneider" <vschneid@redhat.com>,
"Waiman Long" <llong@redhat.com>,
"Michal Koutný" <mkoutny@suse.com>,
"Frederic Weisbecker" <frederic@kernel.org>,
"Mel Gorman" <mgorman@suse.de>, "Hannes Reinecke" <hare@suse.de>,
"Sridhar Balaraman" <sbalaraman@parallelwireless.com>,
"brookxu.cn" <brookxu.cn@gmail.com>,
linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
linux-nvme@lists.infradead.org, megaraidlinux.pdl@broadcom.com,
linux-scsi@vger.kernel.org, storagedev@microchip.com,
virtualization@lists.linux.dev
Subject: Re: [PATCH v4 8/9] blk-mq: use hk cpus only when isolcpus=managed_irq is enabled
Date: Mon, 13 Jan 2025 14:19:14 +0100 [thread overview]
Message-ID: <4047eb51-ee6b-46ae-a67b-ce74c54c41e7@flourine.local> (raw)
In-Reply-To: <Z4Hl_oQhJ2u6g2B3@fedora>
Hi Ming,
On Sat, Jan 11, 2025 at 11:31:10AM +0800, Ming Lei wrote:
> > What about a commit message like:
> >
> > When isolcpus=managed_irq is enabled, and the last housekeeping CPU for
> > a given hardware context goes offline, there is no CPU left which
> > handles the IOs anymore. If isolated CPUs mapped to this hardware
> > context are online and an application running on these isolated CPUs
> > issue an IO this will lead to stalls.
>
> It isn't correct, the in-tree code doesn't have such stall, no matter if
> IO is issued from HK or isolated CPUs since the managed irq is guaranteed to
> live if any mapped CPU is online.
Yes, it has different properties.
> Please see irq_do_set_affinity():
>
> if (irqd_affinity_is_managed(data) &&
> housekeeping_enabled(HK_TYPE_MANAGED_IRQ)) {
> const struct cpumask *hk_mask;
>
> hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
>
> cpumask_and(&tmp_mask, mask, hk_mask);
> if (!cpumask_intersects(&tmp_mask, cpu_online_mask))
> prog_mask = mask;
> else
> prog_mask = &tmp_mask;
> } else {
> prog_mask = mask;
>
> The whole mask which may include isolated CPUs is only programmed to
> hardware if there isn't any online CPU in `irq_mask & hk_mask`.
This is not what I try to achieve here. The main motivation with this
series is that isolated CPUs are never serving IRQs.
> > I was talking about implementing the feature which would remap the
> > isolated CPUs to online hardware context when the current hardware
> > context goes offline. I didn't find a solution which I think would be
> > worth presenting. All involved some sort of locking/refcounting in the
> > hotpath, which I think we should just avoid.
>
> I understand the trouble, but it is still one improvement from user
> viewpoint instead of feature since the interface of 'isolcpus=manage_irq'
> isn't changed.
Ah, I understood you wrong. I didn't want to upset you. I thought you
were fine by changing how managed_irq works.
> > Indeed, I forgot to update the documentation. I'll update it accordingly.
>
> It isn't documentation thing, it breaks the no-regression policy, which crosses
> our red-line.
>
> If you really want to move on, please add one new kernel command
> line with documenting the new usage which requires applications to
> offline CPU in order.
Sure, I'll bring the separate command line option back.
Thanks,
Daniel
next prev parent reply other threads:[~2025-01-13 13:19 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-12-17 18:29 [PATCH v4 0/9] blk: honor isolcpus configuration Daniel Wagner
2024-12-17 18:29 ` [PATCH v4 1/9] lib/group_cpus: let group_cpu_evenly return number of groups Daniel Wagner
2025-01-07 7:51 ` Hannes Reinecke
2025-01-07 8:20 ` Daniel Wagner
2025-01-07 10:35 ` Hannes Reinecke
2025-01-07 10:46 ` Daniel Wagner
2025-01-07 11:35 ` Hannes Reinecke
2024-12-17 18:29 ` [PATCH v4 2/9] sched/isolation: document HK_TYPE housekeeping option Daniel Wagner
2025-01-07 15:39 ` Waiman Long
2024-12-17 18:29 ` [PATCH v4 3/9] blk-mq: add number of queue calc helper Daniel Wagner
2025-01-08 7:04 ` Hannes Reinecke
2024-12-17 18:29 ` [PATCH v4 4/9] nvme-pci: use block layer helpers to calculate num of queues Daniel Wagner
2025-01-08 7:19 ` Hannes Reinecke
2024-12-17 18:29 ` [PATCH v4 5/9] scsi: " Daniel Wagner
2024-12-17 18:29 ` [PATCH v4 6/9] virtio: blk/scsi: " Daniel Wagner
2024-12-19 6:25 ` Christoph Hellwig
2024-12-19 8:31 ` Michael S. Tsirkin
2024-12-17 18:29 ` [PATCH v4 7/9] lib/group_cpus: honor housekeeping config when grouping CPUs Daniel Wagner
2024-12-17 18:29 ` [PATCH v4 8/9] blk-mq: use hk cpus only when isolcpus=managed_irq is enabled Daniel Wagner
2024-12-19 6:26 ` Christoph Hellwig
2024-12-19 9:20 ` Ming Lei
2024-12-19 15:38 ` Daniel Wagner
2024-12-20 8:54 ` Ming Lei
2025-01-10 9:21 ` Daniel Wagner
2025-01-11 3:31 ` Ming Lei
2025-01-13 13:19 ` Daniel Wagner [this message]
2024-12-17 18:29 ` [PATCH v4 9/9] blk-mq: issue warning when offlining hctx with online isolcpus Daniel Wagner
2024-12-19 6:28 ` Christoph Hellwig
2024-12-20 9:04 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4047eb51-ee6b-46ae-a67b-ce74c54c41e7@flourine.local \
--to=dwagner@suse.de \
--cc=GR-QLogic-Storage-Upstream@marvell.com \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=brookxu.cn@gmail.com \
--cc=chandrakanth.patil@broadcom.com \
--cc=costa.shul@redhat.com \
--cc=don.brace@microchip.com \
--cc=eperezma@redhat.com \
--cc=frederic@kernel.org \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=jasowang@redhat.com \
--cc=juri.lelli@redhat.com \
--cc=kashyap.desai@broadcom.com \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-scsi@vger.kernel.org \
--cc=llong@redhat.com \
--cc=martin.petersen@oracle.com \
--cc=megaraidlinux.pdl@broadcom.com \
--cc=mgorman@suse.de \
--cc=ming.lei@redhat.com \
--cc=mkoutny@suse.com \
--cc=mst@redhat.com \
--cc=njavali@marvell.com \
--cc=pbonzini@redhat.com \
--cc=sagi@grimberg.me \
--cc=sbalaraman@parallelwireless.com \
--cc=shivasharan.srikanteshwara@broadcom.com \
--cc=stefanha@redhat.com \
--cc=storagedev@microchip.com \
--cc=sumit.saxena@broadcom.com \
--cc=tglx@linutronix.de \
--cc=virtualization@lists.linux.dev \
--cc=vschneid@redhat.com \
--cc=wagi@kernel.org \
--cc=xuanzhuo@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®