From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7EA0D2EF652 for ; Mon, 20 Jul 2026 15:26:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784561165; cv=none; b=EiE+MWZQCEm4fQIfs5zWPJQbSns2ReUh7oxzHBroNTOuqXFILKaqAUhgaAPFOrvAaLZR4cPCF0gh4SXjWzX4hmJoDsEXcfmzWQmKtrGJs9vbePt5xkguge0gKqeAtdaXZbGrPymECn4gozVKOB5Qdd86Jm+1bpqx+byvgYjjs14= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784561165; c=relaxed/simple; bh=ZfWqs8hv0qfWZDdh27/3z175BvCuBHT+R+8A/Ye4FKM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OZ+fg0fsb8haCBbKPNr1JMorGYelVRlm0tqWwhEKrmeCEK9OKnrOinzsZU5tEoKFUNKszXCtinwhsPQKXFWgnEpUG1Rlinv6/722B/7f2PGl0+5AMY86I4lX+/dot3TWJcl0bUc3CU+muzJ21v2vpJ/x0CyuHxoZmWEsOQwX3lU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OEFVkMmL; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OEFVkMmL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 826831F00A3A; Mon, 20 Jul 2026 15:26:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784561164; bh=h47FluPB0Lx0sJMaFu92NXe4FEFI2mlygxYVFCWL/Rk=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=OEFVkMmL6h8M4mk+Idh+5zhL/PDrcgvHiEgwCwiDtetRFpYnjYpVotLg1jpFp0J9V zo+iF9k1mRpZHOg0negcrk8gowQnjfWsg4a4WL2nRezFfBQow2WiJ00eTHSG7XPGCt qLHsT2sUBOMWNuX+Km8Y7PMPPlzpZYi4MjeBXFFzLxcNG1JQtG/XwwPQu/y9flReQ/ YMPlFF7uxAwku9vaBPNELvqXbLjv4zlIX/0LUec+Wa/qAXi0G6eDGBdpbfqm/Go4pM 5lJwLyP0XcmqNGuyP+fG2J6/1Sw29oTDPR6672ZZEaowWMjkaB7BazTXs+msIKrH/R nAiLlugcJ398A== Date: Mon, 20 Jul 2026 09:25:59 -0600 From: Keith Busch To: Christoph Hellwig Cc: Jun Zeng , axboe@kernel.dk, sagi@grimberg.me, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, andriy.shevchenko@intel.com, gang.cao@intel.com, jun.i.jin@intel.com, yong.hu@intel.com, Liang Fang , Thomas Gleixner Subject: Re: [PATCH v1 1/1] nvme-pci: adaptive interrupt coalescing Message-ID: References: <20260715075703.15578-1-jun1.zeng@intel.com> <20260715075703.15578-2-jun1.zeng@intel.com> <20260720150539.GB17986@lst.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260720150539.GB17986@lst.de> On Mon, Jul 20, 2026 at 05:05:39PM +0200, Christoph Hellwig wrote: > On Thu, Jul 16, 2026 at 02:52:41PM -0600, Keith Busch wrote: > > On Wed, Jul 15, 2026 at 03:57:03PM +0800, Jun Zeng wrote: > > > Add adaptive interrupt coalescing to improve IOPS for high-throughput > > > workloads. Monitor IO pressure periodically and enable/disable coalescing > > > automatically based on IOPS, queue depth, and inflight IO thresholds. > > > This feature is enabled by default, can be controlled through debugfs > > > variable during runtime. > > > > Can't you do all this from user space? You'd maybe need the blk-mq > > debugfs to get access to the individual hctx dispatch numbers, but maybe > > that's okay? > > Having these kinds of interfaces where we rely on a userspace daemon > to be in sync with the kernel implementation are a mess, and really > lock us out of future improvements in this area. We can't stop someone from doing this today, though. > > The atomic_long_inc where you placed it is especially harmful to polled > > queues. > > We really should be able to do this without any counters, as blk-mq keeps > more than enough statistics. And of course for polled queues nothing > related to interrupt coalescing is actually need. > > And we should take a page from the networking playbook and look into > adaptively switching to polling under high load instead of just > mitigating interrupts. An important part of that is to move the CQ > reaping from irq context to thread context. Totally. I had a proposal from I think 7 years ago to converge nvme's threaded IRQ handling: reap the first X entries from hard-irq context, and if there's more, return IRQ_WAKE_THREAD to poll the rest and future completions with that queue's interrupt masked (this is one scenario where NVMe's MSI masking is better than MSI-x, but it's not a big deal). I can't find the patch though, so I guess it was in the gap that infradead lost. Anyway, I think combine that concept with the isolcpus proposal and you can have full control over which CPUs can dispatch new IO without being interrupted from CPUs reaping completions. Then we shouldn't need the spec's coalescing feature. > But modulo all these caveats adaptive interrupt coalescing / > moderation is something we absolute need right now and even more so > going foward. > > Note that a few month ago there also was as an interesting patchset > that does global interrupt moderation at the IRQ controller level. > I don't remember what happened to it and can't find a link to it, > but adding the interrupt maintainer in case he remembers.