From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753535AbcD0Q2N (ORCPT ); Wed, 27 Apr 2016 12:28:13 -0400 Received: from mail-yw0-f194.google.com ([209.85.161.194]:36327 "EHLO mail-yw0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752509AbcD0Q2K (ORCPT ); Wed, 27 Apr 2016 12:28:10 -0400 Date: Wed, 27 Apr 2016 12:11:46 -0400 From: Tejun Heo To: Linus Torvalds Cc: linux-kernel@vger.kernel.org, Roman Pen , peter@hurleysoftware.com, Jens Axboe Subject: [GIT PULL] workqueue fixes for v4.6-rc5 Message-ID: <20160427161146.GO7822@mtj.duckdns.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, Linus. So, it turns out we had a silly bug in the most fundamental part of workqueue for a very long time. AFAICS, this dates back to pre-git era and has quite likely been there from the time workqueue was first introduced. A work item uses its PENDING bit to synchronize multiple queuers. Anyone who wins the PENDING bit owns the pending state of the work item. Whether a queuer wins or loses the race, one thing should be guaranteed - there will soon be at least one execution of the work item - where "after" means that the execution instance would be able to see all the changes that the queuer has made prior to the queueing attempt. Unfortunately, we were missing a smp_rmb() after clearing PENDING for execution, so nothing guaranteed visibility of the changes that a queueing loser has made, which manifested as a reproducible blk-mq stall. Lots of kudos to Roman for debugging the problem. The patch for -stable is the minimal one. For v3.7, Peter is working on a patch to make the code path slightly more efficient and less fragile. Thanks. The following changes since commit f7813ad5cbfd1fab2899914281b72a1ba0805c80: Merge tag 'for-linus-4.6' of git://git.code.sf.net/p/openipmi/linux-ipmi (2016-03-18 10:21:04 -0700) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git for-4.6-fixes for you to fetch changes up to 346c09f80459a3ad97df1816d6d606169a51001a: workqueue: fix ghost PENDING flag while doing MQ IO (2016-04-26 11:23:22 -0400) ---------------------------------------------------------------- Roman Pen (1): workqueue: fix ghost PENDING flag while doing MQ IO kernel/workqueue.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) -- tejun