From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030481AbcIWNYG (ORCPT ); Fri, 23 Sep 2016 09:24:06 -0400 Received: from mail.virtall.com ([178.63.195.102]:60259 "EHLO mail.virtall.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965663AbcIWNXY (ORCPT ); Fri, 23 Sep 2016 09:23:24 -0400 DomainKey-Signature: a=rsa-sha1; c=nofws; d=wpkg.org; h=mime-version :content-type:content-transfer-encoding:date:from:to:subject :in-reply-to:references:message-id; q=dns; s=default; b=CCMQHoaa eqb2H4T87ttCBwpMR5JzChklkCqhr2IgH7/327kokbba2glENxIG/y82jnih18uX LQdwmIHbcdw4PLPOzwn37lQRid5r22qufAQ3MPTzSxENC5sQC1RzblQejz1iZGZ/ vDcStk9Y1d+y4C69dV8AwL7HQ2p21arNslo= MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 23 Sep 2016 22:23:17 +0900 From: Tomasz Chmielewski To: LKML Subject: Re: thousands of kworker processes with 4.7.x and 4.8-rc* In-Reply-To: <35b2acf87bfeaf1d04f2cf3d10b99b6e@admin.virtall.com> References: <35b2acf87bfeaf1d04f2cf3d10b99b6e@admin.virtall.com> Message-ID: <20acd5801291dff03ed56b63919b9249@admin.virtall.com> User-Agent: Roundcube Webmail/1.2.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016-09-19 16:08, Tomasz Chmielewski wrote: > On several servers running 4.7.x and 4.8-rc6/7 kernels I'm seeing > thousands of kworker processes. > # ps auxf|grep -c kworker > 2104 > Load average goes into hundreds on a pretty much idle server (biggest > CPU and RAM consumers are probably SSHD with one user logged in and > rsyslog writing ~1 line per minute): > # uptime > 06:58:56 up 26 min, 1 user, load average: 146.11, 215.46, 105.70 > # uptime > 06:59:48 up 26 min, 1 user, load average: 305.20, 240.84, 120.25 > Sometimes seeing lots of them in "D" state: > root 19474 0.0 0.0 0 0 ? D 06:54 0:00 \_ > [kworker/0:208] > root 19475 0.0 0.0 0 0 ? D 06:54 0:00 \_ > [kworker/0:209] I did some experiments to see when the problem first appeared. Thousands of kworker processes start to show up in 4.7.0-rc5. kernel version | kworker count after boot ------------------------------------------- 4.6.3 37 4.6.4 47 4.6.5 46 4.6.6 49 4.6.7 49 4.7.0-rc1 46 4.7.0-rc2 49 4.7.0-rc3 45 4.7.0-rc4 47 4.7.0-rc5 1592 4.7.0-rc6 1714 4.7.0-rc7 1955 4.7.0 2088 4.7.1 1222 4.7.2 1699 4.7.3 1446 4.7.4 1781 4.8-rc1 (not tested) 4.8-rc2 2012 4.8-rc3 1696 4.8-rc4 1210 4.8-rc5 1890 4.8-rc6 1657 4.8-rc7 1647 Tomasz Chmielewski https://lxadm.com