From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F674C282C0 for ; Fri, 25 Jan 2019 19:48:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5B3AF218B0 for ; Fri, 25 Jan 2019 19:48:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MSrBMA+D" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726604AbfAYTsD (ORCPT ); Fri, 25 Jan 2019 14:48:03 -0500 Received: from mail-lj1-f193.google.com ([209.85.208.193]:42455 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725778AbfAYTsC (ORCPT ); Fri, 25 Jan 2019 14:48:02 -0500 Received: by mail-lj1-f193.google.com with SMTP id l15-v6so9352699lja.9; Fri, 25 Jan 2019 11:48:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=PaL/Nm/0v0Ytkgf6OpFItlBs3opiEJwSabaR9s9+cdA=; b=MSrBMA+DumtrsUm/pQTTTpC1N1PaTbo58IFFJQCNsIHblpupPe11ThukCc4LHwPB9x hFQR6vD0Q0n/vEiBaxxBdXR1sMO2KCZh4vjS8ItJpSEjBWVtmwGkdK3XX+416m2xZaZJ YNcV2OkvHcfrQMGTMuf4962U7Yp+Xbs4WuMDmodHT0VmmxfIdWszOrWEMVztr4Plf75M MN8DxaivndxdcG1Li3uzBJnkOrDRrBaHjM/SeoUWHVijxrR7VC2y9xp2PcRV+wuxAgO3 oD2FaW7qhnBfgegGRYYQ1hp72rSAJ/mt+dTMf9Ve/TCIVJ+hJSqBz+nVY1wFXyrwJR5X JnFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=PaL/Nm/0v0Ytkgf6OpFItlBs3opiEJwSabaR9s9+cdA=; b=Vi2nvKIuNL17mfwzNvOr9XivY21RVogK7mM339r28dHef2jsHNh5NhBLs/LcqLTjC9 Clu8g69V3uktvf+0K8l2+uK8eG4hbXhX3JDu4C2N8YFswEGUCPUVNDEgkJxlJTuY8+Px fpNEjv6Gi1H9plK5fjEvLMwMtAS2DD8A8jyrywNcD/SsPblgFLn3fbSE8Mi1Qe+mh23S 9BYgTxhNeoZ82Gbdz5Y1pIIhIeAx6r9DBUk6mO9xCFUZQZavL3BvrnozNhcFZqn4Z3b2 iYCuCW2wR7mCZg2sPp2OUvq6P6QZHetCoVqRXCxkONjIfxhCijduua7z4q+xgIOyEM7S 8rOA== X-Gm-Message-State: AJcUukdHkc1vl1LA/k6lWrxAoiCKegwlEuduMrPWY5Y3qYIj9Cyk12VF ne12zMFPWf0Sk2EqGZGM/Qs99oeeQU4= X-Google-Smtp-Source: ALg8bN5E91E+HlT5VVEGWOC6UtfvzDJd7FJfIowJ003CcX2TdiaNY/FmtCFoKeLTC4weO5Mr+QAISg== X-Received: by 2002:a2e:4601:: with SMTP id t1-v6mr10068707lja.111.1548445679752; Fri, 25 Jan 2019 11:47:59 -0800 (PST) Received: from amb.local (31-179-17-47.dynamic.chello.pl. [31.179.17.47]) by smtp.gmail.com with ESMTPSA id c15-v6sm1683785ljf.76.2019.01.25.11.47.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 25 Jan 2019 11:47:59 -0800 (PST) Subject: Re: pids.current with invalid value for hours [5.0.0 rc3 git] To: Tejun Heo Cc: cgroups@vger.kernel.org, Aleksa Sarai , Jay Kamat , Roman Gushchin , Michal Hocko , Johannes Weiner , linux-kernel@vger.kernel.org References: <20190117122535.njcbqhlmzozdkncw@mikami> <1d36b181-cbaf-6694-1a31-2f7f55d15675@gmail.com> <96ef6615-a5df-30af-b4dc-417a18ca63f1@gmail.com> <1cdbef13-564d-61a6-95f4-579d2cad243d@gmail.com> <20190125163731.GJ50184@devbig004.ftw2.facebook.com> From: =?UTF-8?Q?Arkadiusz_Mi=c5=9bkiewicz?= Message-ID: Date: Fri, 25 Jan 2019 20:47:57 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <20190125163731.GJ50184@devbig004.ftw2.facebook.com> Content-Type: text/plain; charset=utf-8 Content-Language: pl Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 25/01/2019 17:37, Tejun Heo wrote: > On Fri, Jan 25, 2019 at 08:52:11AM +0100, Arkadiusz Miśkiewicz wrote: >> On 24/01/2019 12:21, Arkadiusz Miśkiewicz wrote: >>> On 17/01/2019 14:17, Arkadiusz Miśkiewicz wrote: >>>> On 17/01/2019 13:25, Aleksa Sarai wrote: >>>>> On 2019-01-17, Arkadiusz Miśkiewicz wrote: >>>>>> Using kernel 4.19.13. >>>>>> >>>>>> For one cgroup I noticed weird behaviour: >>>>>> >>>>>> # cat pids.current >>>>>> 60 >>>>>> # cat cgroup.procs >>>>>> # >>>>> >>>>> Are there any zombies in the cgroup? pids.current is linked up directly >>>>> to __put_task_struct (so exit(2) won't decrease it, only the task_struct >>>>> actually being freed will decrease it). >>>>> >>>> >>>> There are no zombie processes. >>>> >>>> In mean time the problem shows on multiple servers and so far saw it >>>> only in cgroups that were OOMed. >>>> >>>> What has changed on these servers (yesterday) is turning on >>>> memory.oom.group=1 for all cgroups and changing memory.high from 1G to >>>> "max" (leaving memory.max=2G limit only). >>>> >>>> Previously there was no such problem. >>>> >>> >>> I'm attaching reproducer. This time tried on different distribution >>> kernel (arch linux). >>> >>> After 60s pids.current still shows 37 processes even if there are no >>> processes running (according to ps aux). >> >> >> The same test on 5.0.0-rc3-00104-gc04e2a780caf and it's easy to >> reproduce bug. No processes in cgroup but pids.current reports 91. > > Can you please see whether the problem can be reproduced on the > current linux-next? > > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git I can reproduce on next (5.0.0-rc3-next-20190125), too: [root@xps test]# python3 cg.py Created cgroup: /sys/fs/cgroup/test_2501 Start: pids.current: 0 Start: cgroup.procs: 0: pids.current: 65 0: cgroup.procs: 1: pids.current: 44 1: cgroup.procs: 2: pids.current: 44 2: cgroup.procs: 3: pids.current: 44 3: cgroup.procs: 4: pids.current: 44 4: cgroup.procs: 5: pids.current: 44 5: cgroup.procs: 6: pids.current: 44 6: cgroup.procs: 7: pids.current: 44 7: cgroup.procs: 8: pids.current: 44 8: cgroup.procs: 9: pids.current: 44 9: cgroup.procs: 10: pids.current: 44 10: cgroup.procs: 11: pids.current: 44 11: cgroup.procs: [root@xps test]# uname -a Linux xps 5.0.0-rc3-next-20190125 #2 SMP PREEMPT Fri Jan 25 19:11:40 CET 2019 x86_64 Intel(R)_Core(TM)_i9-8950HK_CPU_@_2.90GHz PLD Linux [root@xps test]# mount |grep cgroup2 cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime) I'm booting kernel with cgroup_no_v1=all -- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )