From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753166AbdHXMwH (ORCPT ); Thu, 24 Aug 2017 08:52:07 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:55723 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753130AbdHXMwC (ORCPT ); Thu, 24 Aug 2017 08:52:02 -0400 Date: Thu, 24 Aug 2017 13:51:13 +0100 From: Roman Gushchin To: Michal Hocko CC: , Vladimir Davydov , Johannes Weiner , Tetsuo Handa , David Rientjes , Tejun Heo , , , , Subject: Re: [v6 3/4] mm, oom: introduce oom_priority for memory cgroups Message-ID: <20170824125113.GB15916@castle.DHCP.thefacebook.com> References: <20170823165201.24086-1-guro@fb.com> <20170823165201.24086-4-guro@fb.com> <20170824121054.GI5943@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20170824121054.GI5943@dhcp22.suse.cz> User-Agent: Mutt/1.8.3 (2017-05-23) X-Originating-IP: [2620:10d:c092:200::1:ec5a] X-ClientProxiedBy: AM4PR07CA0006.eurprd07.prod.outlook.com (2603:10a6:205:1::19) To SN2PR15MB1085.namprd15.prod.outlook.com (2603:10b6:804:22::7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: beef81f9-e9c2-4d9c-5a76-08d4eaeed70d X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:SN2PR15MB1085; X-Microsoft-Exchange-Diagnostics: 1;SN2PR15MB1085;3:DFMG5OldC4cu3VWyDBiWD+dJzhWdHnUcixHgKwf3Q+qx9yt0IXviJ6OLJbcsM7XlSbLjjQgqQ+YVowccBDG2U7tjCCXLXRroITwWNQuL6tG7YpmvpUxcYQSwnbjYUv9lLn6orTrvI97DeLxANF6AIP0qAmNUwE+My6yetmuO2V1AOGKPNAYT09AFHSpOvekBG52jGs/dNUueCq/+c3hQBP/NNrpcH+CvBVbguLqn2B/HKCbNCiVnAIsoN06RnhBc;25:0XUPVp9kEUZAOhhleiiT6WB92Juq8uzc0sr0Ke8K92tglfRJURKTfteK10Lf/SvegSadbmdbo/4+O/0d59C3iqumlJqXxriOCjgB1vAD1vuIFgFF7xTHwmz4g0YyDQdrI+V/R72ALTVgfZWdP0lgu9QKmcTQpt9j+HHR7ps5oOon9uVMscYf4mXTUNtTo7bB3qY9Umq857/Y4i/rqPTcL3owbCcEPoebpCyHCkIUT0qQ2GLCniODm3Zennu+rRaZsxQNfvkV17nMcjY64Kjqw6hbD+EyVQfY3Y8rtFfRTRil1D06rMm9x9mNhR50towQUTAp2znApOTc6SxLwo+a3g==;31:2VYSAC2EaeZnfwNGGnbXlH9EefbPAO75jVv7zeinlJixdsAuaDcO11vLYXxrY89sM6yB9o7LS66eU5/T1tI/Bl2u3kgRIZA//+McEOBK0/49BYkL93TlO1bv5fgLMcDW+Lxs6bv0Nd9KFoCGGoPbMIRrWRi9FzuL2dNhBYA3eIqhywX+b1r8ThdP3CxWMeu2TmCOwg1qPalwFVjMhkGzpZdREQ6mhYBpitNDDM6mtN8= X-MS-TrafficTypeDiagnostic: SN2PR15MB1085: X-Microsoft-Exchange-Diagnostics: 1;SN2PR15MB1085;20:5ZMP7xahAkJnhjlurgpiBIzApk7+xa4/5/tmq+QId/4jkJR298MjnEFew/tQzqjTDLqv5sqOjipg7ly2acapQSKaGy3ULdnmm9emnNH6NeLkGeimns7v47u92ExnVr/NnXjKz5JiSYIaSYCCMK5rCzjNW6y8geMlnm+9krV61XNQMzQ7++yRsbctixTd0eFd1k+tJs/tBNkMUiHfaevLM43RpcubUtcBXecS1AmjTapOyRO4UfR3mwCSwQW8hyc2kQ6gpNIGM4EeiLkxLMoMnxmdys4se+m78RCnEhZctWEc1HQd/Ckf2JH5/tQK2RydkopXUc2ee0M/kji1y11qWnY7GBuubpRaXVdQpG/iwOsEgv9mdPtFQ+J2p/5Cx9DH+/D58GRE4TxXXwzq23H3kx1Q4AA+YatCLy1NJYpWO9o2x9pqN2KDrokrrXy1dF2ArqKHdcxji1GUCpyr2X5XjIqKs3bxuRyA8ndEyT5cWEUqWropHj4Qgl+StFmTr1On;4:jF1S5WS5jrhqAhfmeQCyGwDr4FS3MHKbRhZdBZP/ecxZRLiEtgl8096aZ8T9dREb4Cnl1CcFDBbwo/C0tqJSmDVGLyi5MCkj7nUSInI/JIK3+nvjyaSBVycOnlZFykbo0XFCH1dU1eRHVa8KVsg2fOVaiOEzQIbS4QhkZloNBk6W5F1bDFx/h4PLaKq39TgmE86i9o+FD8X5GQgvJIlcbv31avivcezlYsKwWzasEm9tioGx7slXIQYH3A0UrSJGB2Ko0OGJCg9WQcUyqZyKLjTsxRlrUr4Ib2f/qrrebOM= X-Exchange-Antispam-Report-Test: UriScan:(788757137089); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6041248)(20161123562025)(20161123560025)(20161123558100)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:SN2PR15MB1085;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:SN2PR15MB1085; X-Forefront-PRVS: 04097B7F7F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(7370300001)(6009001)(199003)(57704003)(377424004)(24454002)(189002)(47776003)(6666003)(97736004)(23726003)(50466002)(83506001)(6116002)(5660300001)(2950100002)(2906002)(6916009)(7350300001)(8676002)(81166006)(6506006)(54906002)(1076002)(55016002)(4001350100001)(81156014)(86362001)(478600001)(6246003)(42186005)(105586002)(9686003)(101416001)(25786009)(229853002)(4326008)(7416002)(189998001)(33656002)(50986999)(53936002)(68736007)(54356999)(76176999)(305945005)(110136004)(106356001)(7736002)(18370500001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:SN2PR15MB1085;H:castle.DHCP.thefacebook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN2PR15MB1085;23:B36w7hmY4JYhZxIxlpp8E6MqMA01T2V3ptMGfz3Lj?= =?us-ascii?Q?NZt92yCLJKfGeJW9YvLFHbS7hVijXHLUqCKKDB5t+qSlwpcNqGmvLpZLCkHQ?= =?us-ascii?Q?jlCLooTMOt3wJHRBk3NQa8XwoUxCsrvngLNoyi/ss3rGN+JVyoMlGO+CO0Ri?= =?us-ascii?Q?bla/3gRHvSjSKW7cK9PR6mRi1HLCxRaaRMNBAlorjKfUgPwBYevHD+SQWcua?= =?us-ascii?Q?6RJnBYgmHZQsDc5XM0vUSbhTUEDlYh8vWL4ASEgA4NJez9r6HT8Wlidsh/aZ?= =?us-ascii?Q?k4HveLohrxE2tXcxs5dC6gARXxoGIjF1ZL9p+f04I6/F1QweMoZReF+YMvDh?= =?us-ascii?Q?j0XHsehL273+7DPieL2C957sMjgnRADsYH58zyb6Jr+R2dUsIjwt79xQc/sP?= =?us-ascii?Q?zHvhmI1/mNN91IcHmqzD1RK+ePQZPWQvpnWi3XT2/l7KUGltKQS+t4SqC1o3?= =?us-ascii?Q?Jdh9k1zOhtwC60hKSxbA7IQgzVU6U2WKWm1nLHjx/eTmJvJqhDzswJ9dtHf6?= =?us-ascii?Q?lfTq19LjHxenKAIsGnQG4GA5XspKQQOUeiX4Y7Jifl7Ma7jW9ICm86fU890r?= =?us-ascii?Q?DYExit0Pvk+fGbw3ge6WuBSTfdUcy4ELVWrvCOj8LBFPowo5YczfCvZ5ALFO?= =?us-ascii?Q?4LPDGTkNqvZM27J1FZTMdx1uUSVLctrhVmlbw9Z1+Y6d7GFna7P4VrEMt2uu?= =?us-ascii?Q?OcernKf3LN2/8BhfI6xGZ1b9itVJEExQzrsFKgMxKlkLIYbXgFpvW3dk950E?= =?us-ascii?Q?JBCAShTUoTtYJpiG+HO6930vVUoWhUGNgweDy5YDpq+Fy/+dZjsKjHlNZMXg?= =?us-ascii?Q?C6EzaXBYetjgZ2qoNHgnHPT6kYFVpx35gHChME5p7a94GNlGPpPDtclkSiOq?= =?us-ascii?Q?wJv7nUjHGfBsjbGL4oGGO4m6m/uFDbQ1NAo7GkKaGD3y9wY3JPO+uS+zfKyD?= =?us-ascii?Q?8XlE42h3sylxgnLoAI2LSLLseQDBN2hMJ60sASFfaj5YF2qIJZrghx3ftytv?= =?us-ascii?Q?LLBdqQgyz5z0fNaMG9EhKKnLNWlM0+qjsSeD1FAUDPLL7nxa7F/lmswjOndp?= =?us-ascii?Q?An8gbDayZfb8VWeUF2UFnpDdIGo2vdwUTPRKa7SL8lqW0xGQsl9rDPC9nfUa?= =?us-ascii?Q?Lhol+7uZ8VNKTULA2fiAjt1+e4s27M+YDsnr6SkG0ZUU1nrEHM75vKFY2XPJ?= =?us-ascii?Q?ujSbPWU8Vwx8s0NPJ9rlCnu06VbPtnb6V493Yvg0PD5GQK9vxpQaIhjrB3fC?= =?us-ascii?Q?DFt1Q7rORb/JkcJI30=3D?= X-Microsoft-Exchange-Diagnostics: 1;SN2PR15MB1085;6:mp+2yrspHRh63IIhNcpJukfKZGNmmUDAkRcG0GwdpRFNLfNQy0qxUFS33cWp4Sc/hdIUy0vuvRndWMmhmVTTAuwGVRpYm6F3afdWgt78vZfk6no5Si6aWtx6P/Jw30gxfGQ9z6RmVDbIORgQJrawgBwys2xeUa/DM002yWUhPKs5uJGDtlBb5prvjhB2A4EYG+A//XRwhdYy+qpn7PiMp+BZb4QW92jXDImskegUPNJLjeovb4NSYv/69tWteUrgSXwKKTO/u6ZjRDF2i9yOlqEV5HBfOuOtCgd9jBN6YMiA2Ws32yo7dvF2DemKk/D3mrAkAgeQ7mWxJwqbz5Q+Tw==;5:NM46S5Av20ASB8Ecdu6sSC9+iYOFIS7q/1wczBa7MNYJTWTmAONGYoiYkhAoLREt21m+pQoHriSdbIRWjl7kkwopvJRAQvLsG9w9tmPexwG1TjdiA2iAq1x1mBZKBGTw9934Tzk2dyG8rFg+h9WxGcfmGv6/yTA84hrbUBbYEaQ=;24:CMY/XANZCynv6jXfS45ScE5FoI/eLQTneMiq8NMDNO5EVe1orGJ3qwn6RXw03VJZ98WVwOU1FMU92imWMTVs2fHL1u3lRqx9CZNpS2zlvO0=;7:vYoMG8Odpg9lez2NYZLbZwp8ECxGQzflZ6hYGukO1iZautnfT8JZcNQLwbeazyePVQI1w8nSKsd2wYFeunzncVvCX3nZt1d4A4FHsZrPD8ZuGQMCdKPHfMa4T6Q+lf9BgAXGYd7MGgyRM2O1Ht/L3kVpLWQxdFqvKs7hJ347xYQVe42HU4GrYMlmfmOTUWH9kggV+CoCMI0Z0ezHkKuL8vgwKiQMZQAp3Ol9xV2dqV0= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN2PR15MB1085;20:/AnZ5DrTHAPaHIqRQwiULJ9Z3F6N0MIWECGemQxoFP/2PxmhOHM5gqIc78f8Ovh/N4lZ7+ecVS3zdd+gPVeoTijXHWgYgs/P417AUoS9M44d88OMJCOL5BLIEpkQtjdvVmogvNS0//ivJcslKAq9u0W9InM3fKJGiNQ6AuKZZXc= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2017 12:51:27.8110 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR15MB1085 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-08-24_04:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Aug 24, 2017 at 02:10:54PM +0200, Michal Hocko wrote: > On Wed 23-08-17 17:52:00, Roman Gushchin wrote: > > Introduce a per-memory-cgroup oom_priority setting: an integer number > > within the [-10000, 10000] range, which defines the order in which > > the OOM killer selects victim memory cgroups. > > Why do we need a range here? No specific reason, both [INT_MIN, INT_MAX] and [-10000, 10000] will work equally. We should be able to predefine an OOM killing order for any reasonable amount of cgroups. > > > OOM killer prefers memory cgroups with larger priority if they are > > populated with eligible tasks. > > So this is basically orthogonal to the score based selection and the > real size is only the tiebreaker for same priorities? Could you describe > the usecase? Becasuse to me this sounds like a separate oom killer > strategy. I can imagine somebody might be interested (e.g. always kill > the oldest memcgs...) but an explicit range wouldn't fly with such a > usecase very well. The usecase: you have a machine with several containerized workloads of different importance, and some system-level stuff, also in (memory) cgroups. In case of global memory shortage, some workloads should be killed in a first order, others should be killed only if there is no other option. Several workloads can have equal importance. Size-based tiebreaking is very useful to catch memory leakers amongst them. > > That brings me back to my original suggestion. Wouldn't a "register an > oom strategy" approach much better than blending things together and > then have to wrap heads around different combinations of tunables? Well, I believe that 90% of this patchset is still relevant; the only thing you might want to customize/replace size-based tiebreaking with something else (like timestamp-based tiebreaking, mentioned by David earlier). What about tunables, there are two, and they are completely orthogonal: 1) oom_priority allows to define an order, in which cgroups will be OOMed 2) oom_kill_all defines if all or just one task should be killed So, I don't think it's a too complex interface. Again, I'm not against oom strategy approach, it just looks as a much bigger project, and I do not see a big need. Do you have an example, which can't be effectively handled by an approach I'm suggesting? > > [...] > > @@ -2760,7 +2761,12 @@ static void select_victim_memcg(struct mem_cgroup *root, struct oom_control *oc) > > if (iter->oom_score == 0) > > continue; > > > > - if (iter->oom_score > score) { > > + if (iter->oom_priority > prio) { > > + memcg = iter; > > + prio = iter->oom_priority; > > + score = iter->oom_score; > > + } else if (iter->oom_priority == prio && > > + iter->oom_score > score) { > > memcg = iter; > > score = iter->oom_score; > > } > > Just a minor thing. Why do we even have to calculate oom_score when we > use it only as a tiebreaker? Right now it's necessary, because at the same time we do look for per-existing OOM victims. But if we can have a memcg-level counter for it, this can be optimized. Thanks!