From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756194AbcK2Gv5 (ORCPT ); Tue, 29 Nov 2016 01:51:57 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:59685 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754329AbcK2Gvr (ORCPT ); Tue, 29 Nov 2016 01:51:47 -0500 Subject: Re: [RFC 4/4] mm: Ignore cpuset enforcement when allocation flag has __GFP_THISNODE To: Dave Hansen , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <1479824388-30446-1-git-send-email-khandual@linux.vnet.ibm.com> <1479824388-30446-5-git-send-email-khandual@linux.vnet.ibm.com> <8216916c-c3f3-bad9-33cb-b0da2508f3d0@intel.com> Cc: mhocko@suse.com, vbabka@suse.cz, mgorman@suse.de, minchan@kernel.org, aneesh.kumar@linux.vnet.ibm.com, bsingharora@gmail.com, srikar@linux.vnet.ibm.com, haren@linux.vnet.ibm.com, jglisse@redhat.com From: Anshuman Khandual Date: Tue, 29 Nov 2016 12:21:28 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <8216916c-c3f3-bad9-33cb-b0da2508f3d0@intel.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16112906-0012-0000-0000-000001F119D9 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16112906-0013-0000-0000-000006897175 Message-Id: <583D2570.6070109@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-11-29_01:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1611290115 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/29/2016 02:42 AM, Dave Hansen wrote: > On 11/22/2016 06:19 AM, Anshuman Khandual wrote: >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -3715,7 +3715,7 @@ struct page * >> .migratetype = gfpflags_to_migratetype(gfp_mask), >> }; >> >> - if (cpusets_enabled()) { >> + if (cpusets_enabled() && !(alloc_mask & __GFP_THISNODE)) { >> alloc_mask |= __GFP_HARDWALL; >> alloc_flags |= ALLOC_CPUSET; >> if (!ac.nodemask) > > This means now that any __GFP_THISNODE allocation can "escape" the > cpuset. That seems like a pretty major change to how cpusets works. Do > we know that *ALL* __GFP_THISNODE allocations are truly lacking in a > cpuset context that can be enforced? Right, I know its a very blunt change. With the cpuset based isolation of coherent device node for the user space tasks leads to a side effect that a driver or even kernel cannot allocate memory from the coherent device node in the task's own context (ioctl() calls or similar). For non task context allocation (work queues, interrupts, anything async etc) this problem can be fixed by modifying kernel thread's task->mems _allowed to include all nodes of the system including the coherent device nodes. Though I have not figured out the details yet. Whats your thoughts on this ? What we are looking for is a explicit and definite way of allocating from the coherent device node inside the kernel.