From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753373AbdBHQqR (ORCPT ); Wed, 8 Feb 2017 11:46:17 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:51290 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751060AbdBHQqO (ORCPT ); Wed, 8 Feb 2017 11:46:14 -0500 Subject: Re: [RFC V2 12/12] mm: Tag VMA with VM_CDM flag explicitly during mbind(MPOL_BIND) To: Dave Hansen , Anshuman Khandual , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20170130033602.12275-1-khandual@linux.vnet.ibm.com> <20170130033602.12275-13-khandual@linux.vnet.ibm.com> <26a17cd1-dd50-43b9-03b1-dd967466a273@intel.com> Cc: mhocko@suse.com, vbabka@suse.cz, mgorman@suse.de, minchan@kernel.org, aneesh.kumar@linux.vnet.ibm.com, bsingharora@gmail.com, srikar@linux.vnet.ibm.com, haren@linux.vnet.ibm.com, jglisse@redhat.com, dan.j.williams@intel.com From: Anshuman Khandual Date: Wed, 8 Feb 2017 19:43:54 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17020814-7323-0000-0000-000000BFF9BD X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17020814-7324-0000-0000-00000261181C Message-Id: <941c2b32-7bd8-56e7-a8d5-c103cab121d1@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-02-08_09:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1702080141 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/07/2017 11:37 PM, Dave Hansen wrote: >> On 01/30/2017 11:24 PM, Dave Hansen wrote: >>> On 01/29/2017 07:35 PM, Anshuman Khandual wrote: >>>> + if ((new_pol->mode == MPOL_BIND) >>>> + && nodemask_has_cdm(new_pol->v.nodes)) >>>> + set_vm_cdm(vma); >>> So, if you did: >>> >>> mbind(addr, PAGE_SIZE, MPOL_BIND, all_nodes, ...); >>> mbind(addr, PAGE_SIZE, MPOL_BIND, one_non_cdm_node, ...); >>> >>> You end up with a VMA that can never have KSM done on it, etc... Even >>> though there's no good reason for it. I guess /proc/$pid/smaps might be >>> able to help us figure out what was going on here, but that still seems >>> like an awful lot of damage. >> Agreed, this VMA should not remain tagged after the second call. It does >> not make sense. For this kind of scenarios we can re-evaluate the VMA >> tag every time the nodemask change is attempted. But if we are looking for >> some runtime re-evaluation then we need to steal some cycles are during >> general VMA processing opportunity points like merging and split to do >> the necessary re-evaluation. Should do we do these kind two kinds of >> re-evaluation to be more optimal ? > I'm still unconvinced that you *need* detection like this. Scanning big > VMAs is going to be really painful. > > I thought I asked before but I can't find it in this thread. But, we > have explicit interfaces for disabling KSM and khugepaged. Why do we > need implicit ones like this in addition to those? Missed the discussion we had on this last time around I think. My bad, sorry about that. IIUC we can disable KSM through madvise() call, in fact I guess its disabled by default and need to be enabled. We can just have a similar interface to disable auto NUMA for a specific VMA or we can handle it page by page basis with something like this. diff --git a/mm/memory.c b/mm/memory.c index 1099d35..101dfd9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3518,6 +3518,9 @@ static int do_numa_page(struct vm_fault *vmf) goto out; } + if (is_cdm_node(page_to_nid(page))) + goto out; + /* Migrate to the requested node */ migrated = migrate_misplaced_page(page, vma, target_nid); if (migrated) { I am still looking into these aspects. BTW have posted the minimum set of CDM patches which defines and isolates CDM node.