From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Google-Smtp-Source: AH8x2246MuR2MG8wwFRPEd94ZX0E1aTBL6BBNHT4FytC91i/GE1zoKnVMgP4y50bM68v78gJOVrB ARC-Seal: i=1; a=rsa-sha256; t=1516668085; cv=none; d=google.com; s=arc-20160816; b=vO9UFg8B5U5kAvhuB5IlW6gpwrbdOPPRaZZR9K73B4JgAkhYsmI2nFrNCyXgy5l+RL mAlhijZkoUzA5SSGxHY6hsovrV76MWXZoYXiPOar7Sinx5oE0ZhJILjeP11yENFhR47g jQoOJ5sirFFODsmWJfuu6W5Z8cjIPp83O4yoEUUGx2+M7Wj+0Wi1SI9I7ZjqEForwjnz ArO532s/JSxR56mUei45/j12Ve0eUjZbgeN5iFwH8ivGchJV+TOhu9QxXr5TYcuBMUuI GRa2bJafgflNJ5pxnSz3B3ADdBG2VIp05qEDJcJe0qxjcOKLMystD7ZjR2z5FebadeCR g2VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:dkim-signature :arc-authentication-results; bh=/idB2iR57NulghH0Y0xfO1opB4aT8xmZSnCS9+xPDqg=; b=jc/80rJuG2MvjQCm+BdcMuD9xeXTND8qDv73TwoHCOo5BwUHAuJX6Epe6Py34wmMfo pzidYR58dv992F9lFCewffxbqk+kNcDvVKxJ/OknKdo8w3OrCgL6D68I/d3ixbIZsxyt fhMuEp4MnYjvCZMH97hWsmuSpjo4Ua+/se4u6rWxqt2xoYhGKED1glwQGzvGSb1GoSCC uK+zMbfrlmyz9qe1palNhO8ujeLdVRzvH5docamocY8KH4rEJxDHAqScIjEAMTpn0Ocd o7QUfYalnwso5CnzGdOTC+PqA3ts4+yeMKFBPdX8+yrVsah5JwS1G8XgYzWz1Ir1hEmV 934g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2017-10-26 header.b=Z1dKL8sE; spf=pass (google.com: domain of henry.willard@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=henry.willard@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2017-10-26 header.b=Z1dKL8sE; spf=pass (google.com: domain of henry.willard@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=henry.willard@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: [PATCH] mm: numa: Do not trap faults on shared data section pages. From: Henry Willard In-Reply-To: Date: Mon, 22 Jan 2018 16:41:11 -0800 Cc: Mel Gorman , akpm@linux-foundation.org, kstewart@linuxfoundation.org, zi.yan@cs.rutgers.edu, pombredanne@nexb.com, aarcange@redhat.com, gregkh@linuxfoundation.org, aneesh.kumar@linux.vnet.ibm.com, kirill.shutemov@linux.intel.com, jglisse@redhat.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <1516130924-3545-1-git-send-email-henry.willard@oracle.com> <1516130924-3545-2-git-send-email-henry.willard@oracle.com> <20180116212614.gudglzw7kwzd3get@suse.de> <2BEFC6DE-7A47-4CB9-AAE5-CEF70453B46F@oracle.com> To: Christopher Lameter X-Mailer: Apple Mail (2.3445.5.20) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8782 signatures=668655 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=786 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1801230004 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1589778616343155985?= X-GMAIL-MSGID: =?utf-8?q?1590341755138306721?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: > On Jan 19, 2018, at 6:12 PM, Christopher Lameter wrote: >=20 > On Thu, 18 Jan 2018, Henry Willard wrote: >=20 >> If MPOL_MF_LAZY were allowed and specified things would not work >> correctly. change_pte_range() is unaware of and can=E2=80=99t honor = the >> difference between MPOL_MF_MOVE_ALL and MPOL_MF_MOVE. >=20 > Not sure how that relates to what I said earlier... Sorry. Only that CAP_SYS_NICE is not relevant to this patch. >=20 >>=20 >> For the case of auto numa balancing, it may be undesirable for shared >> pages to be migrated whether they are also copy-on-write or not. The >> copy-on-write test was added to restrict the effect of the patch to = the >> specific situation we observed. Perhaps I should remove it, I don=E2=80= =99t >> understand why it would be desirable to modify the behavior via = sysfs. >=20 > I think the most common case of shared pages occurs for pages that = contain > code. In that case a page may be mapped into hundreds if not thousands = of > processes. In particular that is often the case for basic system = libraries > like the c library which may actually be mapped into every binary that = is > running. That is true, but auto numa balancing skips these and similar pages = before it calls change_prot_numa(). They don=E2=80=99t even have to be = actually shared to be skipped.=20 >=20 > It is very difficult and expensive to unmap these pages from all the > processes in order to migrate them. So some sort of limit would be = useful > to avoid unnecessary migration attempts. One example would be to = forbid > migrating pages that are mapped in more than 5 processes. Some sysctl = know > would be useful here to set the boundary. >=20 > Your patch addresses a special case here by forbidding migration of = any > page mapped by more than a single process (mapcount !=3D1). The current patch skips pages that are in copy-on-write VMAs and still = shared. These include pages in the program=E2=80=99s data segment that = are writable. but have not been written to. Once the pages are modified = they are no longer shared and can be migrated. The problem is that in = some cases, the pages are never modified and remain shared. Prior to commit 4b10e7d562c90d0a72f324832c26653947a07381, = change_prot_numa() called change_prot_numa_range(), which tested for = (page_mapcount(page) !=3D 1) and bailed out for any shared pages. This = patch is more selective. A simple test for shared or not seems to be = common. >=20 > That would mean f.e. that the complete migration of a set of processes > that rely on sharing data via a memory segment is impossible because = those > shared pages can never be moved. >=20 > By setting the limit higher that migration would still be possible. >=20 > Maybe we can set that limit by default at 5 and allow a higher setting > if users have applications that require a higher mapcoun? F.e. a > common construct is a shepherd task and N worker threads. If those > tasks each have their own address space and only communicate via > a shared data segment then one may want to set the limit higher than N > in order to allow the migration of the group of processes. This example would be unaffected by this patch, because the patch does = not affect explicitly shared memory. A process with the necessary = capabilities is still able to migrate all pages. Henry