From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752035AbbIONYm (ORCPT ); Tue, 15 Sep 2015 09:24:42 -0400 Received: from mout.kundenserver.de ([212.227.17.10]:56931 "EHLO mout.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751536AbbIONYl convert rfc822-to-8bit (ORCPT ); Tue, 15 Sep 2015 09:24:41 -0400 From: Arnd Bergmann To: linux-arm-kernel@lists.infradead.org Cc: David Vrabel , wei.liu2@citrix.com, ian.campbell@citrix.com, stefano.stabellini@eu.citrix.com, linux-kernel@vger.kernel.org, Julien Grall , xen-devel@lists.xenproject.org, boris.ostrovsky@oracle.com, Roger Pau =?ISO-8859-1?Q?Monn=E9?= Subject: Re: [Xen-devel] [PATCH v4 00/20] xen/arm64: Add support for 64KB page in Linux Date: Tue, 15 Sep 2015 15:24:06 +0200 Message-ID: <1539842.T2XY1FNaDl@wuerfel> User-Agent: KMail/4.11.5 (Linux/3.16.0-10-generic; KDE/4.11.5; x86_64; ; ) In-Reply-To: <55F819A1.8070609@citrix.com> References: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> <2415578.qstFX0Ux2G@wuerfel> <55F819A1.8070609@citrix.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8BIT Content-Type: text/plain; charset="iso-8859-1" X-Provags-ID: V03:K0:NDivzGWQafJM+N9IUgs/BAa9CbQvLGRfTKdnjN6klGjUVEJ+4Eu pgxkAVq8NrW6sN5A+UmxcH8IGYju5H04Y6wEONdAhftdNuonVveqBZuuhoMbFceCMZNdhNv WApIrFA4te/0PRxQBRd8kwwJ0c2Vblad33BeFgEnEQesJNh7w9bj2Eq0ilB6OvIdZR80M5Z vxoRIBpwACE8juQicWA8w== X-UI-Out-Filterresults: notjunk:1;V01:K0:bnc1qSJajZs=:VZlC1wkqEfP93lBnHHl5dc KhCKDDZ23yopR6CieGG+SavM9HUmlrDiZwW/MofJYqu06awYyqWNIfVnN/GawfvRvPHnBaSwB ogSa5z5wU4ZdWGb1u0sD1CwHFusUqILWM5R0nfq8XOw4gqQKNk9JPCczYArIfE9OWBGsJFB8j YlmAxzeIQRmoBpUIEg80dZtn5tD1sFtoJ1OuXSDpnAtfl7buEp7E/0P4tcyWrrOSKTr/xlai8 j8u5KZk3Efd2C6XwaYu6+7b7yc3dkdiosrl3qFf4qKUYB2k04Zo6WX1NsD+gzD0OLeKH+OLgZ bfeP09WncQ/DwZlvT4yyrafTnOxiO9Tokkt8NPhM9dlvjrVEK3TJWhW2nVhIQUopNG0pV+g1p 8E7uvc/0a+Ip2p/jLb8xvSAQVJHIth8xMCCqPcezeZZ6HQQFr10GQ9bRvnPVaS87z3Kl2qB2r yvg9or68Er+KAN8gKpf/RLfO0YBHucjJhmzYXOZKk5kxv79QV5V7Y8QrQnIPmecaKAovRBnFJ v0b7WjfwvgXRQR9V9hkYX8yu2wen0H4WLpyDOFXEjmHGr2Y0NzUYETzM+0oMLR2+Qjo+gqUjL hW+zwoA3NTBlVX9JFNnfDb5Iv/yP18w3aXlnlVXYHaATJcYLZvW8C5cgtWjghYbOEN00U2lsy 6KCIViOnX7fX2ZA5lNuOsflA0A/XT6CrLVBtQ2aRXGZIzKgdKtUron80h0scrupfxaBZnsaQG 2w92awzGeP0THllu Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tuesday 15 September 2015 14:14:09 David Vrabel wrote: > On 14/09/15 12:32, Arnd Bergmann wrote: > > On Monday 14 September 2015 13:04:59 Roger Pau Monné wrote: > >>> TBH, I'm expecting a small impact to the performance. It would be hard > >>> to get the exactly the same performance as today if we keep the helpers > >>> to avoid the backend dealing himself with the splitting and page > >>> granularity. > >>> > >>> Although, if the performance impact is not acceptable, it may be > >>> possible to optimize gnttab_foreach_grant_in_range by moving the > >>> function inline. The current way to the loop is the fastest I've found > >>> (I've wrote a small program to test different way) and we will need it > >>> when different of size will be supported. > >> > >> I don't expect the performance to drop massively with this patches > >> applied, but it would be good to al least have an idea of the impact. > > > > Note that using 64kb pages in Linux tends to destroy performance > > in Linux in any case, as the memory consumption for most workloads > > explodes. In a virtualized environment you already tend to be > > memory constrained, so any measurement should take that into account > > and put the extra overhead into perspective to the massive overhead > > of running 64kb pages when RAM is tight. > > If this is the case, why are some distros using 64 KiB pages then? I believe that IBM started this on PowerPC64 when they found that it performs better in microbenchmarks and for certain database workloads when you do have enough memory and do not work with small files or many processes. I also believe that the AIX memory management code is designed differently enough that it suffers less from large pages (or suffers more from small pages), so the hardware designers pushed Linux to do the same as AIX. Why you'd do it on ARM64 I have no idea at all, other than copying what IBM did on PowerPC. If your TLB is too small, hardware designers may of course try to push the OS to using pages as large as possible for the low-level benchmarks to suck less, but in general, you'd be better off with transparent hugepages and 4kb or 8kb pages for real workloads. Arnd