From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15BCDEB64DD for ; Fri, 28 Jul 2023 22:14:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232107AbjG1WOe (ORCPT ); Fri, 28 Jul 2023 18:14:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229825AbjG1WOb (ORCPT ); Fri, 28 Jul 2023 18:14:31 -0400 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A2AC448D for ; Fri, 28 Jul 2023 15:14:30 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-687087d8ddaso1574760b3a.1 for ; Fri, 28 Jul 2023 15:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1690582469; x=1691187269; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=j23ImFvrdRCadrPSwwknJnJ5y4yBBTyt7l0nVVVM2Yc=; b=ZtiNC4ej7OZeywn8wHEAqgDTQCL96l3JSg/+ffzCXaHNuNTx//E5tRxv+/5LyUCmzy wJU7VJ1iIxDjSGiYmpFU6nsZJLAX9pZspnX9lclWTUB+5LrMVyrOHTTcH7n9g0ri1s6/ jxQOuM+oNZ2mdhzAwawrFQ9KDOH6pNdZsBMXiJbfKH2lCh7pATaIgYWj96/7at6d67Va mRmHHE1OyMNNCbopzrhaqGthSM5xveatfXj9L6soKQqTrUS+/DH4oq1w4/EnW2qpPV1M Jdy66HRsCu5/ykHeyesJo9rLKHxuC2ZI1slvAXgyW67g1fPexWbnVuxbXMH7GbD4QMwG mmUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690582469; x=1691187269; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=j23ImFvrdRCadrPSwwknJnJ5y4yBBTyt7l0nVVVM2Yc=; b=eKyCaDmsyce0qkVjj/DeYEsLIVjAbJFxs8923SkgO54I02Wpe9Trm0MF+szbmlCLTL aggPtDx/EFh4XncDVTu6qvdzWiKoTuMEFiQUmpETB5Bb+xsW4qrWt8Nv42pS1F5sWQN6 4Gm6pWpFrsusNC7FOZ8shxNaBZ8LaM7xLfnoZn0O4DUEgsuNLGZ0RxnGtD2NFSILmwvC CEJOwHtVp34O2OP/L7D9XjT4Rdw3ZUHU9jBWauZfKbC6muzxQfp0nnBmRz7QfX48dFSV E9nQbHavCiw2NRyvipYUlhsD4lzLXW9EtAPmbSPNNkvMvVP3J+opfvOwvr3Cgus3CRIc r97A== X-Gm-Message-State: ABy/qLYZ4ijuzUra0vmsMXdh4upH6HxJp+vFy5HYhCPdhMRfBK2VN0XM gzZGB6IZIPBiEvC4IJrOoL/1IA== X-Google-Smtp-Source: APBJJlF0fafCFeexldp3nrIcTl/TKK4y9sr2ROkSLVlVpnM5bPmorNZnR4SuXn5KRsO7VBS9InuRKQ== X-Received: by 2002:a17:902:988e:b0:1b2:1a79:147d with SMTP id s14-20020a170902988e00b001b21a79147dmr3058566plp.2.1690582469499; Fri, 28 Jul 2023 15:14:29 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id jl21-20020a170903135500b001b531e8a000sm4056368plb.157.2023.07.28.15.14.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Jul 2023 15:14:28 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qPVj8-001vv5-5d; Fri, 28 Jul 2023 19:14:26 -0300 Date: Fri, 28 Jul 2023 19:14:26 -0300 From: Jason Gunthorpe To: David Hildenbrand Cc: Peter Xu , Linus Torvalds , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Andrew Morton , liubo , Matthew Wilcox , Hugh Dickins , John Hubbard Subject: Re: [PATCH v1 0/4] smaps / mm/gup: fix gup_can_follow_protnone fallout Message-ID: References: <20230727212845.135673-1-david@redhat.com> <69a5f457-63b6-2d4f-e5c0-4b3de1e6c9f1@redhat.com> <22262495-c92c-20fa-dddf-eee4ce635b12@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22262495-c92c-20fa-dddf-eee4ce635b12@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jul 28, 2023 at 11:31:49PM +0200, David Hildenbrand wrote: > * vfio triggers FOLL_PIN|FOLL_LONGTERM from a random QEMU thread. > Where should we migrate that page to? Would it actually be counter- > productive to migrate it to the NUMA node of the setup thread? The > longterm pin will turn the page unmovable, yes, but where to migrate > it to? For VFIO & KVM you actively don't get any kind of numa balancing or awareness. In this case qemu should probably strive to put the memory on the numa node of the majorty of CPUs early on because it doesn't get another shot at it. In other cases it depends quite alot. Eg DPDK might want its VFIO buffers to NUMA'd to the node that is close to the device, not the CPU. Or vice versa. There is alot of micro sensitivity here at high data rates. I think people today manually tune this by deliberately allocating the memory to specific numas and then GUP should just leave it alone. FWIW, I'm reading this thread and I have no idea what the special semantic is KVM needs from GUP, so I'm all for better documentation on the GUP flag :) Jason