From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D57D0383C6E for ; Tue, 9 Jun 2026 14:58:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781017115; cv=none; b=lj034wCRUFRnZOYKZGIie16n58taUvEwNvnKlnnoLr+1YT5DHYGLi8fDa9XtSLvReqfpGeeeZ6ihBKqTvQrQyO9bSlVDCkzbgdOHVZbIyAXMCnY7R5Zttw5BoJE+MKgX3dOnQs0ww8PI6ilXZDH21F44q6Ms8/rdNnacHycQA9Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781017115; c=relaxed/simple; bh=89JGksdThCij5+VjGUOcb5O/ZGVRmOdgNkQwA8TGZlE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WwJ1pqyZeospXM28x+RS14lLFuO6hBpwMfklNRyRetk9YvLUZghPuNSu+00eyQCc45bJW7hdCKrW4R7TezISTeJ8urA0B1DlvwtTQ2gP1BsKdPqy0KyiHazNep85OUJX/ArEoNMj+2bozLRQMD/hQwrnWMEQe/15OyBqrEcMbHA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Tdo4Gkb7; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Tdo4Gkb7" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2bf237e1433so63016815ad.1 for ; Tue, 09 Jun 2026 07:58:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781017112; x=1781621912; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=eFoSbz/qa2j2QRMMIew+HXblnpoKHM6VjME/GNmQDGg=; b=Tdo4Gkb70sz1WZpRhmXsMMAbsW5c5IptasOp/N1KcmZvLYbw+P0Tf21pdZb6BjM/rm Cpl886tq8iW0aVwtWQz7V1l8l42mami4zZCtQzu0NO6YLf+BW6k4XgXFk8Q+sY1tk9xP 3My6QwJlLb95bjiFQds12dOvrT9NQRdBxym6IAreSNwKTVv4mrJ/XSY/0yPoc1WSExq0 ZGTQS66UR4kOUBUjSpMtEL+CA5mDrTw/Gj8I1q+9v6EFOB81Qrpg3rTpUgX33aDdScDi 8HySQQf8W9Fz2QJqzOUMd9kLnQRXH8ULJSuuSvFxMDVPQKF9GsqCyvubMJ8rthiFjoal YpFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781017112; x=1781621912; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eFoSbz/qa2j2QRMMIew+HXblnpoKHM6VjME/GNmQDGg=; b=N810jkMTJuo1TKyW3iCkIr9430+BwEoaC6w0+V1x45Q2qhjU83lDuodycjpiTAZF1B /JyNTF/44TnDIwZvK2dFRSBF6Y2eEr7mdQvxEFz9Tq22UWVYiJw1ZdQTZx4u0wrxw/ae wpWi+GQ0hE/QJ2+aC3YNWCXKTIX0dyG5KAjKHGSDv43Q9y/GQj9jwOCSch2UsJ+3zvkL NP1Bec1HVUy7jWpY1etHHn5BE2i+zLLaUIegX74LiY59ZG1p0Wlfphub0TpFwbIJsuTD UZnnyowRdDVsbocKglxmQW5iLhljV6Lpm/rEo8BYj4/NwXSOQWLwul/1Qnhu0mswpJD6 U4ow== X-Forwarded-Encrypted: i=1; AFNElJ8Uno+EDb7bj2ZPwZOxxblYBza6Wj8ZVBmF+fZ64z/bYEHd9tCuMb8FCwg1LnirzDiI9T/9zpUqbG7vrbU=@vger.kernel.org X-Gm-Message-State: AOJu0YwyV7GSNzmsOMhT6fa01ajaUB9k3BijoWHiSggUO6ZzOk6eV5gt sc+UJoBvno6FIThDvMZ+EVOc+KECRvEITOIeJurLW9bp2njp4TZT6/5Q X-Gm-Gg: Acq92OE8+PMFaZp5uj2BWNrTg2FoHOGT7G4G6b4NGV0/YXfCLUie4MMCgO2cfuqTjew dSNnuVtb6U7LFDf8wYpOSPnll80dslH9NpbCIUQPFH2wX2jpULs1fnKO5nk/7KPjSUP8JVq1Oxl OUBGl0R4787R4qePQQXXbygOrB1Gdjv/486HDRPGZJE2dTeJIEv8OywqvZ01HdBKHUeaIeEONVu nHlt+6qaMH2CMGaiWQDk13ufjTFYS2dWhV4G2z8YbJEi2it2e+7jEtbs+8aqOdYfxdc7vQcudAp vTy86jQN8L79LULOZ2UqVmUyPLYmSaT4E54jFDHG3r9bAaCLhkYJGY+lwp4r9PdssYr2CzLRnFv dbL7gzesHDgD4basuxgM+vKcAy0YUEr/giYimEm8Hs5q9Cb5fb7a9hQmLqkhDgV5DtnxsloNhKU Oe3JK2rAe35L+7Qr3g/h+ksPWwq9CnA0sM+IRB+pq8TX74GVzYxJrpMQ== X-Received: by 2002:a17:903:3885:b0:2c2:27be:39a9 with SMTP id d9443c01a7336-2c227be3b30mr199425095ad.9.1781017111918; Tue, 09 Jun 2026 07:58:31 -0700 (PDT) Received: from devvm29614.prn0.facebook.com ([2a03:2880:ff:3::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c164f87920sm219844645ad.24.2026.06.09.07.58.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 07:58:31 -0700 (PDT) Date: Tue, 9 Jun 2026 07:58:29 -0700 From: Bobby Eshleman To: Christian =?iso-8859-1?Q?K=F6nig?= Cc: Donald Hunter , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Simon Horman , Andrew Lunn , Gerd Hoffmann , Vivek Kasireddy , Sumit Semwal , Shuah Khan , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-kselftest@vger.kernel.org, sdf@fomichev.me, razor@blackwall.org, daniel@iogearbox.net, almasrymina@google.com, matttbe@kernel.org, skhawaja@google.com, dw@davidwei.uk, Bobby Eshleman Subject: Re: [PATCH net-next 2/4] udmabuf: emit one sg entry per pinned folio Message-ID: References: <20260603-tcpdm-large-niovs-v1-0-f37a4ac6726c@meta.com> <20260603-tcpdm-large-niovs-v1-2-f37a4ac6726c@meta.com> <0c86f5d3-b5e9-4cac-aa9d-30c5c8ecca66@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Mon, Jun 08, 2026 at 03:59:04PM +0200, Christian König wrote: > On 6/8/26 15:55, Bobby Eshleman wrote: > > > > On Sun, Jun 7, 2026 at 11:42 PM Christian König > wrote: > > > > On 6/5/26 20:44, Bobby Eshleman wrote: > > > On Fri, Jun 05, 2026 at 11:30:07AM +0200, Christian König wrote: > > >> On 6/4/26 02:42, Bobby Eshleman wrote: > > >>> From: Bobby Eshleman > > > >>> > > >>> get_sg_table() emitted one PAGE_SIZE sg entry per page even when the > > >>> underlying folio was larger. > > >>> > > >>> Instead, walk folios[] and emit one sg entry per folio. When folios > > >>> represent large pages (as is for MFD_HUGETLB), each sg entry is a large > > >>> page. Normal PAGE_SIZE sg tables are unchanged. > > >>> > > >>> Required by net/core/devmem to support rx-buf-size > PAGE_SIZE with > > >>> udmabuf. > > >> > > >> That doesn't explain why this is required. > > > > > > Sure, can definitely add. Devmem currently requires dmabuf sg entries to > > > be length and size aligned when it allocates niovs for NIC page pools. > > > Though udmabuf is not violating any dmabuf contract by emitting > > > PAGE_SIZE entries and the above restriction is probably more a > > > shortfalling of devmem, by emitting a single entry per folio this patch > > > allows udmabuf to be used by devmem for large pages. > > > > > >> > > >> Please note that accessing the pages/folio of an sg-table returned by DMA-buf is illegal and strictly forbidden! > > >> > > >> Regards, > > >> Christian. > > > > > > It seems both devmem and io_uring zcrx at least introspect through to > > > the sg-table to build NIC page pools (not accessing the memory itself, > > > however). Is there a better way? > > > > That's an absolute NO-GO! We need to stop that immediately. > > > > Touching the underlying struct page of an DMA-buf exported sg-table is strictly forbidden. > > > > We even have code to wrap the sg_table and hide the struct pages on debug builds to catch those issues, see function dma_buf_wrap_sg_table(). > > > > My last status is that the NIC page pools are build directly from the DMA addresses exposed by the sg_table. > > > > Was there any change I'm not aware of? > > > > Regards, > > Christian. > > > > > > Oh no change, your mental model is still current. > > They just go through each sg and use sg_dma_address() on each. > > Ah, thanks! That was a near heart attack :D > > Yeah that is perfectly correct, question is do you then still really need this udmabuf change? I mean the DMA API usually merges together contiguous DMA addresses. > > Regards, > Christian. > Hey Christian, sorry for the delay I justed want to double check what I'm seeing... I reverted the udmabuf patch and confirmed devmem still runs into 4K pages even for hugepage udmabuf. I see that the dma_map_direct() path is being taken, which if I am reading the code correctly results in the sg_dma_len(sg) inheriting sg->length directly (set by udmabuf's sg_set_folio(..., PAGE_SIZE) call), compared to the iommu_dma_map_phys() path which looks like it does merge when possible. Best, Bobby