From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm2-f13.google.com (mail-wm2-f13.google.com [74.125.225.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21E5C47CA7A for ; Mon, 5 Oct 2026 11:49:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.141 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791200962; cv=none; b=fvNJHjPwp8B7Z8iXVJ/CMbXfdYEwFmKqwh6hczkK4Bh+yGtcpke8Sd2flZZvpzcvdGVrp5tGwTtgSF80XfRoUoHoWWlrGGCQYAeFGnx0chPdpqUuc07aF+Txn+hvw1ijfEfO2lD6sEQtNKkA9L6aFml8jBNfe1P855cxnb4Z2ns= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791200962; c=relaxed/simple; bh=C4rWPxAM+gG+nBkU+Lpq5+tCLNPh5lzEfMeHsRf0DPA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=dtpPpUeh37ze4Zi+t/OIdayKRbyJ8PtDIzQ2j2Qn+BDM4i8jdXdLgiiwhy3I96vcgwkDru4bJMZ7Vvz6sZ2oFq+VmEkJJqg8VjLYDGhQ/yVvZj0VnF3D4CKqQk3C2veP1OoRq/NOtKoSaHUm0kzETeFLtCIGnRRnxani43E8a8A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=oe2e4wNU; arc=none smtp.client-ip=74.125.225.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="oe2e4wNU" Received: by mail-wm2-f13.google.com with SMTP id 5b1f17b1804b1-4a0977b9c20so22168545e9.2 for ; Mon, 05 Oct 2026 04:49:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1791200959; x=1791805759; darn=vger.kernel.org; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:from:to:cc:subject:date:message-id:reply-to :content-type; bh=tN32JBtkwvrpOHkqPOwCz1peQMff+9nW+9FnKuTc/9Y=; b=oe2e4wNU3lig5ckdIY4pln90NFMoBuzqjjQp9ZzJ8X5L77pYYYqB8QGUGhNn5BK7ag teomHdKgtcHPMlz/WiXvHlG4Mqk0rycrnXJ912uIw6GEYYvPKxEuLdPx15bRUCANE2uv ElhJ5VhfFNTy7CEgJukBL5HhoL0XKvCSIG556r/Nj1BH3XRFpc1MA1+5+ZVLdT3ilJvY W0bKiKQ9tO8NasBmu7RcIKz5cRszVG0qmcnUvvCR2Zv0WtGFQyYVQGXvgo0gFlVlHf4X JKNl84nvM/LvHX0h4AUV4kuiFJiMSkSxDwbWQhuoEtIxtOwK7scS6B0zruBGFZZJE8ZU b51w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791200959; x=1791805759; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=tN32JBtkwvrpOHkqPOwCz1peQMff+9nW+9FnKuTc/9Y=; b=1bVy4BcIkx0BDubqydhPJN3XtqFF196qP8+ChhSvscA8yhvmyi+39F1uoG9rF6fwBj r61eioeXtChzyjPEFHKJseBbyrOo1CgCIHxwMYpl/0JK7/AfhdBhLMlh/U66OxHufuyk Aba0bfghIo7dWI0JhlfozSLGXN5PLF7VnlWhKAOEteAl+LWnZ2uSUZH2PekrVULMMLPz uZGqvFfsIpJZIxw5z2fO35Iuf6xzvw/vwNF5JzcOsSYK738gT9L4reKDolA96DS8kNa1 b/a2ShKxKGsFOJZXTeie0fP963SEMev2CXMBVARIkSlL++pAyi/benMoj8IEkxevu9S0 XNTw== X-Forwarded-Encrypted: i=1; AKwUvBx5Z/2hwPDh7EdA5wlm9eMjX3w1bg60R3jGcgN1kdFS102sPYsXBuDugexFwFfNq+O1XflcgA57Z9dDQYo=@vger.kernel.org X-Gm-Message-State: AFuF++kJd0fUIw7SfEjPAOjWaAl5Sp1KYsFXD0/Th9xuZQ2Dr0dOw/jC 3t9wafR8sWVJ65PJBqXWdrAcu/+khYEM5XMCnhw3GysiLmKTrxpBtH1y X-Gm-Gg: AYBFou1tPkx7i3AD5GzGP0aSUJIhOuh2dX4d3rmtK9kE8NXmABEFF+ofX4DOXLzNAob oSOm2RHHDmIKPC943WlxQucc5SjVDw7kux3JmFBKqUWxbQ0VMRXBU+qAQbWjrPc8fzltjHgaT2C tusXL6M2oG9v3x9mLoNpoHjgWcyvpZwsYqm42BsbXM7/6VJ06XGKLzuTyk6l6TPn/Gtp4zhmdbf jsSUhQXR9DnyAu64HdO6JwiqxoaNlJaFy5fUpDpYXMnsHK3XI32foVWQdf7uX9yusQYzJKTOIei n9xp4fnlf8IP+qJyP8k4+LUgZALZKiwrhjdbNGk9JYo3OoEAwWrNtF+vrDxj5MEoWkAzAhkme5O IUnvYAB4B1/tTmCif6Ta+E9XG23gfYi9Gkl3H2yb5A8sS0Lu46SeTKv2/8l44Q8KTQ6J8fkuJTv Ygf4yII3DFSBdFe4VJqnOFews+uiA4pREvtJbss7m/xm8nFct/5UA1ZKsidec11OnMWRlQfwzEN PdrIVqU3150AX2owzI9VH6756/BVkOqIt8I/ULQZIZzLoq0u8i8s1LmxSO2C4M8gO/41XUwE7w= X-Received: by 2002:a05:600c:1c08:b0:4a1:722a:d2a3 with SMTP id 5b1f17b1804b1-4a1722ad405mr52583605e9.23.1791200959268; Mon, 05 Oct 2026 04:49:19 -0700 (PDT) Received: from [10.41.30.1] ([161.12.45.16]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4a027727cecsm346670305e9.9.2026.10.05.04.49.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Oct 2026 04:49:18 -0700 (PDT) Message-ID: Date: Mon, 5 Oct 2026 12:49:15 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 01/13] dma-buf: introduce initial file I/O infrastructure To: Matthew Brost Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, Christoph Hellwig , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Keith Busch , Sagi Grimberg , Alexander Viro , Christian Brauner , Jan Kara , Andrew Morton , Jens Axboe , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , dm-devel@lists.linux.dev References: <5490ee42c4452fd4198b245ead20fcf7438c066e.1789997898.git.asml.silence@gmail.com> <447c04dd-6bfa-4c45-99e7-37d25a7ab539@gmail.com> Content-Language: en-US From: Pavel Begunkov In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 9/30/26 20:43, Matthew Brost wrote: > On Wed, Sep 30, 2026 at 01:48:49PM +0100, Pavel Begunkov wrote: >> On 9/30/26 11:08, Pavel Begunkov wrote: >>> On 9/30/26 04:00, Matthew Brost wrote: >>>> On Mon, Sep 21, 2026 at 02:38:45PM +0100, Pavel Begunkov wrote: >>> ...>> +struct dma_buf_io_map *dma_buf_io_create_map(struct dma_buf_io_ctx *ctx) >>>>> +{ >>>>> +    struct dma_buf *dmabuf = ctx->dmabuf; >>>>> +    struct dma_buf_io_map *map; >>>>> +    long ret; >>>>> + >>>>> +    guard(mutex)(&ctx->map_create_mutex); >>>>> + >>>>> +    scoped_guard(mutex, &ctx->map_mutex) { >>>>> +        if (ctx->maps_killed) >>>>> +            return ERR_PTR(-ENOENT); >>>>> +        /* recheck under the lock in case it has already been re-created */ >>>>> +        map = __dma_buf_io_get_map(ctx); >>>>> +        if (map) >>>>> +            return map; >>>>> +    } >>>>> + >>>>> +    dma_buf_io_wait_active_maps(ctx); >>>>> + >>>>> +    ret = dma_resv_lock_interruptible(dmabuf->resv, NULL); >>>>> +    if (ret) >>>>> +        return ERR_PTR(ret); >>>>> + >>>>> +    ret = dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_KERNEL, >>>>> +                    true, MAX_SCHEDULE_TIMEOUT); >>>>> +    if (ret <= 0) { >>>>> +        if (!ret) >>>>> +            ret = -EAGAIN; >>>>> +        dma_resv_unlock(dmabuf->resv); >>>>> +        return ERR_PTR(ret); >>>>> +    } >>>>> + >>>>> +    map = ctx->dev_ops->map(ctx); >>>> >>>> I'm playing around this code now. >>>> >>>> I think you need the dma_resv_wait_timeout after the 'map'? >>>> >>>> If a device doesn't support p2p ->map() will typically trigger an async >>>> migrate to system memory and data will be moving but the map is valid - >>>> Xe 100% does this, I checked AMDGPU and fairly confident it has the same >>>> async behavior. >>> >>> There is a wait right before because I read somewhere in dma-buf >>> comments that I need to do that, sounds a bit odd if I need to >>> wait on fences before and after. I can add it, just curious how >>> come that other dma_buf_map_attachment() callers don't need to do >>> that. Or maybe they wait somewhere else? > > In GPU drivers, when we receive a foreign object and map it in Xe or > AMDGPU, this logic sits deep in the stack. However, the top-level call > is typically ttm_bo_validate(), which in both drivers eventually > resolves to the TTM ->move() vfunc. That path calls > dma_buf_map_attachment(), which in turn invokes the dma-buf's ->map() > callback. > > That ->map() callback can trigger a asynchronous move on a different > device, where the top-level call is again ttm_bo_validate(). > We then end up back in the TTM ->move() vfunc, where kernel fences are > installed. I realize that's a lot of layers, but the call chain can end > up looking like this. > > Now we have a mapping and an object with kernel fences attached. Before > the object can be used by either the exec IOCTL (batch buffer > submission), the VM bind IOCTL (mapping into the GPU address space), or > a CPU page fault (not relevant for dma-bufs since they cannot be > CPU-mapped, but a useful example of a normal BO move), we wait on those > kernel fences. > > In the case of the exec IOCTL or VM bind IOCTL, the kernel fences are > added as dependencies to a drm_sched job, delaying its execution until > the fences signal. That is where the wait occurs. In the case of a CPU > page fault, the kernel waits directly on the fences before installing > the CPU page mappings. > > So the TL;DR is if you want to immediately hand back a valid mapping, > the kernel fences must be waited on before returning after ->map() call. Thanks for the overview! I'll add the wait after ->map in v8 >> I can't find it, so maybe it was the comment below and I mixed >> sth up back then. I'll move it after ->map(). >> >> >> * Note that for non-dynamic exporters the driver must guarantee that >> * that the memory is available for use and cleared of any old data by >> * the time this function returns. Drivers which pipeline their buffer >> * moves internally must wait for all moves and clears to complete. >> * Dynamic exporters do not need to follow this rule: For non-dynamic >> * importers the buffer is already pinned through @pin, which has the >> * same requirements. Dynamic importers otoh are required to obey the >> * dma_resv fences. >> * -- Pavel Begunkov