mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Danilo Krummrich <dakr@redhat.com>
To: "Thomas Hellström" <thomas.hellstrom@linux.intel.com>,
	airlied@gmail.com, daniel@ffwll.ch, matthew.brost@intel.com,
	sarah.walker@imgtec.com, donald.robson@imgtec.com,
	boris.brezillon@collabora.com, christian.koenig@amd.com,
	faith@gfxstrand.net
Cc: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH drm-misc-next v5 4/6] drm/gpuvm: track/lock/validate external/evicted objects
Date: Wed, 4 Oct 2023 20:24:20 +0200	[thread overview]
Message-ID: <cdb528b1-b53f-bff6-fdf9-260ee4bd897a@redhat.com> (raw)
In-Reply-To: <bbb235037b02336a8c69271d5ff97c69b2e8b597.camel@linux.intel.com>



On 10/4/23 19:57, Thomas Hellström wrote:
> On Wed, 2023-10-04 at 19:17 +0200, Danilo Krummrich wrote:
>> On 10/4/23 17:29, Thomas Hellström wrote:
>>>
>>> On Wed, 2023-10-04 at 14:57 +0200, Danilo Krummrich wrote:
>>>> On 10/3/23 11:11, Thomas Hellström wrote:
>>>>
>>>> <snip>
>>>>
>>>>>>> +
>>>>>>> +/**
>>>>>>> + * drm_gpuvm_bo_evict() - add / remove a &drm_gpuvm_bo to
>>>>>>> /
>>>>>>> from the &drm_gpuvms
>>>>>>> + * evicted list
>>>>>>> + * @vm_bo: the &drm_gpuvm_bo to add or remove
>>>>>>> + * @evict: indicates whether the object is evicted
>>>>>>> + *
>>>>>>> + * Adds a &drm_gpuvm_bo to or removes it from the
>>>>>>> &drm_gpuvms
>>>>>>> evicted list.
>>>>>>> + */
>>>>>>> +void
>>>>>>> +drm_gpuvm_bo_evict(struct drm_gpuvm_bo *vm_bo, bool evict)
>>>>>>> +{
>>>>>>> +    struct drm_gem_object *obj = vm_bo->obj;
>>>>>>> +
>>>>>>> +    dma_resv_assert_held(obj->resv);
>>>>>>> +
>>>>>>> +    /* Always lock list transactions, even if
>>>>>>> DRM_GPUVM_RESV_PROTECTED is
>>>>>>> +     * set. This is required to protect multiple
>>>>>>> concurrent
>>>>>>> calls to
>>>>>>> +     * drm_gpuvm_bo_evict() with BOs with different
>>>>>>> dma_resv.
>>>>>>> +     */
>>>>>>
>>>>>> This doesn't work. The RESV_PROTECTED case requires the
>>>>>> evicted
>>>>>> flag we discussed before. The list is either protected by the
>>>>>> spinlock or the resv. Otherwise a list add could race with a
>>>>>> list
>>>>>> removal elsewhere.
>>>>
>>>> I think it does unless I miss something, but it might be a bit
>>>> subtle
>>>> though.
>>>>
>>>> Concurrent drm_gpuvm_bo_evict() are protected by the spinlock.
>>>> Additionally, when
>>>> drm_gpuvm_bo_evict() is called we hold the dma-resv of the
>>>> corresponding GEM object.
>>>>
>>>> In drm_gpuvm_validate() I assert that we hold *all* dma-resv,
>>>> which
>>>> implies that no
>>>> one can call drm_gpuvm_bo_evict() on any of the VM's objects and
>>>> no
>>>> one can add a new
>>>> one and directly call drm_gpuvm_bo_evict() on it either.
>>>
>>> But translated into how the data (the list in this case) is
>>> protected
>>> it becomes
>>>
>>> "Either the spinlock and the bo resv of a single list item OR the
>>> bo
>>> resvs of all bos that can potentially be on the list",
>>>
>>> while this is certainly possible to assert, any new / future code
>>> that
>>> manipulates the evict list will probably get this wrong and as a
>>> result
>>> the code becomes pretty fragile. I think drm_gpuvm_bo_destroy()
>>> already
>>> gets it wrong in that it, while holding a single resv, doesn't take
>>> the
>>> spinlock.
>>
>> That's true and I don't like it either. Unfortunately, with the dma-
>> resv
>> locking scheme we can't really protect the evict list without the
>> drm_gpuvm_bo::evicted trick properly.
>>
>> But as pointed out in my other reply, I'm a bit worried about the
>> drm_gpuvm_bo::evicted trick being too restrictive, but maybe it's
>> fine
>> doing it in the RESV_PROTECTED case.
> 
> Ah, indeed. I misread that as discussing the current code rather than
> the drm_gpuvm_bo::evicted trick. If validating only a subset, or a
> range, then with the drm_gpuvm_bo::evicted trick would be valid only
> for that subset.
> 
> But the current code would break because the condition of locking "the
> resvs of all bos that can potentially be on the list" doesn't hold
> anymore, and you'd get list corruption.
> 
> What *would* work, though, is the solution currently in xe, The
> original evict list, and a staging evict list whose items are copied
> over on validation. The staging evict list being protected by the
> spinlock, the original evict list by the resv, and they'd use separate
> list heads in the drm_gpuvm_bo, but that is yet another complication.
> 
> But I think if this becomes an issue, those VMs (perhaps OpenGL UMD
> VMs) only wanting to validate a subset, would simply initially rely on
> the current non-RESV solution. It looks like it's only a matter of
> flipping the flag on a per-vm basis.

If such a driver locks a range it can also just validate all locked
objects I guess.

And for everything else, we still have the spinlock protected variant,
where drivers can freely move things around by just taking the spinlock.

I think I will go ahead and add drm_gpuvm_bo::evicted, plus the helpers
I mentioned.

> 
> /Thomas
> 
> 
>>
>>>
>>> So I think that needs fixing, and if keeping that protection I
>>> think it
>>> needs to be documented with the list member and ideally an assert.
>>> But
>>> also note that lockdep_assert_held will typically give false true
>>> for
>>> dma_resv locks; as long as the first dma_resv lock locked in a
>>> drm_exec
>>> sequence  remains locked, lockdep thinks *all* dma_resv locks are
>>> held.
>>> (or something along those lines), so the resv lockdep asserts are
>>> currently pretty useless.
>>>
>>> /Thomas
>>>
>>>
>>>
>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
> 


  reply	other threads:[~2023-10-04 18:25 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-28 19:16 [PATCH drm-misc-next v5 0/6] [RFC] DRM GPUVM features Danilo Krummrich
2023-09-28 19:16 ` [PATCH drm-misc-next v5 1/6] drm/gpuvm: add common dma-resv per struct drm_gpuvm Danilo Krummrich
2023-09-28 19:16 ` [PATCH drm-misc-next v5 2/6] drm/gpuvm: add drm_gpuvm_flags to drm_gpuvm Danilo Krummrich
2023-09-28 19:16 ` [PATCH drm-misc-next v5 3/6] drm/gpuvm: add an abstraction for a VM / BO combination Danilo Krummrich
2023-10-02  6:34   ` kernel test robot
2023-10-05 11:51   ` Thomas Hellström
2023-10-08 23:08     ` Danilo Krummrich
2023-09-28 19:16 ` [PATCH drm-misc-next v5 4/6] drm/gpuvm: track/lock/validate external/evicted objects Danilo Krummrich
2023-10-02 15:40   ` kernel test robot
2023-10-03  8:36   ` Thomas Hellström
2023-10-03  9:11     ` Thomas Hellström
2023-10-04 12:57       ` Danilo Krummrich
2023-10-04 15:29         ` Thomas Hellström
2023-10-04 17:17           ` Danilo Krummrich
2023-10-04 17:57             ` Thomas Hellström
2023-10-04 18:24               ` Danilo Krummrich [this message]
2023-10-03 10:05     ` Boris Brezillon
2023-10-03 12:25       ` Thomas Hellström
2023-10-03 14:21         ` Boris Brezillon
2023-10-03 16:55           ` Danilo Krummrich
2023-10-03 17:37             ` Thomas Hellström
2023-10-04 13:35               ` Danilo Krummrich
2023-10-03 18:57             ` Thomas Hellström
2023-10-05 11:55   ` Thomas Hellström
2023-09-28 19:16 ` [PATCH drm-misc-next v5 5/6] drm/nouveau: make use of the GPUVM's shared dma-resv Danilo Krummrich
2023-09-28 19:16 ` [PATCH drm-misc-next v5 6/6] drm/nouveau: use GPUVM common infrastructure Danilo Krummrich
2023-10-05  9:35 ` [PATCH drm-misc-next v5 0/6] [RFC] DRM GPUVM features Thomas Hellström
2023-10-08 22:48   ` Danilo Krummrich

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cdb528b1-b53f-bff6-fdf9-260ee4bd897a@redhat.com \
    --to=dakr@redhat.com \
    --cc=airlied@gmail.com \
    --cc=boris.brezillon@collabora.com \
    --cc=christian.koenig@amd.com \
    --cc=daniel@ffwll.ch \
    --cc=donald.robson@imgtec.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=faith@gfxstrand.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=matthew.brost@intel.com \
    --cc=nouveau@lists.freedesktop.org \
    --cc=sarah.walker@imgtec.com \
    --cc=thomas.hellstrom@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®