[PATCH v3] mm/migrate_device: Try to handle swapcache pages

[PATCH v3] mm/migrate_device: Try to handle swapcache pages

[mpenttil]

From: Mika Penttilä <mpenttil@redhat.com>

Migrating file pages and swapcache pages into device memory is not supported.
The decision is done based on page_mapping(). For now, swapcache pages are not migrated.

Things can however be improved, for swapcache pages. Try to get rid of the swap cache,
and if successful, go ahead as with other anonymous pages.

Cc: Alistair Popple <apopple@nvidia.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Ralph Campbell <rcampbell@nvidia.com>
Cc: "Huang, Ying" <ying.huang@intel.com>
Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
Reviewed-by: Alistair Popple <apopple@nvidia.com>
Signed-off-by: Mika Penttilä <mpenttil@redhat.com>
---

v3:
  - Adjust comments
  - Add Reviewed-bys

v2:
  - use folio_test_anon() (Huang, Ying)

 mm/migrate_device.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index d30c9de60b0d..f76ebccfe067 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -747,13 +747,23 @@ static void __migrate_device_pages(unsigned long *src_pfns,
 
 		if (is_device_private_page(newpage) ||
 		    is_device_coherent_page(newpage)) {
-			/*
-			 * For now only support anonymous memory migrating to
-			 * device private or coherent memory.
-			 */
+
 			if (mapping) {
-				src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
-				continue;
+				struct folio *folio;
+
+				folio = page_folio(page);
+
+				/*
+				 * For now only support anonymous memory migrating to
+				 * device private or coherent memory.
+				 *
+				 * Try to get rid of swap cache if possible.
+				 *
+				 */
+				if (!folio_test_anon(folio) || !folio_free_swap(folio)) {
+					src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
+					continue;
+				}
 			}
 		} else if (is_zone_device_page(newpage)) {
 			/*
-- 
2.17.1

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[David Hildenbrand]

On 06.06.23 07:01, mpenttil@redhat.com wrote:
> From: Mika Penttilä <mpenttil@redhat.com>
> 
> Migrating file pages and swapcache pages into device memory is not supported.
> The decision is done based on page_mapping(). For now, swapcache pages are not migrated.
> 
> Things can however be improved, for swapcache pages. Try to get rid of the swap cache,
> and if successful, go ahead as with other anonymous pages.
> 
> Cc: Alistair Popple <apopple@nvidia.com>
> Cc: John Hubbard <jhubbard@nvidia.com>
> Cc: Ralph Campbell <rcampbell@nvidia.com>
> Cc: "Huang, Ying" <ying.huang@intel.com>
> Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
> Reviewed-by: Alistair Popple <apopple@nvidia.com>
> Signed-off-by: Mika Penttilä <mpenttil@redhat.com>
> ---
> 
> v3:
>    - Adjust comments
>    - Add Reviewed-bys
> 
> v2:
>    - use folio_test_anon() (Huang, Ying)
> 
>   mm/migrate_device.c | 22 ++++++++++++++++------
>   1 file changed, 16 insertions(+), 6 deletions(-)
> 
> diff --git a/mm/migrate_device.c b/mm/migrate_device.c
> index d30c9de60b0d..f76ebccfe067 100644
> --- a/mm/migrate_device.c
> +++ b/mm/migrate_device.c
> @@ -747,13 +747,23 @@ static void __migrate_device_pages(unsigned long *src_pfns,
>   
>   		if (is_device_private_page(newpage) ||
>   		    is_device_coherent_page(newpage)) {
> -			/*
> -			 * For now only support anonymous memory migrating to
> -			 * device private or coherent memory.
> -			 */
> +
>   			if (mapping) {
> -				src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
> -				continue;
> +				struct folio *folio;
> +
> +				folio = page_folio(page);
> +
> +				/*
> +				 * For now only support anonymous memory migrating to
> +				 * device private or coherent memory.
> +				 *
> +				 * Try to get rid of swap cache if possible.
> +				 *
> +				 */
> +				if (!folio_test_anon(folio) || !folio_free_swap(folio)) {
> +					src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
> +					continue;

I'm pretty sure the folio has to be locked in order to call 
folio_free_swap().

-- 
Cheers,

David / dhildenb

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[David Hildenbrand]

On 06.06.23 09:44, David Hildenbrand wrote:
> On 06.06.23 07:01, mpenttil@redhat.com wrote:
>> From: Mika Penttilä <mpenttil@redhat.com>
>>
>> Migrating file pages and swapcache pages into device memory is not supported.
>> The decision is done based on page_mapping(). For now, swapcache pages are not migrated.
>>
>> Things can however be improved, for swapcache pages. Try to get rid of the swap cache,
>> and if successful, go ahead as with other anonymous pages.
>>
>> Cc: Alistair Popple <apopple@nvidia.com>
>> Cc: John Hubbard <jhubbard@nvidia.com>
>> Cc: Ralph Campbell <rcampbell@nvidia.com>
>> Cc: "Huang, Ying" <ying.huang@intel.com>
>> Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
>> Reviewed-by: Alistair Popple <apopple@nvidia.com>
>> Signed-off-by: Mika Penttilä <mpenttil@redhat.com>
>> ---
>>
>> v3:
>>     - Adjust comments
>>     - Add Reviewed-bys
>>
>> v2:
>>     - use folio_test_anon() (Huang, Ying)
>>
>>    mm/migrate_device.c | 22 ++++++++++++++++------
>>    1 file changed, 16 insertions(+), 6 deletions(-)
>>
>> diff --git a/mm/migrate_device.c b/mm/migrate_device.c
>> index d30c9de60b0d..f76ebccfe067 100644
>> --- a/mm/migrate_device.c
>> +++ b/mm/migrate_device.c
>> @@ -747,13 +747,23 @@ static void __migrate_device_pages(unsigned long *src_pfns,
>>    
>>    		if (is_device_private_page(newpage) ||
>>    		    is_device_coherent_page(newpage)) {
>> -			/*
>> -			 * For now only support anonymous memory migrating to
>> -			 * device private or coherent memory.
>> -			 */
>> +
>>    			if (mapping) {
>> -				src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
>> -				continue;
>> +				struct folio *folio;
>> +
>> +				folio = page_folio(page);
>> +
>> +				/*
>> +				 * For now only support anonymous memory migrating to
>> +				 * device private or coherent memory.
>> +				 *
>> +				 * Try to get rid of swap cache if possible.
>> +				 *
>> +				 */
>> +				if (!folio_test_anon(folio) || !folio_free_swap(folio)) {
>> +					src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
>> +					continue;
> 
> I'm pretty sure the folio has to be locked in order to call
> folio_free_swap().
> 

Oh, staring at the bigger context, I assume we locked the folios via 
migrate_device_range(), correct?

-- 
Cheers,

David / dhildenb

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[Mika Penttilä]

Hi,

On 6.6.2023 10.46, David Hildenbrand wrote:
> On 06.06.23 09:44, David Hildenbrand wrote:
>> On 06.06.23 07:01, mpenttil@redhat.com wrote:
>>> From: Mika Penttilä <mpenttil@redhat.com>
>>>
>>> Migrating file pages and swapcache pages into device memory is not 
>>> supported.
>>> The decision is done based on page_mapping(). For now, swapcache 
>>> pages are not migrated.
>>>
>>> Things can however be improved, for swapcache pages. Try to get rid 
>>> of the swap cache,
>>> and if successful, go ahead as with other anonymous pages.
>>>
>>> Cc: Alistair Popple <apopple@nvidia.com>
>>> Cc: John Hubbard <jhubbard@nvidia.com>
>>> Cc: Ralph Campbell <rcampbell@nvidia.com>
>>> Cc: "Huang, Ying" <ying.huang@intel.com>
>>> Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
>>> Reviewed-by: Alistair Popple <apopple@nvidia.com>
>>> Signed-off-by: Mika Penttilä <mpenttil@redhat.com>
>>> ---
>>>
>>> v3:
>>>     - Adjust comments
>>>     - Add Reviewed-bys
>>>
>>> v2:
>>>     - use folio_test_anon() (Huang, Ying)
>>>
>>>    mm/migrate_device.c | 22 ++++++++++++++++------
>>>    1 file changed, 16 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/mm/migrate_device.c b/mm/migrate_device.c
>>> index d30c9de60b0d..f76ebccfe067 100644
>>> --- a/mm/migrate_device.c
>>> +++ b/mm/migrate_device.c
>>> @@ -747,13 +747,23 @@ static void __migrate_device_pages(unsigned 
>>> long *src_pfns,
>>>            if (is_device_private_page(newpage) ||
>>>                is_device_coherent_page(newpage)) {
>>> -            /*
>>> -             * For now only support anonymous memory migrating to
>>> -             * device private or coherent memory.
>>> -             */
>>> +
>>>                if (mapping) {
>>> -                src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
>>> -                continue;
>>> +                struct folio *folio;
>>> +
>>> +                folio = page_folio(page);
>>> +
>>> +                /*
>>> +                 * For now only support anonymous memory migrating to
>>> +                 * device private or coherent memory.
>>> +                 *
>>> +                 * Try to get rid of swap cache if possible.
>>> +                 *
>>> +                 */
>>> +                if (!folio_test_anon(folio) || 
>>> !folio_free_swap(folio)) {
>>> +                    src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
>>> +                    continue;
>>
>> I'm pretty sure the folio has to be locked in order to call
>> folio_free_swap().
>>
> 
> Oh, staring at the bigger context, I assume we locked the folios via 
> migrate_device_range(), correct?
> 

Yes either that, or when migrating via virtual addresses 
migrate_vma_collect_pmd() has trylock_page() and we migrate only pages 
which succeed locking.

--Mika

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[Christoph Hellwig]

On Tue, Jun 06, 2023 at 08:01:49AM +0300, mpenttil@redhat.com wrote:
> From: Mika Penttilä <mpenttil@redhat.com>
> 
> Migrating file pages and swapcache pages into device memory is not supported.
> The decision is done based on page_mapping(). For now, swapcache pages are not migrated.

Please fix the commit log formatting, it should not exceed 7 lines.

>  		if (is_device_private_page(newpage) ||
>  		    is_device_coherent_page(newpage)) {
> -			/*
> -			 * For now only support anonymous memory migrating to
> -			 * device private or coherent memory.
> -			 */
> +
>  			if (mapping) {

Very nitpicky, but this empty line looks odd.  Also isn't the comment
still (mostly) correct given that file backed memory is still not
supported?

> +				/*
> +				 * For now only support anonymous memory migrating to
> +				 * device private or coherent memory.
> +				 *
> +				 * Try to get rid of swap cache if possible.
> +				 *
> +				 */
> +				if (!folio_test_anon(folio) || !folio_free_swap(folio)) {

Please avoid the overly long lines.

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[Mika Penttilä]

Hi,


On 7.6.2023 17.10, Christoph Hellwig wrote:
> On Tue, Jun 06, 2023 at 08:01:49AM +0300, mpenttil@redhat.com wrote:
>> From: Mika Penttilä <mpenttil@redhat.com>
>>
>> Migrating file pages and swapcache pages into device memory is not supported.
>> The decision is done based on page_mapping(). For now, swapcache pages are not migrated.
> 
> Please fix the commit log formatting, it should not exceed 7 lines.
> 
>>   		if (is_device_private_page(newpage) ||
>>   		    is_device_coherent_page(newpage)) {
>> -			/*
>> -			 * For now only support anonymous memory migrating to
>> -			 * device private or coherent memory.
>> -			 */
>> +
>>   			if (mapping) {
> 
> Very nitpicky, but this empty line looks odd.  Also isn't the comment
> still (mostly) correct given that file backed memory is still not
> supported?

Yes the comment is mostly correct and moved a few lines lower, 
complemented with a comment about the swap cache.

> 
>> +				/*
>> +				 * For now only support anonymous memory migrating to
>> +				 * device private or coherent memory.
>> +				 *
>> +				 * Try to get rid of swap cache if possible.
>> +				 *
>> +				 */
>> +				if (!folio_test_anon(folio) || !folio_free_swap(folio)) {
> 
> Please avoid the overly long lines.
> 

Thanks,
Mika

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[Mika Penttilä]

On 7.6.2023 17.10, Christoph Hellwig wrote:
> On Tue, Jun 06, 2023 at 08:01:49AM +0300, mpenttil@redhat.com wrote:
>> From: Mika Penttilä <mpenttil@redhat.com>
>>
>> Migrating file pages and swapcache pages into device memory is not supported.
>> The decision is done based on page_mapping(). For now, swapcache pages are not migrated.
> 
> Please fix the commit log formatting, it should not exceed 7 lines.

Not sure what you mean should not exceed 7 lines..?

Re: [PATCH v3] mm/migrate_device: Try to handle swapcache pages

[Christoph Hellwig]

On Wed, Jun 07, 2023 at 07:06:06PM +0300, Mika Penttilä wrote:
> 
> 
> On 7.6.2023 17.10, Christoph Hellwig wrote:
> > On Tue, Jun 06, 2023 at 08:01:49AM +0300, mpenttil@redhat.com wrote:
> > > From: Mika Penttilä <mpenttil@redhat.com>
> > > 
> > > Migrating file pages and swapcache pages into device memory is not supported.
> > > The decision is done based on page_mapping(). For now, swapcache pages are not migrated.
> > 
> > Please fix the commit log formatting, it should not exceed 7 lines.
> 
> Not sure what you mean should not exceed 7 lines..?

Sorry, this should be 73.