Re: s2disk hang update

[Alan Jenkins <sourcejedi.lkml@googlemail.com>]

[-- Attachment #1: Type: text/plain, Size: 4346 bytes --]

Rafael J. Wysocki wrote:
> On Friday 19 February 2010, Alan Jenkins wrote:
>   
>> On 2/18/10, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>>     
>>> On Thursday 18 February 2010, Alan Jenkins wrote:
>>>       
>>>> On 2/17/10, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>>>>         
>>>>> On Wednesday 17 February 2010, Alan Jenkins wrote:
>>>>>           
>>>>>> On 2/16/10, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>>>>>>             
>>>>>>> On Tuesday 16 February 2010, Alan Jenkins wrote:
>>>>>>>               
>>>>>>>> On 2/16/10, Alan Jenkins <sourcejedi.lkml@googlemail.com> wrote:
>>>>>>>>                 
>>>>>>>>> On 2/15/10, Rafael J. Wysocki <rjw@sisk.pl> wrote:
>>>>>>>>>                   
>>>>>>>>>> On Tuesday 09 February 2010, Alan Jenkins wrote:
>>>>>>>>>>                     
>>>>>>>>>>> Perhaps I spoke too soon.  I see the same hang if I run too many
>>>>>>>>>>> applications.  The first hibernation fails with "not enough
>>>>>>>>>>> swap"
>>>>>>>>>>> as
>>>>>>>>>>> expected, but the second or third attempt hangs (with the same
>>>>>>>>>>> backtrace
>>>>>>>>>>> as before).
>>>>>>>>>>>
>>>>>>>>>>> The patch definitely helps though.  Without the patch, I see a
>>>>>>>>>>> hang
>>>>>>>>>>> the
>>>>>>>>>>> first time I try to hibernate with too many applications
>>>>>>>>>>> running.
>>>>>>>>>>>                       
>>>>>>>>>> Well, I have an idea.
>>>>>>>>>>
>>>>>>>>>> Can you try to apply the appended patch in addition and see if
>>>>>>>>>> that
>>>>>>>>>> helps?
>>>>>>>>>>
>>>>>>>>>> Rafael
>>>>>>>>>>                     
>>>>>>>>> It doesn't seem to help.
>>>>>>>>>                   
>>>>>>>> To be clear: It doesn't stop the hang when I hibernate with too many
>>>>>>>> applications.
>>>>>>>>
>>>>>>>> It does stop the same hang in a different case though.
>>>>>>>>
>>>>>>>> 1. boot with init=/bin/bash
>>>>>>>> 2. run s2disk
>>>>>>>> 3. cancel the s2disk
>>>>>>>> 4. repeat steps 2&3
>>>>>>>>
>>>>>>>> With the patch, I can run 10s of iterations, with no hang.
>>>>>>>> Without the patch, it soon hangs, (in disable_nonboot_cpus(), as
>>>>>>>> always).
>>>>>>>>
>>>>>>>> That's what happens on 2.6.33-rc7.  On 2.6.30, there is no problem.
>>>>>>>> On 2.6.31 and 2.6.32 I don't get a hang, but dmesg shows an
>>>>>>>> allocation
>>>>>>>> failure after a couple of iterations ("kthreadd: page allocation
>>>>>>>> failure. order:1, mode:0xd0").  It looks like it might be the same
>>>>>>>> stop_machine thread allocation failure that causes the hang.
>>>>>>>>                 
>>>>>>> Have you tested it alone or on top of the previous one?  If you've
>>>>>>> tested it
>>>>>>> alone, please apply the appended one in addition to it and retest.
>>>>>>>
>>>>>>> Rafael
>>>>>>>               
>>>>>> I did test with both patches applied together -
>>>>>>
>>>>>> 1. [Update] MM / PM: Force GFP_NOIO during suspend/hibernation and
>>>>>> resume
>>>>>> 2. "reducing the number of pages that we're going to keep preallocated
>>>>>> by
>>>>>> 20%"
>>>>>>             
>>>>> In that case you can try to reduce the number of preallocated pages even
>>>>> more,
>>>>> ie. change "/ 5" to "/ 2" (for example) in the second patch.
>>>>>           
>>>> It still hangs if I try to hibernate a couple of times with too many
>>>> applications.
>>>>         
>>> Hmm.  I guess I asked that before, but is this a 32-bit or 64-bit system and
>>> how much RAM is there in the box?
>>>
>>> Rafael
>>>       
>> EeePC 701.  32 bit.  512Mb RAM.  350Mb swap file, on a "first-gen" SSD.
>>     
>
> Hmm.  I'd try to make  free_unnecessary_pages() free all of the preallocated
> pages and see what happens.
>   

It still hangs in hibernation_snapshot() / disable_nonboot_cpus().  
After apparently freeing over 400Mb / 100,000 pages of preallocated ram.



There is a change which I missed before.  When I applied your first 
patch ("Force GFP_NOIO during suspend" etc.), it did change the hung 
task backtraces a bit.  I don't know if it tells us anything.

Without the patch, there were two backtraces.  The first backtrace 
suggested a problem allocating pages for a kernel thread (at 
copy_process() / try_to_free_pages()).  The second showed that this 
problem was blocking s2disk (at hibernation_snapshot() / 
disable_nonboot_cpus() / stop_machine_create()). 

With the GFP_NOIO patch, I see only the s2disk backtrace.


Thanks
Alan

[-- Attachment #2: free-all-prealloc.patch --]
[-- Type: text/x-diff, Size: 575 bytes --]

diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index 36cb168..970930a 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -1179,6 +1179,12 @@ static void free_unnecessary_pages(void)
 		to_free_normal -= save_highmem - alloc_highmem;
 	}
 
+	to_free_normal = alloc_normal;
+	printk(KERN_ERR "Freeing all %d preallocated pages\n",
+			to_free_normal);
+	printk(KERN_ERR "(and %d highmem pages, out of %d)\n",
+			alloc_highmem, to_free_highmem);
+
 	memory_bm_position_reset(&copy_bm);
 
 	while (to_free_normal > 0 && to_free_highmem > 0) {