Re: [PATCH 0/2] Kexec jump: The first step to kexec base hibernation

["Rafael J. Wysocki" <rjw@sisk.pl>]

On Friday, 13 July 2007 18:48, Jeremy Maitin-Shepard wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> writes:
> 
> [snip]
> 
> > Okay, I have thought it through and I think that, as an initial step, we can do
> > something like this:
> 
> > - preload the image-saving kernel before hibernation
> > - in the hibernation code path replace device_suspend() with the shutting down
> > of
> >   all devices without unregistering them (not very nice, but should be
> > sufficient
> >   for a while)
> 
> It seems that the effect of what is done by the current hibernate
> implementations is to shutdown all of the devices, but according to
> kernel data structures, have it look like the devices were merely
> suspended (i.e. device_suspend).  Then in the resume path, the "restore
> image" kernel also calls device_suspend just before jumping to the
> hibernated kernel, so all of the devices the "restore image" kernel knew
> about are in the state device_suspend expects them to be in, except that
> they were actually suspended by a different kernel, so they might not be
> in quite the right state.  There is also the issue that the "restore
> image" kernel might not know about all of the devices; for instance, if
> USB support is modular, and, as is likely to be the case, the user
> didn't load the USB modules in the "restore image" kernel from an initrd
> or something, then the USB devices will actually be powered off, rather
> than "suspended".  Despite these apparent discrepancies, it seems that
> for many devices (I'm not sure USB devices are included, though),
> device_restore happens to do the right thing so that the device is
> placed back in the state it needs to be so that the driver can begin
> talking to it as it did before, and the device is recognized as the same
> device as was there before (since otherwise mounted filesystems backed
> by block devices that came back as a different device would cause great
> havoc).
> 
> Since I recall there being issues with USB devices being recognized as
> the same devices post-hibernate-resume, without looking at the code I'm
> inclined to believe that the USB drivers still don't end up resuming
> from hibernate correctly.
> 
> Note that I am describing what is done currently, not what is planned to
> be done (i.e. change device_suspend to quiesce and device_resume to
> unquiesce).  It seems that ironically, despite everyone believing that
> device_suspend/device_resume is incorrect for hibernate, many of the
> things that those functions do (like saving the PCI configuration,
> perhaps, and then restoring it later, or re-initializing the device) are
> actually necessary, especially for modular drivers that won't be loaded
> by the "restore image" kernel.
> 
> What needs to be done is for the devices to be shut down (or possibly
> just quiesced for a select few, but we won't worry about that
> complication until later; in the case of the current implementations,
> they should all be quiesced rather than shut down), but whatever
> information that will be needed later to reinitialize the device
> (ideally the reinitialization should be able to handle the device either
> being in a quiesced state or completely off) and recognize it as the
> same device must be saved.  This probably means they cannot be
> "unregistered", as otherwise there would be nothing with which to
> associate the saved information.
> 
> The resume path needs to use the saved state to reinitialize the device
> and recognize it as the same device.  It seems that the existing
> reprobing code may not be sufficient for this.  Note that exactly the
> same thing must be done on resume for both the current hibernate
> implementations and the kexec approach.  It seems that properly
> restoring the devices should be relatively easy for the devices that
> already get this correct, like IDE devices and basic PCI devices (and
> SATA and SCSI devices as well perhaps?), and possibly harder for
> Firewire or USB devices.

The problem of handling devices during hibernation has been discussed for many
times on linux-pm and we have reached certain agreement.  I wouldn't like to
repeat all of the arguments here.  For details, please refer, for example, to
this thread:

https://lists.linux-foundation.org/pipermail/linux-pm/2007-May/012386.html

> > - when we've called device_power_down() and save_processor_state(), jump to
> >   the image-saving kernel and let it run
> > - make the image-saving kernel set up everything, save the image without
> >   starting any user space (we may use the existing image-saving code for this
> >   purpose, with some modifications) and power off the system (or make it enter
> >   S4)
> 
> I suppose this has the advantage of not requiring that a
> kernel-to-userspace interface be created for this purpose.

Yes, and for now we're avoiding the problems with starting the new user space
from a special place.

> > - use the existing restoration code to load the image and jump to the
> >   hibernated kernel
> 
> This would again avoid the need for a separate userspace-kernelspace
> interface for the purpose, so I agree it could be a useful thing to do
> initially.
> 
> > - in the restore code patch replace device_resume() with the reprobing of all
> >   devices.
> 
> See my comments above.

And please see my reply. :-)

Greetings,
Rafael


-- 
"Premature optimization is the root of all evil." - Donald Knuth