Re: [PATCH v3] livepatch: Clear relocation targets on a module removal

[Joe Lawrence <joe.lawrence@redhat.com>]

On 8/1/22 17:19, Song Liu wrote:
> On Mon, Aug 1, 2022 at 3:25 AM Petr Mladek <pmladek@suse.com> wrote:
>>
>> On Sat 2022-07-30 20:20:22, Song Liu wrote:
>>> On Sat, Jul 30, 2022 at 3:32 PM Song Liu <song@kernel.org> wrote:
>>>>
>>>> On Tue, Jul 26, 2022 at 8:54 PM Song Liu <song@kernel.org> wrote:
>>>>>
>>>>> On Tue, Jul 26, 2022 at 4:33 PM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>>>>>>
>>>>>> On Thu, Jul 21, 2022 at 10:51:47AM -0700, Song Liu wrote:
>>>>>>> From: Miroslav Benes <mbenes@suse.cz>
>>>>>>>
>>>>>>> Josh reported a bug:
>>>>>>>
>>>>>>>   When the object to be patched is a module, and that module is
>>>>>>>   rmmod'ed and reloaded, it fails to load with:
>>>>>>>
>>>>>>>   module: x86/modules: Skipping invalid relocation target, existing value is nonzero for type 2, loc 00000000ba0302e9, val ffffffffa03e293c
>>>>>>>   livepatch: failed to initialize patch 'livepatch_nfsd' for module 'nfsd' (-8)
>>>>>>>   livepatch: patch 'livepatch_nfsd' failed for module 'nfsd', refusing to load module 'nfsd'
>>>>>>>
>>>>>>>   The livepatch module has a relocation which references a symbol
>>>>>>>   in the _previous_ loading of nfsd. When apply_relocate_add()
>>>>>>>   tries to replace the old relocation with a new one, it sees that
>>>>>>>   the previous one is nonzero and it errors out.
>>>>>>>
>>>>>>>   On ppc64le, we have a similar issue:
>>>>>>>
>>>>>>>   module_64: livepatch_nfsd: Expected nop after call, got e8410018 at e_show+0x60/0x548 [livepatch_nfsd]
>>>>>>>   livepatch: failed to initialize patch 'livepatch_nfsd' for module 'nfsd' (-8)
>>>>>>>   livepatch: patch 'livepatch_nfsd' failed for module 'nfsd', refusing to load module 'nfsd'
>>>>>>>
>>>>>> 3) A selftest would be a good idea.
>>>>>
>>>>
>>>> I found it is pretty tricky to run the selftests inside a qemu VM. How about
>>>> we test it with modules in samples/livepatch? Specifically, we can add a
>>>> script try to reload livepatch-shadow-mod.ko.
>>>
>>> Actually, livepatch-shadow-mod.ko doesn't have the reload problem before
>>> the fix. Is this expected?
>>
>> Good question. I am afraid that there is no easy way to prepare
>> the selftest at the moment.
>>
>> There are two situations when a symbol from the livepatched module is
>> relocated:
>>
>>
>> 1. The livepatch might access a symbol exported by the module via
>>    EXPORT_SYMBOL(). In this case, it is "normal" external symbol
>>    and it gets relocated by the module loader.
>>
>>    But EXPORT_SYMBOL() will create an explicit dependency between the
>>    livepatch and livepatched module. As a result, the livepatch
>>    module could be loaded only when the livepatched module is loaded.
>>    And the livepatched module could not be removed when the livepatch
>>    module is loaded.
>>
>>    In this case, the problem will not exist. Well, the developers
>>    of the livepatch module will probably want to avoid this
>>    dependency.
>>
>>
>> 2. The livepatch module might access a non-exported symbol from another
>>    module using the special elf section for klp relocation, see
>>    section, see Documentation/livepatch/module-elf-format.rst
>>
>>    These symbols are relocated in klp_apply_section_relocs().
>>
>>    The problem is that upstream does not have a support to
>>    create this elf section. There is a patchset for this, see
>>    https://lore.kernel.org/all/20220216163940.228309-1-joe.lawrence@redhat.com/
>>    It requires some more review.
>>
>>
>> Resume: I think that we could not prepare the selftest without
>>         upstreaming klp-convert tool.
> 
> Thanks for the explanation! I suspected the same issue, but couldn't
> connect all the logic.
> 
> I guess the selftests can wait until the klp-convert tool.
> 

Hi Song,

Petr is correct about selftests and these relocations.  Let me know if
rebasing the klp-convert patchset would be helpful in your testing.
Otherwise kpatch-build is the only (easy?) way to create klp-relocations
as far as I know.  (For limited arches anyway.)

Thanks,

-- 
Joe