Re: pt_regs->ax == -ENOSYS

[Kees Cook <keescook@chromium.org>]

On Tue, Apr 27, 2021 at 04:51:06PM -0700, Andy Lutomirski wrote:
> Fortunately there is not, and never will be, a syscall -1.  But I
> agree that calling max syscall + 1 should behave identically to calling
> a nonexistent syscall in the middle of the table.

If that happens, we have to separate the meaning of -1L from ptrace,
seccomp, etc. (i.e. we can't just add an "else { result = -ENOSYS; }" to
the syscall table dispatching code, since that'll overwrite any written
return value when the syscall is meant to be skipped with a specific
return value set by ptrace/seccomp.

syscall_trace_enter() will currently return either -1 or the
syscall. Which means someone making a "syscall -1" will get the skip
semantics currently (though the preloaded -ENOSYS results in the
"expected" outcome).

arm64 recently had to untangle this too:

15956689a0e6 arm64: compat: Ensure upper 32 bits of x0 are zero on syscall return
59ee987ea47c arm64: ptrace: Add a comment describing our syscall entry/exit trap ABI
139dbe5d8ed3 arm64: syscall: Expand the comment about ptrace and syscall(-1)
d83ee6e3e75d arm64: ptrace: Use NO_SYSCALL instead of -1 in syscall_trace_enter()

-- 
Kees Cook