[RFC][PATCH 0/2] x86: Unify/extend insn_is_nop()

[Peter Zijlstra <peterz@infradead.org>]

Hi all, welcome to an x86 instruction encoding quest.

I realized I implemented insn_is_nop() twice, obviously they have different bugs.
Additionally, I was made aware of another uprobe use-case.

As such, unify and extend insn_is_nop().

While doing this, I went down the rabbit hole of REX2/*VEX* encoded
instructions (because REX2.b4 NOP). The Intel APX document is clear as mud on
what exactly VEX/EVEX Map0 is, but the way I read it, it is the normal single
byte opcode map, no exceptions. Similarly Map1 is the 0f second byte opcode
map. So VEX prefixed NOPs should just work.

REX2 has a similar Map0/Map1 scheme, except it excludes a number of opcode
ranges -- notable not including NOP/NOPL. So REX2 prefixed NOPs are fine too.

Because I had to figure it all out anyway, I've made the function deal with
them, but perhaps the more prudent option is to simply reject all REX2/VEX
prefixed instructions entirely.

(although EVEX NOPL can hit 11 bytes without additional prefixes, so maybe
someone will start using them at some point).