Re: [PATCH v41 12/24] x86/sgx: Add SGX_IOC_ENCLAVE_CREATE

[Dave Hansen <dave.hansen@intel.com>]

On 11/16/20 9:54 AM, Dave Hansen wrote:
>> ENCLS instructions must be serialized for a given enclave, but holding
>> encl->lock for an entire ioctl() will result in deadlock due to an enclave
>> triggering reclaim on itself.
>>
>> Building an enclave must also be serialized, i.e. userspace can't queue up
>> EADD on multiple threads, because the order in which pages are added to an
>> enclave affects the measurement.  In other words, rejecting the ioctl() as
>> opposed to waiting on a lock is also desirable.
> Sounds like we need should follow up with an add-on patch to get some of
> that into a comment.

Jarkko, first of all, let's rename:

	SGX_ENCL_IOCTL -> SGX_ENCL_IOCTL_LOCK

If it walks like a duck and quacks like a duck...

Sean had a good example of examples of how EADD could go wrong with
multiple threads.  Were there more good examples we could stick in a
changelog?  I seem to recall that there are more than a few SGX
instructions don't even work in parallel and require software
synchronization.  Could we get a list or at least a few more good examples?

I also think we should be much more assertive about multiple ioctl()
callers:

	/* Multi-threaded enclave management is invalid and unsafe: */
	if (test_and_set_bit(SGX_ENCL_IOCTL_LOCK, &encl->flags))
		return -EINVAL;

-EBUSY is saying "everything is OK, just busy, please try again later."
 -EINVAL is saying, "userspace, you screwed up".

Also, does SGX_ENCL_IOCTL_LOCK provide serialization for anything other
than the *hardware* instructions?  I couldn't find much, although:

        encl->attributes_mask |= SGX_ATTR_PROVISIONKEY;

seems to be lacking any other serialization.

sgx_encl_create() also seems like it has no other locking and relies on
SGX_ENCL_IOCTL_LOCK for sanity.