Detecting RWF_NOWAIT support

Detecting RWF_NOWAIT support

[Avi Kivity]

I'm looking to add support for RWF_NOWAIT within a linux-aio iocb. 
Naturally, I need to detect at runtime whether the kernel support 
RWF_NOWAIT or not.


The only method I could find was to issue an I/O with RWF_NOWAIT set, 
and look for errors. This is somewhat less than perfect:

  - from the error, I can't tell whether RWF_NOWAIT was the problem, or 
something else. If I enable a number of new features, I have to run 
through all combinations to figure out which ones are supported and 
which are not.

  - RWF_NOWAIT support is per-filesystem, so I can't just remember not 
to enable RWF_NOWAIT globally, I have to track it per file.


Did I miss another method of detecting RWF_NOWAIT?

Re: Detecting RWF_NOWAIT support

[Goldwyn Rodrigues]

On 12/14/2017 11:38 AM, Avi Kivity wrote:
> I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> Naturally, I need to detect at runtime whether the kernel support
> RWF_NOWAIT or not.
> 
> 
> The only method I could find was to issue an I/O with RWF_NOWAIT set,
> and look for errors. This is somewhat less than perfect:
> 
>  - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> something else. If I enable a number of new features, I have to run
> through all combinations to figure out which ones are supported and
> which are not.

Here is the return codes for RWF_NOWAIT
EINVAL - not supported (older kernel)
EOPNOTSUPP - not supported
EAGAIN - supported but could not complete because I/O will be delayed
0 - supported and I/O completed (success).

> 
>  - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
> enable RWF_NOWAIT globally, I have to track it per file.

Yes, the support is per filesystem. So, the application must know if the
filesystem supports it, possibly by performing a small I/O.

-- 
Goldwyn

Re: Detecting RWF_NOWAIT support

[Avi Kivity]

On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
>
> On 12/14/2017 11:38 AM, Avi Kivity wrote:
>> I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
>> Naturally, I need to detect at runtime whether the kernel support
>> RWF_NOWAIT or not.
>>
>>
>> The only method I could find was to issue an I/O with RWF_NOWAIT set,
>> and look for errors. This is somewhat less than perfect:
>>
>>   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
>> something else. If I enable a number of new features, I have to run
>> through all combinations to figure out which ones are supported and
>> which are not.
> Here is the return codes for RWF_NOWAIT
> EINVAL - not supported (older kernel)
> EOPNOTSUPP - not supported
> EAGAIN - supported but could not complete because I/O will be delayed

Which of these are returned from io_submit() and which are returned in 
the iocb?

> 0 - supported and I/O completed (success).
>
>>   - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
>> enable RWF_NOWAIT globally, I have to track it per file.
> Yes, the support is per filesystem. So, the application must know if the
> filesystem supports it, possibly by performing a small I/O.

So the application must know about filesystem mount points, and be 
prepared to create a file and try to write it (in case the filesystem is 
empty) or alter its behavior during runtime depending on the errors it sees.

Re: Detecting RWF_NOWAIT support

[vcaputo]

On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
> 
> 
> On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
> > 
> > On 12/14/2017 11:38 AM, Avi Kivity wrote:
> > > I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> > > Naturally, I need to detect at runtime whether the kernel support
> > > RWF_NOWAIT or not.
> > > 
> > > 
> > > The only method I could find was to issue an I/O with RWF_NOWAIT set,
> > > and look for errors. This is somewhat less than perfect:
> > > 
> > >   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> > > something else. If I enable a number of new features, I have to run
> > > through all combinations to figure out which ones are supported and
> > > which are not.
> > Here is the return codes for RWF_NOWAIT
> > EINVAL - not supported (older kernel)
> > EOPNOTSUPP - not supported
> > EAGAIN - supported but could not complete because I/O will be delayed
> 
> Which of these are returned from io_submit() and which are returned in the
> iocb?
> 
> > 0 - supported and I/O completed (success).
> > 
> > >   - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
> > > enable RWF_NOWAIT globally, I have to track it per file.
> > Yes, the support is per filesystem. So, the application must know if the
> > filesystem supports it, possibly by performing a small I/O.
> 
> So the application must know about filesystem mount points, and be prepared
> to create a file and try to write it (in case the filesystem is empty) or
> alter its behavior during runtime depending on the errors it sees.

Can't the application simply add a "nowait" flag to its open file
descriptor encapsulation struct, then in the constructor perform a
zero-length RWF_NOWAIT write immediately after opening the fd to set the
flag?  Then all writes branch according to the flag.

According to write(2):

       If  count is zero and fd refers to a regular file, then write()
       may return a failure status if one of the errors below is
       detected.  If no errors are detected,  or error detection is not
       performed, 0 will be returned without causing any other effect.
       If count is zero and fd refers to a file  other than a regular
       file, the results are not specified.

So the zero-length RWF_NOWAIT write should return zero, unless it's not
supported.

Regards,
Vito Caputo

Re: Detecting RWF_NOWAIT support

[vcaputo]

On Sat, Dec 16, 2017 at 10:03:38AM -0800, vcaputo@pengaru.com wrote:
> On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
> > 
> > 
> > On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
> > > 
> > > On 12/14/2017 11:38 AM, Avi Kivity wrote:
> > > > I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> > > > Naturally, I need to detect at runtime whether the kernel support
> > > > RWF_NOWAIT or not.
> > > > 
> > > > 
> > > > The only method I could find was to issue an I/O with RWF_NOWAIT set,
> > > > and look for errors. This is somewhat less than perfect:
> > > > 
> > > >   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> > > > something else. If I enable a number of new features, I have to run
> > > > through all combinations to figure out which ones are supported and
> > > > which are not.
> > > Here is the return codes for RWF_NOWAIT
> > > EINVAL - not supported (older kernel)
> > > EOPNOTSUPP - not supported
> > > EAGAIN - supported but could not complete because I/O will be delayed
> > 
> > Which of these are returned from io_submit() and which are returned in the
> > iocb?
> > 
> > > 0 - supported and I/O completed (success).
> > > 
> > > >   - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
> > > > enable RWF_NOWAIT globally, I have to track it per file.
> > > Yes, the support is per filesystem. So, the application must know if the
> > > filesystem supports it, possibly by performing a small I/O.
> > 
> > So the application must know about filesystem mount points, and be prepared
> > to create a file and try to write it (in case the filesystem is empty) or
> > alter its behavior during runtime depending on the errors it sees.
> 
> Can't the application simply add a "nowait" flag to its open file
> descriptor encapsulation struct, then in the constructor perform a
> zero-length RWF_NOWAIT write immediately after opening the fd to set the
> flag?  Then all writes branch according to the flag.
> 
> According to write(2):
> 
>        If  count is zero and fd refers to a regular file, then write()
>        may return a failure status if one of the errors below is
>        detected.  If no errors are detected,  or error detection is not
>        performed, 0 will be returned without causing any other effect.
>        If count is zero and fd refers to a file  other than a regular
>        file, the results are not specified.
> 
> So the zero-length RWF_NOWAIT write should return zero, unless it's not
> supported.
> 

Oh, I assumed this new flag applied to pwritev2() flags.  Disregard my
comment, I see the ambiguity causing your question Avi and do not know
the best approach.

Cheers,
Vito Caputo

Re: Detecting RWF_NOWAIT support

[Avi Kivity]

On 12/16/2017 08:12 PM, vcaputo@pengaru.com wrote:
> On Sat, Dec 16, 2017 at 10:03:38AM -0800, vcaputo@pengaru.com wrote:
>> On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
>>>
>>> On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
>>>> On 12/14/2017 11:38 AM, Avi Kivity wrote:
>>>>> I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
>>>>> Naturally, I need to detect at runtime whether the kernel support
>>>>> RWF_NOWAIT or not.
>>>>>
>>>>>
>>>>> The only method I could find was to issue an I/O with RWF_NOWAIT set,
>>>>> and look for errors. This is somewhat less than perfect:
>>>>>
>>>>>    - from the error, I can't tell whether RWF_NOWAIT was the problem, or
>>>>> something else. If I enable a number of new features, I have to run
>>>>> through all combinations to figure out which ones are supported and
>>>>> which are not.
>>>> Here is the return codes for RWF_NOWAIT
>>>> EINVAL - not supported (older kernel)
>>>> EOPNOTSUPP - not supported
>>>> EAGAIN - supported but could not complete because I/O will be delayed
>>> Which of these are returned from io_submit() and which are returned in the
>>> iocb?
>>>
>>>> 0 - supported and I/O completed (success).
>>>>
>>>>>    - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
>>>>> enable RWF_NOWAIT globally, I have to track it per file.
>>>> Yes, the support is per filesystem. So, the application must know if the
>>>> filesystem supports it, possibly by performing a small I/O.
>>> So the application must know about filesystem mount points, and be prepared
>>> to create a file and try to write it (in case the filesystem is empty) or
>>> alter its behavior during runtime depending on the errors it sees.
>> Can't the application simply add a "nowait" flag to its open file
>> descriptor encapsulation struct, then in the constructor perform a
>> zero-length RWF_NOWAIT write immediately after opening the fd to set the
>> flag?  Then all writes branch according to the flag.
>>
>> According to write(2):
>>
>>         If  count is zero and fd refers to a regular file, then write()
>>         may return a failure status if one of the errors below is
>>         detected.  If no errors are detected,  or error detection is not
>>         performed, 0 will be returned without causing any other effect.
>>         If count is zero and fd refers to a file  other than a regular
>>         file, the results are not specified.
>>
>> So the zero-length RWF_NOWAIT write should return zero, unless it's not
>> supported.
>>
> Oh, I assumed this new flag applied to pwritev2() flags.  Disregard my
> comment, I see the ambiguity causing your question Avi and do not know
> the best approach.
>

Actually it's not a bad idea. I'm using AIO, not p{read,write}v2, but I 
can assume that the response will be the same and that a zero-length 
read will return immediately.

Re: Detecting RWF_NOWAIT support

[Goldwyn Rodrigues]

On 12/16/2017 08:49 AM, Avi Kivity wrote:
> 
> 
> On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
>>
>> On 12/14/2017 11:38 AM, Avi Kivity wrote:
>>> I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
>>> Naturally, I need to detect at runtime whether the kernel support
>>> RWF_NOWAIT or not.
>>>
>>>
>>> The only method I could find was to issue an I/O with RWF_NOWAIT set,
>>> and look for errors. This is somewhat less than perfect:
>>>
>>>   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
>>> something else. If I enable a number of new features, I have to run
>>> through all combinations to figure out which ones are supported and
>>> which are not.
>> Here is the return codes for RWF_NOWAIT
>> EINVAL - not supported (older kernel)
>> EOPNOTSUPP - not supported
>> EAGAIN - supported but could not complete because I/O will be delayed
> 
> Which of these are returned from io_submit() and which are returned in
> the iocb?

These are returned in iocb.

> 
>> 0 - supported and I/O completed (success).
>>
>>>   - RWF_NOWAIT support is per-filesystem, so I can't just remember
>>> not to
>>> enable RWF_NOWAIT globally, I have to track it per file.
>> Yes, the support is per filesystem. So, the application must know if the
>> filesystem supports it, possibly by performing a small I/O.
> 
> So the application must know about filesystem mount points, and be
> prepared to create a file and try to write it (in case the filesystem is
> empty) or alter its behavior during runtime depending on the errors it
> sees.

Well yes. Hopefully, the application knows what it is doing when it
performs RWF_NOWAIT.

-- 
Goldwyn

Re: Detecting RWF_NOWAIT support

[Avi Kivity]

On 12/18/2017 05:28 AM, Goldwyn Rodrigues wrote:
>
> On 12/16/2017 08:49 AM, Avi Kivity wrote:
>>
>> On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
>>> On 12/14/2017 11:38 AM, Avi Kivity wrote:
>>>> I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
>>>> Naturally, I need to detect at runtime whether the kernel support
>>>> RWF_NOWAIT or not.
>>>>
>>>>
>>>> The only method I could find was to issue an I/O with RWF_NOWAIT set,
>>>> and look for errors. This is somewhat less than perfect:
>>>>
>>>>    - from the error, I can't tell whether RWF_NOWAIT was the problem, or
>>>> something else. If I enable a number of new features, I have to run
>>>> through all combinations to figure out which ones are supported and
>>>> which are not.
>>> Here is the return codes for RWF_NOWAIT
>>> EINVAL - not supported (older kernel)
>>> EOPNOTSUPP - not supported
>>> EAGAIN - supported but could not complete because I/O will be delayed
>> Which of these are returned from io_submit() and which are returned in
>> the iocb?
> These are returned in iocb.

Thanks.

>
>>> 0 - supported and I/O completed (success).
>>>
>>>>    - RWF_NOWAIT support is per-filesystem, so I can't just remember
>>>> not to
>>>> enable RWF_NOWAIT globally, I have to track it per file.
>>> Yes, the support is per filesystem. So, the application must know if the
>>> filesystem supports it, possibly by performing a small I/O.
>> So the application must know about filesystem mount points, and be
>> prepared to create a file and try to write it (in case the filesystem is
>> empty) or alter its behavior during runtime depending on the errors it
>> sees.
> Well yes. Hopefully, the application knows what it is doing when it
> performs RWF_NOWAIT.

This type of interface makes it very hard to consume new kernel 
facilities in a backward compatible way. The kernel should advertise 
what support it provides; for example this support could be advertised 
via statx(2).

For examples of facilities that advertise their capabilities, see 
membarrier(2) and KVM.