question on read_barrier_depends

question on read_barrier_depends

[Oliver Neukum]

Hi,

I am looking at memory ordering and a question hit me.
I was looking at the kfifo code. kfifo_put() has a barrier:

			)[__kfifo->in & __tmp->kfifo.mask] = \
				(typeof(*__tmp->type))__val; \
			smp_wmb(); \
			__kfifo->in++; \

Looking at kfifo_get() 

		__ret = !kfifo_is_empty(__tmp); \
		if (__ret) { \
			*(typeof(__tmp->type))__val = \
				(__is_kfifo_ptr(__tmp) ? \

A thought struck me. There is no corresponding barrier. I cannot
help myself, but I think there needs to be a smp_read_barrier_depends()
between reading kfifo->in (in kfifo_is empty) and reading val.
What do you think?

	Regards
		Oliver

Re: question on read_barrier_depends

[Alan Stern]

On Wed, 16 Apr 2014, Oliver Neukum wrote:

> Hi,
> 
> I am looking at memory ordering and a question hit me.
> I was looking at the kfifo code. kfifo_put() has a barrier:
> 
> 			)[__kfifo->in & __tmp->kfifo.mask] = \
> 				(typeof(*__tmp->type))__val; \
> 			smp_wmb(); \
> 			__kfifo->in++; \
> 
> Looking at kfifo_get() 
> 
> 		__ret = !kfifo_is_empty(__tmp); \
> 		if (__ret) { \
> 			*(typeof(__tmp->type))__val = \
> 				(__is_kfifo_ptr(__tmp) ? \
> 
> A thought struck me. There is no corresponding barrier. I cannot
> help myself, but I think there needs to be a smp_read_barrier_depends()
> between reading kfifo->in (in kfifo_is empty) and reading val.
> What do you think?

I think you are right.

In addition, the following code in kfifo_get() does this:

			*(typeof(__tmp->type))__val = \
				(__is_kfifo_ptr(__tmp) ? \
				((typeof(__tmp->type))__kfifo->data) : \
				(__tmp->buf) \
				)[__kfifo->out & __tmp->kfifo.mask]; \
			smp_wmb(); \
			__kfifo->out++; \

It looks like the smp_wmb() should really be smp_mb(), because it 
separates the _read_ for val from the _write_ of kfifo->out.

Alan Stern

Re: question on read_barrier_depends

[Oliver Neukum]

On Wed, 2014-04-16 at 11:26 -0400, Alan Stern wrote:

> In addition, the following code in kfifo_get() does this:
> 
> 			*(typeof(__tmp->type))__val = \
> 				(__is_kfifo_ptr(__tmp) ? \
> 				((typeof(__tmp->type))__kfifo->data) : \
> 				(__tmp->buf) \
> 				)[__kfifo->out & __tmp->kfifo.mask]; \
> 			smp_wmb(); \
> 			__kfifo->out++; \
> 
> It looks like the smp_wmb() should really be smp_mb(), because it 
> separates the _read_ for val from the _write_ of kfifo->out.

But where is kfifo->out read at all?

	Regards
		Oliver

Re: question on read_barrier_depends

[Alan Stern]

On Thu, 17 Apr 2014, Oliver Neukum wrote:

> On Wed, 2014-04-16 at 11:26 -0400, Alan Stern wrote:
> 
> > In addition, the following code in kfifo_get() does this:
> > 
> > 			*(typeof(__tmp->type))__val = \
> > 				(__is_kfifo_ptr(__tmp) ? \
> > 				((typeof(__tmp->type))__kfifo->data) : \
> > 				(__tmp->buf) \
> > 				)[__kfifo->out & __tmp->kfifo.mask]; \
> > 			smp_wmb(); \
> > 			__kfifo->out++; \
> > 
> > It looks like the smp_wmb() should really be smp_mb(), because it 
> > separates the _read_ for val from the _write_ of kfifo->out.
> 
> But where is kfifo->out read at all?

It is read in kfifo_put(), inside the call to kfifo_is_full() -> 
kfifo_len().

Alan Stern

Re: question on read_barrier_depends

[Oliver Neukum]

On Thu, 2014-04-17 at 11:50 -0400, Alan Stern wrote:
> On Thu, 17 Apr 2014, Oliver Neukum wrote:
> 
> > On Wed, 2014-04-16 at 11:26 -0400, Alan Stern wrote:
> > 
> > > In addition, the following code in kfifo_get() does this:
> > > 
> > > 			*(typeof(__tmp->type))__val = \
> > > 				(__is_kfifo_ptr(__tmp) ? \
> > > 				((typeof(__tmp->type))__kfifo->data) : \
> > > 				(__tmp->buf) \
> > > 				)[__kfifo->out & __tmp->kfifo.mask]; \
> > > 			smp_wmb(); \
> > > 			__kfifo->out++; \
> > > 
> > > It looks like the smp_wmb() should really be smp_mb(), because it 
> > > separates the _read_ for val from the _write_ of kfifo->out.
> > 
> > But where is kfifo->out read at all?
> 
> It is read in kfifo_put(), inside the call to kfifo_is_full() -> 
> kfifo_len().

I had overlooked that. In that case kfifo_put() also needs
smp_read_barrier_depends()

	Regards
		Oliver

Re: question on read_barrier_depends

[Oliver Neukum]

On Thu, 2014-04-17 at 11:50 -0400, Alan Stern wrote:
> On Thu, 17 Apr 2014, Oliver Neukum wrote:
> 
> > On Wed, 2014-04-16 at 11:26 -0400, Alan Stern wrote:
> > 
> > > In addition, the following code in kfifo_get() does this:
> > > 
> > > 			*(typeof(__tmp->type))__val = \
> > > 				(__is_kfifo_ptr(__tmp) ? \
> > > 				((typeof(__tmp->type))__kfifo->data) : \
> > > 				(__tmp->buf) \
> > > 				)[__kfifo->out & __tmp->kfifo.mask]; \
> > > 			smp_wmb(); \
> > > 			__kfifo->out++; \
> > > 
> > > It looks like the smp_wmb() should really be smp_mb(), because it 
> > > separates the _read_ for val from the _write_ of kfifo->out.
> > 
> > But where is kfifo->out read at all?
> 
> It is read in kfifo_put(), inside the call to kfifo_is_full() -> 
> kfifo_len().

I had overlooked that. In that case kfifo_put() also needs
smp_read_barrier_depends()

	Regards
		Oliver

Re: question on read_barrier_depends

[Alan Stern]

On Fri, 18 Apr 2014, Oliver Neukum wrote:

> On Thu, 2014-04-17 at 11:50 -0400, Alan Stern wrote:
> > On Thu, 17 Apr 2014, Oliver Neukum wrote:
> > 
> > > On Wed, 2014-04-16 at 11:26 -0400, Alan Stern wrote:
> > > 
> > > > In addition, the following code in kfifo_get() does this:
> > > > 
> > > > 			*(typeof(__tmp->type))__val = \
> > > > 				(__is_kfifo_ptr(__tmp) ? \
> > > > 				((typeof(__tmp->type))__kfifo->data) : \
> > > > 				(__tmp->buf) \
> > > > 				)[__kfifo->out & __tmp->kfifo.mask]; \
> > > > 			smp_wmb(); \
> > > > 			__kfifo->out++; \
> > > > 
> > > > It looks like the smp_wmb() should really be smp_mb(), because it 
> > > > separates the _read_ for val from the _write_ of kfifo->out.
> > > 
> > > But where is kfifo->out read at all?
> > 
> > It is read in kfifo_put(), inside the call to kfifo_is_full() -> 
> > kfifo_len().
> 
> I had overlooked that. In that case kfifo_put() also needs
> smp_read_barrier_depends()

I don't think so.

kfifo_put() uses the read of kfifo->out to determine whether to perform
the store; i.e., it won't store the data if the kfifo is already full.  
No architectures do speculative writes, so the data can never get
stored before kfifo->out is read, even without any memory barriers.  

smp_read_barrier_depends() is meant to prevent speculative _reads_, or
any similar mechanism.

Alan Stern

Re: question on read_barrier_depends

[Oliver Neukum]

On Wed, 2014-04-16 at 11:26 -0400, Alan Stern wrote:
> On Wed, 16 Apr 2014, Oliver Neukum wrote:
> 
> > Hi,
> > 
> > I am looking at memory ordering and a question hit me.
> > I was looking at the kfifo code. kfifo_put() has a barrier:
> > 
> > 			)[__kfifo->in & __tmp->kfifo.mask] = \
> > 				(typeof(*__tmp->type))__val; \
> > 			smp_wmb(); \
> > 			__kfifo->in++; \
> > 
> > Looking at kfifo_get() 
> > 
> > 		__ret = !kfifo_is_empty(__tmp); \
> > 		if (__ret) { \
> > 			*(typeof(__tmp->type))__val = \
> > 				(__is_kfifo_ptr(__tmp) ? \
> > 
> > A thought struck me. There is no corresponding barrier. I cannot
> > help myself, but I think there needs to be a smp_read_barrier_depends()
> > between reading kfifo->in (in kfifo_is empty) and reading val.
> > What do you think?
> 
> I think you are right.
> 
> In addition, the following code in kfifo_get() does this:
> 
> 			*(typeof(__tmp->type))__val = \
> 				(__is_kfifo_ptr(__tmp) ? \
> 				((typeof(__tmp->type))__kfifo->data) : \
> 				(__tmp->buf) \
> 				)[__kfifo->out & __tmp->kfifo.mask]; \
> 			smp_wmb(); \
> 			__kfifo->out++; \
> 
> It looks like the smp_wmb() should really be smp_mb(), because it 
> separates the _read_ for val from the _write_ of kfifo->out.

On the third hand, I now think wmb() is sufficient, because
there's also a write to __val. It does depend on the read
of buf[out & mask], but if no CPU does speculative writes
it must be correct.

	Regards
		Oliver

Re: question on read_barrier_depends

[Alan Stern]

On Mon, 21 Apr 2014, Oliver Neukum wrote:

> > In addition, the following code in kfifo_get() does this:
> > 
> > 			*(typeof(__tmp->type))__val = \
> > 				(__is_kfifo_ptr(__tmp) ? \
> > 				((typeof(__tmp->type))__kfifo->data) : \
> > 				(__tmp->buf) \
> > 				)[__kfifo->out & __tmp->kfifo.mask]; \
> > 			smp_wmb(); \
> > 			__kfifo->out++; \
> > 
> > It looks like the smp_wmb() should really be smp_mb(), because it 
> > separates the _read_ for val from the _write_ of kfifo->out.
> 
> On the third hand, I now think wmb() is sufficient, because
> there's also a write to __val. It does depend on the read
> of buf[out & mask], but if no CPU does speculative writes
> it must be correct.

You are right; I missed that.  Good point.

Alan Stern