Re: memory barrier question

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Thu, 2010-09-16 at 19:17 +0200, Miklos Szeredi wrote:
> On Thu, 16 Sep 2010, James Bottomley wrote:
> > On Thu, 2010-09-16 at 18:56 +0200, Miklos Szeredi wrote:
> > > On Thu, 16 Sep 2010, Paul E. McKenney wrote:
> > > > On Thu, Sep 16, 2010 at 06:06:53PM +0200, Miklos Szeredi wrote:
> > > > > On Thu, 16 Sep 2010, Paul E. McKenney wrote:
> > > > > > On Thu, Sep 16, 2010 at 03:30:56PM +0100, David Howells wrote:
> > > > > > > Miklos Szeredi <miklos@szeredi.hu> wrote:
> > > > > > > 
> > > > > > > > Is the rmb() really needed?
> > > > > > > > 
> > > > > > > > Take this code from fs/namei.c for example:
> > > > > > > > 
> > > > > > > > 		inode = next.dentry->d_inode;
> > > > > > > > 		if (!inode)
> > > > > > > > 			goto out_dput;
> > > > > > > > 
> > > > > > > > 		if (inode->i_op->follow_link) {
> > > > > > > > 
> > > > > > > > It happily dereferences dentry->d_inode without a barrier after
> > > > > > > > checking it for non-null, while that d_inode might have just been
> > > > > > > > initialized on another CPU with a freshly created inode.  There's
> > > > > > > > absolutely no synchornization with that on this side.
> > > > > > > 
> > > > > > > Perhaps it's not necessary; once set, how likely is i_op to be changed once
> > > > > > > I_NEW is cleared?
> > > > > > 
> > > > > > Are the path_get()s protecting this?
> > > > > 
> > > > > No, when creating a file the dentry will go from negative to positive
> > > > > independently from lookup.  The dentry can get instantiated with an
> > > > > inode between the path_get() and dereferencing ->d_inode.
> > > > > 
> > > > > > If there is no protection, then something like rcu_dereference() is
> > > > > > needed for the assignment from next.dentry->d_inode.
> > > > > 
> > > > > Do I understand correctly that the problem is that a CPU may have a
> > > > > stale cache associated with *inode, one that was loaded before the
> > > > > write barrier took effect?
> > > > 
> > > > Yes, especially if the compiler is aggressively optimizing.
> > > 
> > > How do compiler optimizations make a difference?
> > 
> > There are two types of reorderings that cause problems if you expect the
> > bus visible ordering to matter.  One is CPU issue reordering, where the
> > cpu decides to output loads and stores in a different order than the
> > input instruction stream actually said.  The other is compiler
> > re-ordering where the compiler actually reorders the instructions to
> > execute in a different order from what you'd expect by simply reading
> > the C code.
> > 
> > We have compiler barrier instructions for the latter and barriers which
> > issue CPU primitives for the former.
> 
> Right but in the concrete namei example I can't see how a compiler
> optimization can make a difference.  The order of the loads is quite
> clear:
> 
>    LOAD inode = next.dentry->inode
>    if (inode != NULL)
>    	LOAD inode->f_op
> 
> What is there the compiler can optimize?

In this example, it can't the if is conditional on the previous
executions, so even a speculating CPU is required to do ordering  If the
compiler could prove inode wasn't null (which it can't I think in this
case) it might then re-order.  in your original question, there were
ways the compiler could behave strangely (for instance, it would likely
start by initialising p to &x rather than NULL).

James