mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Nicholas A. Bellinger" <nab@linux-iscsi.org>
To: Christoph Hellwig <hch@lst.de>
Cc: "Nicholas A. Bellinger" <nab@daterainc.com>,
	target-devel <target-devel@vger.kernel.org>,
	linux-scsi <linux-scsi@vger.kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Hannes Reinecke <hare@suse.de>,
	Sagi Grimberg <sagig@mellanox.com>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: Re: [PATCH 01/12] target: Convert se_node_acl->device_list[] to RCU hlist
Date: Tue, 12 May 2015 23:20:12 -0700	[thread overview]
Message-ID: <1431498012.26897.61.camel@haakon3.risingtidesystems.com> (raw)
In-Reply-To: <20150513054646.GA20825@lst.de>

On Wed, 2015-05-13 at 07:46 +0200, Christoph Hellwig wrote:
> On Tue, May 12, 2015 at 09:25:25AM +0000, Nicholas A. Bellinger wrote:
> > @@ -240,18 +237,12 @@ int core_free_device_list_for_node(
> >  {
> >  	struct se_dev_entry *deve;
> >  	struct se_lun *lun;
> > -	u32 i;
> > -
> > -	if (!nacl->device_list)
> > -		return 0;
> > -
> > -	spin_lock_irq(&nacl->device_list_lock);
> > -	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
> > -		deve = nacl->device_list[i];
> > +	u32 mapped_lun;
> >  
> > +	rcu_read_lock();
> > +	hlist_for_each_entry_rcu(deve, &nacl->lun_entry_hlist, link) {
> >  		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS))
> >  			continue;
> > -
> >  		if (!deve->se_lun) {
> >  			pr_err("%s device entries device pointer is"
> >  				" NULL, but Initiator has access.\n",
> > @@ -259,16 +250,14 @@ int core_free_device_list_for_node(
> >  			continue;
> >  		}
> >  		lun = deve->se_lun;
> > +		mapped_lun = deve->mapped_lun;
> > +		rcu_read_unlock();
> >  
> > -		spin_unlock_irq(&nacl->device_list_lock);
> > -		core_disable_device_list_for_node(lun, NULL, deve->mapped_lun,
> > -			TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
> > -		spin_lock_irq(&nacl->device_list_lock);
> > +		core_disable_device_list_for_node(lun, NULL, mapped_lun,
> > +					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
> 
> I don't think this change is a good idea.  Now that you've just switched
> to a list call into core_disable_device_list_for_node with the lock
> instead of retaking it and restart the list walk after it instead of
> encoding the previous wrong behavior with the local mapped_lun
> variable.  Note that this patter is the same for all all but one of the
> callers, and even core_dev_del_initiator_node_lun_acl would benefit
> from being called locked and with an already looked up dev entry.
> 

Ugh, yes.  Fixing up clear_lun_from_tpg + free_device_list_for_node to
use a common caller acquiring se_node_acl->lun_entry_mutex during
se_dev_entry release.

Fixing up target_fabric_mappedlun_unlink() as well.

> Note that if you cherry picked this patch I posted a while ago
> to be before the series one of the callers would already be gone:
> 
> http://git.infradead.org/users/hch/scsi.git/commitdiff/dfb7096ba5ea47cb5b7fb5b6e2f8d7d6436af24f
> 
> > +	spin_lock_irq(&nacl->lun_entry_lock);
> > +	deve = target_nacl_find_deve(nacl, mapped_lun);
> > +	if (deve) {
> > +		if (lun_access & TRANSPORT_LUNFLAGS_READ_WRITE) {
> > +			deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_ONLY;
> > +			deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_WRITE;
> > +		} else {
> > +			deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_WRITE;
> > +			deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_ONLY;
> > +		}
> >  	}
> > -	spin_unlock_irq(&nacl->device_list_lock);
> > +	spin_unlock_irq(&nacl->lun_entry_lock);
> > +
> > +	synchronize_rcu();
> 
> This only updates scalar fields, the synchronize_rcu() calls isn't
> going to buy you anything.
> 
> Btw, it would be good to always document what a synchronize_rcu()
> call code is for.

<nod>, dropping synchronize_rcu() here

> 
> > +
> > +static void target_nacl_deve_callrcu(struct rcu_head *head)
> > +{
> > +	struct se_dev_entry *deve = container_of(head, struct se_dev_entry,
> > +						 rcu_head);
> > +	kfree(deve);
> >  }
> 
> Just use kfree_rcu instead of open coding it.
> 

Done

> > +/*
> > + * Called with rcu_read_lock or nacl->device_list_lock held.
> > + */
> 
> It would be good to assert that.  Paul, is there a good way to assert
> we're called under rcu_read_lock?
> 
> > +	spin_lock_irq(&nacl->device_list_lock);
> > +	orig = target_nacl_find_deve(nacl, mapped_lun);
> > +	if (orig && orig->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
> > +		BUG_ON(orig->se_lun_acl != NULL);
> > +		BUG_ON(orig->se_lun != lun);
> > +
> > +		rcu_assign_pointer(new->se_lun, lun);
> > +		rcu_assign_pointer(new->se_lun_acl, lun_acl);
> > +		hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
> >  		spin_unlock_irq(&nacl->device_list_lock);
> > +		spin_lock_bh(&port->sep_alua_lock);
> > +		list_del(&orig->alua_port_list);
> > +		list_add_tail(&new->alua_port_list, &port->sep_alua_list);
> > +		spin_unlock_bh(&port->sep_alua_lock);
> >  
> > +		return 0;
> >  	}
> 
> The case where we have an original one is the demo mode -> explicit
> change.  So I don't think we actually need the newly allocate dev
> entry here.  Just change lun_flags like in core_update_device_list_access
> and do an rcu_assign_pointer for the lun ACLs.

Will take a look at this.

> 
> > -	deve->creation_time = get_jiffies_64();
> > -	deve->attach_count++;
> > +	rcu_assign_pointer(new->se_lun, lun);
> > +	rcu_assign_pointer(new->se_lun_acl, lun_acl);
> > +	hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
> >  	spin_unlock_irq(&nacl->device_list_lock);
> >  
> >  	spin_lock_bh(&port->sep_alua_lock);
> > -	list_add_tail(&deve->alua_port_list, &port->sep_alua_list);
> > +	list_add_tail(&new->alua_port_list, &port->sep_alua_list);
> >  	spin_unlock_bh(&port->sep_alua_lock);
> >  
> > +	synchronize_rcu();
> 
> Please add a comment why we need the synchronize_rcu here again.  Nothing
> is delete from any list, and nothing is freed so I don't see any need
> to wait for a grace period.
> 

I don't think it's required either.  Dropping.

> > +	core_scsi3_ua_release_all(orig);
> > +	rcu_assign_pointer(orig->se_lun, NULL);
> > +	rcu_assign_pointer(orig->se_lun_acl, NULL);
> 
> Can you document the life time rules that ensure ->se_lun and ->se_lun_acl
> stay around while readers in the RCU grace period may still access them?

Will do.

Thanks HCH.

--nab


  reply	other threads:[~2015-05-13  6:20 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-05-12  9:25 [PATCH 00/12] target: TPG/NodeACL LUN table conversion " Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 01/12] target: Convert se_node_acl->device_list[] " Nicholas A. Bellinger
2015-05-12 20:58   ` Andy Grover
2015-05-13  5:08     ` Nicholas A. Bellinger
2015-05-13  5:32       ` Christoph Hellwig
2015-05-13  5:41         ` Nicholas A. Bellinger
2015-05-13  5:46   ` Christoph Hellwig
2015-05-13  6:20     ` Nicholas A. Bellinger [this message]
2015-05-13  6:48       ` Christoph Hellwig
2015-05-13  6:35   ` Christoph Hellwig
2015-05-13  8:46     ` Nicholas A. Bellinger
2015-05-17 16:51       ` Christoph Hellwig
2015-05-18  7:17         ` Nicholas A. Bellinger
2015-05-18  7:41           ` Christoph Hellwig
2015-05-18  8:01             ` Christoph Hellwig
2015-05-19  6:05               ` Nicholas A. Bellinger
2015-05-19  6:22                 ` Christoph Hellwig
2015-05-21 17:03                   ` Christoph Hellwig
2015-05-21 18:10                     ` Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 02/12] target: Convert REPORT_LUN + MODE_SENSE to RCU reader Nicholas A. Bellinger
2015-05-13  5:47   ` Christoph Hellwig
2015-05-13  8:10     ` Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 03/12] target/configfs: Convert mappedlun + SCSI MIBs " Nicholas A. Bellinger
2015-05-12 20:58   ` Andy Grover
2015-05-13  5:09     ` Nicholas A. Bellinger
2015-05-13  5:49       ` Christoph Hellwig
2015-05-12  9:25 ` [PATCH 04/12] target: Convert UNIT_ATTENTION logic " Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 05/12] target: Convert transport_lookup_*_lun " Nicholas A. Bellinger
2015-05-13  5:55   ` Christoph Hellwig
2015-05-13  7:42     ` Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 06/12] target/pr: Convert se_dev_entry to kref for RCU Nicholas A. Bellinger
2015-05-13  5:59   ` Christoph Hellwig
2015-05-12  9:25 ` [PATCH 07/12] target/pr: Convert registration check to RCU pointer Nicholas A. Bellinger
2015-05-13  6:13   ` Christoph Hellwig
2015-05-12  9:25 ` [PATCH 08/12] target/pr: Change alloc_registration to avoid pr_reg_tg_pt_lun Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 09/12] target: Convert se_portal_group->tpg_lun_list[] to RCU hlist Nicholas A. Bellinger
2015-05-13  6:24   ` Christoph Hellwig
2015-05-13  7:22     ` Juergen Gross
2015-05-13  7:53       ` Christoph Hellwig
2015-05-19  6:46   ` Christoph Hellwig
2015-05-12  9:25 ` [PATCH 10/12] target: Convert se_tpg->acl_node_lock to ->acl_node_mutex Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 11/12] target: Convert core_tpg_deregister to use list splice Nicholas A. Bellinger
2015-05-12  9:25 ` [PATCH 12/12] target: Drop unused se_lun->lun_acl_list Nicholas A. Bellinger
2015-05-13  6:29 ` [PATCH 00/12] target: TPG/NodeACL LUN table conversion to RCU hlist Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1431498012.26897.61.camel@haakon3.risingtidesystems.com \
    --to=nab@linux-iscsi.org \
    --cc=hare@suse.de \
    --cc=hch@lst.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=nab@daterainc.com \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=sagig@mellanox.com \
    --cc=target-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®