Re: [PATCH 12/27] blk-iocost: grab ioc->lock for debt handling

[John Garry <john.g.garry@oracle.com>]

On 01/09/2020 19:52, Tejun Heo wrote:

Background:

I have been trying to solve some block/ sparse issues, and it has led me 
to digging up this old mail.

> Currently, debt handling requires only iocg->waitq.lock. In the future, we
> want to adjust and propagate inuse changes depending on debt status. Let's
> grab ioc->lock in debt handling paths in preparation.
> 
> * Because ioc->lock nests outside iocg->waitq.lock, the decision to grab
>    ioc->lock needs to be made before entering the critical sections.
> 
> * Add and use iocg_[un]lock() which handles the conditional double locking.
> 
> * Add @pay_debt to iocg_kick_waitq() so that debt payment happens only when
>    the caller grabbed both locks.
> 
> This patch is prepatory and the comments contain references to future
> changes.
> 
> Signed-off-by: Tejun Heo<tj@kernel.org>
> ---
>   block/blk-iocost.c | 92 ++++++++++++++++++++++++++++++++++++----------
>   1 file changed, 73 insertions(+), 19 deletions(-)
> 
> diff --git a/block/blk-iocost.c b/block/blk-iocost.c
> index f36988657594..23b173e34591 100644
> --- a/block/blk-iocost.c
> +++ b/block/blk-iocost.c
> @@ -680,6 +680,26 @@ static void iocg_commit_bio(struct ioc_gq *iocg, struct bio *bio, u64 cost)
>   	atomic64_add(cost, &iocg->vtime);
>   }
>   
> +static void iocg_lock(struct ioc_gq *iocg, bool lock_ioc, unsigned long *flags)
> +{
> +	if (lock_ioc) {
> +		spin_lock_irqsave(&iocg->ioc->lock, *flags);
> +		spin_lock(&iocg->waitq.lock);
> +	} else {
> +		spin_lock_irqsave(&iocg->waitq.lock, *flags);
> +	}
> +}

This generates the following sparse warnings on mainline today:

   CHECK   block/blk-iocost.c
block/blk-iocost.c:685:9: warning: context imbalance in 'iocg_lock' -
wrong count at exit
block/blk-iocost.c:696:28: warning: context imbalance in 'iocg_unlock'
- unexpected unlock

If we try to break iocg_lock() into one version for lock_ioc set and 
another for lock_ioc unset, we can solve the sparse issues for those 
functions, but then we get another sparse issue from the callsites for 
those functions:

block/blk-iocost.c:2679:17: warning: context imbalance in
'ioc_rqos_throttle' - different lock contexts for basic block

I tried to solve with a total ioc_rqos_throttle() re-org and much code 
duplication by calling the different lock and unlock versions from 
effectively 2x separate copies of ioc_rqos_throttle(), as sparse seems 
confused with how we call these functions. It's a total no-go.

Any simpler idea to solve these? Or just something to live with?

Thanks,
John

> +
> +static void iocg_unlock(struct ioc_gq *iocg, bool unlock_ioc, unsigned long *flags)
> +{
> +	if (unlock_ioc) {
> +		spin_unlock(&iocg->waitq.lock);
> +		spin_unlock_irqrestore(&iocg->ioc->lock, *flags);
> +	} else {
> +		spin_unlock_irqrestore(&iocg->waitq.lock, *flags);
> +	}
> +}