Re: [PATCH] libcxgb: use raw_cpu_ptr() in ppm_get_cpu_entries()

[netdev-bot+sashiko@kernel.org]

Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 1

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [High] The patch introduces a cpu/pool mismatch race in
  ppm_get_cpu_entries()…
- [Low] The commit is a bug fix for a user-visible 'BUG: sleeping
  function called from invalid context' on PREEMPT_RT.

--

libcxgb: use raw_cpu_ptr() in ppm_get_cpu_entries()

This patch changes ppm_get_cpu_entries(). It drops get_cpu(),
per_cpu_ptr() and put_cpu() and uses raw_smp_processor_id() and
raw_cpu_ptr() instead. The aim is to stop spin_lock_bh() from being taken
with preemption disabled, which triggers a sleeping-in-atomic BUG on
PREEMPT_RT.

> Signed-off-by: Haoran Wang <haoranwangsec@gmail.com>

[Severity: Low]
This fixes a user-visible BUG on PREEMPT_RT, but the commit has no Fixes:
tag. The get_cpu()/per_cpu_ptr()/spin_lock_bh()/put_cpu() sequence has
not changed since it was added in:

Fixes: 47da7e62b391 ("cxgb4: add iSCSI DDP page pod manager")

Should that tag be added, possibly with a Cc: stable?

> diff --git a/drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c b/drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c
> index 2e3973a32d9d..5f68c5d5766e 100644
> --- a/drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c
> +++ b/drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c
> @@ -125,10 +125,9 @@ static int ppm_get_cpu_entries(struct cxgbi_ppm *ppm, unsigned int count,
>  	if (!ppm->pool)
>  		return -EINVAL;
>  
> -	cpu = get_cpu();
> -	pool = per_cpu_ptr(ppm->pool, cpu);
> +	cpu = raw_smp_processor_id();
> +	pool = raw_cpu_ptr(ppm->pool);

[Severity: High]
Can cpu and pool refer to different CPUs here?

The new code reads the current CPU twice, with preemption and migration
both enabled. Say the task moves from CPU A to CPU B between
raw_smp_processor_id() and raw_cpu_ptr(). The bit is then reserved in CPU
B's bitmap under CPU B's lock, but the returned index is built from the
old cpu value:

drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c:ppm_get_cpu_entries() {
    ...
	i += cpu * ppm->pool_index_max;
	ppm_mark_entries(ppm, i, count, caller_data);
    ...
}

The index and the ppod_data[] entry now point into CPU A's slice, while
bit i in CPU A's bitmap is still clear.

Could a task on CPU A then get the same index, hardware page pod and
ppod_data slot while the first transfer is still in flight?

cxgbi_ppm_ppods_reserve() turns this index into hwidx and the DDP tag.
cxgbit_ddp_reserve() and cxgbi_ddp_reserve() then program that tag into
the adapter. The caller_data in ppod_data[] is also what
cxgbi_parse_pdu_itt() uses later to match responses to tasks.

On release, ppm_unmark_entries() gets the pool back from the index:

	cpu = i / ppm->pool_index_max;
	i %= ppm->pool_index_max;

	pool = per_cpu_ptr(ppm->pool, cpu);
	spin_lock_bh(&pool->lock);
	bitmap_clear(pool->bmap, i, count);

That would clear the bit in CPU A's pool, which may free another
transfer's live entry. The bit that was really set in CPU B's pool would
never be cleared.

One preemptible caller that holds no spinlock is the iSCSI target TX
path:

iscsit_send_r2t()
  cxgbit_get_r2t_ttt()
    cxgbit_ddp_reserve()
      cxgbi_ppm_ppods_reserve()
        ppm_get_cpu_entries()

iscsit_get_r2t_from_list() drops r2t_lock before it returns, so this
whole chain runs unlocked. A remote initiator can drive it simply by
sending WRITE commands to a Chelsio-offloaded target.

The raw_ accessors are not checked by CONFIG_DEBUG_PREEMPT, so nothing
would warn. The race exists on any SMP kernel built with CONFIG_PREEMPT,
PREEMPT_LAZY or PREEMPT_RT.

Given that, do these parts of the commit message still hold?

    The preemption disable exists only to stabilize the per-CPU accessor,
    the pool is serialized by pool->lock, not by preemption.

    ...the defect is PREEMPT_RT-specific.

The returned index encodes cpu, so cpu and pool still have to match even
though pool->lock serializes the bitmap.

Would it work to keep the saved cpu and derive the pool from it? For
example:

	cpu = raw_smp_processor_id();
	pool = per_cpu_ptr(ppm->pool, cpu);

Another option would be to wrap the lookup in
migrate_disable()/migrate_enable().

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/179241614797.5916.13928836073297916808%40whr