ppm_get_cpu_entries() must not acquire a sleeping lock while preemption
is disabled: get_cpu() raises preempt_count and spin_lock_bh() may sleep
on PREEMPT_RT kernels.  Read the current CPU with raw_cpu_ptr() instead,
which keeps the per-CPU accessor stable without disabling preemption.

Using the pool of a CPU the thread has since migrated away from is
harmless: the per-CPU pools are shards of the global ppod index space,
and the free path ppm_unmark_entries() derives the owning pool from the
index instead of from the running CPU, so any thread can service any
shard.  Derive the global index base from pool->base so it always
matches the pool that was locked.

Signed-off-by: Haoran Wang <haoranwangsec@gmail.com>

---
Hi David,

thanks for the comments.  v4 below:

- The commit message now explains why using the pool of another CPU is
  harmless, and the PREEMPT_RT problem statement is trimmed to two
  sentences.
- pr_debug() now labels the pool's owner as "pool cpu" and also prints
  the running CPU, so a thread servicing a foreign shard is visible in
  the log.
- I also stress-tested the patched allocator itself: the unmodified
  libcxgb_ppm.c with this patch applied, compiled into an out-of-tree
  module and driven through the real cxgbi_ppm_ppods_reserve() /
  cxgbi_ppm_ppod_release() API by 8 unbound kernel threads on a 2-CPU
  QEMU x86_64 guest, with about half of the releases executed by
  another thread (and so, usually, another CPU).  Across 7 runs
  totalling 1.12M allocations there was no double-issued index, no bad
  free, and every pool drained back to empty each run; 4 allocations
  were naturally served by another CPU's pool because the caller
  migrated inside the allocator, and the accounting stayed exact.  The
  test harness is AI-generated; module and logs are attached.
- On the sharding granularity: agreed, the per-pool split only spreads
  lock contention and cache-line bouncing; on systems with thousands of
  CPUs a coarser sharding would serve the same purpose.  That is a
  separate optimisation and does not affect this fix, so I left it
  untouched here.

Changes since v3:
- Explain in the commit message why a foreign CPU's pool is harmless;
  trim the PREEMPT_RT problem description (review).
- pr_debug(): print the running CPU and the pool-owning CPU separately
  and label them accordingly (review).
- pr_debug(): the owning CPU keeps being derived from pool->base, so it
  is always consistent with the locked pool.
- Attach stress-test evidence for the foreign-pool argument.

Changes since v2:
- Keep the owning CPU in the pr_debug() output, derived from
  pool->base / ppm->pool_index_max (suggested by Sebastian).
- Reword the commit message to describe what the patch does and move
  the review discussion into this changelog; spell PREEMPT_RT.

Changes since v1:
- Drop the cpu variable: read the global-index base from pool->base
  (initialised at pool allocation) instead of a cpu captured before
  the lock, so the base is always consistent with the pool that was
  locked.

 drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c |   15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

--- a/drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c
+++ b/drivers/net/ethernet/chelsio/libcxgb/libcxgb_ppm.c
@@ -119,16 +119,13 @@
 			       unsigned long caller_data)
 {
 	struct cxgbi_ppm_pool *pool;
-	unsigned int cpu;
 	int i;
 
 	if (!ppm->pool)
 		return -EINVAL;
 
-	cpu = get_cpu();
-	pool = per_cpu_ptr(ppm->pool, cpu);
+	pool = raw_cpu_ptr(ppm->pool);
 	spin_lock_bh(&pool->lock);
-	put_cpu();
 
 	i = ppm_find_unused_entries(pool->bmap, ppm->pool_index_max,
 				    pool->next, count, 0);
@@ -144,11 +141,12 @@
 
 	spin_unlock_bh(&pool->lock);
 
-	pr_debug("%s: cpu %u, idx %d + %d (%d), next %u.\n",
-		 __func__, cpu, i, count, i + cpu * ppm->pool_index_max,
-		pool->next);
+	pr_debug("%s: cpu %u, pool cpu %u, idx %d + %d (%d), next %u.\n",
+		 __func__, raw_smp_processor_id(),
+		 pool->base / ppm->pool_index_max, i, count, i + pool->base,
+		 pool->next);
 
-	i += cpu * ppm->pool_index_max;
+	i += pool->base;
 	ppm_mark_entries(ppm, i, count, caller_data);
 
 	return i;
@@ -375,6 +373,7 @@
 
 		memset(ppool, 0, alloc_sz);
 		spin_lock_init(&ppool->lock);
+		ppool->base = count;
 		count += ppmax;
 	}