mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Yury Norov <yury.norov@gmail.com>
To: linux-kernel@vger.kernel.org, Yury Norov <yury.norov@gmail.com>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Rasmus Villemoes <linux@rasmusvillemoes.dk>,
	Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
	Daniel Bristot de Oliveira <bristot@redhat.com>,
	Valentin Schneider <vschneid@redhat.com>
Cc: Jan Kara <jack@suse.cz>,
	Mirsad Todorovac <mirsad.todorovac@alu.unizg.hr>,
	Matthew Wilcox <willy@infradead.org>,
	Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>,
	Alexey Klimov <klimov.linux@gmail.com>,
	Bart Van Assche <bvanassche@acm.org>,
	Sergey Shtylyov <s.shtylyov@omp.ru>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Subject: [PATCH v3 05/35] sched: add cpumask_find_and_set() and use it in __mm_cid_get()
Date: Mon, 11 Dec 2023 18:27:19 -0800	[thread overview]
Message-ID: <20231212022749.625238-6-yury.norov@gmail.com> (raw)
In-Reply-To: <20231212022749.625238-1-yury.norov@gmail.com>

__mm_cid_get() uses __mm_cid_try_get() helper to atomically acquire a
bit in mm cid mask. Now that we have atomic find_and_set_bit(), we can
easily extend it to cpumasks and use in the scheduler code.

cpumask_find_and_set() considers cid mask as a volatile region of memory,
as it actually is in this case. So, if it's changed while search is in
progress, KCSAN wouldn't fire warning on it.

CC: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Yury Norov <yury.norov@gmail.com>
---
 include/linux/cpumask.h | 12 ++++++++++++
 kernel/sched/sched.h    | 14 +++++---------
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index cfb545841a2c..c2acced8be4e 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -271,6 +271,18 @@ unsigned int cpumask_next_and(int n, const struct cpumask *src1p,
 		small_cpumask_bits, n + 1);
 }
 
+/**
+ * cpumask_find_and_set - find the first unset cpu in a cpumask and
+ *			  set it atomically
+ * @srcp: the cpumask pointer
+ *
+ * Return: >= nr_cpu_ids if nothing is found.
+ */
+static inline unsigned int cpumask_find_and_set(volatile struct cpumask *srcp)
+{
+	return find_and_set_bit(cpumask_bits(srcp), small_cpumask_bits);
+}
+
 /**
  * for_each_cpu - iterate over every cpu in a mask
  * @cpu: the (optionally unsigned) integer iterator
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 2e5a95486a42..2ce9112de89b 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -3347,23 +3347,19 @@ static inline void mm_cid_put(struct mm_struct *mm)
 
 static inline int __mm_cid_try_get(struct mm_struct *mm)
 {
-	struct cpumask *cpumask;
-	int cid;
+	struct cpumask *cpumask = mm_cidmask(mm);
+	int cid = nr_cpu_ids;
 
-	cpumask = mm_cidmask(mm);
 	/*
 	 * Retry finding first zero bit if the mask is temporarily
 	 * filled. This only happens during concurrent remote-clear
 	 * which owns a cid without holding a rq lock.
 	 */
-	for (;;) {
-		cid = cpumask_first_zero(cpumask);
-		if (cid < nr_cpu_ids)
-			break;
+	while (cid >= nr_cpu_ids) {
+		cid = cpumask_find_and_set(cpumask);
 		cpu_relax();
 	}
-	if (cpumask_test_and_set_cpu(cid, cpumask))
-		return -1;
+
 	return cid;
 }
 
-- 
2.40.1


  parent reply	other threads:[~2023-12-12  2:28 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-12-12  2:27 [PATCH v3 00/35] bitops: add atomic find_bit() operations Yury Norov
2023-12-12  2:27 ` [PATCH v3 01/35] lib/find: add atomic find_bit() primitives Yury Norov
2023-12-12  2:27 ` [PATCH v3 02/35] lib/find: add test for atomic find_bit() ops Yury Norov
2023-12-12  2:27 ` [PATCH v3 03/35] lib/sbitmap; optimize __sbitmap_get_word() by using find_and_set_bit() Yury Norov
2023-12-12  2:27 ` [PATCH v3 04/35] watch_queue: optimize post_one_notification() by using find_and_clear_bit() Yury Norov
2023-12-12  2:27 ` Yury Norov [this message]
2023-12-12 14:14   ` [PATCH v3 05/35] sched: add cpumask_find_and_set() and use it in __mm_cid_get() Mathieu Desnoyers
2023-12-12  2:27 ` [PATCH v3 06/35] mips: sgi-ip30: optimize heart_alloc_int() by using find_and_set_bit() Yury Norov
2023-12-12  2:27 ` [PATCH v3 07/35] sparc: optimize alloc_msi() " Yury Norov
2023-12-12  2:27 ` [PATCH v3 08/35] perf/arm: use atomic find_bit() API Yury Norov
2023-12-12  2:27 ` [PATCH v3 09/35] drivers/perf: optimize ali_drw_get_counter_idx() by using find_and_set_bit() Yury Norov
2023-12-12  2:27 ` [PATCH v3 10/35] dmaengine: idxd: optimize perfmon_assign_event() Yury Norov
2023-12-12  2:30   ` Fenghua Yu
2023-12-12  2:27 ` [PATCH v3 11/35] ath10k: optimize ath10k_snoc_napi_poll() Yury Norov
2023-12-12  2:27 ` [PATCH v3 12/35] wifi: rtw88: optimize the driver by using atomic iterator Yury Norov
2023-12-12  2:27 ` [PATCH v3 13/35] KVM: x86: hyper-v: optimize and cleanup kvm_hv_process_stimers() Yury Norov
2023-12-12 17:22   ` Sean Christopherson
2023-12-12 17:35     ` Yury Norov
2023-12-12  2:27 ` [PATCH v3 14/35] PCI: hv: Optimize hv_get_dom_num() by using find_and_set_bit() Yury Norov
2023-12-12  2:27 ` [PATCH v3 15/35] scsi: core: optimize scsi_evt_emit() by using an atomic iterator Yury Norov
2023-12-12  2:27 ` [PATCH v3 16/35] scsi: mpi3mr: optimize the driver by using find_and_set_bit() Yury Norov
2023-12-12  2:27 ` [PATCH v3 17/35] scsi: qedi: optimize qedi_get_task_idx() " Yury Norov
2023-12-12  2:27 ` [PATCH v3 18/35] powerpc: optimize arch code by using atomic find_bit() API Yury Norov
2023-12-12  2:27 ` [PATCH v3 19/35] iommu: optimize subsystem " Yury Norov
2023-12-12  2:27 ` [PATCH v3 20/35] media: radio-shark: optimize the driver " Yury Norov
2023-12-12  2:27 ` [PATCH v3 21/35] sfc: " Yury Norov
2023-12-12  2:27 ` [PATCH v3 22/35] tty: nozomi: optimize interrupt_handler() Yury Norov
2023-12-12  2:27 ` [PATCH v3 23/35] usb: cdc-acm: optimize acm_softint() Yury Norov
2023-12-12  2:27 ` [PATCH v3 24/35] block: null_blk: replace get_tag() with a generic find_and_set_bit_lock() Yury Norov
2023-12-12  2:27 ` [PATCH v3 25/35] RDMA/rtrs: optimize __rtrs_get_permit() by using find_and_set_bit_lock() Yury Norov
2023-12-12  2:27 ` [PATCH v3 26/35] mISDN: optimize get_free_devid() Yury Norov
2023-12-12  2:27 ` [PATCH v3 27/35] media: em28xx: cx231xx: optimize drivers by using find_and_set_bit() Yury Norov
2023-12-12  2:27 ` [PATCH v3 28/35] ethernet: rocker: optimize ofdpa_port_internal_vlan_id_get() Yury Norov
2023-12-12  2:27 ` [PATCH v3 29/35] serial: sc12is7xx: optimize sc16is7xx_alloc_line() Yury Norov
2023-12-12  2:27 ` [PATCH v3 30/35] bluetooth: optimize cmtp_alloc_block_id() Yury Norov
2023-12-12  2:27 ` [PATCH v3 31/35] net: smc: optimize smc_wr_tx_get_free_slot_index() Yury Norov
2023-12-12 13:26   ` Wen Gu
2023-12-12  2:27 ` [PATCH v3 32/35] ALSA: use atomic find_bit() functions where applicable Yury Norov
2023-12-12  2:27 ` [PATCH v3 33/35] m68k: optimize get_mmu_context() Yury Norov
2023-12-12  2:27 ` [PATCH v3 34/35] microblaze: " Yury Norov
2023-12-12  2:27 ` [PATCH v3 35/35] sh: mach-x3proto: optimize ilsel_enable() Yury Norov
2023-12-16 21:48 ` [PATCH v3 00/35] bitops: add atomic find_bit() operations Yury Norov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231212022749.625238-6-yury.norov@gmail.com \
    --to=yury.norov@gmail.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=bristot@redhat.com \
    --cc=bsegall@google.com \
    --cc=bvanassche@acm.org \
    --cc=dietmar.eggemann@arm.com \
    --cc=jack@suse.cz \
    --cc=juri.lelli@redhat.com \
    --cc=klimov.linux@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@rasmusvillemoes.dk \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=maxim.kuvyrkov@linaro.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=mirsad.todorovac@alu.unizg.hr \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=s.shtylyov@omp.ru \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®