From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02161269CE6 for ; Sat, 6 Dec 2025 13:20:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765027209; cv=none; b=llwU0UGLJCi9d+NYP42cZLgNFIEPGbdJa3ZC3wN4QZ0dNy0jmE6RQQ50O2+xnj+TgZF4m1a4/R26vAK3ACQYtgHGXnLoPkTUpWX9sbehRuEPGNcKiZH1aO1spl+lJ7U80GZo5qJkOsRCDBD2vd4pw4HvKuH4s6UnbkkjLDao9Yc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765027209; c=relaxed/simple; bh=vcUwS7wZ6okB/Oxs9EwtO96t3oxmDYxYxBJ8X5g56ME=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=c0yfZGmullV4gVKT/U/T7DXA5pmlvm0YR8HRwLYuSIxcU+/ZbUZhjktS4/ViCQgLHkNrw7ueK3uHf56ke9nIwSOPYmAutG9n4ncqHMmszaiYdSZKaYQzM9ETXfM1A95+cVIAVCq+/Tj5bl63GNxBd+QpnqAATnS3xaZ2P0Znw1U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=j8VHNRUF; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="j8VHNRUF" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-477a1c28778so40360745e9.3 for ; Sat, 06 Dec 2025 05:20:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765027204; x=1765632004; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=GV6EbijV0HjN+hquGcR/yvTpNUGs1FS+l/WZr5mJ+5o=; b=j8VHNRUFuGYTPoq/GYyfDBDL1ejtNud009ltncH3J1UL35bG3xVSRHE1FzgLyhq+2f F3yInrQvHJuiPpiS5DtKlyro+ntNsDyYqZ+/44qZvtu0XFw8c+AxVatZsPMLgCgodjHG sHZCktzXZtpUUWhkLwjZQ/MgQY67nDjg5i/9qYvUnMX1PHt1AO0FFUyjQ71v9yrvsMO5 Fio01eSJAn/Cwxm8Zky6w9VBUl1F4un7nd3PgNgH9p2J/awsIxWyskKpC0A7epm36dvC Gs1Tkwvog2EyePsJLwEmzxDDULmACXloqbo9TjC6EN258gr6mOSA2enQWBUBzexMleI6 SsYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765027204; x=1765632004; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GV6EbijV0HjN+hquGcR/yvTpNUGs1FS+l/WZr5mJ+5o=; b=bl0NsVK9gsZ45/0TRDQTrXGbN2ZoXnLyiwtZu3K0A74cWx+npU+Zcek1GqJtafaRYK YbPZQMqG4pSf+6XIVbu2tB5UCbD+4tB6lRp4NFVgkfQX8bJXXRp4FmNd4p1+dVsgfSjK MLVLqaYLgjz8zEHxQpnuJSZZu6wSMpzbINC+m76xhA1FPknGItGIEnxTApNvhYhZEv17 bSZYWEV3LCcSWdcuMzP/y9cu3PJF8kfnrvODsuUVwAhXNgdqSQmL7H1jjhlzonJRRuvr I0hj5DMCxXf6fmIbiN04lJCga+0qIso3JEX3nJFntll8w9QASu/xzvIaMYjN3EawCsJ4 Kwig== X-Forwarded-Encrypted: i=1; AJvYcCWegGy3oCv/fuqu8YqicbwmWFB/rBsPyYE5bSQSx2vKppvfXQBDF8WYI1G3xJtxQoDENK/LZad1EMfKVD4=@vger.kernel.org X-Gm-Message-State: AOJu0YyDbB3O09CPr7vvdjSFIBwIecLcXAMIsu5q7l0EDxxVjciz8RbE Qv53rtO/qAFaygdOUiCTxzQosDMkUgslNT3buNIY3cDJqErqFJVTfGgF9eURDw== X-Gm-Gg: ASbGnctGRXQ2UlkthNDnWDPaqqpVfSTnETdhPUrQKMqrR+Mp4gBqxzgMksAVujnMviA ex+ZoQWPzivYN4wztDNWoXKprzzI/ssfWNygHkd5CHJ2H4GIG0Vw3v37iuu8IW7kdBUtuEFe5rp 13q8A9sv3cSmZ4b0BjF3f1Hk2QPvERUSpCXkorywjJnmvpb0818K2j26KcizIB1z59z2UMwE5Yg Ln8UqgUwuQm1ZrXgyzT9CMMnCsP04CeYrE68c2/ttFQN8PmxFbS8SajBjl5ZvMB6CoXFS4ojWtH cEuDJCAy3Eu+GAw7E4mFvxiVc3PMIA5efbO3LVsuroO5VwfXKr29Z++ZAsOey2+0pauBWLb3Q7H nyLXaNm1KmrHPkVybot6q41gDSGlqQmOnbt2AemWk2HrdjciDvSTx1+kwIPEuCyc2ixl3K5V07w S53kBC3E0QT7Mw+3NyF09ZzXj5Dk1HBgtxt5zWRdyOOPGwWrn+xK2PSG7Eluw= X-Google-Smtp-Source: AGHT+IEtzHd2QjOH/hxJu9cWk5GtETWkHsIS8LyXnaLojWjaBsdHQmrCjEull4s7jMKp054EN3S1Tw== X-Received: by 2002:a05:600c:4e15:b0:475:dd89:abc with SMTP id 5b1f17b1804b1-47939df80eemr24700315e9.11.1765027204083; Sat, 06 Dec 2025 05:20:04 -0800 (PST) Received: from f.. (cst-prg-14-82.cust.vodafone.cz. [46.135.14.82]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4792afd3b31sm110871215e9.0.2025.12.06.05.20.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 06 Dec 2025 05:20:03 -0800 (PST) From: Mateusz Guzik To: oleg@redhat.com Cc: brauner@kernel.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, willy@infradead.org, Mateusz Guzik Subject: [PATCH v3 0/2] further damage-control lack of clone scalability Date: Sat, 6 Dec 2025 14:19:53 +0100 Message-ID: <20251206131955.780557-1-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit When spawning and killing threads in separate processes in parallel the primary bottleneck on the stock kernel is pidmap_lock, largely because of a back-to-back acquire in the common case. Benchmark code at the end. With this patchset alloc_pid() only takes the lock once and consequently alleviates the problem. While scalability improves, the lock remains the primary bottleneck by a large margin. I believe idr is a poor choice for the task at hand to begin with, but sorting out that out beyond the scope of this patchset. At the same time any replacement would be best evaluated against a state where the above relock problem is fixed. Performance improvement varies between reboots. When benchmarking with 20 processes creating and killing threads in a loop, the unpatched baseline hovers around 465k ops/s, while patched is anything between ~510k ops/s and ~560k depending on false-sharing (which I only minimally sanitized). So this is at least 10% if you are unlucky. bench from will-it-scale: #include #include char *testcase_description = "Thread creation and teardown"; static void *worker(void *arg) { return (NULL); } void testcase(unsigned long long *iterations, unsigned long nr) { pthread_t thread[1]; int error; while (1) { for (int i = 0; i < 1; i++) { error = pthread_create(&thread[i], NULL, worker, NULL); assert(error == 0); } for (int i = 0; i < 1; i++) { error = pthread_join(thread[i], NULL); assert(error == 0); } (*iterations)++; } } v3: - fix some whitespace and one typo - slightly reword the ENOMEM comment - move i-- in the first loop towards the end for consistency with the other loop - 2 extra unlikely for initial error conditions I retained Oleg's r-b as the changes don't affect behavior v2: - cosmetic fixes from Oleg - drop idr_preload_many, relock pidmap + call idr_preload again instead - write a commit message Mateusz Guzik (2): ns: pad refcount pid: only take pidmap_lock once on alloc include/linux/ns/ns_common_types.h | 4 +- kernel/pid.c | 134 ++++++++++++++++++----------- 2 files changed, 89 insertions(+), 49 deletions(-) -- 2.48.1