From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25E352DE70A for ; Wed, 3 Dec 2025 09:29:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764754144; cv=none; b=AeCzaqAiHebe8g0B/rsBYI/ZRzkYnnHIW2Uxp+gC1HlfxqpQrRS/rn/hpx3Y+5Sswb8Qfda7INOwjdoM+2eYkmO3xa1/I36ybxjbY5ZbZJTdqcvXJdOc5TwW6wgIIEW+6yVBOGhFVq7sS/JZLaEpUtB6rRFRfyeEkSTJoOU/wGQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764754144; c=relaxed/simple; bh=Si5ely/081Qndi8DlAgpHw1X5adswpymMpLK//gxnWk=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=f8WR22Nym9oDOU5vr9+m29n83TjGwkUq+nzPpw12HGrphFulKr0OAm25l9sG+gGpqsnG8L1wWrpVugzNQaiD6niZCdlb6J1jPLxgBl6K27XtymE4FWHO7/3P5w8AsPrybBW2YvoF1y2ZgrWyNRGBDnPqHuGZ07yxuBxC9Qhr/6g= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=X7t+TAWw; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="X7t+TAWw" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-6417313bddaso10446092a12.3 for ; Wed, 03 Dec 2025 01:29:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1764754140; x=1765358940; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=OJl4vJuwyDpSRQ6QgxZUD7yK7xjBETvAgfX6HralMRU=; b=X7t+TAWwDRL+F+8NxNiQsU3AABOD50ChnsNT3ffxO4HW8xKTpM9bYuy0f3Z/hPpG2p Rxm+78+nsuFfzRo0XjyZPIWEHFCUHgkLUMqv0dLDiDQqNSCqsdKA3U6Bk7zS2DyMIoOW GRxxce38HtMcY8CYjm/y4W5wXRq1izWweH1+qK3Ci4cSDqU/B9lO7vEDgjFhMP49PhmH bbl+7U0LjA0T2ql0B4UlhudZySvvlLOEYsBvMejc8VcodHQ9qvj54KB3kRdHqtLOQbxN LEhOPsyBRI9Ldk3gg44aBaHVtDgY4tcElpKNOPqXkXaxVuYTtM+LImOXCrSTp22XiuC/ QpPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764754140; x=1765358940; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=OJl4vJuwyDpSRQ6QgxZUD7yK7xjBETvAgfX6HralMRU=; b=qaGdk+c27KT7+P2PCVta5qLQnPGdmRodf0Uo3pEp/9NwHs3S6yquZA8c0fqigr6biR FSCzkdN7XuULB342L0yowmA6fC6UON7wwA83AlZ3N5K6S83LzUxoR+5sf27hEVI+dZG4 jSfVoKz0IUIhs++Wqcc6SOivhYneNsDHMjIr5IJqYHAfMIvSaqMAu9oLplDKCkHBVoiJ G9BD/pQxh2Fxu+/w0c7dMtQpkgug6Ni+1mnfbLM9shQiwdrhkqQeH7OfLNdGy5UlA0QV 4hFJlGt1NTe+O9pK15b36XFLNz9K7qMin3tu3uUnZDSn0F/R8fPMm1aqYVh6GfPVo04k qwdg== X-Forwarded-Encrypted: i=1; AJvYcCVAThuNaR1mqTO+28nzwDKpik5C4iTS+9wINyBiMfsLQBFrmd2S9W+efQDpzrRmZ60nX7sLnmbJJ7qHnes=@vger.kernel.org X-Gm-Message-State: AOJu0YxL9TvhMtYVEjgXYeQ/KnzMG4aVNgbN3Fk+Jy+B2qOeqRckNEWG fqAWvZPdzw5L4Ni/KaNHQtb16fypWiAzwt4NYkSzMruy6CtgJ1vISQlV X-Gm-Gg: ASbGncsppa6FjjiaGTE65bIHeW8YJcxrJ75QiNwZ+5pU5HdReRAZtbUGbI7ORp5V7qx eB7nx8T5TLgWmqHT+r2eHen/y2DWPz3YgYR8f/JeMKEebZQOaAOdtvyKfkq1jNUzLNBUOn3K5+k ntbuWlAles8klfKNKljArwootAb4JhDGU8/XdMoGoIN3m3mqkiIaKYK/QvO3kcC8k/mfL2qVpkg G5y3Pzze1mN8t28fd9NuenYw8vRH782EbK7NPTTEMsEri5WskT2HGdoH1VCioJRnZOqDiKRcCuF cs73owX0xO/tVYpkxZ6XOE0sQEG+T1wNY4qlRtBC/pHAxxsSMUXpBlUizJ+n8WYVIHsVECnsjcH JN6NquK0eCeJ4dE0Xk+rBwBBjM3ojcKoQfstitsNG4acnP8v+8acseyBUxxJ+PBKSUElE4NnS8q aXV/H9gu6QW5sgX0CMEGddRcWhRoiY8c6/+q12ID16/0AsGGPV47g2q6XnZW4= X-Google-Smtp-Source: AGHT+IF3+F+5LRAC6bKH9Pe1Zn5vprjhF1qCd3gmpOuk/TlFgRTWsY7rZUFwTPZeDr5lXTrvlKrTYA== X-Received: by 2002:a05:6402:26c3:b0:640:cc76:ae35 with SMTP id 4fb4d7f45d1cf-6479c48bb3bmr1046073a12.21.1764754139666; Wed, 03 Dec 2025 01:28:59 -0800 (PST) Received: from f.. (cst-prg-14-82.cust.vodafone.cz. [46.135.14.82]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-64750a90d14sm17859981a12.10.2025.12.03.01.28.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Dec 2025 01:28:58 -0800 (PST) From: Mateusz Guzik To: oleg@redhat.com Cc: brauner@kernel.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, willy@infradead.org, Mateusz Guzik Subject: [PATCH v2 0/2] further damage-control lack of clone scalability Date: Wed, 3 Dec 2025 10:28:49 +0100 Message-ID: <20251203092851.287617-1-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit When spawning and killing threads in separate processes in parallel the primary bottleneck on the stock kernel is pidmap_lock, largely because of a back-to-back acquire in the common case. Benchmark code at the end. With this patchset alloc_pid() only takes the lock once and consequently alleviates the problem. While scalability improves, the lock remains the primary bottleneck by a large margin. I believe idr is a poor choice for the task at hand to begin with, but sorting out that out beyond the scope of this patchset. At the same time any replacement would be best evaluated against a state where the above relock problem is fixed. Performance improvement varies between reboots. When benchmarking with 20 processes creating and killing threads in a loop, the unpatched baseline hovers around 465k ops/s, while patched is anything between ~510k ops/s and ~560k depending on false-sharing (which I only minimally sanitized). So this is at least 10% if you are unlucky. bench from will-it-scale: #include #include char *testcase_description = "Thread creation and teardown"; static void *worker(void *arg) { return (NULL); } void testcase(unsigned long long *iterations, unsigned long nr) { pthread_t thread[1]; int error; while (1) { for (int i = 0; i < 1; i++) { error = pthread_create(&thread[i], NULL, worker, NULL); assert(error == 0); } for (int i = 0; i < 1; i++) { error = pthread_join(thread[i], NULL); assert(error == 0); } (*iterations)++; } } v2: - cosmetic fixes from Oleg - drop idr_preload_many, relock pidmap + call idr_preload again instead - write a commit message for the alloc pid patch Mateusz Guzik (2): ns: pad refcount pid: only take pidmap_lock once on alloc include/linux/ns/ns_common_types.h | 4 +- kernel/pid.c | 131 +++++++++++++++++++---------- 2 files changed, 88 insertions(+), 47 deletions(-) -- 2.48.1