From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA51621FF5F for ; Sun, 23 Nov 2025 06:31:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763879469; cv=none; b=nW7NEE5/5CrTyNB6ZgthzLMJBQSzYbN+mbn/rubKO3emIBR3bCOD/hTGdIRIsBm/cl3OmF/KH3zYoIjs/RctKAHAYo95jrSGbP7pG+VdFmxj03Wwz5tsGktxoJJANh+7aW65AkssjkNzWZbSYxmgl6nofx1y7I3L5iUNthlWAoc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763879469; c=relaxed/simple; bh=02VJuadQx5eN9Nut5ln9u2uI86mNcg9jf1rotiMVF8Y=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=dk6hnA/wND1Yo0IQwjYLEd3UGHhM4EZuvMU+wehUtXAcIxILZBAr1BJGT4obfF+6bFJguZn3gzTqSj9Lca4xBXI9NvcxMdXdBHer2iVR8LOGKAA2Y1dk8geH9OnTOdRKKUuE9jiQ9Pm3N06yLnAcfOPJyBSMKIwaKvSQoa0HQGs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eYlY62q8; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eYlY62q8" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-b736ffc531fso477634666b.1 for ; Sat, 22 Nov 2025 22:31:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763879465; x=1764484265; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=zlnQWv1xCLYBTLtdCEO30f5cxKERrZdCiEwscZcR3Kw=; b=eYlY62q8eVqPWmAQp3sFOdFIPdtrUHdfXb2ZhtyuwOVpOagUuENUsjsde4h1Nl2NaV 7A+jOO0Px0/r36QJ6wReNDuHYW6SMn/g9+wRhb38QxqfD6qnbmAgx+s2qejsP/bhV5Tx OeiuPzw3a4zGDl7Nd7hEYTzyaLy1oYu9N014mK96OEBfj2hqYRhjIqzVASU1dv4IebbY VyFLPh35C4pgqs/CGFsxPZFkD3g6lq/mq5G6Hmsi61OjLE8qdszIUvb/KByZJN3GFUWn H3wO30CM597gAtCiqKwy5oIYAQill+pDxE7guZrT3PUKIq8IYrYsOiNpSs6KNVGwmzNH Gjug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763879465; x=1764484265; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=zlnQWv1xCLYBTLtdCEO30f5cxKERrZdCiEwscZcR3Kw=; b=lPdXlrNzy/U5GqffMJFUDqtDRpTqr7dJY3XNZ7p01shRDUDwRc/Dxh0LEUsB0VDbmo aYLksYpJSaXT/Qf9IvmFMVBmdBMeqmGT83rB06RKhsFKowLZb29JoIs8vDwFS/POM97W UAewOL/PfDT5cLHs4K9RYeyj1FGqEykgyXPqELm3KYpC0sAl7QCvND5FYCouGzt5+ms8 QLGUKYTdKOWWyrClKNyywZcM5IekJGtDpYpJ2b+y66aK6gZrX5drkxurXudpys2ziS6G Nq/LDsQGq3C4BgA2sHM3Sjrw/5eg6L82m9vHo0KyjvU9DEBlRJcSQzAwnnkkc5Go5dw9 GvaQ== X-Forwarded-Encrypted: i=1; AJvYcCVjhP7fhF5EKRD9Qj8+t4pjr1a6qV0D0NX0y/1dFgPbKxuGXikmPdl8+ZuSQuHvx4s/iF51/tZFaMMjeFk=@vger.kernel.org X-Gm-Message-State: AOJu0Yxr7+UlPv81ryA7tPoTEncX/HEfyCnlrqA+tqk8z1dbedxitv39 opk9KyoNZOesEmhDd7tXThFw5ZxZsp+DYjKtlJqBR89KKQTK2c1fj+XE X-Gm-Gg: ASbGnctDdSlD7hg00bZL8h9u7W2/+RRIPkiQF+pMFaFyrCymnEUPt+lAx7MevM6YaWK Py0ywP/d0GziG1XASiNBfEZCzJCm+w7bvg89+SZwVRNpz7AM2ZltXjcdXOFf9KhwJzTkSL12Sj8 jCt4SfpeeeWrPd8TiLVuF1bS7VKjFMprvDaJpcbsujRBqxf9aiO4vEGD+u7fsoo1Iub6o39j3sX dSaT3XI05mOz7DAF7OB4giKyp5jqwTKvaspshyELTOpl3pWkd9tVd5rkOslE/d2PnNwlO4gslPB r+TBiEAyALXqNQCmV/vJTXay393J+6YSGPX+BYEs8J0YtnQdmK/zBwveqdf58Al/T1wh5UXKFU/ 7H+OeWR1hw+LDovbWd1RxjhZOS5bCr53D8sITcgMPxKrKGyAORAs1QDM9QSFERjYMp7vSOxR0BF owcdmsLfAbN9WtD9ERJlYp75P/T4lTtDyS1/HiUONI937eTl8MKs7ZSPVu+RE= X-Google-Smtp-Source: AGHT+IEk2qV1I9dbcykpasrU40jJ8DeXhyPRy9Tsf7CPWcP8hKXihzFc7lGHCIdXQ34J061Aq57Mig== X-Received: by 2002:a17:907:97d0:b0:b73:6b85:1a8f with SMTP id a640c23a62f3a-b767170ae15mr867540666b.49.1763879464915; Sat, 22 Nov 2025 22:31:04 -0800 (PST) Received: from f.. (cst-prg-14-82.cust.vodafone.cz. [46.135.14.82]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-b7654cf0435sm929432866b.4.2025.11.22.22.31.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 22 Nov 2025 22:31:04 -0800 (PST) From: Mateusz Guzik To: oleg@redhat.com Cc: brauner@kernel.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, Mateusz Guzik Subject: [PATCH 0/3] further damage-control lack of clone scalability Date: Sun, 23 Nov 2025 07:30:51 +0100 Message-ID: <20251123063054.3502938-1-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit When spawning and killing threads in separate processes in parallel the primary bottleneck on the stock kernel is pidmap_lock, largely because of a back-to-back acquire in the common case. Benchmark code at the end. With this patchset alloc_pid() only takes the lock once and consequently alleviates the problem. While scalability improves, the lock remains the primary bottleneck by a large margin. I believe idr is a poor choice for the task at hand to begin with, but sorting out that out beyond the scope of this patchset. At the same time any replacement would be best evaluated against a state where the above relock problem is fixed. Performance improvement varies between reboots. When benchmarking with 20 processes creating and killing threads in a loop, the unpatched baseline hovers around 465k ops/s, while patched is anything between ~510k ops/s and ~560k depending on false-sharing (which I only minimally sanitized). So this is at least 10% if you are unlucky. bench from will-it-scale: #include #include char *testcase_description = "Thread creation and teardown"; static void *worker(void *arg) { return (NULL); } void testcase(unsigned long long *iterations, unsigned long nr) { pthread_t thread[1]; int error; while (1) { for (int i = 0; i < 1; i++) { error = pthread_create(&thread[i], NULL, worker, NULL); assert(error == 0); } for (int i = 0; i < 1; i++) { error = pthread_join(thread[i], NULL); assert(error == 0); } (*iterations)++; } } Mateusz Guzik (3): idr: add idr_prealloc_many ns: pad refcount pid: only take pidmap_lock once on alloc include/linux/idr.h | 1 + include/linux/ns/ns_common_types.h | 4 +- kernel/pid.c | 99 +++++++++++++++++------------- lib/radix-tree.c | 19 +++++- 4 files changed, 76 insertions(+), 47 deletions(-) -- 2.48.1