From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E00F197A68 for ; Mon, 27 Jan 2025 20:23:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738009438; cv=none; b=WdbxOmkVrHbuYWk/gJrhZ4YP/pDb4THbtDhJvgFDr0kKa8yvlx1yB/zwn49nA2hIUNkHpLkPIsXpFZ+Vl0rZFwZZLct2jqJrFLYEPykd0kJh0hPgELQ+HPsdmcK5Lv4d55038Wf6wYMRrgIa4nhGmU/vZ53Zx6qICLKNq4edI18= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738009438; c=relaxed/simple; bh=7EHWlex+CN1OdVBUyDN/GiCX7s6OLsDz0BRIS75VFqM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=N96OS5GGIdve6Fj+5uh12CKzOcqWPnSBda0V9Mfc47+kFDgtpOhIPShQPNS+aJPbLHB2r/LcAWkSAtogM4x7kWJoyd8fqc8YO+/q+rgEi7lHj5uyscKNKEq1QsaJyRY5l/m/DjNjyrpGgPfTByddTWruCBUnZEvk6XSTn6RsiBo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TkAKHg+E; arc=none smtp.client-ip=209.85.208.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TkAKHg+E" Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-5d0d32cd31aso7151550a12.0 for ; Mon, 27 Jan 2025 12:23:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738009435; x=1738614235; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=7e+2wHGN3Pp6q7dFqDtLcOtOJoQxDMmhsrb9ED/JLVY=; b=TkAKHg+EO9M5KJDZBgRHtDvJIJ5/24k1RTz6cB7d2wMf5oNr+Kx4jsCb54qxna8ltD ogFFg27Hn/VC0asSllvjH+O5l9d0LZEAeOLDhv8bvHuMsVJITtRz55vNAemaXmYwMUnR HcA0cqwveSBV1wBUrC+iSATZMsk8Tx3JpcFeoZRPhAGcSM74t2NU3M+vXbg9gIj7Fk24 Mu7qzxxkBQDQymHH6htvc7dw61TGFfDgcBryImOZ45BubmZb8RLY5PRGoAVvcJHtrMYr RnhXzy8N4JhiD84W7pFs64stDDEc8cXItwlFUC/1Mj+TNOEVZEXHixMhK4nd8TVhmoCZ 8hnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738009435; x=1738614235; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7e+2wHGN3Pp6q7dFqDtLcOtOJoQxDMmhsrb9ED/JLVY=; b=EDKjsHUPBqJVa3HvoE44MzyHkbCVpLOolMPQYb2f9bIQPTQ+4GqCXJ0sgpmF1BJJFl bC2kMI8QPIDfUs0UUhJO8CTzm2SIa55L/1jLej+Cpgea9/aV54jC+d8eLEfwLGG4TBnD 5JXB9Ut5o3aznHlt4oH1et7mq68rXKqyC/8ryVE4IrG10px4Wh1IGNJVVxcupbHCRTmc kZxQlg1EPRvMyBVx1owZEvBisfmN5LCnHIqtDLLZnprqMTBgcUEASTKARnx2K9D9Cpva YfeHNkuyBOAnvk0f0EzXEaeqG2DHqjQAo1h4h+FnXtvK6hR7ZzGTvZwHBg+hsBUJ797m lhsg== X-Forwarded-Encrypted: i=1; AJvYcCV4TX1YLlCYcvXfvvkSvf4kdywFvuEmqhL0RVtRv3bWZHyTtCKSyEvkAmI0GCz3JSs52aU27Hqp+85eNhw=@vger.kernel.org X-Gm-Message-State: AOJu0YyUWoDaqr1tVnUqCcd1s+TVEOmecLr3cxoDhDyWZ2BjlF+eWoHc I2KgKY9P2y4phplzXnpCmzKo+yKMZa26uIv7BnvPO7QeM4eCQSYLEccmNg0Y X-Gm-Gg: ASbGnctELRmvje7c/gVzyt5kHMeQgCLf74lWK1haDaoakGPdTXKuJFoufwPr2Jnd1N7 A9xw05XBcuQItPyiegzVkS1TbwtRadr1m2HG/EUpVNKZiyHw+SB7flbvTp69Wl7j+PXGBSDUYG0 cZi/oexUQiJTXHF+SgkLb1XY5efWT3y0Ba5pfzsMBSZKKibCiYiQB58cjMARzt05fk5pijcHUCa rtPyZ426X308wmX8GX0nJ6nEtbb6yjleVnLy5d0sHrfepW+pYpPkM9XsqR+KkwQEGv/e/FZW14V o9yjLg== X-Google-Smtp-Source: AGHT+IHY/VGwKOWXuQ+QiJq5REmrQV8M3DnBCVAY2eoQ40JKqvkDx09Zti93fML0J4ZJ5D2nMVaVCg== X-Received: by 2002:a05:6402:518a:b0:5d3:ba42:e9e3 with SMTP id 4fb4d7f45d1cf-5db7d2f5ec0mr103782166a12.13.1738009435161; Mon, 27 Jan 2025 12:23:55 -0800 (PST) Received: from [192.168.1.100] ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5dc18638ed2sm5915926a12.36.2025.01.27.12.23.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Jan 2025 12:23:54 -0800 (PST) Message-ID: Date: Mon, 27 Jan 2025 21:23:53 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: [RFC PATCH 2/6] zsmalloc: make zspage lock preemptible To: Sergey Senozhatsky , Andrew Morton , Minchan Kim , Johannes Weiner , Yosry Ahmed , Nhat Pham Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20250127080254.1302026-1-senozhatsky@chromium.org> <20250127080254.1302026-3-senozhatsky@chromium.org> Content-Language: en-US From: Uros Bizjak In-Reply-To: <20250127080254.1302026-3-senozhatsky@chromium.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 27. 01. 25 08:59, Sergey Senozhatsky wrote: > Switch over from rwlock_t to a atomic_t variable that takes > negative value when the page is under migration, or positive > values when the page is used by zsmalloc users (object map, > etc.) Using a rwsem per-zspage is a little too memory heavy, > a simple atomic_t should suffice, after all we only need to > mark zspage as either used-for-write or used-for-read. This > is needed to make zsmalloc preemtible in the future. > > Signed-off-by: Sergey Senozhatsky > --- > mm/zsmalloc.c | 112 +++++++++++++++++++++++++++++--------------------- > 1 file changed, 66 insertions(+), 46 deletions(-) > > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c > index 817626a351f8..28a75bfbeaa6 100644 > --- a/mm/zsmalloc.c > +++ b/mm/zsmalloc.c > @@ -257,6 +257,9 @@ static inline void free_zpdesc(struct zpdesc *zpdesc) > __free_page(page); > } > > +#define ZS_PAGE_UNLOCKED 0 > +#define ZS_PAGE_WRLOCKED -1 > + > struct zspage { > struct { > unsigned int huge:HUGE_BITS; > @@ -269,7 +272,7 @@ struct zspage { > struct zpdesc *first_zpdesc; > struct list_head list; /* fullness list */ > struct zs_pool *pool; > - rwlock_t lock; > + atomic_t lock; > }; > > struct mapping_area { > @@ -290,11 +293,53 @@ static bool ZsHugePage(struct zspage *zspage) > return zspage->huge; > } > > -static void migrate_lock_init(struct zspage *zspage); > -static void migrate_read_lock(struct zspage *zspage); > -static void migrate_read_unlock(struct zspage *zspage); > -static void migrate_write_lock(struct zspage *zspage); > -static void migrate_write_unlock(struct zspage *zspage); > +static void zspage_lock_init(struct zspage *zspage) > +{ > + atomic_set(&zspage->lock, ZS_PAGE_UNLOCKED); > +} > + > +static void zspage_read_lock(struct zspage *zspage) > +{ > + atomic_t *lock = &zspage->lock; > + int old; > + > + while (1) { > + old = atomic_read(lock); > + if (old == ZS_PAGE_WRLOCKED) { > + cpu_relax(); > + continue; > + } > + > + if (atomic_cmpxchg(lock, old, old + 1) == old) > + return; You can use atomic_try_cmpxchg() here: if (atomic_try_cmpxchg(lock, &old, old + 1)) return; > + > + cpu_relax(); > + } > +} > + > +static void zspage_read_unlock(struct zspage *zspage) > +{ > + atomic_dec(&zspage->lock); > +} > + > +static void zspage_write_lock(struct zspage *zspage) > +{ > + atomic_t *lock = &zspage->lock; > + int old; > + > + while (1) { > + old = atomic_cmpxchg(lock, ZS_PAGE_UNLOCKED, ZS_PAGE_WRLOCKED); > + if (old == ZS_PAGE_UNLOCKED) > + return; Also, the above code can be rewritten as: while (1) { old = ZS_PAGE_UNLOCKED; if (atomic_try_cmpxchg (lock, &old, ZS_PAGE_WRLOCKED)) return; > + > + cpu_relax(); > + } > +} The above change will result in a slightly better generated asm. Uros.