From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D99D219CCFC for ; Wed, 29 Jan 2025 08:14:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738138454; cv=none; b=sTvlfj5vxIKrNW3ga1xBUzkNEfzBwi0HIxn+CyRZIJMH2xqFqMed7pgvu1McRdxqAEuuj+fKd6A0dGz+nP98+rLN8zwwuvplMn5JYtZ6YbcA3fpk2jLYJOa17WlqQhrtapgQw6tMhXiWQt6Q/cTaKtnduxgIC/Ths9hrQkmYYsw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738138454; c=relaxed/simple; bh=O2oR4asarDrt6PbE4adZ8hyBG4rzzF8Lg7K2V13a7KI=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=Q+jHAYLUej3+6qaMK5/wh4zyIKYlKO/AtYf8QUt3noot5RyEmdl5E/nJafnTjwRZLyk+6A/pnsdzBK5XtXwTqsXRVkNAqHVuU7al5+We2nweqRAxxA6K23VY/7QCv+MeUb1wOAli3TRqViz6s8ZEKdarfAICJAhnlIo98a9Qth4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=dkGP1ZCC; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="dkGP1ZCC" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-216426b0865so109193555ad.0 for ; Wed, 29 Jan 2025 00:14:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1738138452; x=1738743252; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=eGcNqLBqVaGSGR+tLjKdq+471J/9pRKvhfVPep5UlRs=; b=dkGP1ZCCypykq6bxqd6ISkuKuImZYil67Do5CfoWtqv6RVom7X7qpx4HpfA1URzWiO LO3xBUGMXCwM03XQu0AOEV/vWlpPzMG6I8JCrdvrQh+EOsTniTY49weAuzqHES1fEQuN ex9jcEKvHd/TZmOZftLrnzbBVhgdwNBz1vvPxefk1+j2eDUzsjoHOGU/1A6fP7LMdP7K m2BETTOiMZF2KKhTmnesTMD1g0KIB+691GCjSjRgZRdGzZYzTlhECpeMpChmgDGPRP83 aI5QeS2X8Kbhhguhcb9KllVSAkt2JRFOv6EjFbTeI+tzhuw35pKJ4rb+/3z3oW/glcZf hGTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738138452; x=1738743252; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eGcNqLBqVaGSGR+tLjKdq+471J/9pRKvhfVPep5UlRs=; b=khEsm8PaSh0ziEyy9m2uUJzNvKaGPArqhDjSEAkMVBYrLqWIUBO8BbBd3/ajZPN9bS 3wQvFyVLw7XV7r1JOpbrLsL2eJDg7kKm/6yyMZQ91Fpo8RYZv+QhAeK2o8bQLlmNnoS5 vUOM1dYLBy+BeRVwfVEbhdPATFNwOGfhq9Ij/vMUHAGtQG1k0ihkNFFV+qkvjUaaL7Zt qaceWQE+ZDhqcvd6TV/xFqlbr4Pan9dwgd7lDKMNuO9voW5rj2oq8RdlB6UZU///t6E5 hPI96rsCmbCcEkDFRkD63+FmE8xbx2OlB3TTZvsVBEm9R3XVGcuAEXaHCVjmzbttF0I1 wZag== X-Forwarded-Encrypted: i=1; AJvYcCWORH5NXrWsupGwLfNzl/sdbS0vps8hqpwMaJuTmqbIeR3FWPBPXrQLjVAhO9gdElxeanbf7hoRNI0yE9I=@vger.kernel.org X-Gm-Message-State: AOJu0YyqfUJKJKI+cankTBmneOxD12DZOYcD/z1JOJ7+6nAVjMC7Xs7T 8r4yAQkv07e9IRbqMAhsY704eqqpCA122ZPZBsFAJ0C+FhTagqVbPFRH9hXd0Vg= X-Gm-Gg: ASbGncuSAJifo9RuJ2hSQ0fN+Gio20IM4XDSwpxUxxL299rTdxHA3/nnc164k8Slurp 6yShjbyZhWMLPCkpKJpi9ARzLfRNeS98sp+cfXet+w81Z6Z4VSDuDprQYJLoJiqvNtsbtlBRJDY X0FcccHcI4WGPW0Ti4hmx7qhYKxGUsSrxpAgCH2XKmxeWuiwiKns2w3SmPCYrO7IMXqL9lQAgj9 VSJbt3ABK0MdROOqvjxKGST4wklIKmkZQthjO4EdC9t1UPlYUKY2Viq4uNvRguDE/qcleaTWIvp UdR/doO8zutyD5gdANxcASqZahg94BLfc8s6sERY+1z01yw+dJMJdaxiD8veZy18oZf9H8ldbCj 8GSZ36HiuiYeVLco= X-Google-Smtp-Source: AGHT+IG2BvQPppbJyzYGHLMoCmyEyD+zahGffGSvJ1pYnxwgl4UNrpz3F6voAb0czs5u8mYiykQ34w== X-Received: by 2002:a17:902:ecc7:b0:215:9d58:6f35 with SMTP id d9443c01a7336-21dd7c55feamr31993245ad.1.1738138452026; Wed, 29 Jan 2025 00:14:12 -0800 (PST) Received: from ?IPV6:240e:370:8b10:5140:d8c5:7248:cf0a:d072? ([240e:370:8b10:5140:d8c5:7248:cf0a:d072]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21da3faab7csm93660025ad.103.2025.01.29.00.14.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 29 Jan 2025 00:14:11 -0800 (PST) Message-ID: <14159fb4-0c59-4653-9265-73f415e70063@bytedance.com> Date: Wed, 29 Jan 2025 16:14:01 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [linus:master] [x86] 4817f70c25: stress-ng.mmapaddr.ops_per_sec 63.0% regression Content-Language: en-US To: Rik van Riel Cc: Peter Zijlstra , David Hildenbrand , kernel test robot , oe-lkp@lists.linux.dev, lkp@intel.com, linux-kernel@vger.kernel.org, Andrew Morton , Dave Hansen , Andy Lutomirski , Catalin Marinas , David Rientjes , Hugh Dickins , Jann Horn , Lorenzo Stoakes , Matthew Wilcox , Mel Gorman , Muchun Song , Peter Xu , Will Deacon , Zach O'Keefe , Dan Carpenter References: <202501281734.d408a35b-lkp@intel.com> <46ea4509-0f38-4b22-bd91-f05c68db2562@redhat.com> <20250128113100.GB7145@noisy.programming.kicks-ass.net> <2c599f1f-f63b-4504-a026-46bf73954124@redhat.com> <20250128132847.GB505@noisy.programming.kicks-ass.net> <18172465-37e3-48a9-9d67-0d44fdcb93bd@redhat.com> <8b661116-0a85-4928-91ed-3c01ebbf8d39@bytedance.com> <2212111cad3180948cf388a7e5c8689df0fdda08.camel@surriel.com> From: Qi Zheng In-Reply-To: <2212111cad3180948cf388a7e5c8689df0fdda08.camel@surriel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Rik, On 2025/1/29 02:35, Rik van Riel wrote: > On Wed, 2025-01-29 at 01:06 +0800, Qi Zheng wrote: >> >> I did reproduce the performance regression using the following test >> program: >> >> stress-ng --timeout 60 --times --verify --metrics --no-rand-seed >> --mmapaddr 64 >> >> And its call stack is as follows: >> >> bpftrace -e 'k:_raw_spin_unlock_irqrestore {@[kstack,comm]=count();} >> interval:s:1 {exit();}' >> >> @[ >> _raw_spin_unlock_irqrestore+5 >> free_one_page+85 >> rcu_do_batch+424 >> rcu_core+401 >> handle_softirqs+204 >> irq_exit_rcu+208 > > That looks like the RCU freeing somehow bypassing the > per-cpu-pages, and hitting the zone->lock at page free > time, while regular freeing usually puts pages in the > CPU-local free page cache, without the lock? Take the following call stack as an example: @[ _raw_spin_unlock_irqrestore+5 free_one_page+85 tlb_remove_table_rcu+140 rcu_do_batch+424 rcu_core+401 handle_softirqs+204 irq_exit_rcu+208 sysvec_apic_timer_interrupt+113 asm_sysvec_apic_timer_interrupt+26 _raw_spin_unlock_irqrestore+29 get_page_from_freelist+2014 __alloc_frozen_pages_noprof+364 alloc_pages_mpol+123 alloc_pages_noprof+14 get_free_pages_noprof+17 __x64_sys_mincore+141 do_syscall_64+98 entry_SYSCALL_64_after_hwframe+118 , stress-ng-mmapa]: 5301 It looks like the following happened: get_page_from_freelist --> rmqueue --> rmqueue_pcplist --> pcp_spin_trylock (hold the pcp lock) __rmqueue_pcplist --> rmqueue_bulk --> spin_lock_irqsave(&zone->lock) __rmqueue spin_unlock_irqrestore(&zone->lock) tlb_remove_table_rcu --> free_frozen_pages --> pcp = pcp_spin_trylock (failed!!!) if (!pcp) free_one_page It seems that the pcp lock is held when doing tlb_remove_table_rcu(), so trylock fails, then bypassing PCP and calling free_one_page() directly, which leads to the hot spot of zone lock. As for the regular freeing, since the freeing operation will not be performed in the softirq, the above situation will not occur. Right? > > I'm not quite sure why this would be happening, though. > > Maybe the RCU batches are too big for the PCPs to > hold them? > > If that is the case, chances are more code paths are > hitting that issue, and we should just fix it, rather > than trying to bypass it. > > Maybe the reason is more simple than that? > > I have not found a place where it explicitly bypasses > the PCPs, but who knows? >