From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D6A2C432C3 for ; Thu, 14 Nov 2019 14:43:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EDA512070E for ; Thu, 14 Nov 2019 14:43:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1573742602; bh=vs4o9uhBZ8ZZarCFGMsrxyCFr4QaYvMaNYpXSmY+IJw=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=Zgb6R3MbOMBkhrfED9wziE4MGaL0Y54B5oTWwOEl2p8bjkcJSrYxXugs+9Q2akvq6 BIUMpiDskzapeee3f98rkBA0MDf6cX/zOoydOTFD+Jx6Uo20W/tOxN35vpW3sQkyvM AzmKcC3GLnJIZuRil18dVXGyJLD/eDvVNCzUxcIw= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726750AbfKNOnU (ORCPT ); Thu, 14 Nov 2019 09:43:20 -0500 Received: from mx2.suse.de ([195.135.220.15]:56294 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726276AbfKNOnU (ORCPT ); Thu, 14 Nov 2019 09:43:20 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 755A8ADFB; Thu, 14 Nov 2019 14:43:18 +0000 (UTC) Date: Thu, 14 Nov 2019 15:43:17 +0100 From: Michal Hocko To: Shaokun Zhang Cc: linux-kernel@vger.kernel.org, yuqi jin , Andrew Morton , Mike Rapoport , Paul Burton , Michael Ellerman , Anshuman Khandual , netdev@vger.kernel.org Subject: Re: [PATCH v3] lib: optimize cpumask_local_spread() Message-ID: <20191114144317.GJ20866@dhcp22.suse.cz> References: <1573091048-10595-1-git-send-email-zhangshaokun@hisilicon.com> <20191108103102.GF15658@dhcp22.suse.cz> <20191112115630.GD2763@dhcp22.suse.cz> <00856999-739f-fd73-eddd-d71e4e94962e@hisilicon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <00856999-739f-fd73-eddd-d71e4e94962e@hisilicon.com> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed 13-11-19 10:46:05, Shaokun Zhang wrote: [...] > >> available: 4 nodes (0-3) > >> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 > >> node 0 size: 63379 MB > >> node 0 free: 61899 MB > >> node 1 cpus: 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 > >> node 1 size: 64509 MB > >> node 1 free: 63942 MB > >> node 2 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 > >> node 2 size: 64509 MB > >> node 2 free: 63056 MB > >> node 3 cpus: 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 > >> node 3 size: 63997 MB > >> node 3 free: 63420 MB > >> node distances: > >> node 0 1 2 3 > >> 0: 10 16 32 33 > >> 1: 16 10 25 32 > >> 2: 32 25 10 16 > >> 3: 33 32 16 10 [...] > before patch > Euler:/sys/bus/pci/devices/0000:7d:00.2 # cat numa_node > 2 > Euler:/sys/bus/pci # cat /proc/irq/345/smp_affinity_list > 48 node 2 > Euler:/sys/bus/pci # cat /proc/irq/369/smp_affinity_list > 0 node 0 > Euler:/sys/bus/pci # cat /proc/irq/393/smp_affinity_list > 24 node 1 > Euler:/sys/bus/pci # > > after patch > Euler:/sys/bus/pci/devices/0000:7d:00.2 # cat numa_node > 2 > Euler:/sys/bus/pci # cat /proc/irq/345/smp_affinity_list > 48 node 2 > Euler:/sys/bus/pci # cat /proc/irq/369/smp_affinity_list > 72 node 3 > Euler:/sys/bus/pci # cat /proc/irq/393/smp_affinity_list > 24 node 1 So few more questions. The only difference seems to be IRQ369 moving from 0 to 3 and having the device affinity to node 2 makes some sense because node 3 is closer. So far so good. I still have a large gap to get the whole picture. Namely why those other IRQs are not using any of the existing CPUs on the node 2. Could you explain that please? Btw. this all should be in the changelog. -- Michal Hocko SUSE Labs