* [PATCH 0/4] x86 smpboot: optimize cpu_up() a bit more... @ 2015-08-16 15:45 Len Brown 2015-08-16 15:45 ` [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map Len Brown 0 siblings, 1 reply; 9+ messages in thread From: Len Brown @ 2015-08-16 15:45 UTC (permalink / raw) To: x86, linux-pm, linux-kernel This set of patches optimizes cpu_up() speed. Patch 3/4 depends on init_udelay, which was recently added to smpboot. (and yes, it should havde been in Usec all along -- it seems we somehow checked-in part of an old version of that patch) cheers, -Len ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map 2015-08-16 15:45 [PATCH 0/4] x86 smpboot: optimize cpu_up() a bit more Len Brown @ 2015-08-16 15:45 ` Len Brown 2015-08-16 15:45 ` [PATCH 2/4] x86 smpboot: remove udelay(100) when polling cpu_callin_map Len Brown ` (3 more replies) 0 siblings, 4 replies; 9+ messages in thread From: Len Brown @ 2015-08-16 15:45 UTC (permalink / raw) To: x86, linux-pm, linux-kernel; +Cc: Len Brown From: Len Brown <len.brown@intel.com> After the BSP sends the APIC INIT/SIPI/SIPI to the AP, it waits for the AP to come up and indicate that it is alive by setting its own bit in the cpu_initialized_mask. Linux polls for up to 10 seconds for this to happen. Each polling loop has a udelay(100) and a call to schedule(). The udelay(100) adds no value. For example, on my desktop, the BSP waits for the other 3 CPUs to come on line at boot for 305, 404, 405 usec. For resume from S3, it waits 317, 404, 405 usec. But when the udelay(100) is removed, the BSP waits 305, 310, 306 for boot, and 305, 307, 306 for resume. So for both boot and resume, removing the udelay(100) speeds online by about 100us in 2 of 3 cases. Signed-off-by: Len Brown <len.brown@intel.com> --- arch/x86/kernel/smpboot.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index b1f3ed9c..9ad88fb 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -898,7 +898,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) if (!boot_error) { /* - * Wait 10s total for a response from AP + * Wait 10s total for first sign of life from AP */ boot_error = -1; timeout = jiffies + 10*HZ; @@ -911,7 +911,6 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) boot_error = 0; break; } - udelay(100); schedule(); } } -- 2.5.0.330.g130be8e ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 2/4] x86 smpboot: remove udelay(100) when polling cpu_callin_map 2015-08-16 15:45 ` [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map Len Brown @ 2015-08-16 15:45 ` Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove " tip-bot for Len Brown 2015-08-16 15:45 ` [PATCH 3/4] x86 smpboot: remove SIPI delays from cpu_up() Len Brown ` (2 subsequent siblings) 3 siblings, 1 reply; 9+ messages in thread From: Len Brown @ 2015-08-16 15:45 UTC (permalink / raw) To: x86, linux-pm, linux-kernel; +Cc: Len Brown From: Len Brown <len.brown@intel.com> After the BSP sends INIT/SIPI/SIP to the AP and sees the AP in the cpu_initialized_map, it sets the AP loose via the cpu_callout_map, and waits for it via the cpu_callin_map. The BSP polls the cpu_callin_map with a udelay(100) and a schedule() in each iteration. The udelay(100) adds no value. For example, on my 4-CPU dekstop, the AP finishes cpu_callin() in under 70 usec and sets the cpu_callin_mask. The BSP, however, doesn't see that setting until over 30 usec later, because it was still running its udelay(100) when the AP finished. Deleting the udelay(100) in the cpu_callin_mask polling loop, saves from 0 to 100 usec per Application Processor. Signed-off-by: Len Brown <len.brown@intel.com> --- arch/x86/kernel/smpboot.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 9ad88fb..310b6f0 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -926,7 +926,6 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) * for the MTRR work(triggered by the AP coming online) * to be completed in the stop machine context. */ - udelay(100); schedule(); } } -- 2.5.0.330.g130be8e ^ permalink raw reply [flat|nested] 9+ messages in thread
* [tip:x86/boot] x86/smpboot: Remove udelay(100) when polling cpu_callin_map 2015-08-16 15:45 ` [PATCH 2/4] x86 smpboot: remove udelay(100) when polling cpu_callin_map Len Brown @ 2015-08-17 16:28 ` tip-bot for Len Brown 0 siblings, 0 replies; 9+ messages in thread From: tip-bot for Len Brown @ 2015-08-17 16:28 UTC (permalink / raw) To: linux-tip-commits Cc: tglx, arjan, zhugh.fnst, jschoenh, luto, hpa, peterz, bp, mingo, boris.ostrovsky, torvalds, dave.hansen, len.brown, imammedo, linux-kernel, paulmck Commit-ID: 2d99af8e8fd6c2dea11ab539f7aba69c37b845b4 Gitweb: http://git.kernel.org/tip/2d99af8e8fd6c2dea11ab539f7aba69c37b845b4 Author: Len Brown <len.brown@intel.com> AuthorDate: Sun, 16 Aug 2015 11:45:46 -0400 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Mon, 17 Aug 2015 10:42:27 +0200 x86/smpboot: Remove udelay(100) when polling cpu_callin_map After the BSP sends INIT/SIPI/SIP to the AP and sees the AP in the cpu_initialized_map, it sets the AP loose via the cpu_callout_map, and waits for it via the cpu_callin_map. The BSP polls the cpu_callin_map with a udelay(100) and a schedule() in each iteration. The udelay(100) adds no value. For example, on my 4-CPU dekstop, the AP finishes cpu_callin() in under 70 usec and sets the cpu_callin_mask. The BSP, however, doesn't see that setting until over 30 usec later, because it was still running its udelay(100) when the AP finished. Deleting the udelay(100) in the cpu_callin_mask polling loop, saves from 0 to 100 usec per Application Processor. Signed-off-by: Len Brown <len.brown@intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Jan H. Schönherr <jschoenh@amazon.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Zhu Guihua <zhugh.fnst@cn.fujitsu.com> Link: http://lkml.kernel.org/r/0aade12eabeb89a688c929fe80856eaea0544bb7.1439739165.git.len.brown@intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/kernel/smpboot.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 9ad88fb..310b6f0 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -926,7 +926,6 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) * for the MTRR work(triggered by the AP coming online) * to be completed in the stop machine context. */ - udelay(100); schedule(); } } ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 3/4] x86 smpboot: remove SIPI delays from cpu_up() 2015-08-16 15:45 ` [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map Len Brown 2015-08-16 15:45 ` [PATCH 2/4] x86 smpboot: remove udelay(100) when polling cpu_callin_map Len Brown @ 2015-08-16 15:45 ` Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove " tip-bot for Len Brown 2015-08-16 15:45 ` [PATCH 4/4] x86 smpboot: remove APIC.wait_for_init_deassert and atomic init_deasserted Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove udelay(100) when polling cpu_initialized_map tip-bot for Len Brown 3 siblings, 1 reply; 9+ messages in thread From: Len Brown @ 2015-08-16 15:45 UTC (permalink / raw) To: x86, linux-pm, linux-kernel; +Cc: Len Brown From: Len Brown <len.brown@intel.com> MPS 1.4 example code shows the following delays during processor on-line: INIT udelay(10,000) SIPI udelay(200) SIPI udelay(200) /* Linux actually implements this as udelay(300) */ Linux skips the udelay(10,000) on modern processors. This patch removes the udelay(200) after each SIPI on those same processors. All three legacy delays can be restored by the cmdline "cpu_init_udelay=10000". As measured by analyze_suspend.py, this patch speeds processor resume time on my desktop from 2.4ms to 1.8ms, per AP. Signed-off-by: Len Brown <len.brown@intel.com> --- arch/x86/kernel/smpboot.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 310b6f0..3d992b6 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -665,7 +665,8 @@ wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) /* * Give the other CPU some time to accept the IPI. */ - udelay(300); + if (init_udelay != 0) + udelay(300); pr_debug("Startup point 1\n"); @@ -675,7 +676,8 @@ wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) /* * Give the other CPU some time to accept the IPI. */ - udelay(200); + if (init_udelay != 0) + udelay(200); if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */ apic_write(APIC_ESR, 0); -- 2.5.0.330.g130be8e ^ permalink raw reply [flat|nested] 9+ messages in thread
* [tip:x86/boot] x86/smpboot: Remove SIPI delays from cpu_up() 2015-08-16 15:45 ` [PATCH 3/4] x86 smpboot: remove SIPI delays from cpu_up() Len Brown @ 2015-08-17 16:28 ` tip-bot for Len Brown 0 siblings, 0 replies; 9+ messages in thread From: tip-bot for Len Brown @ 2015-08-17 16:28 UTC (permalink / raw) To: linux-tip-commits Cc: peterz, hpa, mingo, paulmck, linux-kernel, boris.ostrovsky, zhugh.fnst, jschoenh, tglx, luto, arjan, imammedo, len.brown, dave.hansen, bp, torvalds Commit-ID: a9bcaa02a5104ace6a9d9e4a9cd9192a9e7744d6 Gitweb: http://git.kernel.org/tip/a9bcaa02a5104ace6a9d9e4a9cd9192a9e7744d6 Author: Len Brown <len.brown@intel.com> AuthorDate: Sun, 16 Aug 2015 11:45:47 -0400 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Mon, 17 Aug 2015 10:42:27 +0200 x86/smpboot: Remove SIPI delays from cpu_up() MPS 1.4 example code shows the following required delays during processor on-lining: INIT udelay(10,000) SIPI udelay(200) SIPI udelay(200) /* Linux actually implements this as udelay(300) */ Linux skips the udelay(10,000) on modern processors. This patch removes the udelay(200) after each SIPI on those same processors. All three legacy delays can be restored by the cmdline "cpu_init_udelay=10000". As measured by analyze_suspend.py, this patch speeds processor resume time on my desktop from 2.4ms to 1.8ms, per AP. Signed-off-by: Len Brown <len.brown@intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Jan H. Schönherr <jschoenh@amazon.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Zhu Guihua <zhugh.fnst@cn.fujitsu.com> Link: http://lkml.kernel.org/r/a5dfdbc8fbfdd813784da204aad5677fe459ac37.1439739165.git.len.brown@intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/kernel/smpboot.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 310b6f0..6740264 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -665,7 +665,8 @@ wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) /* * Give the other CPU some time to accept the IPI. */ - udelay(300); + if (init_udelay) + udelay(300); pr_debug("Startup point 1\n"); @@ -675,7 +676,8 @@ wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) /* * Give the other CPU some time to accept the IPI. */ - udelay(200); + if (init_udelay) + udelay(200); if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */ apic_write(APIC_ESR, 0); ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 4/4] x86 smpboot: remove APIC.wait_for_init_deassert and atomic init_deasserted 2015-08-16 15:45 ` [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map Len Brown 2015-08-16 15:45 ` [PATCH 2/4] x86 smpboot: remove udelay(100) when polling cpu_callin_map Len Brown 2015-08-16 15:45 ` [PATCH 3/4] x86 smpboot: remove SIPI delays from cpu_up() Len Brown @ 2015-08-16 15:45 ` Len Brown 2015-08-17 16:29 ` [tip:x86/boot] x86/smpboot: Remove " tip-bot for Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove udelay(100) when polling cpu_initialized_map tip-bot for Len Brown 3 siblings, 1 reply; 9+ messages in thread From: Len Brown @ 2015-08-16 15:45 UTC (permalink / raw) To: x86, linux-pm, linux-kernel; +Cc: Len Brown From: Len Brown <len.brown@intel.com> Both the per-APIC flag ".wait_for_init_deassert", and the global atomic_t "init_deasserted" are dead code -- remove them. For all APIC types, "wait_for_master()" prevents an AP from proceeding until the BSP has set cpu_callout_mask, making "init_deasserted" {unnecessary}: BSP: <de-assert INIT> ... BSP: {set init_deasserted} AP: wait_for_master() set cpu_initialized_mask wait for cpu_callout_mask BSP: test cpu_initialized_mask BSP: set cpu_callout_mask AP: test cpu_callout_mask AP: {wait for init_deasserted} ... AP: <touch APIC> Deleting the {dead code} above is necessary to enable some parallelism in a future patch. Signed-off-by: Len Brown <len.brown@intel.com> --- arch/x86/include/asm/apic.h | 2 -- arch/x86/kernel/apic/apic_flat_64.c | 2 -- arch/x86/kernel/apic/apic_noop.c | 1 - arch/x86/kernel/apic/apic_numachip.c | 2 -- arch/x86/kernel/apic/bigsmp_32.c | 1 - arch/x86/kernel/apic/probe_32.c | 1 - arch/x86/kernel/apic/x2apic_cluster.c | 1 - arch/x86/kernel/apic/x2apic_phys.c | 1 - arch/x86/kernel/apic/x2apic_uv_x.c | 2 -- arch/x86/kernel/smpboot.c | 16 +++------------- 10 files changed, 3 insertions(+), 26 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index c839363..ebf6d5e 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -313,7 +313,6 @@ struct apic { /* wakeup_secondary_cpu */ int (*wakeup_secondary_cpu)(int apicid, unsigned long start_eip); - bool wait_for_init_deassert; void (*inquire_remote_apic)(int apicid); /* apic ops */ @@ -378,7 +377,6 @@ extern struct apic *__apicdrivers[], *__apicdrivers_end[]; * APIC functionality to boot other CPUs - only used on SMP: */ #ifdef CONFIG_SMP -extern atomic_t init_deasserted; extern int wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip); #endif diff --git a/arch/x86/kernel/apic/apic_flat_64.c b/arch/x86/kernel/apic/apic_flat_64.c index de918c4..f92ab36 100644 --- a/arch/x86/kernel/apic/apic_flat_64.c +++ b/arch/x86/kernel/apic/apic_flat_64.c @@ -191,7 +191,6 @@ static struct apic apic_flat = { .send_IPI_all = flat_send_IPI_all, .send_IPI_self = apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, @@ -299,7 +298,6 @@ static struct apic apic_physflat = { .send_IPI_all = physflat_send_IPI_all, .send_IPI_self = apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/apic_noop.c b/arch/x86/kernel/apic/apic_noop.c index b205cdb..0d96749 100644 --- a/arch/x86/kernel/apic/apic_noop.c +++ b/arch/x86/kernel/apic/apic_noop.c @@ -152,7 +152,6 @@ struct apic apic_noop = { .wakeup_secondary_cpu = noop_wakeup_secondary_cpu, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = noop_apic_read, diff --git a/arch/x86/kernel/apic/apic_numachip.c b/arch/x86/kernel/apic/apic_numachip.c index 017149c..b548fd3 100644 --- a/arch/x86/kernel/apic/apic_numachip.c +++ b/arch/x86/kernel/apic/apic_numachip.c @@ -92,7 +92,6 @@ static int numachip_wakeup_secondary(int phys_apicid, unsigned long start_rip) write_lcsr(CSR_G3_EXT_IRQ_GEN, int_gen.v); - atomic_set(&init_deasserted, 1); return 0; } @@ -235,7 +234,6 @@ static const struct apic apic_numachip __refconst = { .send_IPI_self = numachip_send_IPI_self, .wakeup_secondary_cpu = numachip_wakeup_secondary, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, /* REMRD not supported */ .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/bigsmp_32.c b/arch/x86/kernel/apic/bigsmp_32.c index c4a8d63..971cf88 100644 --- a/arch/x86/kernel/apic/bigsmp_32.c +++ b/arch/x86/kernel/apic/bigsmp_32.c @@ -186,7 +186,6 @@ static struct apic apic_bigsmp = { .send_IPI_all = bigsmp_send_IPI_all, .send_IPI_self = default_send_IPI_self, - .wait_for_init_deassert = true, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/probe_32.c b/arch/x86/kernel/apic/probe_32.c index bda4886..7694ae6 100644 --- a/arch/x86/kernel/apic/probe_32.c +++ b/arch/x86/kernel/apic/probe_32.c @@ -111,7 +111,6 @@ static struct apic apic_default = { .send_IPI_all = default_send_IPI_all, .send_IPI_self = default_send_IPI_self, - .wait_for_init_deassert = true, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/x2apic_cluster.c b/arch/x86/kernel/apic/x2apic_cluster.c index ab3219b..1b6c1a4 100644 --- a/arch/x86/kernel/apic/x2apic_cluster.c +++ b/arch/x86/kernel/apic/x2apic_cluster.c @@ -272,7 +272,6 @@ static struct apic apic_x2apic_cluster = { .send_IPI_all = x2apic_send_IPI_all, .send_IPI_self = x2apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = native_apic_msr_read, diff --git a/arch/x86/kernel/apic/x2apic_phys.c b/arch/x86/kernel/apic/x2apic_phys.c index 3ffd925..662e915 100644 --- a/arch/x86/kernel/apic/x2apic_phys.c +++ b/arch/x86/kernel/apic/x2apic_phys.c @@ -128,7 +128,6 @@ static struct apic apic_x2apic_phys = { .send_IPI_all = x2apic_send_IPI_all, .send_IPI_self = x2apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = native_apic_msr_read, diff --git a/arch/x86/kernel/apic/x2apic_uv_x.c b/arch/x86/kernel/apic/x2apic_uv_x.c index c8d9295..4a13946 100644 --- a/arch/x86/kernel/apic/x2apic_uv_x.c +++ b/arch/x86/kernel/apic/x2apic_uv_x.c @@ -248,7 +248,6 @@ static int uv_wakeup_secondary(int phys_apicid, unsigned long start_rip) APIC_DM_STARTUP; uv_write_global_mmr64(pnode, UVH_IPI_INT, val); - atomic_set(&init_deasserted, 1); return 0; } @@ -414,7 +413,6 @@ static struct apic __refdata apic_x2apic_uv_x = { .send_IPI_self = uv_send_IPI_self, .wakeup_secondary_cpu = uv_wakeup_secondary, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = native_apic_msr_read, diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 3d992b6..70276bc 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -97,8 +97,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map); DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); EXPORT_PER_CPU_SYMBOL(cpu_info); -atomic_t init_deasserted; - static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip) { unsigned long flags; @@ -146,16 +144,11 @@ static void smp_callin(void) /* * If waken up by an INIT in an 82489DX configuration - * we may get here before an INIT-deassert IPI reaches - * our local APIC. We have to wait for the IPI or we'll - * lock up on an APIC access. - * - * Since CPU0 is not wakened up by INIT, it doesn't wait for the IPI. + * cpu_callout_mask guarantees we don't get here before + * an INIT_deassert IPI reaches our local APIC, so it is + * now safe to touch our local APIC. */ cpuid = smp_processor_id(); - if (apic->wait_for_init_deassert && cpuid) - while (!atomic_read(&init_deasserted)) - cpu_relax(); /* * (This works even if the APIC is not enabled.) @@ -620,7 +613,6 @@ wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) send_status = safe_apic_wait_icr_idle(); mb(); - atomic_set(&init_deasserted, 1); /* * Should we send STARTUP IPIs ? @@ -861,8 +853,6 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) * the targeted processor. */ - atomic_set(&init_deasserted, 0); - if (get_uv_system_type() != UV_NON_UNIQUE_APIC) { pr_debug("Setting warm reset code and vector.\n"); -- 2.5.0.330.g130be8e ^ permalink raw reply [flat|nested] 9+ messages in thread
* [tip:x86/boot] x86/smpboot: Remove APIC.wait_for_init_deassert and atomic init_deasserted 2015-08-16 15:45 ` [PATCH 4/4] x86 smpboot: remove APIC.wait_for_init_deassert and atomic init_deasserted Len Brown @ 2015-08-17 16:29 ` tip-bot for Len Brown 0 siblings, 0 replies; 9+ messages in thread From: tip-bot for Len Brown @ 2015-08-17 16:29 UTC (permalink / raw) To: linux-tip-commits Cc: len.brown, arjan, linux-kernel, peterz, tglx, mingo, paulmck, jschoenh, zhugh.fnst, dave.hansen, luto, imammedo, bp, hpa, boris.ostrovsky, torvalds Commit-ID: 656bba306827a44ed73b3f93f75bb3147de17fae Gitweb: http://git.kernel.org/tip/656bba306827a44ed73b3f93f75bb3147de17fae Author: Len Brown <len.brown@intel.com> AuthorDate: Sun, 16 Aug 2015 11:45:48 -0400 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Mon, 17 Aug 2015 10:42:28 +0200 x86/smpboot: Remove APIC.wait_for_init_deassert and atomic init_deasserted Both the per-APIC flag ".wait_for_init_deassert", and the global atomic_t "init_deasserted" are dead code -- remove them. For all APIC types, "wait_for_master()" prevents an AP from proceeding until the BSP has set cpu_callout_mask, making "init_deasserted" {unnecessary}: BSP: <de-assert INIT> ... BSP: {set init_deasserted} AP: wait_for_master() set cpu_initialized_mask wait for cpu_callout_mask BSP: test cpu_initialized_mask BSP: set cpu_callout_mask AP: test cpu_callout_mask AP: {wait for init_deasserted} ... AP: <touch APIC> Deleting the {dead code} above is necessary to enable some parallelism in a future patch. Signed-off-by: Len Brown <len.brown@intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Jan H. Schönherr <jschoenh@amazon.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Zhu Guihua <zhugh.fnst@cn.fujitsu.com> Link: http://lkml.kernel.org/r/de4b3a9bab894735e285870b5296da25ee6a8a5a.1439739165.git.len.brown@intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/include/asm/apic.h | 2 -- arch/x86/kernel/apic/apic_flat_64.c | 2 -- arch/x86/kernel/apic/apic_noop.c | 1 - arch/x86/kernel/apic/apic_numachip.c | 2 -- arch/x86/kernel/apic/bigsmp_32.c | 1 - arch/x86/kernel/apic/probe_32.c | 1 - arch/x86/kernel/apic/x2apic_cluster.c | 1 - arch/x86/kernel/apic/x2apic_phys.c | 1 - arch/x86/kernel/apic/x2apic_uv_x.c | 2 -- arch/x86/kernel/smpboot.c | 16 +++------------- 10 files changed, 3 insertions(+), 26 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index c839363..ebf6d5e 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -313,7 +313,6 @@ struct apic { /* wakeup_secondary_cpu */ int (*wakeup_secondary_cpu)(int apicid, unsigned long start_eip); - bool wait_for_init_deassert; void (*inquire_remote_apic)(int apicid); /* apic ops */ @@ -378,7 +377,6 @@ extern struct apic *__apicdrivers[], *__apicdrivers_end[]; * APIC functionality to boot other CPUs - only used on SMP: */ #ifdef CONFIG_SMP -extern atomic_t init_deasserted; extern int wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip); #endif diff --git a/arch/x86/kernel/apic/apic_flat_64.c b/arch/x86/kernel/apic/apic_flat_64.c index de918c4..f92ab36 100644 --- a/arch/x86/kernel/apic/apic_flat_64.c +++ b/arch/x86/kernel/apic/apic_flat_64.c @@ -191,7 +191,6 @@ static struct apic apic_flat = { .send_IPI_all = flat_send_IPI_all, .send_IPI_self = apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, @@ -299,7 +298,6 @@ static struct apic apic_physflat = { .send_IPI_all = physflat_send_IPI_all, .send_IPI_self = apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/apic_noop.c b/arch/x86/kernel/apic/apic_noop.c index b205cdb..0d96749 100644 --- a/arch/x86/kernel/apic/apic_noop.c +++ b/arch/x86/kernel/apic/apic_noop.c @@ -152,7 +152,6 @@ struct apic apic_noop = { .wakeup_secondary_cpu = noop_wakeup_secondary_cpu, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = noop_apic_read, diff --git a/arch/x86/kernel/apic/apic_numachip.c b/arch/x86/kernel/apic/apic_numachip.c index 017149c..b548fd3 100644 --- a/arch/x86/kernel/apic/apic_numachip.c +++ b/arch/x86/kernel/apic/apic_numachip.c @@ -92,7 +92,6 @@ static int numachip_wakeup_secondary(int phys_apicid, unsigned long start_rip) write_lcsr(CSR_G3_EXT_IRQ_GEN, int_gen.v); - atomic_set(&init_deasserted, 1); return 0; } @@ -235,7 +234,6 @@ static const struct apic apic_numachip __refconst = { .send_IPI_self = numachip_send_IPI_self, .wakeup_secondary_cpu = numachip_wakeup_secondary, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, /* REMRD not supported */ .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/bigsmp_32.c b/arch/x86/kernel/apic/bigsmp_32.c index c4a8d63..971cf88 100644 --- a/arch/x86/kernel/apic/bigsmp_32.c +++ b/arch/x86/kernel/apic/bigsmp_32.c @@ -186,7 +186,6 @@ static struct apic apic_bigsmp = { .send_IPI_all = bigsmp_send_IPI_all, .send_IPI_self = default_send_IPI_self, - .wait_for_init_deassert = true, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/probe_32.c b/arch/x86/kernel/apic/probe_32.c index bda4886..7694ae6 100644 --- a/arch/x86/kernel/apic/probe_32.c +++ b/arch/x86/kernel/apic/probe_32.c @@ -111,7 +111,6 @@ static struct apic apic_default = { .send_IPI_all = default_send_IPI_all, .send_IPI_self = default_send_IPI_self, - .wait_for_init_deassert = true, .inquire_remote_apic = default_inquire_remote_apic, .read = native_apic_mem_read, diff --git a/arch/x86/kernel/apic/x2apic_cluster.c b/arch/x86/kernel/apic/x2apic_cluster.c index ab3219b..1b6c1a4 100644 --- a/arch/x86/kernel/apic/x2apic_cluster.c +++ b/arch/x86/kernel/apic/x2apic_cluster.c @@ -272,7 +272,6 @@ static struct apic apic_x2apic_cluster = { .send_IPI_all = x2apic_send_IPI_all, .send_IPI_self = x2apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = native_apic_msr_read, diff --git a/arch/x86/kernel/apic/x2apic_phys.c b/arch/x86/kernel/apic/x2apic_phys.c index 3ffd925..662e915 100644 --- a/arch/x86/kernel/apic/x2apic_phys.c +++ b/arch/x86/kernel/apic/x2apic_phys.c @@ -128,7 +128,6 @@ static struct apic apic_x2apic_phys = { .send_IPI_all = x2apic_send_IPI_all, .send_IPI_self = x2apic_send_IPI_self, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = native_apic_msr_read, diff --git a/arch/x86/kernel/apic/x2apic_uv_x.c b/arch/x86/kernel/apic/x2apic_uv_x.c index c8d9295..4a13946 100644 --- a/arch/x86/kernel/apic/x2apic_uv_x.c +++ b/arch/x86/kernel/apic/x2apic_uv_x.c @@ -248,7 +248,6 @@ static int uv_wakeup_secondary(int phys_apicid, unsigned long start_rip) APIC_DM_STARTUP; uv_write_global_mmr64(pnode, UVH_IPI_INT, val); - atomic_set(&init_deasserted, 1); return 0; } @@ -414,7 +413,6 @@ static struct apic __refdata apic_x2apic_uv_x = { .send_IPI_self = uv_send_IPI_self, .wakeup_secondary_cpu = uv_wakeup_secondary, - .wait_for_init_deassert = false, .inquire_remote_apic = NULL, .read = native_apic_msr_read, diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 6740264..c15d007 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -97,8 +97,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map); DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); EXPORT_PER_CPU_SYMBOL(cpu_info); -atomic_t init_deasserted; - static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip) { unsigned long flags; @@ -146,16 +144,11 @@ static void smp_callin(void) /* * If waken up by an INIT in an 82489DX configuration - * we may get here before an INIT-deassert IPI reaches - * our local APIC. We have to wait for the IPI or we'll - * lock up on an APIC access. - * - * Since CPU0 is not wakened up by INIT, it doesn't wait for the IPI. + * cpu_callout_mask guarantees we don't get here before + * an INIT_deassert IPI reaches our local APIC, so it is + * now safe to touch our local APIC. */ cpuid = smp_processor_id(); - if (apic->wait_for_init_deassert && cpuid) - while (!atomic_read(&init_deasserted)) - cpu_relax(); /* * (This works even if the APIC is not enabled.) @@ -620,7 +613,6 @@ wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) send_status = safe_apic_wait_icr_idle(); mb(); - atomic_set(&init_deasserted, 1); /* * Should we send STARTUP IPIs ? @@ -861,8 +853,6 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) * the targeted processor. */ - atomic_set(&init_deasserted, 0); - if (get_uv_system_type() != UV_NON_UNIQUE_APIC) { pr_debug("Setting warm reset code and vector.\n"); ^ permalink raw reply [flat|nested] 9+ messages in thread
* [tip:x86/boot] x86/smpboot: Remove udelay(100) when polling cpu_initialized_map 2015-08-16 15:45 ` [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map Len Brown ` (2 preceding siblings ...) 2015-08-16 15:45 ` [PATCH 4/4] x86 smpboot: remove APIC.wait_for_init_deassert and atomic init_deasserted Len Brown @ 2015-08-17 16:28 ` tip-bot for Len Brown 3 siblings, 0 replies; 9+ messages in thread From: tip-bot for Len Brown @ 2015-08-17 16:28 UTC (permalink / raw) To: linux-tip-commits Cc: torvalds, luto, mingo, boris.ostrovsky, zhugh.fnst, peterz, imammedo, bp, paulmck, jschoenh, tglx, len.brown, arjan, linux-kernel, dave.hansen, hpa Commit-ID: 6e38f1e79d16f4fa9e5cf06792500e11c96a6f84 Gitweb: http://git.kernel.org/tip/6e38f1e79d16f4fa9e5cf06792500e11c96a6f84 Author: Len Brown <len.brown@intel.com> AuthorDate: Sun, 16 Aug 2015 11:45:45 -0400 Committer: Ingo Molnar <mingo@kernel.org> CommitDate: Mon, 17 Aug 2015 10:42:27 +0200 x86/smpboot: Remove udelay(100) when polling cpu_initialized_map After the BSP sends the APIC INIT/SIPI/SIPI to the AP, it waits for the AP to come up and indicate that it is alive by setting its own bit in the cpu_initialized_mask. Linux polls for up to 10 seconds for this to happen. Each polling loop has a udelay(100) and a call to schedule(). The udelay(100) adds no value. For example, on my desktop, the BSP waits for the other 3 CPUs to come on line at boot for 305, 404, 405 usec. For resume from S3, it waits 317, 404, 405 usec. But when the udelay(100) is removed, the BSP waits 305, 310, 306 for boot, and 305, 307, 306 for resume. So for both boot and resume, removing the udelay(100) speeds online by about 100us in 2 of 3 cases. Signed-off-by: Len Brown <len.brown@intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Jan H. Schönherr <jschoenh@amazon.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Zhu Guihua <zhugh.fnst@cn.fujitsu.com> Link: http://lkml.kernel.org/r/33ef746c67d2489cad0a9b1958cf71167232ff2b.1439739165.git.len.brown@intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/kernel/smpboot.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index b1f3ed9c..9ad88fb 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -898,7 +898,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) if (!boot_error) { /* - * Wait 10s total for a response from AP + * Wait 10s total for first sign of life from AP */ boot_error = -1; timeout = jiffies + 10*HZ; @@ -911,7 +911,6 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) boot_error = 0; break; } - udelay(100); schedule(); } } ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2015-08-17 16:30 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-08-16 15:45 [PATCH 0/4] x86 smpboot: optimize cpu_up() a bit more Len Brown 2015-08-16 15:45 ` [PATCH 1/4] x86 smpboot: remove udelay(100) when polling cpu_initialized_map Len Brown 2015-08-16 15:45 ` [PATCH 2/4] x86 smpboot: remove udelay(100) when polling cpu_callin_map Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove " tip-bot for Len Brown 2015-08-16 15:45 ` [PATCH 3/4] x86 smpboot: remove SIPI delays from cpu_up() Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove " tip-bot for Len Brown 2015-08-16 15:45 ` [PATCH 4/4] x86 smpboot: remove APIC.wait_for_init_deassert and atomic init_deasserted Len Brown 2015-08-17 16:29 ` [tip:x86/boot] x86/smpboot: Remove " tip-bot for Len Brown 2015-08-17 16:28 ` [tip:x86/boot] x86/smpboot: Remove udelay(100) when polling cpu_initialized_map tip-bot for Len Brown
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
Powered by JetHome