`calibrate_APIC_clock()` takes 100 ms on AMD systems

`calibrate_APIC_clock()` takes 100 ms on AMD systems

[Paul Menzel]

Dear Linux folks,


Trying to decrease the boot time, I noticed on AMD systems 
`calibrate_APIC_clock()` takes around 100 ms.

     [    0.004764] Freeing SMP alternatives memory: 32K
     [    0.107676] smpboot: CPU0: AMD Ryzen 3 2200G with Radeon Vega 
Graphics (family: 0x17, model: 0x11, stepping: 0x0)

On a different system with `apic=verbose`:

     [    0.209161] Freeing SMP alternatives memory: 28K
     [    0.212830] Using local APIC timer interrupts.
            calibrating APIC timer ...
     [    0.320928] ... lapic delta = 625044
     [    0.320928] ... PM-Timer delta = 357959
     [    0.320928] ... PM-Timer result ok
     [    0.320929] ..... delta 625044
     [    0.320930] ..... mult: 26844619
     [    0.320930] ..... calibration result: 400028
     [    0.320931] ..... CPU clock speed is 4400.1228 MHz.
     [    0.320931] ..... host bus clock speed is 100.0028 MHz.
     [    0.320936] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD 
Graphics (family: 0x15, model: 0x13, stepping: 0x1)

On Intel systems TSC deadline is used, and the function returns right away.

         if (boot_cpu_has(X86_FEATURE_TSC_DEADLINE_TIMER))
                 return 0;

     [    0.078373] Freeing SMP alternatives memory: 32K
     [    0.080373] TSC deadline timer enabled
     [    0.080375] smpboot: CPU0: Intel(R) Core(TM) i7-5600U CPU @ 
2.60GHz (family: 0x6, model: 0x3d, stepping: 0x4)

100 ms is almost 10 % percent of the total one second the Linux kernel 
takes here on the AMD systems, so I wonder if there are ways to reduce that.

     #define LAPIC_CAL_LOOPS         (HZ/10)

is 100 on my system, so the while loop below runs 100 times.

         while (lapic_cal_loops <= LAPIC_CAL_LOOPS) {
                 /* Wait for a tick to elapse */
                 while (1) {
                         if (tsc_khz) {
                                 u64 tsc_now = rdtsc();
                                 if ((tsc_now - tsc_start) >= tsc_perj) {
                                         tsc_start += tsc_perj;
                                         break;
                                 }
                         } else {
                                 unsigned long jif_now = READ_ONCE(jiffies);

                                 if (time_after(jif_now, jif_start)) {
                                         jif_start = jif_now;
                                         break;
                                 }
                         }
                         cpu_relax();
                 }

                 /* Invoke the calibration routine */
                 local_irq_disable();
                 lapic_cal_handler(NULL);
                 local_irq_enable();
         }

Can you give more information, why HZ/10 is chosen? Is there a way to 
break out earlier?

Do AMD devices feature an alternatives like for Intel? What other 
possibilities are there? (Please excuse my ignorance.) For example, 
taking the calibration values from the firmware, or could they be passed 
on the command line, if the user knows, the system has not changed?


Kind regards,

Paul

Re: `calibrate_APIC_clock()` takes 100 ms on AMD systems

[Peter Zijlstra]

On Fri, Apr 24, 2020 at 04:59:21PM +0200, Paul Menzel wrote:
> Do AMD devices feature an alternatives like for Intel?

https://developer.amd.com/resources/developer-guides-manuals/

Let us know if you find something...