[MAYBE REGRESSION] kbuild time of kernel compiling

[MAYBE REGRESSION] kbuild time of kernel compiling

[Jingbo Xu]

Hi,

I noticed a regression of kbuild time in v6.7, especially when running
`make` command when all images have already been compiled.  In v6.6 it
takes ~3s, while in v6.7 it takes ~18s.

I'm not sure if it's a known issue, or an extra action configurable with
a "CONFIG_XX" option.


Following is the kbuild time in v6.6 versus v7.7, with
arch/x86/configs/x86_64_defconfig used here:


v6.6
```
make clean
make olddefconfig

# first full compiling
$time make bzImage -j128 -s

real	1m9.896s
user	36m56.153s
sys	4m21.748s

# second time with image already compiled
$time make bzImage -j128 -s

real	0m2.776s
user	0m13.823s
sys	0m3.936s
```


v6.7
```
make clean
make olddefconfig

# first full compiling
$time make bzImage -j128 -s

real	1m22.865s
user	37m25.977s
sys	4m22.094s

# second time with image already compiled
$time make bzImage -j128 -s

real	0m18.209s
user	0m29.243s
sys	0m4.330s
```


I tried to bisect, while commit d4e175f2c460 ("Merge tag 'vfs-6.7.super'
of gitolite.kernel.org:pub/scm/linux/kernel/git/vfs/vfs") is identified
as the first bad commit, which is obviously wrong.  As I'm not familiar
with the structure of the merge commit, the further investigation has
not been made yet.


Besides, it seems that it will take most of the time before "CALL
scripts/checksyscalls.sh" is printed (with "INSTALL libsubcmd_headers"
has already printed).

$time make bzImage -j128
  DESCEND objtool
  INSTALL libsubcmd_headers

  CALL    scripts/checksyscalls.sh
  BUILD   arch/x86/boot/bzImage
Kernel: arch/x86/boot/bzImage is ready  (#437)

-- 
Thanks,
Jingbo

Re: [MAYBE REGRESSION] kbuild time of kernel compiling

[Linux regression tracking (Thorsten Leemhuis)]

On 22.01.24 09:32, Jingbo Xu wrote:
> 
> I noticed a regression of kbuild time in v6.7, especially when running
> `make` command when all images have already been compiled.  In v6.6 it
> takes ~3s, while in v6.7 it takes ~18s.
> 
> I'm not sure if it's a known issue, or an extra action configurable with
> a "CONFIG_XX" option.
> [...]

Not my area of expertise, so take this with a gain of salt, as I might
mislead you. Nevertheless:

One problem with compile times in 6.7 was noticed and fixed in mainline;
I wonder if that is the same issue as yours:

https://lore.kernel.org/lkml/18c6df0d-45ed-450c-9eda-95160a2bbb8e@gmail.com/
https://git.kernel.org/torvalds/c/31e97d7c9ae3de072d7b424b2cf706a03ec10720

I think there was another issue, but can't remember it from the top of
my head. Maybe simply checking if 6.8-rc1 fixes things might be the best.

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.

Re: [MAYBE REGRESSION] kbuild time of kernel compiling

[Jingbo Xu]

On 1/22/24 6:50 PM, Linux regression tracking (Thorsten Leemhuis) wrote:
> On 22.01.24 09:32, Jingbo Xu wrote:
>>
>> I noticed a regression of kbuild time in v6.7, especially when running
>> `make` command when all images have already been compiled.  In v6.6 it
>> takes ~3s, while in v6.7 it takes ~18s.
>>
>> I'm not sure if it's a known issue, or an extra action configurable with
>> a "CONFIG_XX" option.
>> [...]
> 
> Not my area of expertise, so take this with a gain of salt, as I might
> mislead you. Nevertheless:
> 
> One problem with compile times in 6.7 was noticed and fixed in mainline;
> I wonder if that is the same issue as yours:
> 
> https://lore.kernel.org/lkml/18c6df0d-45ed-450c-9eda-95160a2bbb8e@gmail.com/
> https://git.kernel.org/torvalds/c/31e97d7c9ae3de072d7b424b2cf706a03ec10720
> 
> I think there was another issue, but can't remember it from the top of
> my head. Maybe simply checking if 6.8-rc1 fixes things might be the best.

Yeah this issue doesn't reproduce anymore after pulling 6.8-rc1.  I
didn't test the latest 6.8-rc1 as our internal mirror repo is one day
later than the upstream, while my machine can not connect to github at
the time...

Anyway thanks for the feedback and sorry for the noise.


> 
> Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
> --
> Everything you wanna know about Linux kernel regression tracking:
> https://linux-regtracking.leemhuis.info/about/#tldr
> If I did something stupid, please tell me, as explained on that page.

-- 
Thanks,
Jingbo

Re: [MAYBE REGRESSION] kbuild time of kernel compiling

[Masahiro Yamada]

On Mon, Jan 22, 2024 at 5:32 PM Jingbo Xu <jefflexu@linux.alibaba.com> wrote:
>
> Hi,
>
> I noticed a regression of kbuild time in v6.7, especially when running
> `make` command when all images have already been compiled.  In v6.6 it
> takes ~3s, while in v6.7 it takes ~18s.
>
> I'm not sure if it's a known issue, or an extra action configurable with
> a "CONFIG_XX" option.
>
>
> Following is the kbuild time in v6.6 versus v7.7, with
> arch/x86/configs/x86_64_defconfig used here:
>
>
> v6.6
> ```
> make clean
> make olddefconfig
>
> # first full compiling
> $time make bzImage -j128 -s
>
> real    1m9.896s
> user    36m56.153s
> sys     4m21.748s
>
> # second time with image already compiled
> $time make bzImage -j128 -s
>
> real    0m2.776s
> user    0m13.823s
> sys     0m3.936s
> ```
>
>
> v6.7
> ```
> make clean
> make olddefconfig
>
> # first full compiling
> $time make bzImage -j128 -s
>
> real    1m22.865s
> user    37m25.977s
> sys     4m22.094s
>
> # second time with image already compiled
> $time make bzImage -j128 -s
>
> real    0m18.209s
> user    0m29.243s
> sys     0m4.330s
> ```
>
>
> I tried to bisect, while commit d4e175f2c460 ("Merge tag 'vfs-6.7.super'
> of gitolite.kernel.org:pub/scm/linux/kernel/git/vfs/vfs") is identified
> as the first bad commit, which is obviously wrong.  As I'm not familiar
> with the structure of the merge commit, the further investigation has
> not been made yet.




If you use olddefconfig, git-bisect will not produce the correct result.

'make olddefconfig' updates the existing .config.
The resulting .config depends on what you had
before running 'make olddefconfig'.

You need to use a deterministic target (e.g. defconfig)
during the git-bisect.





>
>
> Besides, it seems that it will take most of the time before "CALL
> scripts/checksyscalls.sh" is printed (with "INSTALL libsubcmd_headers"
> has already printed).
>
> $time make bzImage -j128
>   DESCEND objtool
>   INSTALL libsubcmd_headers
>
>   CALL    scripts/checksyscalls.sh
>   BUILD   arch/x86/boot/bzImage
> Kernel: arch/x86/boot/bzImage is ready  (#437)
>
> --
> Thanks,
> Jingbo
>


--
Best Regards
Masahiro Yamada