Linus Torvalds writes: > On Tue, Feb 8, 2011 at 12:28 PM, Eric W. Biederman > wrote: >> >> A quick heads up.  2.6.38-rc4 looks like the worst kernel I've ever tried to >> test.  It boots up properly and looks ok, but I can't get it to even >> compile the programs I usually test with. 2.6.38-rc3 at least managed >> that yesterday. > > gcc dying with ICE or SIGSEGV? Or what? Seriously lacking information > here. > > Are you using btrfs or cifs (the two filesystems that had > bigger-than-average changes)? > > Any chance to bisect it (even if just partially - a couple of > bisection runs would already narrow it down quite a bit)? What I have been able to do today on this one is, narrow it down a little and provide a reasonable description of one of the problem. In a fedora 12 world in a subdirectory on ext4 with the applications pinned into a mount namespace. I am seeing compiles of gcc-4.4.3 fail. A quick survey of the compile failures makes it look like these compiles failures are deterministic, at least I see the same error messages in multiple failed builds. The machine is a dual socket quad core non hyperthread machine with 10G of RAM. The cpus are: Intel(R) Xeon(R) CPU E5420 @ 2.50GHz In the case that fails I am simultaneously compiling a 32bit gcc cross compiler for x86_64 and i386, for the same source but with different object directories. The compiles failures happen exactly as before if I rerun the build commands. I'm still trying to figure out why the individual files are failing. I expect if I do that will give me a much bigger clue. What is interesting is that this fails exactly the same way in at least three invocations of the command and I expect all of them. With failures taking about 30 minutes. I might try a bisect tomorrow. The unfortunate thing is that this weird compile failure during a parallel build doesn't tell me much yet about what the root cause is. I have attached my build scripts in case that helps. To understand. At the moment I am still trying to break this down into something smaller so I can see just what the heck is going on. And I will keep poking at it until I get somewhere. The annoying thing is that looking at the other failures I was seeing before the compile fell apart. I think I am might be on top of half a dozen other regressions as well. I don't have a clue how I am going to get through all of those regressions before before 2.6.38 is out. Eric The failures look like: $ make -C gcc-stage1-x86_64 all-gcc all-target-libgcc install-gcc install-target-libgcc DESTDIR=/bld/Across/Across-2.0.0/sysroot-x86_64 gcc -qlanglvl=ansi -c -DHAVE_CONFIG_H -g -O2 -I. -I/bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/../include -W -Wall -Wwrite-strings -Wc++-compat -Wstrict-prototypes /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c -o fibheap.o gcc: unrecognized option '-qlanglvl=ansi' /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c: In function ‘fibheap_union’: /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:151: warning: implicit declaration of function ‘free’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:151: warning: incompatible implicit declaration of built-in function ‘free’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:156: warning: incompatible implicit declaration of built-in function ‘free’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:172: warning: incompatible implicit declaration of built-in function ‘free’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c: In function ‘fibheap_extract_min’: /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:190: warning: incompatible implicit declaration of built-in function ‘free’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c: In function ‘fibheap_delete_node’: /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:258: error: ‘LONG_MIN’ undeclared (first use in this function) /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:258: error: (Each undeclared identifier is reported only once /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:258: error: for each function it appears in.) /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c: In function ‘fibheap_delete’: /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:269: warning: incompatible implicit declaration of built-in function ‘free’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c: In function ‘fibheap_consolidate’: /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:360: warning: implicit declaration of function ‘memset’ /bld/Across/Across-2.0.0/gcc-4.4.3/libiberty/fibheap.c:360: warning: incompatible implicit declaration of built-in function ‘memset’ $ make -C gcc-stage1-i386 all-gcc all-target-libgcc install-gcc install-target-libgcc DESTDIR=/bld/Across/Across-2.0.0/sysroot-i386 make: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386' make[1]: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/libiberty' make[2]: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/libiberty/testsuite' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/libiberty/testsuite' make[1]: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/libiberty' make[1]: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/intl' make[1]: Nothing to be done for `all'. make[1]: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/intl' make[1]: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/build-i686-pc-linux-gnu/libiberty' make[2]: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/build-i686-pc-linux-gnu/libiberty/testsuite' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/build-i686-pc-linux-gnu/libiberty/testsuite' make[1]: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/build-i686-pc-linux-gnu/libiberty' make[1]: Entering directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/build-i686-pc-linux-gnu/fixincludes' gcc -g -O2 -o fixincl fixincl.o fixtests.o fixfixes.o server.o procopen.o fixlib.o fixopts.o ../libiberty/libiberty.a fixincl.o: In function `quoted_file_exists': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:624: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:624: undefined reference to `_sch_istable' fixincl.o: In function `egrep_test': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:600: undefined reference to `xregexec' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:600: undefined reference to `xregexec' fixincl.o: In function `test_for_changes': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:1211: undefined reference to `xregexec' fixincl.o: In function `extract_quoted_files': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:711: undefined reference to `xregexec' fixincl.o: In function `initialize': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:245: undefined reference to `_sch_istable' fixincl.o: In function `main': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:139: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixincl.c:157: undefined reference to `_sch_istable' fixtests.o: In function `machine_name_test': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixtests.c:79: undefined reference to `xregexec' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixtests.c:104: undefined reference to `xregexec' fixfixes.o: In function `gnu_type_fix': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:681: undefined reference to `xregexec' fixfixes.o: In function `emit_gnu_type': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:144: undefined reference to `_sch_toupper' fixfixes.o: In function `wrap_fix': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:599: undefined reference to `xregexec' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:617: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:621: undefined reference to `_sch_toupper' fixfixes.o: In function `machine_name_fix': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:508: undefined reference to `xregexec' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:539: undefined reference to `xregexec' fixfixes.o: In function `format_fix': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:282: undefined reference to `xregexec' fixfixes.o: In function `format_write': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:198: undefined reference to `_sch_istable' fixfixes.o: In function `char_macro_use_fix': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:332: undefined reference to `xregexec' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:439: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:448: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:451: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:453: undefined reference to `_sch_istable' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixfixes.c:448: undefined reference to `_sch_istable' server.o:/bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/server.c:124: more undefined references to `_sch_istable' follow fixlib.o: In function `compile_re': /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixlib.c:193: undefined reference to `xregcomp' /bld/Across/Across-2.0.0/gcc-4.4.3/fixincludes/fixlib.c:198: undefined reference to `xregerror' collect2: ld returned 1 exit status make[1]: *** [full-stamp] Error 1 make[1]: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386/build-i686-pc-linux-gnu/fixincludes' make: *** [all-build-fixincludes] Error 2 make: Leaving directory `/bld/Across/Across-2.0.0/gcc-stage1-i386'