kbuild updates

kbuild updates

[Sam Ravnborg]

Hi Linus, Andrew, lkml.

Here follows a few kbuild updates.
None of these are critical fixes so they may live in -mm for a while,
but safe to add to mainline.

o Adaptions for solaris and cygwin
  - I would be happy to hear reports of outstanding issues when
  building a kernel on non-linux platforms
  - The primary target group for this is embedded people that
  often use non-Linux environements.
  
o new static analyser tool 'namespacecheck' from Keith Owens.
  - Try it on your code and fix the issues
  - The kernel shall be build before running the tool:
    -> make namespacecheck
    Read comments in scripts/namespacecheck.pl to interpret the output
  - The output is comprehensive: ~8000 lines for allmodconfig kernel

Patches not previously posted will be posted to lkml.

Everything pushed to:
bk://linux-sam.bkbits.net/kbuild

	Sam

kbuild: Add namespacecheck

[Sam Ravnborg]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/08/29 10:17:02+02:00 kaos@ocs.com.au 
#   kbuild: Add 'make namespacecheck'
#   
#   make namespacecheck lists globally visible symbols that are not used
#   outside the file that defines them.  These symbols are candidates for
#   static declarations.  It also lists multiply defined symbols.
#   namespace.pl knows about lots of special cases in the kernel code,
#   including exported symbols and conglomerate objects.
#   
#   The patch also corrects the usage of scripts/reference*.pl, they need
#   $(src).
#   
#   Signed-off-by: Keith Owens <kaos@ocs.com.au>
#   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
#   
#   Index: 2.6.9-rc1/Makefile
#   ===================================================================
# 
# Makefile
#   2004/08/29 09:27:51+02:00 kaos@ocs.com.au +5 -2
#   kbuild: Add 'make namespacecheck'
# 
# scripts/namespace.pl
#   2004/08/29 09:53:21+02:00 kaos@ocs.com.au +416 -0
#   kbuild: Add 'make namespacecheck'
# 
# scripts/namespace.pl
#   2004/08/29 09:53:21+02:00 kaos@ocs.com.au +0 -0
#   BitKeeper file /home/sam/bk/kbuild/scripts/namespace.pl
# 
diff -Nru a/Makefile b/Makefile
--- a/Makefile	2004-08-30 21:25:24 +02:00
+++ b/Makefile	2004-08-30 21:25:24 +02:00
@@ -1123,8 +1123,11 @@
 		| xargs $(PERL) -w scripts/checkversion.pl
 
 buildcheck:
-	$(PERL) scripts/reference_discarded.pl
-	$(PERL) scripts/reference_init.pl
+	$(PERL) $(src)/scripts/reference_discarded.pl
+	$(PERL) $(src)/scripts/reference_init.pl
+
+namespacecheck:
+	$(PERL) $(src)/scripts/namespace.pl
 
 endif #ifeq ($(config-targets),1)
 endif #ifeq ($(mixed-targets),1)
diff -Nru a/scripts/namespace.pl b/scripts/namespace.pl
--- /dev/null	Wed Dec 31 16:00:00 196900
+++ b/scripts/namespace.pl	2004-08-30 21:25:24 +02:00
@@ -0,0 +1,416 @@
+#!/usr/bin/perl -w
+#
+#	namespace.pl.  Sun Aug 29 2004
+#
+#	Perform a name space analysis on the linux kernel.
+#
+#	Copyright Keith Owens <kaos@ocs.com.au>.  GPL.
+#
+#	Invoke by changing directory to the top of the kernel object
+#	tree then namespace.pl, no parameters.
+#
+#	Tuned for 2.1.x kernels with the new module handling, it will
+#	work with 2.0 kernels as well.  Last change 2.6.9-rc1.
+#
+#	The source must be compiled/assembled first, the object files
+#	are the primary input to this script.  Incomplete or missing
+#	objects will result in a flawed analysis.  Compile both vmlinux
+#	and modules.
+#
+#	Even with complete objects, treat the result of the analysis
+#	with caution.  Some external references are only used by
+#	certain architectures, others with certain combinations of
+#	configuration parameters.  Ideally the source should include
+#	something like
+#
+#	#ifndef CONFIG_...
+#	static
+#	#endif
+#	symbol_definition;
+#
+#	so the symbols are defined as static unless a particular
+#	CONFIG_... requires it to be external.
+#
+
+require 5;	# at least perl 5
+use strict;
+use File::Find;
+
+my $nm = "/usr/bin/nm -p";
+my $objdump = "/usr/bin/objdump -s -j .comment";
+
+if ($#ARGV != -1) {
+	print STDERR "usage: $0 takes no parameters\n";
+	die("giving up\n");
+}
+
+my %nmdata = ();	# nm data for each object
+my %def = ();		# all definitions for each name
+my %ksymtab = ();	# names that appear in __ksymtab_
+my %ref = ();		# $ref{$name} exists if there is a true external reference to $name
+my %export = ();	# $export{$name} exists if there is an EXPORT_... of $name
+
+&find(\&linux_objects, '.');	# find the objects and do_nm on them
+&list_multiply_defined();
+&resolve_external_references();
+&list_extra_externals();
+
+exit(0);
+
+sub linux_objects
+{
+	# Select objects, ignoring objects which are only created by
+	# merging other objects.  Also ignore all of modules, scripts
+	# and compressed.  Most conglomerate objects are handled by do_nm,
+	# this list only contains the special cases.  These include objects
+	# that are linked from just one other object and objects for which
+	# there is really no permanent source file.
+	my $basename = $_;
+	$_ = $File::Find::name;
+	s:^\./::;
+	if (/.*\.o$/ &&
+		! (
+		m:/built-in.o$:
+		|| m:/piggy.o$: || m:/bootsect.o$:
+		|| m:/boot/setup.o$: || m:^modules/: || m:^scripts/:
+		|| m:/compressed/: || m:/vmlinux-obj.o$:
+		|| m:boot/bbootsect.o$: || m:boot/bsetup.o$:
+		|| m:arch/ia64/scripts/check_gas_for_hint.o$:
+		|| m:arch/ia64/lib/__divsi3.o$:
+		|| m:arch/ia64/lib/__udivsi3.o$:
+		|| m:arch/ia64/lib/__modsi3.o$:
+		|| m:arch/ia64/lib/__umodsi3.o$:
+		|| m:arch/ia64/lib/__divdi3.o$:
+		|| m:arch/ia64/lib/__udivdi3.o$:
+		|| m:arch/ia64/lib/__moddi3.o$:
+		|| m:arch/ia64/lib/__umoddi3.o$:
+		|| m:arch/ia64/ia32/ia32.o$:
+		|| m:net/sched/sched.o$:
+		|| m:fs/romfs/romfs.o$:
+		|| m:fs/ramfs/ramfs.o$:
+		|| m:fs/nls/nls.o$:
+		|| m:drivers/video/video.o$:
+		|| m:drivers/scsi/sd_mod.o$:
+		|| m:drivers/media/media.o$:
+		|| m:drivers/ide/ide-detect.o$:
+		|| m:drivers/ide/pci/idedriver-pci.o$:
+		|| m:drivers/cdrom/driver.o$:
+		|| m:net/netlink/netlink.o$:
+		|| m:fs/vfat/vfat.o$:
+		|| m:fs/hugetlbfs/hugetlbfs.o$:
+		|| m:fs/exportfs/exportfs.o$:
+		|| m:fs/devpts/devpts.o$:
+		|| m:arch/ia64/sn/kernel/xp.o$:
+		|| m:arch/ia64/kernel/gate-syms.o$:
+		|| m:^\.tmp_:
+		|| m:^.*/\.tmp_:
+		|| m:sound/pci/snd-intel8x0.o$:
+		|| m:sound/drivers/mpu401/snd-mpu401-uart.o$:
+		|| m:sound/core/snd-timer.o$:
+		|| m:sound/core/snd-rawmidi.o$:
+		|| m:sound/core/seq/snd-seq-device.o$:
+		|| m:sound/core/seq/snd-seq-midi-event.o$:
+		|| m:sound/core/seq/snd-seq-midi.o$:
+		|| m:sound/core/oss/snd-mixer-oss.o$:
+		|| m:sound/core/snd-rtctimer.o$:
+		|| m:sound/core/seq/snd-seq-dummy.o$:
+		|| m:sound/core/seq/snd-seq-virmidi.o$:
+		|| m:sound/drivers/snd-dummy.o$:
+		|| m:sound/drivers/snd-virmidi.o$:
+		|| m:sound/drivers/snd-serial-u16550.o$:
+		|| m:sound/drivers/snd-mtpav.o$:
+		|| m:sound/drivers/mpu401/snd-mpu401.o$:
+		|| m:init/mounts.o$:
+		|| m:fs/msdos/msdos.o$:
+		|| m:arch/i386/kernel/vsyscall-syms.o$:
+		)
+	) {
+		do_nm($basename, $_);
+	}
+	$_ = $basename;		# File::Find expects $_ untouched (undocumented)
+}
+
+sub do_nm
+{
+	my ($basename, $fullname) = @_;
+	my ($source, $type, $name);
+	if (! -e $basename) {
+		printf STDERR "$basename does not exist\n";
+		return;
+	}
+	if ($fullname !~ /\.o$/) {
+		printf STDERR "$fullname is not an object file\n";
+		return;
+	}
+	$source = $basename;
+	$source =~ s/\.o$//;
+	if (! -e "$source.c" && ! -e "$source.S") {
+		# No obvious source, exclude the object if it is conglomerate
+		if (! open(OBJDUMPDATA, "$objdump $basename|")) {
+			printf STDERR "$objdump $fullname failed $!\n";
+			return;
+		}
+		my $comment;
+		while (<OBJDUMPDATA>) {
+			chomp();
+			if (/^In archive/) {
+				# Archives are always conglomerate
+				$comment = "GCC:GCC:";
+				last;
+			}
+			next if (! /^[ 0-9a-f]{5,} /);
+			$comment .= substr($_, 43);
+		}
+		close(OBJDUMPDATA);
+		if (!defined($comment) || $comment !~ /GCC\:.*GCC\:/m) {
+			printf STDERR "No source file found for $fullname\n";
+		}
+		return;
+	}
+	if (! open(NMDATA, "$nm $basename|")) {
+		printf STDERR "$nm $fullname failed $!\n";
+		return;
+	}
+	my @nmdata;
+	while (<NMDATA>) {
+		chop;
+		($type, $name) = (split(/ +/, $_, 3))[1..2];
+		# Expected types
+		# B weak external reference to data that has been resolved
+		# C global variable, uninitialised
+		# D global variable, initialised
+		# G global variable, initialised, small data section
+		# R global array, initialised
+		# S global variable, uninitialised, small bss
+		# T global label/procedure
+		# U external reference
+		# W weak external reference to text that has been resolved
+		# a assembler equate
+		# b static variable, uninitialised
+		# d static variable, initialised
+		# g static variable, initialised, small data section
+		# r static array, initialised
+		# s static variable, uninitialised, small bss
+		# t static label/procedures
+		# w weak external reference to text that has not been resolved
+		# ? undefined type, used a lot by modules
+		if ($type !~ /^[BCDGRSTUWabdgrstw?]$/) {
+			printf STDERR "nm output for $fullname contains unknown type '$_'\n";
+		}
+		elsif ($name =~ /\./) {
+			# name with '.' is local static
+		}
+		else {
+			$type = 'R' if ($type eq '?');	# binutils replaced ? with R at one point
+			$name =~ s/_R[a-f0-9]{8}$//;	# module versions adds this
+			if ($type =~ /[BCDGRSTW]/ &&
+				$name ne 'init_module' &&
+				$name ne 'cleanup_module' &&
+				$name ne 'Using_Versions' &&
+				$name !~ /^Version_[0-9]+$/ &&
+				$name !~ /^__parm_/ &&
+				$name !~ /^__kstrtab/ &&
+				$name !~ /^__ksymtab/ &&
+				$name !~ /^__exitcall_/ &&
+				$name !~ /^__initcall_/ &&
+				$name !~ /^__kdb_initcall_/ &&
+				$name !~ /^__kdb_exitcall_/ &&
+				$name !~ /^__module_/ &&
+				$name !~ /^__mod_/ &&
+				$name ne '__this_module' &&
+				$name ne 'kernel_version') {
+				if (!exists($def{$name})) {
+					$def{$name} = [];
+				}
+				push(@{$def{$name}}, $fullname);
+			}
+			push(@nmdata, "$type $name");
+			if ($name =~ /^__ksymtab_/) {
+				$name = substr($name, 10);
+				if (!exists($ksymtab{$name})) {
+					$ksymtab{$name} = [];
+				}
+				push(@{$ksymtab{$name}}, $fullname);
+			}
+		}
+	}
+	close(NMDATA);
+	if ($#nmdata < 0) {
+		if (
+			$fullname ne "lib/brlock.o"
+			&& $fullname ne "lib/dec_and_lock.o"
+			&& $fullname ne "fs/xfs/xfs_macros.o"
+			&& $fullname ne "drivers/ide/ide-probe-mini.o"
+			&& $fullname ne "usr/initramfs_data.o"
+			&& $fullname ne "drivers/acpi/executer/exdump.o"
+			&& $fullname ne "drivers/acpi/resources/rsdump.o"
+			&& $fullname ne "drivers/acpi/namespace/nsdumpdv.o"
+			&& $fullname ne "drivers/acpi/namespace/nsdump.o"
+			&& $fullname ne "arch/ia64/sn/kernel/sn2/io.o"
+			&& $fullname ne "arch/ia64/kernel/gate-data.o"
+			&& $fullname ne "drivers/ieee1394/oui.o"
+			&& $fullname ne "security/capability.o"
+			&& $fullname ne "sound/core/wrappers.o"
+			&& $fullname ne "fs/ntfs/sysctl.o"
+		) {
+			printf "No nm data for $fullname\n";
+		}
+		return;
+	}
+	$nmdata{$fullname} = \@nmdata;
+}
+
+sub drop_def
+{
+	my ($object, $name) = @_;
+	my $nmdata = $nmdata{$object};
+	my ($i, $j);
+	for ($i = 0; $i <= $#{$nmdata}; ++$i) {
+		if ($name eq (split(' ', $nmdata->[$i], 2))[1]) {
+			splice(@{$nmdata{$object}}, $i, 1);
+			my $def = $def{$name};
+			for ($j = 0; $j < $#{$def{$name}}; ++$j) {
+				if ($def{$name}[$j] eq $object) {
+					splice(@{$def{$name}}, $j, 1);
+				}
+			}
+			last;
+		}
+	}
+}
+
+sub list_multiply_defined
+{
+	my ($name, $module);
+	foreach $name (keys(%def)) {
+		if ($#{$def{$name}} > 0) {
+			# Special case for cond_syscall
+			if ($#{$def{$name}} == 1 && $name =~ /^sys_/ &&
+			    ($def{$name}[0] eq "kernel/sys.o" ||
+			     $def{$name}[1] eq "kernel/sys.o")) {
+				&drop_def("kernel/sys.o", $name);
+				next;
+			}
+			# Special case for i386 entry code
+			if ($#{$def{$name}} == 1 && $name =~ /^__kernel_/ &&
+			    $def{$name}[0] eq "arch/i386/kernel/vsyscall-int80.o" &&
+			    $def{$name}[1] eq "arch/i386/kernel/vsyscall-sysenter.o") {
+				&drop_def("arch/i386/kernel/vsyscall-sysenter.o", $name);
+				next;
+			}
+			printf "$name is multiply defined in :-\n";
+			foreach $module (@{$def{$name}}) {
+				printf "\t$module\n";
+			}
+		}
+	}
+}
+
+sub resolve_external_references
+{
+	my ($object, $type, $name, $i, $j, $kstrtab, $ksymtab, $export);
+	printf "\n";
+	foreach $object (keys(%nmdata)) {
+		my $nmdata = $nmdata{$object};
+		for ($i = 0; $i <= $#{$nmdata}; ++$i) {
+			($type, $name) = split(' ', $nmdata->[$i], 2);
+			if ($type eq "U" || $type eq "w") {
+				if (exists($def{$name}) || exists($ksymtab{$name})) {
+					# add the owning object to the nmdata
+					$nmdata->[$i] = "$type $name $object";
+					# only count as a reference if it is not EXPORT_...
+					$kstrtab = "R __kstrtab_$name";
+					$ksymtab = "R __ksymtab_$name";
+					$export = 0;
+					for ($j = 0; $j <= $#{$nmdata}; ++$j) {
+						if ($nmdata->[$j] eq $kstrtab ||
+						    $nmdata->[$j] eq $ksymtab) {
+							$export = 1;
+							last;
+						}
+					}
+					if ($export) {
+						$export{$name} = "";
+					}
+					else {
+						$ref{$name} = ""
+					}
+				}
+				elsif (    $name ne "mod_use_count_"
+					&& $name ne "__initramfs_end"
+					&& $name ne "__initramfs_start"
+					&& $name ne "_einittext"
+					&& $name ne "_sinittext"
+					&& $name ne "kallsyms_names"
+					&& $name ne "kallsyms_num_syms"
+					&& $name ne "kallsyms_addresses"
+					&& $name ne "__this_module"
+					&& $name ne "_etext"
+					&& $name ne "_edata"
+					&& $name ne "_end"
+					&& $name ne "__bss_start"
+					&& $name ne "_text"
+					&& $name ne "_stext"
+					&& $name ne "__gp"
+					&& $name ne "ia64_unw_start"
+					&& $name ne "ia64_unw_end"
+					&& $name ne "__setup_start"
+					&& $name ne "__setup_end"
+					&& $name ne "__init_begin"
+					&& $name ne "__init_end"
+					&& $name ne "__bss_stop"
+					&& $name ne "__nosave_begin"
+					&& $name ne "__nosave_end"
+					&& $name ne "pg0"
+					&& $name ne "__module_text_address"
+					&& $name !~ /^__sched_text_/
+					&& $name !~ /^__start_/
+					&& $name !~ /^__end_/
+					&& $name !~ /^__stop_/
+					&& $name !~ /^__scheduling_functions_.*_here/
+					&& $name !~ /^__.*initcall_/
+					&& $name !~ /^__.*per_cpu_start/
+					&& $name !~ /^__.*per_cpu_end/
+					&& $name !~ /^__alt_instructions/
+				) {
+
+					printf "Cannot resolve ";
+					printf "weak " if ($type eq "w");
+					printf "reference to $name from $object\n";
+				}
+			}
+		}
+	}
+}
+
+sub list_extra_externals
+{
+	my %noref = ();
+	my ($name, @module, $module, $export);
+	foreach $name (keys(%def)) {
+		if (! exists($ref{$name})) {
+			@module = @{$def{$name}};
+			foreach $module (@module) {
+				if (! exists($noref{$module})) {
+					$noref{$module} = [];
+				}
+				push(@{$noref{$module}}, $name);
+			}
+		}
+	}
+	if (%noref) {
+		printf "\nExternally defined symbols with no external references\n";
+		foreach $module (sort(keys(%noref))) {
+			printf "  $module\n";
+			foreach (sort(@{$noref{$module}})) {
+				if (exists($export{$_})) {
+					$export = " (export only)";
+				}
+				else {
+					$export = "";
+				}
+				printf "    $_$export\n";
+			}
+		}
+	}
+}

kbuild: Updates to namespacecheck

[Sam Ravnborg]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/08/30 20:51:06+02:00 sam@mars.ravnborg.org 
#   kbuild: Updates to namespacecheck.pl
#   
#   From: Keith Owens <kaos@ocs.com.au>
#   
#   This now supports the absolute symbols from modversions, handles
#   recent binutils changes and supports O=.
#   
#   Signed-off-by: Keith Owens <kaos@ocs.com.au>
#   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
# 
# scripts/namespace.pl
#   2004/08/30 20:50:49+02:00 sam@mars.ravnborg.org +86 -53
#   This now supports the absolute symbols from modversions, handles
#   recent binutils changes and supports O=.
# 
diff -Nru a/scripts/namespace.pl b/scripts/namespace.pl
--- a/scripts/namespace.pl	2004-08-30 21:26:04 +02:00
+++ b/scripts/namespace.pl	2004-08-30 21:26:04 +02:00
@@ -1,6 +1,6 @@
 #!/usr/bin/perl -w
 #
-#	namespace.pl.  Sun Aug 29 2004
+#	namespace.pl.  Mon Aug 30 2004
 #
 #	Perform a name space analysis on the linux kernel.
 #
@@ -10,7 +10,10 @@
 #	tree then namespace.pl, no parameters.
 #
 #	Tuned for 2.1.x kernels with the new module handling, it will
-#	work with 2.0 kernels as well.  Last change 2.6.9-rc1.
+#	work with 2.0 kernels as well.
+#
+#	Last change 2.6.9-rc1, adding support for separate source and object
+#	trees.
 #
 #	The source must be compiled/assembled first, the object files
 #	are the primary input to this script.  Incomplete or missing
@@ -31,6 +34,33 @@
 #	so the symbols are defined as static unless a particular
 #	CONFIG_... requires it to be external.
 #
+#	A symbol that is suffixed with '(export only)' has these properties
+#
+#	* It is global.
+#	* It is marked EXPORT_SYMBOL or EXPORT_SYMBOL_GPL, either in the same
+#	  source file or a different source file.
+#	* Given the current .config, nothing uses the symbol.
+#
+#	The symbol is a candidate for conversion to static, plus removal of the
+#	export.  But be careful that a different .config might use the symbol.
+#
+#
+#	Name space analysis and cleanup is an iterative process.  You cannot
+#	expect to find all the problems in a single pass.
+#
+#	* Identify possibly unnecessary global declarations, verify that they
+#	  really are unnecessary and change them to static.
+#	* Compile and fix up gcc warnings about static, removing dead symbols
+#	  as necessary.
+#	* make clean and rebuild with different configs (especially
+#	  CONFIG_MODULES=n) to see which symbols are being defined when the
+#	  config does not require them.  These symbols bloat the kernel object
+#	  for no good reason, which is frustrating for embedded systems.
+#	* Wrap config sensitive symbols in #ifdef CONFIG_foo, as long as the
+#	  code does not get too ugly.
+#	* Repeat the name space analysis until you can live with with the
+#	  result.
+#
 
 require 5;	# at least perl 5
 use strict;
@@ -38,6 +68,10 @@
 
 my $nm = "/usr/bin/nm -p";
 my $objdump = "/usr/bin/objdump -s -j .comment";
+my $srctree = "";
+my $objtree = "";
+$srctree = "$ENV{'srctree'}/" if (exists($ENV{'srctree'}));
+$objtree = "$ENV{'objtree'}/" if (exists($ENV{'objtree'}));
 
 if ($#ARGV != -1) {
 	print STDERR "usage: $0 takes no parameters\n";
@@ -71,58 +105,49 @@
 	if (/.*\.o$/ &&
 		! (
 		m:/built-in.o$:
-		|| m:/piggy.o$: || m:/bootsect.o$:
-		|| m:/boot/setup.o$: || m:^modules/: || m:^scripts/:
-		|| m:/compressed/: || m:/vmlinux-obj.o$:
-		|| m:boot/bbootsect.o$: || m:boot/bsetup.o$:
-		|| m:arch/ia64/scripts/check_gas_for_hint.o$:
+		|| m:arch/i386/kernel/vsyscall-syms.o$:
+		|| m:arch/ia64/ia32/ia32.o$:
+		|| m:arch/ia64/kernel/gate-syms.o$:
+		|| m:arch/ia64/lib/__divdi3.o$:
 		|| m:arch/ia64/lib/__divsi3.o$:
-		|| m:arch/ia64/lib/__udivsi3.o$:
+		|| m:arch/ia64/lib/__moddi3.o$:
 		|| m:arch/ia64/lib/__modsi3.o$:
-		|| m:arch/ia64/lib/__umodsi3.o$:
-		|| m:arch/ia64/lib/__divdi3.o$:
 		|| m:arch/ia64/lib/__udivdi3.o$:
-		|| m:arch/ia64/lib/__moddi3.o$:
+		|| m:arch/ia64/lib/__udivsi3.o$:
 		|| m:arch/ia64/lib/__umoddi3.o$:
-		|| m:arch/ia64/ia32/ia32.o$:
-		|| m:net/sched/sched.o$:
-		|| m:fs/romfs/romfs.o$:
-		|| m:fs/ramfs/ramfs.o$:
-		|| m:fs/nls/nls.o$:
-		|| m:drivers/video/video.o$:
-		|| m:drivers/scsi/sd_mod.o$:
-		|| m:drivers/media/media.o$:
+		|| m:arch/ia64/lib/__umodsi3.o$:
+		|| m:arch/ia64/scripts/check_gas_for_hint.o$:
+		|| m:arch/ia64/sn/kernel/xp.o$:
+		|| m:boot/bbootsect.o$:
+		|| m:boot/bsetup.o$:
+		|| m:/bootsect.o$:
+		|| m:/boot/setup.o$:
+		|| m:/compressed/:
+		|| m:drivers/cdrom/driver.o$:
+		|| m:drivers/char/drm/tdfx_drv.o$:
 		|| m:drivers/ide/ide-detect.o$:
 		|| m:drivers/ide/pci/idedriver-pci.o$:
-		|| m:drivers/cdrom/driver.o$:
-		|| m:net/netlink/netlink.o$:
-		|| m:fs/vfat/vfat.o$:
-		|| m:fs/hugetlbfs/hugetlbfs.o$:
-		|| m:fs/exportfs/exportfs.o$:
+		|| m:drivers/media/media.o$:
+		|| m:drivers/scsi/sd_mod.o$:
+		|| m:drivers/video/video.o$:
 		|| m:fs/devpts/devpts.o$:
-		|| m:arch/ia64/sn/kernel/xp.o$:
-		|| m:arch/ia64/kernel/gate-syms.o$:
-		|| m:^\.tmp_:
-		|| m:^.*/\.tmp_:
-		|| m:sound/pci/snd-intel8x0.o$:
-		|| m:sound/drivers/mpu401/snd-mpu401-uart.o$:
-		|| m:sound/core/snd-timer.o$:
-		|| m:sound/core/snd-rawmidi.o$:
-		|| m:sound/core/seq/snd-seq-device.o$:
-		|| m:sound/core/seq/snd-seq-midi-event.o$:
-		|| m:sound/core/seq/snd-seq-midi.o$:
-		|| m:sound/core/oss/snd-mixer-oss.o$:
-		|| m:sound/core/snd-rtctimer.o$:
-		|| m:sound/core/seq/snd-seq-dummy.o$:
-		|| m:sound/core/seq/snd-seq-virmidi.o$:
-		|| m:sound/drivers/snd-dummy.o$:
-		|| m:sound/drivers/snd-virmidi.o$:
-		|| m:sound/drivers/snd-serial-u16550.o$:
-		|| m:sound/drivers/snd-mtpav.o$:
-		|| m:sound/drivers/mpu401/snd-mpu401.o$:
-		|| m:init/mounts.o$:
+		|| m:fs/exportfs/exportfs.o$:
+		|| m:fs/hugetlbfs/hugetlbfs.o$:
 		|| m:fs/msdos/msdos.o$:
-		|| m:arch/i386/kernel/vsyscall-syms.o$:
+		|| m:fs/nls/nls.o$:
+		|| m:fs/ramfs/ramfs.o$:
+		|| m:fs/romfs/romfs.o$:
+		|| m:fs/vfat/vfat.o$:
+		|| m:init/mounts.o$:
+		|| m:^modules/:
+		|| m:net/netlink/netlink.o$:
+		|| m:net/sched/sched.o$:
+		|| m:/piggy.o$:
+		|| m:^scripts/:
+		|| m:sound/.*/snd-:
+		|| m:^.*/\.tmp_:
+		|| m:^\.tmp_:
+		|| m:/vmlinux-obj.o$:
 		)
 	) {
 		do_nm($basename, $_);
@@ -142,8 +167,12 @@
 		printf STDERR "$fullname is not an object file\n";
 		return;
 	}
-	$source = $basename;
-	$source =~ s/\.o$//;
+	($source = $fullname) =~ s/\.o$//;
+	if (-e "$objtree$source.c" || -e "$objtree$source.S") {
+		$source = "$objtree$source";
+	} else {
+		$source = "$srctree$source";
+	}
 	if (! -e "$source.c" && ! -e "$source.S") {
 		# No obvious source, exclude the object if it is conglomerate
 		if (! open(OBJDUMPDATA, "$objdump $basename|")) {
@@ -176,6 +205,7 @@
 		chop;
 		($type, $name) = (split(/ +/, $_, 3))[1..2];
 		# Expected types
+		# A absolute symbol
 		# B weak external reference to data that has been resolved
 		# C global variable, uninitialised
 		# D global variable, initialised
@@ -194,7 +224,7 @@
 		# t static label/procedures
 		# w weak external reference to text that has not been resolved
 		# ? undefined type, used a lot by modules
-		if ($type !~ /^[BCDGRSTUWabdgrstw?]$/) {
+		if ($type !~ /^[ABCDGRSTUWabdgrstw?]$/) {
 			printf STDERR "nm output for $fullname contains unknown type '$_'\n";
 		}
 		elsif ($name =~ /\./) {
@@ -202,8 +232,10 @@
 		}
 		else {
 			$type = 'R' if ($type eq '?');	# binutils replaced ? with R at one point
+			# binutils keeps changing the type for exported symbols, force it to R
+			$type = 'R' if ($name =~ /^__ksymtab/ || $name =~ /^__kstrtab/);
 			$name =~ s/_R[a-f0-9]{8}$//;	# module versions adds this
-			if ($type =~ /[BCDGRSTW]/ &&
+			if ($type =~ /[ABCDGRSTW]/ &&
 				$name ne 'init_module' &&
 				$name ne 'cleanup_module' &&
 				$name ne 'Using_Versions' &&
@@ -211,12 +243,14 @@
 				$name !~ /^__parm_/ &&
 				$name !~ /^__kstrtab/ &&
 				$name !~ /^__ksymtab/ &&
+				$name !~ /^__kcrctab_/ &&
 				$name !~ /^__exitcall_/ &&
 				$name !~ /^__initcall_/ &&
 				$name !~ /^__kdb_initcall_/ &&
 				$name !~ /^__kdb_exitcall_/ &&
 				$name !~ /^__module_/ &&
 				$name !~ /^__mod_/ &&
+				$name !~ /^__crc_/ &&
 				$name ne '__this_module' &&
 				$name ne 'kernel_version') {
 				if (!exists($def{$name})) {
@@ -252,6 +286,7 @@
 			&& $fullname ne "security/capability.o"
 			&& $fullname ne "sound/core/wrappers.o"
 			&& $fullname ne "fs/ntfs/sysctl.o"
+			&& $fullname ne "fs/jfs/jfs_debug.o"
 		) {
 			printf "No nm data for $fullname\n";
 		}
@@ -354,8 +389,6 @@
 					&& $name ne "__gp"
 					&& $name ne "ia64_unw_start"
 					&& $name ne "ia64_unw_end"
-					&& $name ne "__setup_start"
-					&& $name ne "__setup_end"
 					&& $name ne "__init_begin"
 					&& $name ne "__init_end"
 					&& $name ne "__bss_stop"
@@ -372,8 +405,8 @@
 					&& $name !~ /^__.*per_cpu_start/
 					&& $name !~ /^__.*per_cpu_end/
 					&& $name !~ /^__alt_instructions/
+					&& $name !~ /^__setup_/
 				) {
-
 					printf "Cannot resolve ";
 					printf "weak " if ($type eq "w");
 					printf "reference to $name from $object\n";

kbuild: Update help text

[Sam Ravnborg]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/08/30 21:23:24+02:00 sam@mars.ravnborg.org 
#   kbuild: Add stactic analyser tools to make help
#   
#   Added the tools that seems to be maintained.
#   There is a bunch that has not been touched for a while - ignore them for now.
#   
#   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
# 
# Makefile
#   2004/08/30 21:23:05+02:00 sam@mars.ravnborg.org +9 -3
#   Updated help and use $(srctree) as replacement for $(src)
# 
diff -Nru a/Makefile b/Makefile
--- a/Makefile	2004-08-30 21:26:19 +02:00
+++ b/Makefile	2004-08-30 21:26:20 +02:00
@@ -963,7 +963,13 @@
 	@echo  '  rpm		  - Build a kernel as an RPM package'
 	@echo  '  tags/TAGS	  - Generate tags file for editors'
 	@echo  '  cscope	  - Generate cscope index'
+	@echo  ''
+	@echo  'Static analysers'
+	@echo  '  buildcheck      - List dangling references to vmlinux discarded sections'
+	@echo  '                    and init sections from non-init sections'
 	@echo  '  checkstack      - Generate a list of stack hogs'
+	@echo  '  namespacecheck  - Name space analysis on compiled kernel'
+	@echo  ''
 	@echo  'Kernel packaging:'
 	@$(MAKE) -f $(package-dir)/Makefile help
 	@echo  ''
@@ -1123,11 +1129,11 @@
 		| xargs $(PERL) -w scripts/checkversion.pl
 
 buildcheck:
-	$(PERL) $(src)/scripts/reference_discarded.pl
-	$(PERL) $(src)/scripts/reference_init.pl
+	$(PERL) $(srctree)/scripts/reference_discarded.pl
+	$(PERL) $(srctree)/scripts/reference_init.pl
 
 namespacecheck:
-	$(PERL) $(src)/scripts/namespace.pl
+	$(PERL) $(srctree)/scripts/namespace.pl
 
 endif #ifeq ($(config-targets),1)
 endif #ifeq ($(mixed-targets),1)

Re: Kbuild updates

[Sam Ravnborg]

On Thu, Aug 19, 2004 at 01:02:52AM +0200, Sam Ravnborg wrote:
> Another bunch of smallish kbuild updates.

Everything pushed to:
bk://linux-sam.bkbits.net/kbuild for BK users.

	Sam

Kbuild updates

[Sam Ravnborg]

Another bunch of smallish kbuild updates.

o Fix parrallel build
o Add make C=2 to force check of C code with sparse
o Fiw warnings in binoffset
o Removed dreadful check for undefined symbols in vmlinux
o Delete last occurence of HEAD in kbuild
o Add comments to Makefile.clean
o Introduce CHECKFLAGS enabling make CHECK=mysparse

Patches follows (most have been sent to lkml before).


Significant outstanding items in kbuild land is mainly package
handling.
Several issues needs to be adressed:
- Rename debian directory to deb/debian
- Utilise parallel build with make rpm
- Add one more prm package variant
- Add tar-pkg, targz.pkg
- Find a better way to identify relevant binaries to include in rpm, deb
	- Current KBUILD_IMAGE proves not to be good enough


And a patch relate to control alignment in gcc produced code.
Needs to land some consolidation stuff first. Planning to rename
check_gcc to gcc_option, and include a new one named gcc_option_ok.


For ppc we have a problem with as trashing /dev/null.
This is apperantly due to this code:
BAD_AGCC_AS :=  $(shell echo mftb 5 | $(AS) -mppc -many -o /dev/null >/dev/null 2>&1 && echo 0 || echo 1)
 
It seem like as deletes /dev/null and recreate it.
Not good if run as root - which is needed for modules_install for example.

Would it be considered acceptable to start using mktemp to generate temporary
filenames as part of a normal kernel build?
I do not fully understand the eventual security issues around this!

	Sam

kbuild updates

[Sam Ravnborg]

This is the accumulated list of kbuild patches.
Mostly trivial stuff but also a bit of interest:

o replaced host-progs with hostprogs-y
o Use a shell script to generate System.map
	Includes check for undefined symbols in vmlinux
	Filter out __crc_ from System.map
  -> This replaces two other patches in -mm
	- rmk's ldchk function
	- wli btfixup patch, number 13 in his serie
  This patch replaces both
o Randy's patch to put version in .config
o One patch to a few files so we use POSIX header in some of the
  tools. 
  This is a slow start on improving the build on non-linux platforms.

This does in no way clear my kbuild queue - but the rest
is put on hold for a while.

Everything pushed to:
bk://linux-sam.bkbits.net/kbuild

When the patches has been in -mm for a while they will be pushed
to Linus.
None of this is pre 2.6.8 material.

Patches will follow in seperate mails.

	Sam

 
 

Files touched:

 Documentation/DocBook/Makefile     |    4 
 Documentation/kbuild/makefiles.txt |   50 ++++++++----
 Makefile                           |   37 ++++++--
 arch/alpha/boot/Makefile           |    2 
 arch/i386/boot/Makefile            |    4 
 arch/ppc/boot/Makefile             |    4 
 arch/ppc64/boot/Makefile           |    4 
 arch/sparc/boot/Makefile           |   42 +++++++---
 arch/sparc64/boot/Makefile         |    2 
 arch/um/sys-i386/util/Makefile     |    4 
 arch/x86_64/boot/Makefile          |    2 
 drivers/atm/Makefile               |    2 
 drivers/md/Makefile                |    2 
 drivers/media/dvb/ttpci/Makefile   |    2 
 drivers/pci/Makefile               |    2 
 drivers/zorro/Makefile             |    2 
 lib/Makefile                       |    2 
 scripts/Makefile                   |   17 ++--
 scripts/Makefile.build             |   94 ++--------------------
 scripts/Makefile.clean             |   32 +++++--
 scripts/Makefile.host              |  154 +++++++++++++++++++++++++++++++++++++
 scripts/Makefile.lib               |   49 -----------
 scripts/basic/Makefile             |    4 
 scripts/basic/fixdep.c             |    2 
 scripts/genksyms/Makefile          |    4 
 scripts/kconfig/Makefile           |    2 
 scripts/kconfig/confdata.c         |   17 +++-
 scripts/lxdialog/Makefile          |    4 
 scripts/mksysmap                   |   61 ++++++++++++++
 scripts/mod/Makefile               |    4 
 scripts/mod/sumversion.c           |    2 
 scripts/package/Makefile           |    6 -
 sound/oss/Makefile                 |    2 
 usr/Makefile                       |    2 
 34 files changed, 406 insertions(+), 217 deletions(-)

through these ChangeSets:

<sam@mars.ravnborg.org> (04/08/13 1.1912)
   kbuild: __crc_* symbols in System.map
   
   David S. Miller <davem@redhat.com> wrote:
   Shouldn't we be grepping __crc_ symbols out of the System.map file?
   
   For one thing, these can confuse readprofile.  It's algorithm is
   to start at _stext, then stop when it sees a line in the System.map
   which is not text (mode is one of 'T' 't' 'W' or 'w')
   
   It will exit early if there are some intermixed __crc_* things in
   there (since they are are mode 'A').
   
   For example, in my current sparc64 kernel I have this:
   
   00000000004cef80 t do_split
   00000000004cf2a0 t add_dirent_to_buf
   00000000004cf5a7 A __crc_init_special_inode
   00000000004cf640 t make_indexed_dir
   00000000004cf900 t ext3_add_entry
   
   So no symbols after add_dirent_to_buf will be shown in the profiling
   output of readprofile.
   
   Implementation ported to mksysmap by Sam.
   Included two System.map related fixes:
   - Print "SYSMAP  System.map" during build
   - Sort symbols in System.map
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/10 1.1911)
   kbuild: Use POSIX headers for ntoh functions
   
   From: Benno <benjl@cse.unsw.edu.au>
   When compiling Linux on Mac OSX I had trouble with scripts/sumversion.c.
   It includes <netinet/in.h> to obtain to definitions of htonl and ntohl.
   
   On Mac OSX these are found in <arpa/inet.h>. After checking the POSIX
   specification it appears that this is the correct place to get
   the definitons for these functions.
   
   (http://www.opengroup.org/onlinepubs/009695399/functions/htonl.html)
   
   Using this header also appears to work on Linux (at least with
   Glibc-2.3.2).
   
   It seems clearer to me to go with the POSIX standard than implementing
   #if __APPLE__ style macros, but if such an approach is preferred I can
   supply patches for that instead.
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/10 1.1910)
   kbuild: Fix hostprogs-y
   
   Allow the same target to be specified more than once without causing a warnign from make.
   The same target may be specified twice when using the following pattern:
   hostprogs-$(CONFIG_FOO) += program
   hostprogs-$(CONFIG_BAR) += program
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/10 1.1909)
   kbuild: Replace host-progs with hostprogs-y
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/10 1.1908)
   kbuild: Introduce hostprogs-y, deprecate host-progs
   
   Introducing hostprogs-y allows a user to use the typical Kbuild
   pattern in a Kbuild file:
   hostprogs-$(CONFIG_KALLSYMS) += ...
   
   And then during cleaning the referenced file are still deleted.
   Deprecate the old host-progs assignment but kept the functionlity.
   
   External modules will continue to use host-progs for a while - drawback is
   that they now see a warning.
   Workaround - just assign both variables:
   hostprogs-y := foo
   host-progs  := $(hostprogs-y)
   
   All in-kernel users will be converted in next patch.
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/10 1.1907)
   kbuild: Separate out host-progs handling
   
   Concentrating all host-progs functionality in one file made a more
   readable Makefile.lib - and allow for potential reuse of host-progs
   functionality.
   Processing of host-progs related stuff are avoided when no host-progs are specified. 
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/09 1.1906)
   kbuild: Accept absolute paths in clean-files and introduce clean-dirs
   
   Teach kbuild to accept absolute paths in clean-files. This avoids using
   clean-rules in several places.
   Introduced clean-dirs to delete complete directories.
   Kept clean-rule - but do not print anything when used.
   Cleaned up a few places now the infrastructure are improved.
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/08 1.1905)
   kbuild: Use LINUXINCLUDE to specify include/ directory
   
   Peter Chubb <peterc@gelato.unsw.edu.au> reported that building i386
   on a non-i386 platform failed, because gcc could not locate boot.h.
   Root cause was the extra include2 directory used when using O=
   to specify the output directory.
   Added LINUXINCLUDE as a portable way to specify the include/
   directory, and changed the two users.
   This avoids hardcoding 'include2' in non-kbuild core files.
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/08 1.1904)
   kbuild: Selective compile of targets in scripts/
   
   Do not build executables unless needed.
   Same goes for scripts/mod/, descend only when CONFIG_MODULES are enabled.
   With inputs form: Russell King <rmk+lkml@arm.linux.org.uk> and  Brian Gerst <bgerst@quark.didntduck.org>
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<rddunlap@osdl.org> (04/08/08 1.1903)
   kconfig: save kernel version in .config file
   
   Save kernel version info and date when writing .config file.
   Tested with 'make {menuconfig|xconfig|gconfig}'.
   
   Signed-off-by: Randy Dunlap <rddunlap@osdl.org>
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/08 1.1902)
   kbuild/sparc: Use new generic mksysmap script to generate System.map
   
   o Introduced usage of the mksysmap script.
   o Improved the non-verbose output to look like this:
     BTFIX   arch/sparc/boot/btfix.S
     AS      arch/sparc/boot/btfix.o
     LD      arch/sparc/boot/image
     SYSMAP  arch/sparc/boot/System.map
   
   o No longer generate System.map for each build
   o Use normal AS rule to compile btfix.S
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>

<sam@mars.ravnborg.org> (04/08/07 1.1901)
   kbuild: Check for undefined symbols in vmlinux
   
   At least one bin-utils version for ARM is know to ignore undefined
   symbols when performing the final link of vmlinux.
   Add an explicit check for undefined symbols to catch this.
   The check is made in combination with generating the System.map file
   and the actual algorithm is moved to a small shell script - mksysmap.
   
   External symbols with three leading underscores are ignored - sparc
   uses them for the BTFIXUP logic.
   
   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>