Kernel Make system and linking static libraries on kernel version s2.6.14+

Kernel Make system and linking static libraries on kernel version s2.6.14+

[Puvvada, Vijay B.]

Hello, 

This is a "I'm stumped kind of problem" and desperately need some
guidance with regards to the kernel make system..

I had also started a thread at 
http://forums.fedoraforum.org/showthread.php?p=428551#post428551 with
regards to this, where I describe the context of the problem.  

In a nutshell, I am trying to compile the Nortel VPN client (which is
written as part driver and part app) against the 2.6.14 kernel and I am
getting the following warning. 

Warning: could not
find /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../.libmishim-2.6.a.cmd
for /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../libmishim-2.6.a

For some reason, the kernel make system does not seem to know what to do
with static libraries.  It doesn't seem to create a .cmd for these
files.

1.  Is this a normal warning when trying to link in static libraries
with the kernel make system?  ie, the kernel make system does not create
a .cmd file for static libraries/archives.

2.  Do they all still get linked in or is this a real problem?  It seems
to warn on the first static library so I am assuming the rest of the
line is just abandoned.  Is this a correct interpretation?

3.  How can I clean up the warning?  Is there a proper way to specify
static libraries to the kernel make system?

So far as I can tell, this is an interaction between the kernel make
system and the application I have and not distro-specific.  

If you require any information, please do not hesitate to ask.

Vij

The full makefile for the kernel driver portion is below:


Makefile:
obj-m         += mishim.o nlvcard.o
mishim-libs   += ../libmishim-2.6.a ../libnlcmp.a ../libz.a ../liblzs.a
mishim-c-objs += linux_wrapper.o
mishim-objs   += $(mishim-c-objs) $(mishim-libs)

EXTRA_CFLAGS += -fno-common -w

ifeq ($(KERNELRELEASE),)

KVER            = $(shell uname -r)

ifeq (1,1)
KDIR            = /lib/modules/$(KVER)/build
else
KDIR            = /usr/src/linux-$(KVER)
endif

PWD             = $(shell pwd)

mishim-cfiles = ${mishim-c-objs:.o=.c}


kmod_build:: $(mishim-libs) $(mishim-cfiles)
        $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
        @cp mishim.ko ../mishim.o
        @cp nlvcard.ko ../nlvcard.o

%.c:
        @ln -s ../linux_wrapper.c

clean:
        -rm -rf *.o *.ko *.mod.* .??*

install: all
        @echo "Installing VPN agent."
        cd .. &&  ./install.sh

uninstall: 
        @echo "Uninstalling VPN agent."
        cd .. && ./uninstall.sh

endif

Re: Kernel Make system and linking static libraries on kernel version s2.6.14+

[Arjan van de Ven]

On Mon, 2006-01-02 at 09:51 -0500, Puvvada, Vijay B. wrote:
> Hello, 
> 
> This is a "I'm stumped kind of problem" and desperately need some
> guidance with regards to the kernel make system..
> 
> I had also started a thread at 
> http://forums.fedoraforum.org/showthread.php?p=428551#post428551 with
> regards to this, where I describe the context of the problem.  
> 
> In a nutshell, I am trying to compile the Nortel VPN client (which is
> written as part driver and part app) against the 2.6.14 kernel and I am
> getting the following warning. 
> 
> Warning: could not
> find /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../.libmishim-2.6.a.cmd
> for /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../libmishim-2.6.a


it'll be a LOT easier for you to rewrite the Makefile such that you
don't get .a files but a .o file similar to how the rest of kbuild
operates.... while we do use .a files for linking into the vmlinux.. for
modules it's probably not the best of ideas
(and there's not really an advantage do doing so either anyway)

Re: Kernel Make system and linking static libraries on kernel version s2.6.14+

[John Stoffel]

Vijay> In a nutshell, I am trying to compile the Nortel VPN client
Vijay> (which is written as part driver and part app) against the
Vijay> 2.6.14 kernel and I am getting the following warning.

This software is from a company called 'Apani Networks' and they are
slow to update their code to reflect kernel changes.  I too am stuck
with Nortal VPN boxes at work and it's a pain to get them working.
What's worse, is their NetLock software is also a pain to get working
as well...

Try dropping back to the latest kernel they have supported in their
library and compile that.  I was certainly able to make it compile
against 2.6.12 as I recall... but since I was running 2.6.13+ at the
time, it didn't do me much good...

Again, double check the README that came with the software and try to
do all the compile work by hand, and not using their canned solutions
for Redhat, etc.  I was able to get it to compile, but never managed
to get it working on my main home machine unfortunately.  

Can you post the log of all the steps you took to compile the code,
the kernel version you're using, and any output of the make commands?
That would help.

John

Re: Kernel Make system and linking static libraries on kernelvers ion s2.6.14+

[Puvvada, Vijay B.]

> 
> Can you post the log of all the steps you took to compile the code, 
> the kernel version you're using, and any output of the make commands? 
> That would help.
> 
> John
> 

Hey John,

A full account of my personal crying game is already posted at
http://forums.fedoraforum.org/showthread.php?t=88606&goto=newpost
including the output to all the calls of make.  Below is a repost of my
original message:

Last week, I got a licensed copy of Apini's (Nortel) VPN client for
linux cvc_linux-rh-gcc3-3.3 and I tried to install it against my fedora
core 4 laptop with all the stable updates (including the kernel).
Currently, I am at kernel version 2.6.14.xx.

Out of the box, when I invoked the make on Fedora Core 4 with 2.6.14-1,
I got:

[root@hadji cvc_linux-rh-gcc3-3.3]# make all
cd src && make all
make[1]: Entering directory
`/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src'
cd k2.6 && make
make[2]: Entering directory
`/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6'
make -C /lib/modules/2.6.14-1.1644_FC4/build
SUBDIRS=/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6 modules
make[3]: Entering directory `/usr/src/kernels/2.6.14-1.1644_FC4-i686'
CC
[M] /usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.o
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:
In function ‘nl_ip_rcv’:
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:418:
error: too few arguments to function ‘ip_rcv’
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:
In function ‘nl_skb_dup’:
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:526:
error: ‘struct sk_buff’ has no member named ‘stamp’
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:526:
error: ‘struct sk_buff’ has no member named ‘stamp’
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:527:
error: ‘struct sk_buff’ has no member named ‘security’
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:527:
error: ‘struct sk_buff’ has no member named ‘security’
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:
In function ‘nl_skb_hdr_copy’:
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:553:
error: ‘struct sk_buff’ has no member named ‘stamp’
/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.c:553:
error: ‘struct sk_buff’ has no member named ‘stamp’
make[4]: ***
[/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.o]
Error 1
make[3]: ***
[_module_/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6] Error
2
make[3]: Leaving directory `/usr/src/kernels/2.6.14-1.1644_FC4-i686'
make[2]: *** [kmod_build] Error 2
make[2]: Leaving directory
`/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src/k2.6'
make[1]: *** [all] Error 2
make[1]: Leaving directory
`/usr/src/redhat/TARBALLS/cvc_linux-rh-gcc3-3.3/src'
make: *** [all] Error 2

The compilation errors are due to the fact that sk_buff has undergone
some changes between 2.6.13 and 2.6.14, namely, stamp has been redefined
as tstamp (for time stamp) and security has been redefined to sp (for
security path). I am guessing for clarity ?? Since the functions that
use the sk_buff attributes are simply doing a copy from sk_buff
structure to sk_buff structure, I felt it was safe to just make a global
substitution for these attributes. 

The other change in the 2.6.14 kernel is that ip_rcv takes an extra
pointer to sk_buff than it had previously. If I'm not mistaken, this
change was made several releases prior? The current prototype looks like
this: 

extern int ip_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev);

I initially thought by passing the same pointer twice would solve the
problem. When I did this, I got...

[root@hadji cvc_linux-rh-gcc3-3.3]# make all
cd src && make all
make[1]: Entering directory `/usr/local/cvc_linux-rh-gcc3-3.3/src'
cd k2.6 && make
make[2]: Entering directory `/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6'
make -C /lib/modules/2.6.14-1.1644_FC4/build
SUBDIRS=/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6 modules
make[3]: Entering directory `/usr/src/kernels/2.6.14-1.1644_FC4-i686'
Building modules, stage 2.
MODPOST
Warning: could not
find /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../.libmishim-2.6.a.cmd
for /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../libmishim-2.6.a
*** Warning:
"ip_rcv" [/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/mishim.ko]
undefined!
make[3]: Leaving directory `/usr/src/kernels/2.6.14-1.1644_FC4-i686'
make[2]: Leaving directory `/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6'
make[1]: Leaving directory `/usr/local/cvc_linux-rh-gcc3-3.3/src'

Unfortunately, the problem with that was that the new kernel (in
ip_input.c) the EXPORT_SYMBOL(ip_rcv) was removed so that modules cannot
directly call on ip routines. I noticed in the kernel mailing lists
somewhere that modules should use netif calls instead (in this case,
netif_rx which takes a single argument, the pointer to sk_buff). After
making that change, I invoked make again and got:

[root@hadji cvc_linux-rh-gcc3-3.3]# make all
cd src && make all
make[1]: Entering directory `/usr/local/cvc_linux-rh-gcc3-3.3/src'
cd k2.6 && make
make[2]: Entering directory `/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6'
make -C /lib/modules/2.6.14-1.1653_FC4/build
SUBDIRS=/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6 modules
make[3]: Entering directory `/usr/src/kernels/2.6.14-1.1653_FC4-i686'
CC [M] /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/linux_wrapper.o
LD [M] /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/mishim.o
CC [M] /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/nlvcard.o
Building modules, stage 2.
MODPOST
Warning: could not
find /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../.libmishim-2.6.a.cmd
for /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/../libmishim-2.6.a
CC /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/mishim.mod.o
LD [M] /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/mishim.ko
CC /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/nlvcard.mod.o
LD [M] /usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6/nlvcard.ko
make[3]: Leaving directory `/usr/src/kernels/2.6.14-1.1653_FC4-i686'
make[2]: Leaving directory `/usr/local/cvc_linux-rh-gcc3-3.3/src/k2.6'
make[1]: Leaving directory `/usr/local/cvc_linux-rh-gcc3-3.3/src'

This last error is a little puzzling to me. At this current moment, I am
kind of leaning to a problem in the makefile. Maybe someone can shed
some light on it. So far as I can tell, the makefile is trying to link
in some static libraries (archives) but for some reason, the kernel make
system does not seem to know how handle them or that they are not being
specified correctly in the makefile such that the kernel make system can
generate a proper command for linking in these static libraries.

Below is a recursive diff of my "linux_wrapper.c" file which is the only
file in the tarball that I changed so far (at the end of the day, the
changes are rather trivial).


diff -C3 --recursive cvc_linux-rh-gcc3-3.3/src/linux_wrapper.c
reference/cvc_linux-rh-gcc3-3.3/src/linux_wrapper.c
*** cvc_linux-rh-gcc3-3.3/src/linux_wrapper.c   2006-01-02
10:42:07.000000000 -0500
--- reference/cvc_linux-rh-gcc3-3.3/src/linux_wrapper.c 2005-07-12
19:52:40.000000000 -0400
***************
*** 415,422 ****
  
  int nl_ip_rcv(struct sk_buff *skb, struct packet_type *pt)
  {
!       /* return ip_rcv(skb, skb->dev, pt); */
!       return netif_rx(skb);
  }
  
  void nl_ip_send_check(struct iphdr *iph)
--- 415,421 ----
  
  int nl_ip_rcv(struct sk_buff *skb, struct packet_type *pt)
  {
!       return ip_rcv(skb, skb->dev, pt);
  }
  
  void nl_ip_send_check(struct iphdr *iph)
***************
*** 524,531 ****
      memcpy(new_skb->cb, skb->cb, sizeof(skb->cb));
      new_skb->priority = skb->priority;
      new_skb->protocol = skb->protocol;
!     new_skb->tstamp = skb->tstamp;
!     new_skb->sp = skb->sp;
  #if ((LINUX_VERSION_CODE >= 0x020200) && (LINUX_VERSION_CODE <
0x020300))
      new_skb->used = skb->used;
  #endif
--- 523,530 ----
      memcpy(new_skb->cb, skb->cb, sizeof(skb->cb));
      new_skb->priority = skb->priority;
      new_skb->protocol = skb->protocol;
!     new_skb->stamp = skb->stamp;
!     new_skb->security = skb->security;
  #if ((LINUX_VERSION_CODE >= 0x020200) && (LINUX_VERSION_CODE <
0x020300))
      new_skb->used = skb->used;
  #endif
***************
*** 551,557 ****
    memcpy(skb_to->cb, skb_from->cb, sizeof(skb_from->cb));
    skb_to->priority = skb_from->priority;
    skb_to->protocol = skb_from->protocol;
!   skb_to->tstamp = skb_from->tstamp;
  #if ((LINUX_VERSION_CODE >= 0x020200) && (LINUX_VERSION_CODE <
0x020300))
    skb_to->used = skb_from->used;
  #else
--- 550,556 ----
    memcpy(skb_to->cb, skb_from->cb, sizeof(skb_from->cb));
    skb_to->priority = skb_from->priority;
    skb_to->protocol = skb_from->protocol;
!   skb_to->stamp = skb_from->stamp;
  #if ((LINUX_VERSION_CODE >= 0x020200) && (LINUX_VERSION_CODE <
0x020300))
    skb_to->used = skb_from->used;
  #else