bonding driver issues: slave interface not coming up

[John Jasen <jjasen@realityfailure.org>]

Hello;

In short, using redhat enterprise 4, we're attempting to bond two tg3
gigabit cards in XOR mode.

These cards are plugged into a cisco 2970 switch, configured with
etherchannel bonding in src-dst-mac, which looks to be mode 2.

The problem area is, upon boot, the first slave NIC does not come up.
Furthermore, running ethtool against the slave yields the following message:

[root@XXX ~]# ethtool eth0
Settings for eth0:
Cannot get device settings: Resource temporarily unavailable
         Supports Wake-on: g
         Wake-on: d
         Current message level: 0x000000ff (255)
         Link detected: no

Now, curiously enough, after boot, logging in and bringing down the
bonding interface, then bringing it back up will activate both
interfaces. ie:

ifdown bond0
ifup bond0

ethtool eth0
[root@XXX ~]# ethtool eth0
Settings for eth0:
         Supported ports: [ MII ]
         Supported link modes:   10baseT/Half 10baseT/Full
                                 100baseT/Half 100baseT/Full
                                 1000baseT/Half 1000baseT/Full
         Supports auto-negotiation: Yes
         Advertised link modes:  Not reported
         Advertised auto-negotiation: No
         Speed: 1000Mb/s
         Duplex: Full
         Port: Twisted Pair
         PHYAD: 1
         Transceiver: internal
         Auto-negotiation: off
         Supports Wake-on: g
         Wake-on: d
         Current message level: 0x000000ff (255)
         Link detected: yes

Attempts at debugging that we've taken and their results:

duplex/speed forced on both switch and card: no effect
turning off spanning tree protocol: no effect
enabling fastport: no effect
switching from tg3 to bcm5700 drivers: no effect
attempting to down and up the interface in /etc/rc.d/init.d/network: no
effect
attempting a script that runs after network and irqbalance to do the
same thing: no effect
putting something in rc.local to do the same: works, our current ugly
workaround
breaking the bond, using the interfaces as two independent NICS: works
restarting the bond0 interface manually: works

This problem also manifests on more than one system, which would seem to
preclude possible cable, host, or port problems.

Any ideas, or any further information I can send you to help diagnose?
Any help would be much appreciated.

-- 
-- John E. Jasen (jjasen@realityfailure.org)
-- No one will sorrow for me when I die, because those who would
-- are dead already. -- Lan Mandragoran, The Wheel of Time, New Spring