Re: New network driver interface changes, README

Re: New network driver interface changes, README

[David S. Miller]

   From: Henner Eisen <eis@baty.hanse.de>
   Date: 11 Feb 2000 20:57:58 +0100

   >>>>> "David" == David S Miller <davem@redhat.com> writes:

       David> 5) dev->hard_start_xmit()

	   [:]

       David> 	This method will be called with local _software_
						   ^^^^^^^^^^^^^^^^
       David> interrupts disabled.  If you need local hw IRQs disabled or
	      ^^^^^^^^^^^^^^^^^^^

   Is this part of the interface specification or just a feature of the
   current implementation?

   Should driver authors rely or better not rely on this if they want to
   reduce the probability of trouble in the future (future network changes)?

That's a very good question.  Here is how I would like driver authors
to think about this:

	The network queueing layer guarentees that your
	hard_start_xmit() method is not re-entrable.  This
	means that only one thread can be running in that
	method at once for a particular device instance.

	That is to say, if dev1->hard_start_xmit() is currently
	running, it will not be invoked for "dev1" anywhere
	else in the system until that thread returns from the
	function.

	You may not depend upon the mechanism used by the
	network queueing layer to achieve this.  It is only
	the property achieved which may be depended upon.

Does this clear up the issue?

Actually, we thought a lot about (and are still considering)
taking away this assumption as well, and making the atomicity
guarentee be a problem of the driver.  You may ask yourself
why we would do that, since it would make even more work for
the driver authors (well, to be really SMP safe you have to
interlock with your TX complete interrupt anyways).  Here are
the reasons:

Some software network devices require none of the locking, they
are self-consistent.  One great example is loopback.  It does not
need dev->xmit_lock nor the local software interrupt disabling.
All it does is pass the packet back to the input side of the
networking, via netif_rx(skb) which works only on cpu local state.

Secondly, since most ethernet/fddi/etc. drivers will need to do
their own spinlocking and hw IRQ disabling to protect against their
TX complete interrupt, we're just using twice as much locking overhead
by providing the non-reentrancy dev->xmit_lock buisness.

It's something to consider, but I probably won't allow this change
to happen for 2.4.x

Later,
David S. Miller
davem@redhat.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: New network driver interface changes, README

[jamal]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 225 bytes --]


Dave,
Back in december i wrote a HOWTO on this (attached) 
Only a few people paid attention to it then. Its great to see this finally
making it in! 
So consider this to be another description of the changes.

cheers,
jamal


[-- Attachment #2: Type: APPLICATION/octet-stream, Size: 4748 bytes --]

New network driver interface changes, README

[David S. Miller]

	Hello folks.  A major interface change for the networking
drivers just occurred in 2.3.43-7 and at the request of Linus I'm
going to post here a brief description of what needs to be done to
fix up the drivers which are still not fixed.  This is all for the
better, and it's going to expose (and fix) a lot of bugs drivers
have now, and simplify them as well.

1) The tbusy, start, and interrupt members are now gone.
   They are replaced with clean interfaces which do the
   queueing/state work these members use to be used for.

   Many drivers manipulated them incorrectly, many used
   them internally for state tracking, etc.  And deleting
   these members forces people to fix the drivers and also
   abstracts out the operations so we can modify the network
   packet queueing implementation without changing all the drivers in
   the future should a change in the implementation be needed.

2) dev->tbusy conversions:

	Here are the basic transformations for fix up a driver's
	dev->tbusy references:

	Old way				New way

	dev->tbusy = 1;			netif_stop_queue(dev);

	dev->tbusy = 0;			netif_start_queue(dev);

	dev->tbusy = 0;			netif_wake_queue(dev);
	mark_bh(NET_BH);

	The on/off state of the tx queue is held in the
	state bit LINK_STATE_XOFF of dev->flags.  So you
	may explicitly query it with:

		test_bit(LINK_STATE_XOFF, &dev->flags);

	You absolutely may _NOT_ modify this bit with
	{set,clear,change}_bit().  You must go through the
	netif_*_queue() interfaces to make such a state
	change.

3) dev->start

	It is now represented by the LINK_STATE_START
	bit flag in dev->start, and you may query it.
	For example:

	  if (dev->start)

	becomes:

	  if (test_bit(LINK_STATE_START, &dev->state))

	Drivers are _HIGHLY_ encouraged not to mess with it's
	setting.  Drivers should only test it, the generic
	device up/down/config infrastructure sets it properly
	for you.

	For example, in many drivers there is a "dev->start = 1"
	line in the dev->open() method.  Remove it, it is wrong
	and it will be set for you by the generic network device
	layer when it invokes you.

4) dev->interrupt

	First, just delete all references to it from your driver.

	Some tried to use it as an SMP locking mechanism,
	most if not all did it incorrectly.  If you need
	internal locking in your driver, use your device
	private structure to hold spinlocks and other
	correct locking primitives needed to achieve this.

5) dev->hard_start_xmit()

	Some drivers assumed it was hw IRQ protected, some
	didn't, etc.  Massive disagreement.

	Now it is stated, this method is protected from
	re-entrancy via a dev->xmit_lock which is grabbed
	by the caller, you need not and should not make
	any refernece to this lock.

	This method will be called with local _software_
	interrupts disabled.  If you need local hw IRQs
	disabled or spinlocks on SMP to synchronize with
	the TX handling in your driver IRQ handler, you
	must do this yourself.

	And furthermore, by implication this method being
	called means that LINK_STATE_XOFF is _NOT_ set
	and the queue is on.

	This last point means that you should remove the
	"TX timeout" detection garbage from these methods
	in the driver.  Which leads me to #6.

6) TX timeouts are explicitly handled for you, you just
   need to provide a dev->tx_timeout handler and the
   timeout value you'd like the generic networking to use.

   The type of this method is:

	void my_driver_timeout(struct net_device *dev)

   It should make note of the timeout, try to unstuck
   the hardware, and if this process unblocks the
   transmit queue, you should call netif_wake_queue(dev)
   before returning.

   Set this up at driver init time like so:

	dev->tx_timeout = my_driver_timeout;
	dev->watchdog_timeo = TX_TIMEOUT;

I've begun to try to shape up the drivers/net/skeleton.c example
driver file to show how these mechanisms are meant to be used.

I would suggest that for more information folks look in there.
For most common cards which have a "TX descriptor ring of transmit
buffers" scheme, you want to pay particular attention to the
"#if TX_RING" sections.

You can also look at the changes made to several of the already
converted drivers to get even more of a feel of how things need
to be adjusted/fixed.  Personally I've done the most testing
with the sunhme.c driver.

Later,
David S. Miller
davem@redhat.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/