Kernel drops ethernet packets during disk writes

["Tiemen Schut" <tschut@gmail.com>]

Summary: The linux kernel appears to drop raw ethernet packets if
another process is writing to disk.

Test environment: Used a p4 1.7 GHz with gigabit interface
point-to-point connection to another p4 (windows pc). This windows PC
generates raw ethernet frames holding a counter and sends 'm on to the
linux PC, at a transfer rate of 350 Mbit/s. When not writing to disk,
everything goes quite fine. I can check the counters at the linux
side, and will notice a minimal packet loss (<  0.001 % or so).

However, for my application I want to write each and every frame to
disk. So, I created a second app, and through a fifo the receiver app
and the disk writer app communicate. Now I'm losing like 80% of my
packets :o Reducing the throughput on the network doesn't really help
(though it does help a little).

Note: I tried everything in the same app first, but my guess was that
the write operation delayed the app, so I decided to put everything in
two apps to give the scheduler some work ;) Didn't work though.

My guess would be that the kernel drops the packets because the write
operation takes to long (how long can it take, it's just a stupid 1512
bytes frame). Anyway, I tried to enlarge
.sys.net.core.netdev_max_backlog, but that didn't do the trick.

This problem occures on both 2.6.13 and 2.4.idontremember.

It kinda sucks, what's the use of receiving traffic if you can't write
it to disk?

I'm sending the packets using the winpcap library, and I'm receiving
the packets using the pcap library.

Any help would be _greatly_ appreciated, and if neccessary, please ask
for additional information/used software/test results/etc.

Tiemen Schut


PS: Personal CC's are okay with me, but if it's to much trouble never mind :)