Re: Only ~85MiB/s (e1000e/write) to ~310MiB/s RAID-0 on ATOM board?

[Justin Piszcz <jpiszcz@lucidpixels.com>]

On Sun, 16 Jan 2011, Ian E. Morgan wrote:

> Justin,
>
> I have two Supermicro 5015a-EHF-D525's. They use the same Intel 82574L
> NICs as your slightly older board.
> I asked myself the same questions as you just recently, and my
> research into multi-queue NIC support led to:
>
> Receive Packet Steering
>
> Take a look at the output of NET_TX and NET_RX in /proc/softirq.
> The interrupts were not balanced across the cpus until I did:
>
> echo f > /sys/devices/pci0000:00/0000:00:1c.1/0000:02:00.0/net/eth0/queues/rx-0/rps_cpus
> ("f" being a bitmask of cpus)
>
> Then the interrupts started balancing across all cpus.
> I haven't had a chance to try this in a high throughput transfer
> situation yet, as I just happened to look into this last night.
>
> Hope it leads you down the right path, and please let me know if it
> does or doesn't help, or any other solutions you've found or come up
> with.
>
> --
> Ian Morgan


Hi Ian,

Thanks for the response & suggestion!

Tried it on kernel 2.6.37:

Made sure it was enabled:
# grep RPS .config
CONFIG_RPS=y

Only achieved 79.98M/s on > 4 GB of data via FTP (downloading), does not
appear to change much.

# cat /sys/class/net/eth1/queues/rx-0/rps_cpus
0

# echo f > /sys/class/net/eth1/queues/rx-0/rps_cpus
# cat /sys/class/net/eth1/queues/rx-0/rps_cpus
f
#

# cat /proc/softirqs
                     CPU0       CPU1       CPU2       CPU3
           HI:          0          0          0          0
        TIMER:     508490     507869     508325     508253
       NET_TX:      22933      24274      23800      22124
       NET_RX:      94412     218903      93273      95787
        BLOCK:      63945      22646      21324       8084
BLOCK_IOPOLL:          0          0          0          0
      TASKLET:          8          4          2          5
        SCHED:     244572     218617     262071     242491
      HRTIMER:          0          0          0          0
          RCU:      52859      53519      53715      55199


What speed(s) do you achieve with the D525s?

If anyone finds out why these boards cannot achieve > 90-100MiB/s to disk,
I'd be curious to know why, thanks.

Justin.