RAID Sync Speeds

RAID Sync Speeds

[Rainer Shiz]

Hi,
    I have a general question on operating software RAIDs in linux.
    I am running md RAID in linux 2.6.12 kernel. I observed that the
sync speeds
were always much closer to /proc/sys/dev/raid/speed_limit_min value,
than
/proc/sys/dev/raid/speed_limit_max value. I agree that disk activity
(I/O) and
other system usage will bring down sync speeds. But I want sync speeds
to be
higher at whatever cost. So I thought I will just set max value to
around
500000 and leave the min at 5000. But with idle disk activity and
otherwise
idle system usage, I still see sync speeds around 5300Kb/s only.

So Is the 2.6 kernel designed to sync at speeds closer to min than max?

Please advise.

Please copy on your replies.

Thanks

Re: RAID Sync Speeds

[Chris Boot]

On 22 May 2006, at 14:05, Rainer Shiz wrote:

> Hi,
>    I have a general question on operating software RAIDs in linux.
>    I am running md RAID in linux 2.6.12 kernel. I observed that the
> sync speeds
> were always much closer to /proc/sys/dev/raid/speed_limit_min value,
> than
> /proc/sys/dev/raid/speed_limit_max value. I agree that disk activity
> (I/O) and
> other system usage will bring down sync speeds. But I want sync speeds
> to be
> higher at whatever cost. So I thought I will just set max value to
> around
> 500000 and leave the min at 5000. But with idle disk activity and
> otherwise
> idle system usage, I still see sync speeds around 5300Kb/s only.
>
> So Is the 2.6 kernel designed to sync at speeds closer to min than  
> max?
>
> Please advise.

I must admit I keep my settings on the default (1000/200000) and get  
whatever my disks can handle, usually 50-70MB/sec. All this is on a  
completely idle system, doing anything disk bound at all will lower  
these numbers significantly. This is on various SATA controllers, all  
with Seagate drives.

HTH,
Chris

-- 
Chris Boot
bootc@bootc.net
http://www.bootc.net/

Re: RAID Sync Speeds

[Neil Brown]

On Monday May 22, rainer.shiz@gmail.com wrote:
> 
> So Is the 2.6 kernel designed to sync at speeds closer to min than max?
> 

If there is other detectable activity, the sync speed will be kept at
or below the min.
If there no other activity, the sync speed will be kept at or below
the max.

NeilBrown

Re: RAID Sync Speeds

[Rainer Shiz]

Thanks Neil and Chris for your quick replies.

Neil, well when you say 'detectable activity' what do you exactly refer to.
Because my system is for all purposes 'idle'. It is not on the network,
just booted it up, no apps running, only linux got booted and I am
logging in through console (text only mode - no X). So I wish the sync
speeds were atleast somewhere inbetween min and max values.
But I see it hovering around min (min being 5000, speed was around 5320 kb/s).
My max was around 250000.

But once I changed min value to 20000, my speeds were around 10000kb/s.
Max being same 250000.
I heard that speed values are averaged out for about 5 minutes or so.
Hence I did try these experiments for about 1 hour each.

This is what made me wonder if there is anything in the 2.6.12 kernel which is
causing this behavior!

One more question while I am at it, if a RAID Set is newly created and
syncing, and now I change the /proc/sys/dev/raid/speed_limit_min and
max values will this reflect on new RAID Sets that are created henceforth
or even the existing RAID Sets which are being synced currently?

And Chris, you are right - I understand that doing any other extraneous
disk I/o or fetching will cause the sync speeds to slow down or
in the extreme case there will be a limit on the SATA bus and how
many other hard disks share it and what i/o is happening on them too,
but in my case I am running these experiments keeping this in mind
and keeping all hard disk activity at almost zero levels.

And yes, I am too using Seagate hard drives. Does mixing up of
hard drives (vendors) affect this sync process.? (I presume not but please
correct me if I am wrong).

Thanks again.
Rainer

On 5/22/06, Neil Brown <neilb@suse.de> wrote:
> On Monday May 22, rainer.shiz@gmail.com wrote:
> >
> > So Is the 2.6 kernel designed to sync at speeds closer to min than max?
> >
>
> If there is other detectable activity, the sync speed will be kept at
> or below the min.
> If there no other activity, the sync speed will be kept at or below
> the max.
>
> NeilBrown
>

Re: RAID Sync Speeds

[Neil Brown]

On Tuesday May 23, rainer.shiz@gmail.com wrote:
> Thanks Neil and Chris for your quick replies.
> 
> Neil, well when you say 'detectable activity' what do you exactly
> refer to.

It should mean any IO request to any of the physical devices, whether
though the raid array or otherwise.  Sometimes in can get confused,
but in 2.6.12 (If it a kernel.org 2.6.12) got it pretty right I
think.  Current kernels may have occasional issues that I really need
to sort out one day...

Maybe if you give me more precise details about your setup 
(e.g. cat /proc/mdstat) something might occur to me.

> I heard that speed values are averaged out for about 5 minutes or so.

They are averaged over 30 seconds.

> 
> One more question while I am at it, if a RAID Set is newly created and
> syncing, and now I change the /proc/sys/dev/raid/speed_limit_min and
> max values will this reflect on new RAID Sets that are created henceforth
> or even the existing RAID Sets which are being synced currently?

These numbers apply globally to all arrays, and can be changed at any
time.
More recent kernels have similar numbers in /sys/block/mdX/md/..
which allow the same settings to be applied to individual arrays.

> 
> And yes, I am too using Seagate hard drives. Does mixing up of
> hard drives (vendors) affect this sync process.? (I presume not but please
> correct me if I am wrong).

Mixing things shouldn't have an unexpected effect.  Obviously the
total speed will be limited by the slowest device, but no other
effects that I can thing of.  Certainly not what you are experiencing.

NeilBrown