Main CPU- I/O CPU interaction

Main CPU- I/O CPU interaction

[tony osborne]

Hello,

I wish to be personally CC'ed the answers/comments posted to the list in 
response to this post .


The I/O devices are equipped with dedicated processor to free the  main CPU 
from doing the low level I/O operations. However, if i am editing and 
updating a big size file and i want to save
it afterwards, i  notice my PC getting blocked while saving the file which 
theoritically should NOT happen as it is up to the I/O device processor and 
not the main CPU to save the data into the disk; the main CPU could switch 
to another process after giving the high level command -save-to the device 
processor; so why the main CPU is blocked while saving such big size files

thanks

_________________________________________________________________
It's fast, it's easy and it's free. Get MSN Messenger today! 
http://www.msn.co.uk/messenger

Re: Main CPU- I/O CPU interaction

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On Mon, Jan 03, 2005 at 12:44:36AM +0000, tony osborne wrote:
> The I/O devices are equipped with dedicated processor to free the  main CPU 
> from doing the low level I/O operations. However, if i am editing and 
                 ^^^^^^^^^^^^^
> updating a big size file and i want to save
> it afterwards, i  notice my PC getting blocked while saving the file which 
> theoritically should NOT happen as it is up to the I/O device processor and 
> not the main CPU to save the data into the disk; the main CPU could switch 
> to another process after giving the high level command -save-to the device 
                                      ^^^^^^^^^^^^^^^^^^
> processor; so why the main CPU is blocked while saving such big size files

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: Main CPU- I/O CPU interaction

[David Schwartz]

> from doing the low level I/O operations. However, if i am editing and 
> updating a big size file and i want to save
> it afterwards, i  notice my PC getting blocked while saving the 
> file which 
> theoritically should NOT happen as it is up to the I/O device 
> processor and 

	What does "PC getting blocked" mean?

	DS

RE: Main CPU- I/O CPU interaction

[tony osborne]

sorry for not being clear. i meant no other application running while saving 
the data, i.e. CPU idle?


>From: "David Schwartz" <davids@webmaster.com>
>Reply-To: davids@webmaster.com
>To: <linux-kernel@vger.kernel.org>
>CC: <tonyosborne_a@hotmail.com>
>Subject: RE: Main CPU- I/O CPU interaction
>Date: Mon, 3 Jan 2005 10:51:43 -0800
>
>
> > from doing the low level I/O operations. However, if i am editing and
> > updating a big size file and i want to save
> > it afterwards, i  notice my PC getting blocked while saving the
> > file which
> > theoritically should NOT happen as it is up to the I/O device
> > processor and
>
>	What does "PC getting blocked" mean?
>
>	DS
>
>

_________________________________________________________________
It's fast, it's easy and it's free. Get MSN Messenger today! 
http://www.msn.co.uk/messenger

RE: Main CPU- I/O CPU interaction

[David Schwartz]

> sorry for not being clear. i meant no other application running
> while saving
> the data, i.e. CPU idle?

	I still have no idea what you're talking about and I don't think anyone
else does either. For example, by CPU being idle, do you mean that it's
doing nothing? If so, how did you establish this? Are you talking about the
output of 'top', for example?

	You are going to have to be precise about what your observing. You are
stating your observations in the form of conclusions, and this makes it
impossible for us to tell what you're talking about. It could be that your
misconceptions are coloring your conclusions and others would reach
different conclusions from those same observations.

	Example: You are asking, "why does cutting off a frog's legs make it deaf"?
We have no idea what you're talking about. Then you explain your
*observations* which are that if you cut off all of a frog's legs, it no
longer jumps when you yell at it. We then understand that you drew the wrong
conclusion from your observation. But from just the conclusion, there's no
way to tell what you actually are seeing.

	"CPU idle" is a conclusion. Something made you think your CPU was idle.
What? Same with your entire post.

	DS

Re: Main CPU- I/O CPU interaction

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1604 bytes --]

On Mon, 03 Jan 2005 00:44:36 GMT, tony osborne said:

> The I/O devices are equipped with dedicated processor to free the  main CPU 
> from doing the low level I/O operations.

Even a "smart" RAID/SCS/ATA controller is pretty stupid - at best it can be
trusted to find block number N and either read or write it.  If you're *really*
lucky, the controller knows about partitions.  Any further intelligence is the
realm of IBM-style big iron (where the 'set extent' and 'search key high/low/
equal' CCW commands can offload a significant amount of work).  But even there,
the hardware has no filesystem awareness.

>                                          However, if i am editing and 
> updating a big size file and i want to save
> it afterwards, i  notice my PC getting blocked while saving the file which 
> theoritically should NOT happen as it is up to the I/O device processor and 
> not the main CPU to save the data into the disk;

What sort of I/O device processor is (a) supported by Linux *and* (b) filesystem
aware?  Unless it meets both criteria, the main CPU(s) will still have to do
all the work of block allocation, inode creation, and all the rest of that
stuff.

So what sort of hardware were you thinking of, where the "I/O processor" is
smart enough to handle saving a file (from within an application, no less)
without CPU interference?  (and if you're saving the file from within the
application, what API is used to tell the I/O processor whether to use an
"open and fseek to 0" style overwrite, or "rename, creat, write, close, unlink
old" style - all without CPU interference?)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Main CPU- I/O CPU interaction

[Alan Cox]

On Llu, 2005-01-03 at 22:27, Valdis.Kletnieks@vt.edu wrote:
> What sort of I/O device processor is (a) supported by Linux *and* (b) filesystem
> aware?  Unless it meets both criteria, the main CPU(s) will still have to do
> all the work of block allocation, inode creation, and all the rest of that
> stuff.

NetApp 8)

The offloading file system stuff has been proposed and kicked around and
failed repeatedly in different circles. What has had some success is
offloading layout so that the I/O device thinks in terms of

	handle = allocate(blocks, near_this, and_this, ....)
	data = read(handle, offset, len)
	write(handle, offset, len, data)
	free(handle)

type API's, allowing the storage subsystem to do physical
mirror/relocation/HSM in private.

Re: Main CPU- I/O CPU interaction

[Tom Felker]

On Sunday 02 January 2005 06:44 pm, tony osborne wrote:
> Hello,
>
> I wish to be personally CC'ed the answers/comments posted to the list in
> response to this post .
>
>
> The I/O devices are equipped with dedicated processor to free the  main CPU
> from doing the low level I/O operations. However, if i am editing and
> updating a big size file and i want to save
> it afterwards, i  notice my PC getting blocked while saving the file which
> theoritically should NOT happen as it is up to the I/O device processor and
> not the main CPU to save the data into the disk; the main CPU could switch
> to another process after giving the high level command -save-to the device
> processor; so why the main CPU is blocked while saving such big size files
>
> thanks

Check to make sure DMA is on:

hdparm /dev/hda

If not, try turning it on:

hdparm -d 1 /dev/hda

When it's not on, you'll get bad performance (I usually get around 2 MB/s 
according to "hdparm -t /dev/hda") and high system CPU usage.  Because the 
rate is so low, for big files, the high CPU usage starves other processes for 
long enough that you notice.  With DMA, the transfer will be much faster and 
the CPU usage will be minimal.

-- 
Tom Felker, <tcfelker@mtco.com>
<http://vlevel.sourceforge.net> - Stop fiddling with the volume knob.

If nature has made any one thing less susceptible than all others of exclusive 
property, it is the action of the thinking power called an idea.