Here is a patch that adds I/O priorities to the CFQ I/O scheduler in 
2.6.8.1. Its a forward port of the version Jens had posted a while 
back (http://www.ussg.iu.edu/hypermail/linux/kernel/0311.1/0024.html)

The patch sustains 2 diskhogs running at different I/O priorities for 
over 10 minutes and pounding on an IDE disk.

The patch was developed to continue CKRM's I/O scheduler work. Since 
this scheduler is needed for CKRM, any feedback from testing will be 
much appreciated. Please cc: ckrm-tech@lists.sf.net.

A subsequent patch will export per-priority level statistics of 
interest but is not required for correct functioning of this patch.

-- Shailabh