Disk access scheduling - FS cache dispensing - process priorities

[Oran Agra <oran@monfort.co.il>]

Hi,
I'll try to be brief.

In the context of CPU time scheduling and process priorities the disk access scheduler does not seem to consider the requesting process priority when it handles disk access requests.
This can create a situation where a 'real-time like' high priority process will be stuck or have reduced performance due to a low priority process being a write-hog (or read-hog).

Furthermore, a similar issue resides in the file-system cache dispensing mechanism.
In a system with low amount of RAM, a low priority read/write hog might squeeze a high priority process's data out of the disk-cache resulting in poor performance for the high priority process.

This is even more severe since the cache is used to memory map the executable code into the RAM, and the RSS of that mapping is being reduced. 
In that case the performance of the high priority process will be reduced even if the user code doesn't issue any file system requests, but since the RSS portion of its executable memory mapping was reduced, thrashing will occur when the process calls an executable area that isn't currently cached.

Fist of all, am I missing something? 
Is there any way for a user to overcome this behavior and improve on the I/O scheduling and cache dispensing of a prioritized process?
If not:

[Suggestions]
I want to suggest a few options if these mechanisms are ever re-worked:
- Disk access scheduling based on requesting process priority.
- File system cache dispensing based on process priority.
- Ability (system call api, or /proc entry) to do any of the following:
 1) Limit a certain process's (the hog) maximum cache consumption.
 2) Reserve a certain amount of cache for a certain (high priority) process.
 3) Limit the amount of cache consumption for a certain folder (e.g logs).
 4) Reserve an amount of cache for a file or folder (executable, libraries).

[Reading is optional]
Maybe a little bit of background on the scenario where I stumbled upon this will assist your understanding:
Life assist medical device running Linux kernel (2.6.17.9) + busybox etc, running on ARM9 + 64MB of RAM, no swap space, and flash based disk.
One high priority process handles the critical tasks, and one low priority process handles the GUI.
When the GUI starts TAR-ring log files onto a removable flash drive, it suffocates the high priority critical process.
Although the high priority process doesn't issue any disk requests, the RAM is SO tight that the memory mapped portions of the executable are squeezed out of the cache.

Many thanks for a great work, and for reading thoughts.
Oran Agra.