Re: why VM_SHM has been removed from mm.h?

[Hugh Dickins <hugh@veritas.com>]

On Wed, 1 Mar 2006, Kamran Karimi wrote:
> 
> Thank you Hugh for the reply. Last time I used VM_SHM was in 2.2.x kernels. 
> I have a programme called DIPC which makes System V shared memory segments 
> (and also messages and semaphores) work over a network.
> 
> In the arch/xyz/mm/fault.c file, it checks the VM_SHM flag and then calls 
> its logic. As a substitute I've been trying this ad-hoc code to see if a vma 
> represents a Sys V shm:
> 
>       file = vma->vm_file;
>       if(file && (file->f_dentry) && (file->f_dentry->d_inode) &&
>          (id = file->f_dentry->d_inode->i_ino)) {
>               shp = shm_lock(id);
>               if(shp == NULL)
>                       return 0; // not a Sys V shm
>       }
>       else return 0; // not a Sys V shm
> 
> But the kernel hangs with an invalid-pointer error message. Any suggestions?

It's not obvious to me why the kernel would hang with an invalid pointer
error message there: ipc_lock appears to have good safety against being
passed a random id.  Perhaps the invalid pointer message comes from
other code you've not shown (for example, I hope you shm_unlock(shp)
and return 1 when shm_lock succeeds), or perhaps I'm misreading.

But what you're doing there looks entirely weird and meaningless to me:
if shm_lock happens to succeed or fail on the inode number of some file
on some filesystem, that tells you nothing about whether that file is
SysV shm or not.  Ah, I see ipc/shm.c saves id in i_ino: so if you're
dealing with a SysV shm file, then indeed that ought to tell whether
you're dealing with a SysV shm file - but that hasn't helped much!

Since you're already patching base kernel source (you mention
arch/xyz/mm/fault.c), why don't you just patch your own VM_SYSVSHM
into include/linux/mm.h, and set it on the vma in ipc/shm.c?

Hugh