A hole in kernel space!

A hole in kernel space!

[Pannaga Bhushan]

Hi all,
        I am looking for a setup where I need to have a certain amount
of data always available to the kernel. The size of data I am looking at
is abt
40MB(preferably, but I will settle for 20MB too) . So the normal kmalloc
will not help me. So what I did was, I created a hole in kernel space by
putting
the following line in vmlinux.lds

    ALIGN(4096);
 __hole_start = .;
    . = . + 0xmy_size;
 __hole_end = .;

First, I put these lines in code segment and found that  'my_size'
cannot go beyond 0x500000(5MB) . Any larger value , the kernel image
refuses to
boot up. I found the same problem with these lines being in data segment
or in the bss segment.

But putting these line after

_end = .;

line in vmlinux.lds, I am able to give 0x1700000(17MB) to my_size and
still boot with that kernel image.

My questions are :

1.   Is there any other way I can get to keep 40MB(or even 20MB) of
contiguous kernel memory space ?

2.    Abt the 17MB hole, I am able to use after the   _end = .;
....     is this 17MB really there in kernel image?('cos it isn't in any
segment and also it
appears after _end).
        if yes, are the pages corresponding to this region swappable or
is it that since this hole appears in kernel image, it is locked to a
physical space
and this is never swapped. (basically, i want by data in kernel space
always available to kernel without having to bother abt swapping the
pages back)

Thanx in advance,
Pannaga Bhushan

Re: A hole in kernel space!

[Abraham vd Merwe]

[-- Attachment #1: Type: text/plain, Size: 2584 bytes --]

Hi Pannaga!

Use the mem= parameter on the kernel command-line to specify less memory
than what is available. Then you can use the last unused chunk of memory,
e.g. if you have 256M memory and you want to "steal" a 40M chunk at the end:

mem=216M

> Hi all,
>         I am looking for a setup where I need to have a certain amount
> of data always available to the kernel. The size of data I am looking at
> is abt
> 40MB(preferably, but I will settle for 20MB too) . So the normal kmalloc
> will not help me. So what I did was, I created a hole in kernel space by
> putting
> the following line in vmlinux.lds
> 
>     ALIGN(4096);
>  __hole_start = .;
>     . = . + 0xmy_size;
>  __hole_end = .;
> 
> First, I put these lines in code segment and found that  'my_size'
> cannot go beyond 0x500000(5MB) . Any larger value , the kernel image
> refuses to
> boot up. I found the same problem with these lines being in data segment
> or in the bss segment.
> 
> But putting these line after
> 
> _end = .;
> 
> line in vmlinux.lds, I am able to give 0x1700000(17MB) to my_size and
> still boot with that kernel image.
> 
> My questions are :
> 
> 1.   Is there any other way I can get to keep 40MB(or even 20MB) of
> contiguous kernel memory space ?
> 
> 2.    Abt the 17MB hole, I am able to use after the   _end = .;
> ....     is this 17MB really there in kernel image?('cos it isn't in any
> segment and also it
> appears after _end).
>         if yes, are the pages corresponding to this region swappable or
> is it that since this hole appears in kernel image, it is locked to a
> physical space
> and this is never swapped. (basically, i want by data in kernel space
> always available to kernel without having to bother abt swapping the
> pages back)
> 
> Thanx in advance,
> Pannaga Bhushan
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 

Regards
 Abraham

UNIX enhancements aren't.

__________________________________________________________
 Abraham vd Merwe - 2d3D, Inc.

 Device Driver Development, Outsourcing, Embedded Systems

  Cell: +27 82 565 4451         Snailmail:
   Tel: +27 21 761 7549            Block C, Aintree Park
   Fax: +27 21 761 7648            Doncaster Road
 Email: abraham@2d3d.co.za         Kenilworth, 7700
  Http: http://www.2d3d.com        South Africa


[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: A hole in kernel space!

[Matti Aarnio]

On Wed, Nov 06, 2002 at 05:27:50PM +0530, Pannaga Bhushan wrote:
> Hi all,
>         I am looking for a setup where I need to have a certain amount
> of data always available to the kernel. The size of data I am looking at
> is abt 40MB(preferably, but I will settle for 20MB too) . So the normal
> kmalloc will not help me.

  Will  vmalloc()  help ?  The allocated memory is in special
  vmalloc address space (which is limited!) within the kernel,
  but accesses to it go via page-mapping tables, thus while it
  is virtually contiguous, physically it might be well scattered.

  What properties do you need, to be exact ?

  You are writing some sort of a device driver for a device that
  has somewhat stupid direct-memory-access facilities ?

> if yes, are the pages corresponding to this region swappable or
> is it that since this hole appears in kernel image, it is locked to a
> physical space and this is never swapped. (basically, i want by data
> in kernel space always available to kernel without having to bother
> abt swapping the pages back)

  Both  kmalloc()  and  vmalloc()  allocate kernel space memory that
  is unswappable, and therefore can not be allocated in excessive
  amounts.

  Doing things in BIGMEM fashion will let you have gigabytes of
  in-core memory, but to access it, you must go thru the hoops
  (in i686 architecture) each time you access a page of it.

> Thanx in advance,
> Pannaga Bhushan

/Matti Aarnio

Re: A hole in kernel space!

[Marc A. Volovic]

Quoth Pannaga Bhushan:

> Hi all,
>         I am looking for a setup where I need to have a certain amount
> of data always available to the kernel. The size of data I am looking at
> is abt
> 40MB(preferably, but I will settle for 20MB too) . So the normal kmalloc

I wrote a driver which steals a certain amount of memory from the kernel
and makes it available to userspace (somewhat like the rd driver).
If you want - I can send it to you.

It exports a small device hierarchy which can be read, written and
mmap'ed. The memory is contiguous. Not VERY elegant, but works quite
well.

The driver steals a certain amount of memory from the kernel at boot
time, a-la the mem= parameter. I have used "holes" of up to 1GB in size.


Marc

-- 
---MAV
                        Linguists do it cunningly
Marc A. Volovic                                             marc@bard.org.il

Re: A hole in kernel space!

[Richard B. Johnson]

On Wed, 6 Nov 2002, Marc A. Volovic wrote:

> Quoth Pannaga Bhushan:
> 
> > Hi all,
> >         I am looking for a setup where I need to have a certain amount
> > of data always available to the kernel. The size of data I am looking at
> > is abt
> > 40MB(preferably, but I will settle for 20MB too) . So the normal kmalloc
> 
> I wrote a driver which steals a certain amount of memory from the kernel
> and makes it available to userspace (somewhat like the rd driver).
> If you want - I can send it to you.
> 
> It exports a small device hierarchy which can be read, written and
> mmap'ed. The memory is contiguous. Not VERY elegant, but works quite
> well.
> 
> The driver steals a certain amount of memory from the kernel at boot
> time, a-la the mem= parameter. I have used "holes" of up to 1GB in size.
> 
> 
> Marc

You know, I hope, that you can kmalloc(GFP_KERNEL) something in
init_module() and it will always be resident in the kernel until
you kfree() it in cleanup_module(). You do not have to
allocate/deallocate every time your module does something.

If you need a physical location, mmap() and ioremap() take care
of that. With these capabilities, I don't think you need "holes"
even if you are trying to map-out bad RAM. I think you need to
re-think your requirements. 

Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
   Bush : The Fourth Reich of America

Re: A hole in kernel space!

[Marc A. Volovic]

Quoth Richard B. Johnson:

> You know, I hope, that you can kmalloc(GFP_KERNEL) something in
> init_module() and it will always be resident in the kernel until
> you kfree() it in cleanup_module(). You do not have to
> allocate/deallocate every time your module does something.

I do not alloc/dealloc every time ;-). I allocate at boot and do not
bother deallocating at all. The module just uses that allocated memory
and maps it into the filesystem namespace.

>    Bush : The Fourth Reich of America

Errr, not that I care, but as far as I know, America had not yet had
even its first Reich. ;-)... Germany had three, so a forth Reich might
have been there...

-- 
---MAV
                        Linguists do it cunningly
Marc A. Volovic                                             marc@bard.org.il

Re: A hole in kernel space!

[Adrian Bunk]

On Wed, Nov 06, 2002 at 06:09:34PM +0200, Marc A. Volovic wrote:
> 
> I do not alloc/dealloc every time ;-). I allocate at boot and do not
> bother deallocating at all. The module just uses that allocated memory
> and maps it into the filesystem namespace.
>...

Read pages 221-223 in

   Alessandro Rubini and Jonathan Corbet.  Linux device drivers.
   O'Reilly, second edition, 2001.  Online version:
   http://www.xml.com/ldd/chapter/book/index.html


cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed