Re: Compression algorithm in cloop

Re: Compression algorithm in cloop

[Phillip Lougher]

 > Hello,
 >
 > I am trying to do some experiment on compression ratios with cloop. I
 > know that currently cloop uses zlib. How can I change it to other
 > algorithms?

Changing the algorithm from gzip is going to be probably unpopular.
Cloop uses the gzip deflate library inside the kernel shared by a
large number of other programs.  To change the algorithm you'll have
to add more (private) decompression code to the kernel.  This is a retrograde
step because the shared library was only introduced in 2.4.17 to avoid lots
of private copies of gzip.

 > Where should I start from? Really a newbie to this,
 > appreciate any comments and suggestions!!

There has been discussion on this list before about adding
bzip2 support to the kernel.  Do a search on the list for this.

Regards

Phillip

Re: Compression algorithm in cloop

[Grzegorz Kulewski]

On Wed, 11 Aug 2004, Phillip Lougher wrote:

>> Where should I start from? Really a newbie to this,
>> appreciate any comments and suggestions!!
>
> There has been discussion on this list before about adding
> bzip2 support to the kernel.  Do a search on the list for this.

Long, long ago there was something called e2compr. They had bzip2 in the 
kernel. I remember 40 seconds mc start time on disk compressed with bzip2 
on i486-DX 40 MHz... :-)


Grzegorz Kulewski

Re: Compression algorithm in cloop

[Randy.Dunlap]

On Wed, 11 Aug 2004 19:21:50 +0200 (CEST) Grzegorz Kulewski wrote:

| On Wed, 11 Aug 2004, Phillip Lougher wrote:
| 
| >> Where should I start from? Really a newbie to this,
| >> appreciate any comments and suggestions!!
| >
| > There has been discussion on this list before about adding
| > bzip2 support to the kernel.  Do a search on the list for this.
| 
| Long, long ago there was something called e2compr. They had bzip2 in the 
| kernel. I remember 40 seconds mc start time on disk compressed with bzip2 
| on i486-DX 40 MHz... :-)

still at
  http://e2compr.sourceforge.net/

--
~Randy

Re: Compression algorithm in cloop

[Lei Yang]

If I don't want to build a new compression library and port it to 
kernel, all I want to do is to try out the idea -- whether it could work 
on cloop, whether the compression ratio is acceptable, can I just change 
the cloop code? In other words, I just need a loopback block device with 
other compression scheme.

I know that adding the new algorithm support to kernel would be standard 
way to do this and would be helpful to more people, it's just that I 
don't have that time :(

Phillip Lougher wrote:
>  > Hello,
>  >
>  > I am trying to do some experiment on compression ratios with cloop. I
>  > know that currently cloop uses zlib. How can I change it to other
>  > algorithms?
> 
> Changing the algorithm from gzip is going to be probably unpopular.
> Cloop uses the gzip deflate library inside the kernel shared by a
> large number of other programs.  To change the algorithm you'll have
> to add more (private) decompression code to the kernel.  This is a 
> retrograde
> step because the shared library was only introduced in 2.4.17 to avoid lots
> of private copies of gzip.
> 
>  > Where should I start from? Really a newbie to this,
>  > appreciate any comments and suggestions!!
> 
> There has been discussion on this list before about adding
> bzip2 support to the kernel.  Do a search on the list for this.
> 
> Regards
> 
> Phillip
> 

Re: Compression algorithm in cloop

[Paulo Marques]

Lei Yang wrote:
> 
> If I don't want to build a new compression library and port it to 
> kernel, all I want to do is to try out the idea -- whether it could work 
> on cloop, whether the compression ratio is acceptable, can I just change 
> the cloop code? In other words, I just need a loopback block device with 
> other compression scheme.

If you just want to try out some wacky ideas, I suggest you take a look 
at the network block device.

It is a block device that sends all read/write requests through a socket 
  so that this requests can be fulfilled by a user space deamon.

This is not very good in terms of performance, but it is much easier to 
develop this kind of thing in userspace, without all the stack / memory 
/ atomicity / concurrency problems :)

-- 
Paulo Marques - www.grupopie.com
"In a world without walls and fences who needs windows and gates?"

Re: Compression algorithm in cloop

[Lei Yang]

Great! That certainly should help!

The problem is, how can I put the algorithm in the kernel? Is there a 
forum or mailing list for this kind of work?

Thanks!
Lei

Luca Barbato wrote:
> Lei Yang wrote:
> 
>> Hello,
>>
>> I am trying to do some experiment on compression ratios with cloop. I 
>> know that currently cloop uses zlib. How can I change it to other 
>> algorithms? Where should I start from? Really a newbie to this, 
>> appreciate any comments!!
>>
>> TIA.
>> Lei
>>
> I made a variation from cloop called gcloop it could use any algorithm 
> you can put in the kernel (I use an hack to use ucl right now)
> 
> You can find it at dev.gentoo.org/~lu_zero/gcloop
> 
> I hope it helps.
> 
> lu
> 

Compression algorithm in cloop

[Lei Yang]

Hello,

I am trying to do some experiment on compression ratios with cloop. I 
know that currently cloop uses zlib. How can I change it to other 
algorithms? Where should I start from? Really a newbie to this, 
appreciate any comments and suggestions!!

TIA.
Lei