[PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Richard Purdie]

Upgrade the zlib_inflate implementation in the kernel from a patched
version 1.1.3 to a patched 1.2.3. 

The code in the kernel is about seven years old and I noticed that the
external zlib library's inflate performance was significantly faster
(~50%) than the code in the kernel on ARM (and faster again on x86_32).

For comparison the newer deflate code is 20% slower on ARM and 50%
slower on x86_32 but gives an approx 1% compression ratio improvement. I
don't consider this to be an improvement for kernel use so have no plans
to change the zlib_deflate code.

Various changes have been made to the zlib code in the kernel, the most
significant being the extra functions/flush option used by ppp_deflate.
This update reimplements the features PPP needs to ensure it continues
to work.

This code has been tested on ARM under both JFFS2 (with zlib compression
enabled) and ppp_deflate and on x86_32. JFFS2 sees an approx. 10% real
world file read speed improvement.

This patch also removes ZLIB_VERSION as it no longer has a correct
value. We don't need version checks anyway as the kernel's module
handling will take care of that for us. This removal is also more in
keeping with the zlib author's wishes
(http://www.zlib.net/zlib_faq.html#faq24) and I've added something to
the zlib.h header to note its a modified version.

Signed-off-by: Richard Purdie <rpurdie@rpsys.net>
---

 include/linux/zconf.h           |   12
 include/linux/zlib.h            |  209 ++++---
 include/linux/zutil.h           |   12
 lib/zlib_deflate/deflate.c      |   25
 lib/zlib_deflate/deflate_syms.c |    3
 lib/zlib_inflate/Makefile       |    4
 lib/zlib_inflate/infblock.c     |  365 -------------
 lib/zlib_inflate/infblock.h     |   48 -
 lib/zlib_inflate/infcodes.c     |  202 -------
 lib/zlib_inflate/infcodes.h     |   33 -
 lib/zlib_inflate/inffast.c      |  462 ++++++++++------
 lib/zlib_inflate/inffast.h      |   12
 lib/zlib_inflate/inffixed.h     |   94 +++
 lib/zlib_inflate/inflate.c      | 1114 +++++++++++++++++++++++++++++++---------
 lib/zlib_inflate/inflate.h      |  107 +++
 lib/zlib_inflate/inflate_syms.c |    3
 lib/zlib_inflate/inflate_sync.c |  152 -----
 lib/zlib_inflate/inftrees.c     |  677 ++++++++++--------------
 lib/zlib_inflate/inftrees.h     |   99 +--
 lib/zlib_inflate/infutil.c      |   88 ---
 lib/zlib_inflate/infutil.h      |  178 ------
 21 files changed, 1886 insertions(+), 2013 deletions(-)

The patch is ~160kb so I'll link to it:

http://www.o-hand.com/~richard/zlib_inflate-r3.patch

Re: [PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Jörn Engel]

On Tue, 4 April 2006 16:18:08 +0100, Richard Purdie wrote:
> 
> Upgrade the zlib_inflate implementation in the kernel from a patched
> version 1.1.3 to a patched 1.2.3. 

s/1.1.3/1.1.4/

I once pulled all the bugfixes between the versions into the kernel.

> The code in the kernel is about seven years old and I noticed that the
> external zlib library's inflate performance was significantly faster
> (~50%) than the code in the kernel on ARM (and faster again on x86_32).
> 
> For comparison the newer deflate code is 20% slower on ARM and 50%
> slower on x86_32 but gives an approx 1% compression ratio improvement. I
> don't consider this to be an improvement for kernel use so have no plans
> to change the zlib_deflate code.
> 
> Various changes have been made to the zlib code in the kernel, the most
> significant being the extra functions/flush option used by ppp_deflate.
> This update reimplements the features PPP needs to ensure it continues
> to work.
> 
> This code has been tested on ARM under both JFFS2 (with zlib compression
> enabled) and ppp_deflate and on x86_32. JFFS2 sees an approx. 10% real
> world file read speed improvement.
> 
> This patch also removes ZLIB_VERSION as it no longer has a correct
> value. We don't need version checks anyway as the kernel's module
> handling will take care of that for us. This removal is also more in
> keeping with the zlib author's wishes
> (http://www.zlib.net/zlib_faq.html#faq24) and I've added something to
> the zlib.h header to note its a modified version.

Sounds good.

> +#if 0
> +int zlib_inflatePrime(z_streamp strm, int bits, int value)

Was this code dead in 1.2.3 as well?

> +#ifndef PKZIP_BUG_WORKAROUND

For the kernel, we can remove compat code against DOS compilers, etc.
In this particular case, I believe we can just consider data
compressed with PKZIP to be illegal and throw an error.

Overall the patch doesn't contain any obvious problems and seems to do
what you described.  If testing agrees as well, I'm all for merging
it.

Jörn

-- 
It is better to die of hunger having lived without grief and fear,
than to live with a troubled spirit amid abundance.
-- Epictetus

Re: [PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Richard Purdie]

On Tue, 2006-04-04 at 17:33 +0200, Jörn Engel wrote:
> On Tue, 4 April 2006 16:18:08 +0100, Richard Purdie wrote: 
> > Upgrade the zlib_inflate implementation in the kernel from a patched
> > version 1.1.3 to a patched 1.2.3. 
> 
> s/1.1.3/1.1.4/
>
> I once pulled all the bugfixes between the versions into the kernel.

That's not what the header says :)

> > +#if 0
> > +int zlib_inflatePrime(z_streamp strm, int bits, int value)
> 
> Was this code dead in 1.2.3 as well?

This code was commented out in the kernel by Adrian Bunk as it has no
users and I just followed that example when updating  I'm sure if
someone needed it, it could be enabled.

> > +#ifndef PKZIP_BUG_WORKAROUND
> 
> For the kernel, we can remove compat code against DOS compilers, etc.
> In this particular case, I believe we can just consider data
> compressed with PKZIP to be illegal and throw an error.

Agreed, there are places it can be tidied up a bit further. I was trying
not to make too many changes so any future updates against zlib are
easier.

Richard

Re: [PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Jörn Engel]

On Tue, 4 April 2006 16:56:51 +0100, Richard Purdie wrote:
> On Tue, 2006-04-04 at 17:33 +0200, Jörn Engel wrote:
> > On Tue, 4 April 2006 16:18:08 +0100, Richard Purdie wrote: 
> > > Upgrade the zlib_inflate implementation in the kernel from a patched
> > > version 1.1.3 to a patched 1.2.3. 
> > 
> > s/1.1.3/1.1.4/
> >
> > I once pulled all the bugfixes between the versions into the kernel.
> 
> That's not what the header says :)

Well, the complete patch from 1.1.3 to 1.1.4 was rather big.  Every
single copyright line got updated to a current date, without any other
changes in those files.  I found it rather silly and took the bugfixes
only. ;)

> > > +#if 0
> > > +int zlib_inflatePrime(z_streamp strm, int bits, int value)
> > 
> > Was this code dead in 1.2.3 as well?
> 
> This code was commented out in the kernel by Adrian Bunk as it has no
> users and I just followed that example when updating  I'm sure if
> someone needed it, it could be enabled.

My thinking was rather that we could remove it.  #if 0 has an
advantage in case you made a mistake - easy to revert.  But after a
while...

> > > +#ifndef PKZIP_BUG_WORKAROUND
> > 
> > For the kernel, we can remove compat code against DOS compilers, etc.
> > In this particular case, I believe we can just consider data
> > compressed with PKZIP to be illegal and throw an error.
> 
> Agreed, there are places it can be tidied up a bit further. I was trying
> not to make too many changes so any future updates against zlib are
> easier.

Fair enough.

Jörn

-- 
Data expands to fill the space available for storage.
-- Parkinson's Law

Re: [PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Jan Engelhardt]

>
>Upgrade the zlib_inflate implementation in the kernel from a patched
>version 1.1.3 to a patched 1.2.3. 
>
>The code in the kernel is about seven years old and I noticed that the
>external zlib library's inflate performance was significantly faster
>(~50%) than the code in the kernel on ARM (and faster again on x86_32).
>
Any plans to move to something newer, e.g. bzip2 or LZMA?


Jan Engelhardt
-- 

Re: [PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Jörn Engel]

On Tue, 4 April 2006 17:38:56 +0200, Jan Engelhardt wrote:
> >
> >Upgrade the zlib_inflate implementation in the kernel from a patched
> >version 1.1.3 to a patched 1.2.3. 
> >
> >The code in the kernel is about seven years old and I noticed that the
> >external zlib library's inflate performance was significantly faster
> >(~50%) than the code in the kernel on ARM (and faster again on x86_32).
> >
> Any plans to move to something newer, e.g. bzip2 or LZMA?

For PPP?  Write a new RFC suggesting it and explaining how to deal
with the "chunkiness" and additional memory overhead.  For JFFS2?
Bzlib compresses worse than zlib when dealing with small blocks.
For...

There may be a couple of uses where zlib could be replaced by
something else.  But even ignoring the compatibility issue it isn't
always that simple.

Jörn

-- 
With a PC, I always felt limited by the software available. On Unix, 
I am limited only by my knowledge.
-- Peter J. Schoenster

Re: [PATCH RFC/testing] Upgrade the zlib_inflate library code to a recent version

[Jörn Engel]

On Tue, 4 April 2006 16:18:08 +0100, Richard Purdie wrote:
> 
> Signed-off-by: Richard Purdie <rpurdie@rpsys.net>

Acked-by: Joern Engel <joern@wh.fh-wedel.de>

Jörn

-- 
There is no worse hell than that provided by the regrets
for wasted opportunities.
-- Andre-Louis Moreau in Scarabouche