Re: OT: character encodings (was: Linux 2.6.20-rc4)

[Adrian Bunk <bunk@stusta.de>]

On Sun, Jan 07, 2007 at 09:48:34PM +0100, Willy Tarreau wrote:
> On Sun, Jan 07, 2007 at 08:11:38PM +0100, Jan Engelhardt wrote:
> > 
> > On Jan 7 2007 17:06, Russell King wrote:
> > >On Mon, Jan 08, 2007 at 12:29:05AM +0800, David Woodhouse wrote:
> > >
> > >$ git log | head -n 1000 | tail -n 200 > o
> > >$ file -i o
> > >o: text/plain; charset=us-ascii
> > >$ git log | head -n 1000 | tail -n 300 > o
> > >$ file -i o
> > >o: text/plain; charset=us-ascii
> > >$ git log | head -n 1000 | tail -n 400 > o
> > >$ file -i o
> > >o: text/plain; charset=utf-8
> > 
> > I am inclined to say that "file" does not count, because it tries to guess an
> > ambiguous mapping from bytes to character set. Even more, file should be
> > _unable at all_ to distinguish an iso-8859-1 from an iso-8859-2 (or worse: 15)
> > file. This program is soo... forget it, it's not an argument. It works well for
> > headerful files, but text files don't really contain one. The next best thing
> > would be html, with a proper <meta http-equiv=Content> tag.
> 
> The stupidity from the start up with those character sets is that they
> consider that a whole file is written with a given set. In fact, the
> charset should apply to characters themselves. At least, the
> quoted-printable, non-human friendly, encoding was the least stupid.

I doubt doing this would really be worth the effort.

In the 21st century, people should simply use UTF-8.

> Now that UTF8 comes everywhere, everyone receives tons of mangled mails,
> and even mailers which correctly support UTF8 and use it by default manage
> to shoot themselves in the foot when they reply to, or forward a mail. The
> system is completely broken because limited by design, and we have to learn
> to live with this brokenness.

Only if MUAs have broken charset support or don't set a correct 
"charset" header in the mails they are sending.

If some software still can't handle UTF-8 correctly more than 10 years 
after it was introduced, that's not a brokenness you can blame on UTF-8.

> Willy

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed