The kernel's ctype

The kernel's ctype

[Rasmus Villemoes]

Hi,

The kernel's ctype is almost, but not quite, equivalent to latin1. Apart
from whether one wants to include the C1 control chars (0x80-0x9f),
there are a few other differences. For example, 0xb5 (MICRO SIGN) is, at
least according to glibc, both alpha and lower, while the kernel
classifies it as punct. A slightly surprising quirk of the kernel's
ctype implementation is that toupper() is not idempotent: Both 0xdf
(LATIN SMALL LETTER SHARP S) and 0xff (LATIN SMALL LETTER Y WITH
DIAERESIS) are correctly classified as lower, but since neither
character's uppercase version is representable in latin1, correct
toupper() behaviour would be to return the character itself. Instead, we
have toupper(0xff) == 0xdf and toupper(0xdf) == 0xbf.

Digging in pre-git history, I see that ctype.c was originally ASCII-only,
which I think is the only sane choice. It was changed around 1996, but
the commit log that I've found just says "Import 2.0.1", so it's hard to
tell what the intention was.

What would break if ctype.c was changed back to ASCII?

Rasmus

Re: The kernel's ctype

[Frans Klaver]

On Wed, Feb 4, 2015 at 12:18 PM, Rasmus Villemoes
<linux@rasmusvillemoes.dk> wrote:
> Hi,
>
> The kernel's ctype is almost, but not quite, equivalent to latin1. Apart
> from whether one wants to include the C1 control chars (0x80-0x9f),
> there are a few other differences. For example, 0xb5 (MICRO SIGN) is, at
> least according to glibc, both alpha and lower, while the kernel
> classifies it as punct. A slightly surprising quirk of the kernel's
> ctype implementation is that toupper() is not idempotent: Both 0xdf
> (LATIN SMALL LETTER SHARP S) and 0xff (LATIN SMALL LETTER Y WITH
> DIAERESIS) are correctly classified as lower, but since neither
> character's uppercase version is representable in latin1, correct
> toupper() behaviour would be to return the character itself. Instead, we
> have toupper(0xff) == 0xdf and toupper(0xdf) == 0xbf
>
> Digging in pre-git history, I see that ctype.c was originally ASCII-only,
> which I think is the only sane choice. It was changed around 1996, but
> the commit log that I've found just says "Import 2.0.1", so it's hard to
> tell what the intention was.
>
> What would break if ctype.c was changed back to ASCII?

The implementation of toupper() and tolower() still seems to be
assuming that we're dealing with ascii only, so in that regard I don't
think that much would break, as it should be broken already.

Shouldn't coccinelle be able to detect ctype usage and use of
non-ascii or userland input values?

Frans