Re: Not MTRR !? was: ISSUE: very slow (factor 100) 4-way 16GByte server, with 2.4.2

Re: Not MTRR !? was: ISSUE: very slow (factor 100) 4-way 16GByte server, with 2.4.2

[David Wragg]

Rik van Riel <riel@conectiva.com.br> writes:
> On Wed, 28 Mar 2001, Robert Suetterlin wrote:
> > reg00: base=0xfb000000 (4016MB), size=  16MB: uncachable, count=1
> > reg01: base=0xfc000000 (4032MB), size=  64MB: uncachable, count=1
> > reg02: base=0x00000000 (   0MB), size=8192MB: write-back, count=1
> > reg03: base=0x200000000 (8192MB), size=4096MB: write-back, count=1
> > reg04: base=0x300000000 (12288MB), size=2048MB: write-back, count=1
> > reg05: base=0x380000000 (14336MB), size=1024MB: write-back, count=1
> > reg06: base=0x3c0000000 (15360MB), size= 512MB: write-back, count=1
> > reg07: base=0x3e0000000 (15872MB), size= 256MB: write-back, count=1
>                                            ------  +
>                                           15.75 GB
> 
> It looks like the last 256MB isn't cached (well, it doesn't
> have an MTRR at all) and Linux starts loading programs from
> the top of memory ...

It looks like the BIOS ran out of MTRRs.  I suspect this is one of the
reasons that Intel changed the PPro spec to allow overlapping MTRRs in
some cases, with uncached taking precedence.  The following sequence
of /proc/mtrr commands should give the same uncachable range with all
memory write-back cached:

# cat >/proc/mtrr
disable=2
disable=3
disable=4
disable=5
disable=6
disable=7
base=0 size=0x400000000 type=write-back
base=0x400000000 size=0x4000000 type=write-back
base=0x404000000 size=0x1000000 type=write-back
^D

(I think all those zeros are correct. 0x400000000 == 16GB, 0x4000000
== 64MB, 0x1000000 == 16MB)

It's probably also worth seeing if a BIOS update is available.


Dave Wragg

Solved with MTRR was: ISSUE: very slow (factor 100) 4-way 16GByte server, with 2.4.2

[Robert Suetterlin]

Thanks everyone,

	especially David, for explaining the MTRR problem to me in detail.

I could get the machine to work nicely by mapping all memory in MTRRs.

There are only two corrections I would like to make to Davids eMail:
> # cat >/proc/mtrr
> disable=2
> disable=3
> disable=4
> disable=5
> disable=6
> disable=7
> base=0 size=0x400000000 type=write-back
> base=0x400000000 size=0x4000000 type=write-back
> base=0x404000000 size=0x1000000 type=write-back
> ^D

1. it is cat >| /proc/mtrr
2. I was not allowed to do `base=0 size=0x400000000 type=write-back`, because of the overlap with the memory range at base=0x0fb000000.  So what I do is only disable 3-7, and then base=0x400000000 size=0x400000000.


Thanks alot,

	Robert S.

Re: Solved with MTRR was: ISSUE: very slow (factor 100) 4-way 16GByte server, with 2.4.2

[David Wragg]

Robert Suetterlin <sutter@robert2.mpe-garching.mpg.de> writes:
> 2. I was not allowed to do `base=0 size=0x400000000
> type=write-back`, because of the overlap with the memory range at
> base=0x0fb000000. 

/proc/mtrr does allow overlapping regions in some cases, but the
conditions turned out to be stricter than I remembered.  You have to
create the enclosing range first, which makes the facility useless in
this case (perhaps in all potentially useful cases).

> So what I do is only disable 3-7, and then
> base=0x400000000 size=0x400000000.

Yes, that solution should be safe.