Re: PCI-DMA: high address but no IOMMU - nForce4

[Matti Aarnio <matti.aarnio@zmailer.org>]

On Wed, Nov 02, 2005 at 04:38:20PM +0100, Andreas Kleen wrote:
> Date:	Wed, 2 Nov 2005 16:38:20 +0100 (CET)
> From:	Andreas Kleen <ak@suse.de>
> To:	Marc Perkel <marc@perkel.com>
> Subject: Re: PCI-DMA: high address but no IOMMU - nForce4
> 
> [full quote]

[severe clipping]
 
> Am Di 01.11.2005 06:21 schrieb Marc Perkel <marc@perkel.com>:
> 
> > > On Fri, Oct 28, 2005 at 12:16:51AM +0200, Andi Kleen wrote:
> > >
> > > I will attach my own; A brand new Amd64 dual-core thing.
> > > Works fine with mem=2500M, but blows up with mem=3G or
> > > without any override and full 4G complement in use.
> > >
> > > This board (ASUS A8N-SLI) does use NVIDIA nForce4 chipset with
> > > bios-option to map (hoist) "excess memory" out from first 4G to
> > > higher physical addresses so that it can be accessed by the
> > > processor.
> > >
> > > This board has no AGP at all in it, but it does have lots
> > > of PCIE, and a bit of PCI-X thrown in for "legacy cards".
> > > Somehow that detail breaks things when the machine really
> > > should use bounce-buffering, or something similar -- I don't
> > > know if Nvidia nForce4 chipset does have IOMMU, though...
> >
> > For what it's worth I have almost the exact same hardware and got the
> > same
> > error. Athlon X2 4400 with the same ASUS board. Reverting to 2.6.13.2
> > kernel works.
>  
> This sounds like the PCI-X BIOS misconfiguration issue that Andy C.
> recently tracked down. Andy do you agree?
>  
> If yes it's a BIOS problem, but we can probably work around it with a
> quirk.
> 
> -Andi

Well..  sort of.   Thinking goes probably like this:
  "The GART is AGP thingie, it shall be initialized minimally
   or not at all in a system that has no AGP."

which naturally backfires, when indeed all IO bus things will
use GART, not only AGP..


I am using Fedora Core development kernels for x86_64, and am able
to get system to work (mostly) with all 4 GB of memory by using
following boot parameters:

    iommu=soft swiotlb=65536

with all of 2.6.14 series kernel versions for past week (or two).
I did try some other options as well, like:

    iommu=noagp,force,memaper=3

but that didn't succeed in booting my box all the way, and always
in the end crashing with the message in $SUBJECT, above.

Plain 2.4.14 works completely with   mem=2600M  boot-option, but
it does still tell:

    Checking aperture...
    CPU 0: aperture @ 0 size 32 MB
    No AGP bridge found

and somewhere (AMD documents?) I spotted a mention that having
aperture overlaying memory is "a bad idea", thus when memory
starts at zero, and aperture starts there... 

A quirk, very least.



There is separate issue with recent change in  mm/memory.c
contained   get_user_pages()  function (?) that fail my
BTTV card access  --  and I might be wrong again,  I did
suspect I2C at first, but strace shows fault to happen with
VIDIOC_QBUF  ioctl...

   /Matti Aarnio