Re: 2.6.25 DMA: Out of SW-IOMMU space - Asus M2N32 AMD 8GB memory

Re: 2.6.25 DMA: Out of SW-IOMMU space - Asus M2N32 AMD 8GB memory

[Gerardo Exequiel Pozzi]

(sorry, my english, again, very BAD!)

(not in mailing-list, please CC to me)


Jari Aalto <jari.aalto <at> cante.net> writes:

>> When booted kernel with no iommu parameter, it uses the iommu gart
>> (from amd) and the size of the aperture is 64MB You can try to grow
>> this size with iommu=memaper=2 for 128MB or 3 for 256MB. I don't have
>> experience with this, but i read that this, solves the problem.
>
>See my original bug report, which included experiences with
>memaper. Unofrtunately they didn't help:
>
>    REF: http://article.gmane.org/gmane.linux.kernel/725293 (thread >start)
>
>but I'll try that once more with this 2.6.26 kernel to double check.

No your options are bad,

You have mixed options for "swiotlb" and "amd gart", see the
Documentation/x86_64/boot-options.txt

iommu=soft,memaper=3 swiotlb=65536 BAD.

memaper=3 don't have any effects here, memaper is only for "AMD GART"

The noagp option in this motherboard is redundant.

iommu=noagp,noaperture,off swiotlb=512M BAD combination options!
OFF, OFF, OFF, ON: The result are unknown.

You need to use the IOMMU (from AMD GART) that with your options: you
never used.

When you boot with no iommu options, kernels uses the AMD GART in this
system, with an aperture of 64MB (too small) to fix, you need only one
option: iommu=memaper=3

¡Buena suerte!



>
>> I have the similar hardware, but when boot in 64 bits, limit the
>> memory with mem=4G (discarded 0.5G that are (remaped from pci space)
>> beyond 4G) and iommu=noaperture. (This is not option for you, will be
>> discard 4G of RAM)
>
>Interestingly 'noaperture' together with 'noagp' has been the best
>option so far. It gives workign system for about week, but eventually
>the "out of IOMMU space" messages start appear.

>The system has always seen the full 8 GB memory without problems.


-- 
Gerardo Exequiel Pozzi ( djgera )
http://www.djgera.com.ar
KeyID: 0x1B8C330D
Key fingerprint = 0CAA D5D4 CD85 4434 A219  76ED 39AB 221B 1B8C 330D

Re: 2.6.25 DMA: Out of SW-IOMMU space - Asus M2N32 AMD 8GB memory

[Jari Aalto]

Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> writes:

>>    REF: http://article.gmane.org/gmane.linux.kernel/725293 (thread >start)
>>
>>but I'll try that once more with this 2.6.26 kernel to double check.
>
> You have mixed options for "swiotlb" and "amd gart", see the
> Documentation/x86_64/boot-options.txt

It's difficult to find the pointer that could explain the correct use of
the options and their combinations.

> iommu=soft,memaper=3 swiotlb=65536 BAD.
>
> memaper=3 don't have any effects here, memaper is only for "AMD GART"
>
> The noagp option in this motherboard is redundant.
>
> iommu=noagp,noaperture,off swiotlb=512M BAD combination options!
> OFF, OFF, OFF, ON: The result are unknown.
>
> You need to use the IOMMU (from AMD GART) that with your options: you
> never used.
>
> When you boot with no iommu options, kernels uses the AMD GART in this
> system, with an aperture of 64MB (too small) to fix, you need only one
> option: iommu=memaper=3

This sort of advice wouldbe wonderful if it wereincluded in the spec,
which as of now is too technical for mere mortals:

    http://www.mjmwired.net/kernel/Documentation/x86_64/boot-options.txt

I'm not surprised that end user cannot know what combination of options
make or does not make sense. I'll try at next boot with plain:

    iommu=memaper=3

Thank you,
Jari

Re: 2.6.25 DMA: Out of SW-IOMMU space - Asus M2N32 AMD 8GB memory (SOLVED)

[Jari Aalto]

Jari Aalto <jari.aalto@cante.net> writes:

> Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> writes:
>>>
>>>    REF: http://article.gmane.org/gmane.linux.kernel/725293 (thread >start)
>>
>> You have mixed options for "swiotlb" and "amd gart", see the
>> Documentation/x86_64/boot-options.txt
>
> I'll try at next boot with plain:
>
>     iommu=memaper=3

[This is a summary post in case aonyone else needs the information]

It appears that with AMD BE-2400 and ASUS MB[1] plus Debian kernel
2.6.26, the following boot options solve "out of SW-IOMMU" issue:

   iommu=noagp,memaper=3

If the 'iommu' option is left out to kernel to figure out GART defaults,
then the system freezed in short period of time.

Thank you for all the participants, especially to Gerardo,
Jari

[1]
 CPU
        $ cat /proc/cpuinfo
        model name      : AMD Athlon(tm) X2 Dual Core Processor BE-2400
        stepping        : 2
        cpu MHz         : 2310.518
        cache size      : 512 KB
 MB

        Asus M2N32-SLI Deluxe/Wireless Edition
        - nvidia nForce 590 SLI chipset MCP
        - 2 x PCIe (SLI x16), 1 x PCI (x4), 1 x PCI (x1), 2 x PCI 2.2
        - Socket AM2

        http://www.asus.com/products.aspx?l1=3&l2=101&l3=300&model=1163&modelmenu=1

Debian Compiled Kernel

        $ grep -Ei 'iommu|agp' /boot/config-2.6.26-1-amd64
        CONFIG_GART_IOMMU=y
        CONFIG_CALGARY_IOMMU=y
        CONFIG_CALGARY_IOMMU_ENABLED_BY_DEFAULT=y
        CONFIG_IOMMU_HELPER=y
        CONFIG_AGP=y
        CONFIG_AGP_AMD64=y
        CONFIG_AGP_INTEL=m
        CONFIG_AGP_SIS=m
        CONFIG_AGP_VIA=m
        # CONFIG_IOMMU_DEBUG is not set