mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Nikolas Joshua Britton <nbritton@exabit.io>
To: "Mario Limonciello (AMD)" <superm1@kernel.org>
Cc: mario.limonciello@amd.com,
	"Florent DELAHAYE" <kernelorg@undead.fr>,
	Michel-Ekimia <contact@ekimia.fr>,
	"Konrad J Hambrick" <kjhambrick@gmail.com>,
	"Werner Sembach" <wse@tuxedocomputers.com>,
	"Benoit Grégoire" <benoitg@coeus.ca>,
	mumblingdrunkard <mumblingdrunkard@protonmail.com>,
	"Matt Hansen" <2lprbe78@duck.com>,
	"Tilman Vaeth" <tilman-vaeth@web.de>,
	"Christian Homberg" <Christianhomberg@gmail.com>,
	mccricardo <mcc.ricardo@gmail.com>,
	bnewell <bnewell@fullaware.com>,
	"Bernhard Übelacker" <bernhardu@mailbox.org>,
	"Ton Gur" <kvantorrr@mail.ru>, Alessandro <sandro.it@gmail.com>,
	"Pedro Rodrigues" <pedro.pfxr@gmail.com>,
	"Amaury TC" <amaurytc@gmail.com>, crysman <crysman@seznam.cz>,
	Moataz <Moataz.zakaria@gmail.com>, wilk <j.cubizolles@free.fr>,
	"Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>,
	"Bradley Morgan" <brads@mainlining.org>,
	"Doug Calobrisi" <doug@calobrisi.com>,
	"Helge Deller" <deller@gmx.de>,
	"Rodrigo Vivi" <rodrigo.vivi@intel.com>,
	"Alexey Fomenko" <alexey.fomenko@intel.com>,
	"Eric Auger" <eric.auger@redhat.com>,
	"Maciej Grochowski" <Maciej.Grochowski@sony.com>,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-efi@vger.kernel.org
Subject: Re: [RFT] Change E820 memory map handling
Date: Tue, 17 Nov 2026 14:12:29 +0000	[thread overview]
Message-ID: <20261117141229.11971-1-nbritton@exabit.io> (raw)
In-Reply-To: <20261110-superm1-genoa-mess-v1-0-bdcb3a6f2463@kernel.org>

On Tue, 10 Nov 2026, Mario Limonciello (AMD) wrote:
> Here is the flow I would like to see:
>  1. Test 7.4-rc2 (vanilla - no changes please)
>  2. Test this branch (superm1/genoa-mess)
>  3. If this branch fails, please revert this commit from the branch and test again.
>     commit 35c9c60ea8c1c ("Revert "x86/PCI: Add E820 quirks for Clevo NL4XLU and Lenovo Yoga C940"")
>
> In all cases please check whether your original issue exists and comment if it does.

I tested on the Mac Pro 7,1 from my bridge window report [1]: two Radeon
Pro Vega II Duo cards, four Vega 20 dies behind two root ports, BAR0
resized to 32G on every die. Both kernels were built from the same
.config.

1. v7.4-rc2: the original issue exists. Both root port windows stay at
   32770M, the second die's branch (08:10.0 and 16:10.0) gets no
   prefetchable window, and 0e:00.0 and 1c:00.0 are left without BARs:

  pci 0000:0e:00.0: BAR 0 [mem size 0x800000000 64bit pref]: can't assign; no space
  pci 0000:0e:00.0: BAR 2 [mem size 0x00200000 64bit pref]: can't assign; no space

2. superm1/genoa-mess (35c9c60ea8c1): the original issue is gone. Every
   BAR is assigned, both root port windows are 96G, and all four dies
   get their 32G BAR0. The layout is identical to the one I reported
   for v5 on v7.2.15 [2]. The first root port:

  pci 0000:06:00.0: bridge window [mem 0x9e800000000-0x9ffffffffff 64bit pref]: assigned
  pci 0000:07:00.0: bridge window [mem 0x9e800000000-0x9ffffffffff 64bit pref]: assigned
  pci 0000:08:08.0: bridge window [mem 0x9f7ffe00000-0x9ffffffffff 64bit pref]: assigned
  pci 0000:08:10.0: bridge window [mem 0x9e800000000-0x9f0001fffff 64bit pref]: assigned
  pci 0000:0b:00.0: BAR 0 [mem 0x9f800000000-0x9ffffffffff 64bit pref]: assigned
  pci 0000:0b:00.0: BAR 2 [mem 0x9f7ffe00000-0x9f7ffffffff 64bit pref]: assigned
  pci 0000:0e:00.0: BAR 0 [mem 0x9e800000000-0x9efffffffff 64bit pref]: assigned
  pci 0000:0e:00.0: BAR 2 [mem 0x9f000000000-0x9f0001fffff 64bit pref]: assigned

   amdgpu binds all four dies, and compute and peer-to-peer transfers
   across them work.

3. Not needed, since the branch works.

As for E820 itself, 07eab0901ede removes a single descriptor on this
machine:

  efi: Remove mem126: MMIO range=[0x60000000-0x600fffff] (1MB) from e820 map

That is the first 1MB of ECAM (0x60000000-0x6fffffff), below all host
bridge windows. On the branch the line is gone and /proc/iomem keeps
5e000000-600fffff as Reserved. The firmware is dated 2026, so both
kernels print "PCI: Ignoring E820 reservations for host bridge windows"
and the restored entry could not clip a window here anyway. Apart from
the efi/e820 lines and what follows from the BAR assignment (amdgpu
binding two more dies), the kernel logs of the two boots do not differ.

Also, the boot failure on v7.3 and later that I mentioned in the v5
thread does not reproduce with v7.4-rc2, so both kernels here are
plain upstream. It does not reproduce with Ubuntu 26.10's 7.3.0-10
kernel either, and I have since tested v5 on that kernel as well.
Unpatched, it shows the original issue (two of the four dies without
BARs). With the 14 patches applied (they apply without fuzz and need no
prerequisites on 7.3), all four dies get their BARs with the same
layout as above, and a multi-hour inference benchmark across the four
dies showed no regression against our reference runs.

Tested-by: Nikolas Joshua Britton <nbritton@exabit.io>

[1] https://lore.kernel.org/linux-pci/20260903063124.9316-1-nbritton@exabit.io/
[2] https://lore.kernel.org/linux-pci/20261106152642.23118-1-nbritton@exabit.io/

  reply	other threads:[~2026-11-17 14:12 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 18:50 Mario Limonciello (AMD)
2026-11-17 14:12 ` Nikolas Joshua Britton [this message]
2026-11-17 18:44 ` Werner Sembach

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261117141229.11971-1-nbritton@exabit.io \
    --to=nbritton@exabit.io \
    --cc=2lprbe78@duck.com \
    --cc=Christianhomberg@gmail.com \
    --cc=Maciej.Grochowski@sony.com \
    --cc=Moataz.zakaria@gmail.com \
    --cc=alexey.fomenko@intel.com \
    --cc=amaurytc@gmail.com \
    --cc=benoitg@coeus.ca \
    --cc=bernhardu@mailbox.org \
    --cc=bnewell@fullaware.com \
    --cc=brads@mainlining.org \
    --cc=contact@ekimia.fr \
    --cc=crysman@seznam.cz \
    --cc=deller@gmx.de \
    --cc=doug@calobrisi.com \
    --cc=eric.auger@redhat.com \
    --cc=ilpo.jarvinen@linux.intel.com \
    --cc=j.cubizolles@free.fr \
    --cc=kernelorg@undead.fr \
    --cc=kjhambrick@gmail.com \
    --cc=kvantorrr@mail.ru \
    --cc=linux-efi@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=mario.limonciello@amd.com \
    --cc=mcc.ricardo@gmail.com \
    --cc=mumblingdrunkard@protonmail.com \
    --cc=pedro.pfxr@gmail.com \
    --cc=rodrigo.vivi@intel.com \
    --cc=sandro.it@gmail.com \
    --cc=superm1@kernel.org \
    --cc=tilman-vaeth@web.de \
    --cc=wse@tuxedocomputers.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®