mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Shuai Xue <xueshuai@linux.alibaba.com>
To: lukas@wunner.de, linux-pci@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org
Cc: bhelgaas@google.com, tony.luck@intel.com, bp@alien8.de,
	xueshuai@linux.alibaba.com
Subject: [PATCH v2 0/2] PCI: hotplug: Add a generic RAS tracepoint for hotplug event
Date: Tue, 12 Nov 2024 19:58:50 +0800	[thread overview]
Message-ID: <20241112115852.52980-1-xueshuai@linux.alibaba.com> (raw)

Hotplug events are critical indicators for analyzing hardware health,
particularly in AI supercomputers where surprise link downs can
significantly impact system performance and reliability. The failure
characterization analysis illustrates the significance of failures
caused by the Infiniband link errors. Meta observes that 2% in a machine
learning cluster and 6% in a vision application cluster of Infiniband
failures co-occur with GPU failures, such as falling off the bus, which
may indicate a correlation with PCIe.[1]

PATCH 1/2: Add a generic RAS tracepoint for hotplug event to help healthy check. 
PATCH 2/2: Generate tracepoints for PCIe hotplug event

The output like below:

$ echo 1 > /sys/kernel/debug/tracing/events/hotplug/pci_hp_event/enable
$ cat /sys/kernel/debug/tracing/trace_pipe
           <...>-206     [001] .....    40.373870: pci_hp_event: 0000:00:02.0 slot:10, trans_state:Link Down

           <...>-206     [001] .....    40.374871: pci_hp_event: 0000:00:02.0 slot:10, trans_state:Card not present

[1]https://arxiv.org/abs/2410.21680


Shuai Xue (2):
  PCI: hotplug: Add a generic RAS tracepoint for hotplug event
  pci: pciehp: Generate tracepoints for hotplug event

 drivers/pci/hotplug/pciehp_ctrl.c | 33 ++++++++++++---
 drivers/pci/hotplug/trace.h       | 68 +++++++++++++++++++++++++++++++
 include/uapi/linux/pci.h          |  7 ++++
 3 files changed, 102 insertions(+), 6 deletions(-)
 create mode 100644 drivers/pci/hotplug/trace.h

-- 
2.39.3


             reply	other threads:[~2024-11-12 11:59 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-12 11:58 Shuai Xue [this message]
2024-11-12 11:58 ` [PATCH v2 1/2] " Shuai Xue
2024-11-13 16:27   ` Lukas Wunner
2024-11-14  1:39     ` Shuai Xue
2024-11-12 11:58 ` [PATCH v2 2/2] pci: pciehp: Generate tracepoints " Shuai Xue

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241112115852.52980-1-xueshuai@linux.alibaba.com \
    --to=xueshuai@linux.alibaba.com \
    --cc=bhelgaas@google.com \
    --cc=bp@alien8.de \
    --cc=linux-edac@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lukas@wunner.de \
    --cc=tony.luck@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®