From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41DAB2FD7B4 for ; Tue, 2 Dec 2025 06:01:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764655290; cv=none; b=MQbMsruuLwAXPoXfwoW99isawDAjWFw5fRzUb1V7GRGlml2bJ/qcUbwOHe+NKg12TKjd8s1dIPing2/BN7Ptlpdmm7ypeZECoAxokgFegamZZIuMAwZko13dxP89Kd8EvAgTjFBI9mwFrDJNJVcfJ6P8gZTJj1iB7XmRdWDxzO8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764655290; c=relaxed/simple; bh=f4LGkA6Yq8MzP5nlL4hiuLTNAyEORZwEq1TX6k1jSk8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YrrgET9pemoF1JxlNTtfWxI3lVydjT3C8WQPelE0i5sPzoNNt1kYTbqss6i1eY7PUoSyrP6xnbNpMRoyMLbG+j7QF480Uglxqy6P4NCS8nnZ9AcRPeaA+OmmlqQleOM6xtq8auU7TDP8hhlOGA3Y7XWDlwMdG6ymtGgr62ILxaM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=dK2lXqpP; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="dK2lXqpP" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-34585428e33so5246684a91.3 for ; Mon, 01 Dec 2025 22:01:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1764655287; x=1765260087; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=VjMzCp0YdhbklZdy04cAQHKZQn0920G7A9CAjRSvCOU=; b=dK2lXqpPHSrgdiAxEgOWpgVyBo1O3K468ught3vluZIXmEDyZBf+Z2cSmal3G9SOAj Zx2fHhKGHsisTKKRfSOwwXbngBonE5l2bWiPva3vwFBpzHS1EPRM0t6bfl3OjdD4tT/J UdLOb9wkxzLBJcDXl/Bhg1znU166jRB/KR7zPfIz6vCFpiGwMW4spj+r0nF01tnkpXk3 vPB7GkVae/unv8vWUIuenpQV2SrPeESiCJ+Z8d+llnBedgCRr8kEflLYkEYskIkv4358 3/VXP9ArCQx+ZBaWgUZ2E+ppul6YjmEjwynlqH47LMoXGxCEs3GPw3pGS4VEgv5ZE2eK 3Z3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764655287; x=1765260087; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=VjMzCp0YdhbklZdy04cAQHKZQn0920G7A9CAjRSvCOU=; b=bIeu7hyWkzC7xauym7/Evtz7z3/B8ZHQChhfiPd4qbXXKhVrPN15ZqfI+f4/B7dvb7 S+5OyUMVvOkfs29u971MZ0baYJ6+BhrfeLP7cd+0IUK65oI2OF0tknt5xBJal4Qhgxqj gAoE/HlEpD7IeDOkhhepVIPOqNZwTDxEGUDFHvNuum9HM157WN7L6aQsa0LzBr6iMJkp SJG8B/04B3/e5m9o4CS5PV12B03EHVJ/nt5IgnYP66uGF5WhuZN35EuwbL4ASmEnRTO1 ltZxnmbce+6yuHfp1ckSVzsM7N2rphN1J0zXEIETSM/EtZLGL7XlWCZSfCGQy5C1ZNRA QjxQ== X-Forwarded-Encrypted: i=1; AJvYcCWpu8WPbO8ViQ2bR4K3QCPVXgyoZJQQQdNcLe3P8BRs5gEQ8I8y+v04qrnW+H3QRCt4pFhcJCgDQdVtkeo=@vger.kernel.org X-Gm-Message-State: AOJu0YwBwzoGM/KU4NU87bRhVbINCQFJRw4Ycky/xP2JU/KGf67K30L1 mdjR8n9YIMrcNDiaiBk24gZN2gGXzyqUmvWl6V5Che53RtcBkEU04x7niKogQxEEoV4= X-Gm-Gg: ASbGnctanW46QYH8vKKiUImp+7NK7UG507x2TbR9Vo40qofKR5HiAGR1VeoonLJrOz7 u+2VhaQr1mtNWON0lPQUXapN6WB2IxzX7PDJcedrHdVfBfIw/TAOxp0oP9WjsGXNN24wezNkTYZ c6HJvJ/sNeUd/BxnyfTg1ruZkvSDtloArWxjN+WDfg+o/r7ep2rSIGmDc0y8RhN8z51aa8VVQIa dDd/OzhJRsR9lwiowfje9d6Yj7830uyyGrRHxUCJwg23l5RhS9HBIt6MIxKBvGUAAmpNTAeRad/ DwCuikPCurjMMg3bWx5iep7RC6MXZKyIPOccp4odBkGz5oaV9Ln/aYxLcjJnq2C96R27cyQi9Qw kFyeLbxvxfYHB0EDOlG9Upr++tKLK7TkMyw+H8KhDQtrvff7XFro9Sn1Ky0YVaAcGMnoF4VCu+t gPqPKxFpXc X-Google-Smtp-Source: AGHT+IEUKPl00bYyajTuOTnh93gNZhE3b7XRi8C2K56+OOSTWlUpe0tn8sPwti3RRAIYAehK3weEQA== X-Received: by 2002:a17:90b:3b86:b0:340:e521:bc73 with SMTP id 98e67ed59e1d1-34733e2d536mr38766998a91.5.1764655287162; Mon, 01 Dec 2025 22:01:27 -0800 (PST) Received: from [127.0.1.1] ([210.176.154.34]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3476a547483sm19083810a91.4.2025.12.01.22.01.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Dec 2025 22:01:26 -0800 (PST) From: Nick Hu Date: Tue, 02 Dec 2025 14:07:41 +0800 Subject: [PATCH v3 2/2] irqchip/riscv-aplic: Preserve APLIC states across suspend/resume Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20251202-preserve-aplic-imsic-v3-2-1844fbf1fe92@sifive.com> References: <20251202-preserve-aplic-imsic-v3-0-1844fbf1fe92@sifive.com> In-Reply-To: <20251202-preserve-aplic-imsic-v3-0-1844fbf1fe92@sifive.com> To: Anup Patel , Thomas Gleixner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Yong-Xuan Wang , Cyan Yang , Nutty Liu , Nick Hu X-Mailer: b4 0.14.2 The APLIC states might be reset when the platform enters a low power state, but the register states are not being preserved and restored, which prevents interrupt delivery after the platform resumes. Solve this by adding a syscore ops and a power management notifier to preserve and restore the APLIC states on suspend and resume. Reviewed-by: Yong-Xuan Wang Reviewed-by: Cyan Yang Reviewed-by: Nutty Liu Signed-off-by: Nick Hu --- drivers/irqchip/irq-riscv-aplic-direct.c | 10 ++ drivers/irqchip/irq-riscv-aplic-main.c | 166 ++++++++++++++++++++++++++++++- drivers/irqchip/irq-riscv-aplic-main.h | 19 ++++ 3 files changed, 194 insertions(+), 1 deletion(-) diff --git a/drivers/irqchip/irq-riscv-aplic-direct.c b/drivers/irqchip/irq-riscv-aplic-direct.c index c2a75bf3d20c64c1e1f0e863486633071ab115a7..5a9650225dd805539a2cdf7ca64c7addb4fea39d 100644 --- a/drivers/irqchip/irq-riscv-aplic-direct.c +++ b/drivers/irqchip/irq-riscv-aplic-direct.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -171,6 +172,15 @@ static void aplic_idc_set_delivery(struct aplic_idc *idc, bool en) writel(de, idc->regs + APLIC_IDC_IDELIVERY); } +void aplic_direct_restore_states(struct aplic_priv *priv) +{ + struct aplic_direct *direct = container_of(priv, struct aplic_direct, priv); + int cpu; + + for_each_cpu(cpu, &direct->lmask) + aplic_idc_set_delivery(per_cpu_ptr(&aplic_idcs, cpu), true); +} + static int aplic_direct_dying_cpu(unsigned int cpu) { if (aplic_direct_parent_irq) diff --git a/drivers/irqchip/irq-riscv-aplic-main.c b/drivers/irqchip/irq-riscv-aplic-main.c index 93e7c51f944abe4cb96fc7427c1f36ab7d2cbc4d..b760942e57f916d86b2d424dca8a3c6008f064e6 100644 --- a/drivers/irqchip/irq-riscv-aplic-main.c +++ b/drivers/irqchip/irq-riscv-aplic-main.c @@ -12,10 +12,165 @@ #include #include #include +#include +#include #include +#include #include "irq-riscv-aplic-main.h" +static LIST_HEAD(aplics); + +static void aplic_restore_states(struct aplic_priv *priv) +{ + struct aplic_saved_regs *saved_regs = &priv->saved_hw_regs; + struct aplic_src_ctrl *srcs; + void __iomem *regs; + u32 nr_irqs, i; + + regs = priv->regs; + writel(saved_regs->domaincfg, regs + APLIC_DOMAINCFG); +#ifdef CONFIG_RISCV_M_MODE + writel(saved_regs->msiaddr, regs + APLIC_xMSICFGADDR); + writel(saved_regs->msiaddrh, regs + APLIC_xMSICFGADDRH); +#endif + /* + * The sourcecfg[i] has to be restored prior to the target[i], interrupt-pending and + * interrupt-enable bits. The AIA specification states that "Whenever interrupt source i is + * inactive in an interrupt domain, the corresponding interrupt-pending and interrupt-enable + * bits within the domain are read-only zeros, and register target[i] is also read-only + * zero." + */ + nr_irqs = priv->nr_irqs; + for (i = 0; i < nr_irqs; i++) { + srcs = &priv->saved_hw_regs.srcs[i]; + writel(srcs->sourcecfg, regs + APLIC_SOURCECFG_BASE + i * sizeof(u32)); + writel(srcs->target, regs + APLIC_TARGET_BASE + i * sizeof(u32)); + } + + for (i = 0; i <= nr_irqs; i += 32) { + srcs = &priv->saved_hw_regs.srcs[i]; + writel(-1U, regs + APLIC_CLRIE_BASE + (i / 32) * sizeof(u32)); + writel(srcs->ie, regs + APLIC_SETIE_BASE + (i / 32) * sizeof(u32)); + + /* Re-trigger the interrupts if it forwards interrupts to target harts by MSIs */ + if (!priv->nr_idcs) + writel(readl(regs + APLIC_CLRIP_BASE + (i / 32) * sizeof(u32)), + regs + APLIC_SETIP_BASE + (i / 32) * sizeof(u32)); + } + + if (priv->nr_idcs) + aplic_direct_restore_states(priv); +} + +static void aplic_save_states(struct aplic_priv *priv) +{ + struct aplic_src_ctrl *srcs; + void __iomem *regs; + u32 i, nr_irqs; + + regs = priv->regs; + nr_irqs = priv->nr_irqs; + /* The valid interrupt source IDs range from 1 to N, where N is priv->nr_irqs */ + for (i = 0; i < nr_irqs; i++) { + srcs = &priv->saved_hw_regs.srcs[i]; + srcs->target = readl(regs + APLIC_TARGET_BASE + i * sizeof(u32)); + + if (i % 32) + continue; + + srcs->ie = readl(regs + APLIC_SETIE_BASE + (i / 32) * sizeof(u32)); + } + + /* Save the nr_irqs bit if needed */ + if (!(nr_irqs % 32)) { + srcs = &priv->saved_hw_regs.srcs[nr_irqs]; + srcs->ie = readl(regs + APLIC_SETIE_BASE + (nr_irqs / 32) * sizeof(u32)); + } +} + +static int aplic_syscore_suspend(void) +{ + struct aplic_priv *priv; + + list_for_each_entry(priv, &aplics, head) + aplic_save_states(priv); + + return 0; +} + +static void aplic_syscore_resume(void) +{ + struct aplic_priv *priv; + + list_for_each_entry(priv, &aplics, head) + aplic_restore_states(priv); +} + +static struct syscore_ops aplic_syscore_ops = { + .suspend = aplic_syscore_suspend, + .resume = aplic_syscore_resume, +}; + +static int aplic_pm_notifier(struct notifier_block *nb, unsigned long action, void *data) +{ + struct aplic_priv *priv = container_of(nb, struct aplic_priv, genpd_nb); + + switch (action) { + case GENPD_NOTIFY_PRE_OFF: + aplic_save_states(priv); + break; + case GENPD_NOTIFY_ON: + aplic_restore_states(priv); + break; + default: + break; + } + + return 0; +} + +static void aplic_pm_remove(void *data) +{ + struct aplic_priv *priv = data; + struct device *dev = priv->dev; + + list_del(&priv->head); + if (dev->pm_domain) + dev_pm_genpd_remove_notifier(dev); +} + +static int aplic_pm_add(struct device *dev, struct aplic_priv *priv) +{ + struct aplic_src_ctrl *srcs; + int ret; + + srcs = devm_kzalloc(dev, (priv->nr_irqs + 1) * sizeof(*srcs), GFP_KERNEL); + if (!srcs) + return -ENOMEM; + + priv->saved_hw_regs.srcs = srcs; + list_add(&priv->head, &aplics); + if (dev->pm_domain) { + priv->genpd_nb.notifier_call = aplic_pm_notifier; + ret = dev_pm_genpd_add_notifier(dev, &priv->genpd_nb); + if (ret) + goto remove_head; + + ret = devm_pm_runtime_enable(dev); + if (ret) + goto remove_notifier; + } + + return devm_add_action_or_reset(dev, aplic_pm_remove, priv); + +remove_notifier: + dev_pm_genpd_remove_notifier(dev); +remove_head: + list_del(&priv->head); + return ret; +} + void aplic_irq_unmask(struct irq_data *d) { struct aplic_priv *priv = irq_data_get_irq_chip_data(d); @@ -60,6 +215,8 @@ int aplic_irq_set_type(struct irq_data *d, unsigned int type) sourcecfg += (d->hwirq - 1) * sizeof(u32); writel(val, sourcecfg); + priv->saved_hw_regs.srcs[d->hwirq - 1].sourcecfg = val; + return 0; } @@ -82,6 +239,7 @@ int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base, void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode) { + struct aplic_saved_regs *saved_regs = &priv->saved_hw_regs; u32 val; #ifdef CONFIG_RISCV_M_MODE u32 valh; @@ -95,6 +253,8 @@ void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode) valh |= FIELD_PREP(APLIC_xMSICFGADDRH_HHXS, priv->msicfg.hhxs); writel(val, priv->regs + APLIC_xMSICFGADDR); writel(valh, priv->regs + APLIC_xMSICFGADDRH); + saved_regs->msiaddr = val; + saved_regs->msiaddrh = valh; } #endif @@ -106,6 +266,8 @@ void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode) writel(val, priv->regs + APLIC_DOMAINCFG); if (readl(priv->regs + APLIC_DOMAINCFG) != val) dev_warn(priv->dev, "unable to write 0x%x in domaincfg\n", val); + + saved_regs->domaincfg = val; } static void aplic_init_hw_irqs(struct aplic_priv *priv) @@ -176,7 +338,7 @@ int aplic_setup_priv(struct aplic_priv *priv, struct device *dev, void __iomem * /* Setup initial state APLIC interrupts */ aplic_init_hw_irqs(priv); - return 0; + return aplic_pm_add(dev, priv); } static int aplic_probe(struct platform_device *pdev) @@ -209,6 +371,8 @@ static int aplic_probe(struct platform_device *pdev) if (rc) dev_err_probe(dev, rc, "failed to setup APLIC in %s mode\n", msi_mode ? "MSI" : "direct"); + else + register_syscore_ops(&aplic_syscore_ops); #ifdef CONFIG_ACPI if (!acpi_disabled) diff --git a/drivers/irqchip/irq-riscv-aplic-main.h b/drivers/irqchip/irq-riscv-aplic-main.h index b0ad8cde69b1314c6db5ef3cc7070c843c381790..2d8ad7138541ad68a08349d9ed59a2554e520753 100644 --- a/drivers/irqchip/irq-riscv-aplic-main.h +++ b/drivers/irqchip/irq-riscv-aplic-main.h @@ -23,7 +23,25 @@ struct aplic_msicfg { u32 lhxw; }; +struct aplic_src_ctrl { + u32 sourcecfg; + u32 target; + u32 ie; +}; + +struct aplic_saved_regs { + u32 domaincfg; +#ifdef CONFIG_RISCV_M_MODE + u32 msiaddr; + u32 msiaddrh; +#endif + struct aplic_src_ctrl *srcs; +}; + struct aplic_priv { + struct list_head head; + struct notifier_block genpd_nb; + struct aplic_saved_regs saved_hw_regs; struct device *dev; u32 gsi_base; u32 nr_irqs; @@ -40,6 +58,7 @@ int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base, unsigned long *hwirq, unsigned int *type); void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode); int aplic_setup_priv(struct aplic_priv *priv, struct device *dev, void __iomem *regs); +void aplic_direct_restore_states(struct aplic_priv *priv); int aplic_direct_setup(struct device *dev, void __iomem *regs); #ifdef CONFIG_RISCV_APLIC_MSI int aplic_msi_setup(struct device *dev, void __iomem *regs); -- 2.34.1