[RFC PATCH 0/4] spi: cadence-xspi: add ACMD PIO support for NAND and NOR

[RFC PATCH 0/4] spi: cadence-xspi: add ACMD PIO support for NAND and NOR

[Fei Xie]

From: "fei.xie" <fei.xie@horizon.auto>

The Cadence XSPI controller provides an automatic command (ACMD) engine.
It can execute complete flash transactions, including command sequences,
device-ready polling and master-DMA data transfers.  Software submits a PIO
command containing the flash address, DMA address and transfer length.

For SPI NAND, for example, a read transaction combines PAGE READ, status
polling and READ CACHE in hardware.  Program and erase transactions also
combine write enable, data or erase commands, and status polling.  This
avoids submitting each command separately through STIG mode.

This RFC contains a working controller-local implementation for both SPI
NAND and SPI NOR.  Unsupported operations continue to use STIG mode.

There is an important layering issue in this implementation.  In order to
program the hardware sequences, the controller driver obtains upper-layer
driver data and directly uses struct spinand_device or struct spi_nor.  It
also keeps state across individual spi_mem_exec_op() calls in order to
recognize a complete NAND transaction.  We do not consider either property
desirable for a final implementation.

The hardware nevertheless needs information which is not available in one
spi_mem_op.  A NAND read is represented by separate PAGE READ, status-poll
and READ CACHE operations, while the ACMD engine needs all three before the
transaction is started.  NAND geometry is also needed to translate row and
column addresses for the ACMD address space.

We would appreciate guidance on the preferred interface boundary:

  1. Is a controller-local implementation acceptable because these command
     sequences are specific to the Cadence ACMD engine?

  2. Should SPI mem instead provide an interface for submitting an ordered
     group of operations, including status-poll semantics, with SPI NAND
     and SPI NOR constructing the group and the controller translating it
     into hardware sequences?

  3. If the latter is preferred, should NAND geometry be supplied through
     the sequence interface, or should it be exposed to controllers
     separately?

We have prototyped the second approach locally by adding sequence callbacks
to drivers/spi/spi-mem.c and constructing the NAND transactions in
drivers/mtd/nand/spi/core.c.  It removes the controller dependency on MTD
private structures, but the initial version duplicated page/OOB preparation
and made the generic sequence interface carry NAND-specific geometry.  That
prototype is therefore not included here; we would first like agreement on
the interface.

Performance
===========

The SPI NAND path was tested on a Horizon Robotics J6B platform with a
GigaDevice GD5F4GM8RE device.  A vendor Linux 6.1 backport of the same ACMD
implementation was used for the hardware tests.

Under the same 80 MHz transfer configuration, the sequential-read results
are:

  STIG using readq:  8.24 MB/s (7.86 MiB/s)
  ACMD PIO + MDMA:  10.08 MB/s (9.61 MiB/s), about 22% higher
  STIG read CPU:    59.74% / 40.87%
  ACMD read CPU:    approximately 16%

Using the lower STIG CPU figure for a conservative comparison, ACMD reduced
read CPU usage by 24.87 percentage points, or approximately 61% relative.

The CPU reduction comes from moving device-ready polling and command
sequencing into the controller.  Software no longer needs to submit and
wait for each STIG command while a page is being transferred.

The ACMD program path averaged 2.94 MiB/s over ten 128 KiB writes,
excluding erase time.  A 128 KiB random-data erase/program/read test
produced identical SHA-256 hashes and byte-for-byte comparison.  Read,
program and erase were also traced to confirm that the ACMD path was used.

The mainline series has been compile-tested for arm64.  The SPI NAND
hardware tests were performed with the Linux 6.1 backport because the test
platform currently runs the vendor 6.1 kernel.  SPI NOR has been
compile-tested but has not yet been tested on hardware.

This series is intended to start the API and layering discussion rather
than to propose the current upper-layer coupling as the final design.

fei.xie (4):
  dt-bindings: spi: cdns,xspi: add SPI NAND compatible
  spi: cadence-xspi: add ACMD support for SPI NAND
  spi: cadence-xspi: factor out reusable ACMD helpers
  spi: cadence-xspi: add ACMD support for SPI NOR

 .../devicetree/bindings/spi/cdns,xspi.yaml    |    3 +-
 drivers/spi/spi-cadence-xspi.c                | 1259 ++++++++++++++++-
 2 files changed, 1260 insertions(+), 2 deletions(-)

-- 
2.34.1

[RFC PATCH 1/4] dt-bindings: spi: cdns,xspi: add SPI NAND compatible

[Fei Xie]

From: "fei.xie" <fei.xie@horizon.auto>

Document the Cadence XSPI controller compatible used with SPI NAND.

Signed-off-by: fei.xie <fei.xie@horizon.auto>
---
 Documentation/devicetree/bindings/spi/cdns,xspi.yaml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/spi/cdns,xspi.yaml b/Documentation/devicetree/bindings/spi/cdns,xspi.yaml
index 38a5795589de2..3decf45d1b484 100644
--- a/Documentation/devicetree/bindings/spi/cdns,xspi.yaml
+++ b/Documentation/devicetree/bindings/spi/cdns,xspi.yaml
@@ -13,11 +13,12 @@ maintainers:
 description: |
   The XSPI controller allows SPI protocol communication in
   single, dual, quad or octal wire transmission modes for
-  read/write access to slaves such as SPI-NOR flash.
+  read/write access to slaves such as SPI-NOR and SPI-NAND flash.
 
 properties:
   compatible:
     enum:
+      - cdns,xspi-nand
       - cdns,xspi-nor
       - marvell,cn10-xspi-nor
 
-- 
2.34.1

Re: [RFC PATCH 1/4] dt-bindings: spi: cdns,xspi: add SPI NAND compatible

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 512 bytes --]

On Mon, Sep 21, 2026 at 05:36:58PM +0800, Fei Xie wrote:
> From: "fei.xie" <fei.xie@horizon.auto>
> 
> Document the Cadence XSPI controller compatible used with SPI NAND.

Please submit patches using subject lines reflecting the style for the
subsystem, this makes it easier for people to identify relevant patches.
Look at what existing commits in the area you're changing are doing and
make sure your subject lines visually resemble what they're doing.
There's no need to resubmit to fix this alone.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[RFC PATCH 2/4] spi: cadence-xspi: add ACMD support for SPI NAND

[Fei Xie]

From: "fei.xie" <fei.xie@horizon.auto>

Add PIO master-DMA support for the controller auto-command mode.
Configure the SPI NAND read, program, erase, status and reset
sequences, retaining STIG for operations not consumed by ACMD.

Signed-off-by: fei.xie <fei.xie@horizon.auto>
---
 drivers/spi/spi-cadence-xspi.c | 782 +++++++++++++++++++++++++++++++++
 1 file changed, 782 insertions(+)

diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c
index 39c868a5b171c..0fedf6cd6a590 100644
--- a/drivers/spi/spi-cadence-xspi.c
+++ b/drivers/spi/spi-cadence-xspi.c
@@ -4,6 +4,7 @@
 
 #include <linux/completion.h>
 #include <linux/delay.h>
+#include <linux/dma-mapping.h>
 #include <linux/err.h>
 #include <linux/errno.h>
 #include <linux/interrupt.h>
@@ -11,6 +12,8 @@
 #include <linux/iopoll.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/mtd/spinand.h>
+#include <linux/overflow.h>
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
 #include <linux/property.h>
@@ -54,7 +57,33 @@
 #define CDNS_XSPI_CMD_REG_4			0x0010
 #define CDNS_XSPI_CMD_REG_5			0x0014
 
+/* Auto command fields in command register 0 */
+#define CDNS_XSPI_ACMD_MODE			GENMASK(31, 30)
+#define CDNS_XSPI_ACMD_TRD_NUM			GENMASK(26, 24)
+#define CDNS_XSPI_ACMD_BANK_NUM			GENMASK(22, 20)
+#define CDNS_XSPI_ACMD_DMA_SEL			BIT(19)
+#define CDNS_XSPI_ACMD_INT_EN			BIT(18)
+#define CDNS_XSPI_ACMD_CMD_TYPE			GENMASK(15, 0)
+
+#define CDNS_XSPI_ACMD_READ_OP			0x2200
+#define CDNS_XSPI_ACMD_PROG_OP			0x2100
+#define CDNS_XSPI_ACMD_ERASE_OP			0x1000
+#define CDNS_XSPI_ACMD_RESET_OP			0x1100
+#define CDNS_XSPI_ACMD_DATA_THREAD		7
+#define CDNS_XSPI_ACMD_ERASE_THREAD		5
+#define CDNS_XSPI_ACMD_TIMEOUT_MS		1000
+#define CDNS_XSPI_ACMD_MODE_PIO			1
+
+#define CDNS_XSPI_NAND_OP_GET_FEATURE		0x0f
+#define CDNS_XSPI_NAND_OP_WRITE_ENABLE		0x06
+#define CDNS_XSPI_NAND_OP_PROGRAM_EXECUTE	0x10
+#define CDNS_XSPI_NAND_OP_PAGE_READ		0x13
+#define CDNS_XSPI_NAND_OP_BLOCK_ERASE		0xd8
+#define CDNS_XSPI_NAND_OP_RESET			0xff
+#define CDNS_XSPI_NAND_STATUS_REG		0xc0
+
 /* Command status registers */
+#define CDNS_XSPI_CMD_STATUS_PTR_REG	0x0040
 #define CDNS_XSPI_CMD_STATUS_REG		0x0044
 
 /* Controller status register */
@@ -143,10 +172,126 @@
 #define CDNS_XSPI_CMD_STATUS_CRC_ERROR		BIT(2)
 #define CDNS_XSPI_CMD_STATUS_BUS_ERROR		BIT(1)
 #define CDNS_XSPI_CMD_STATUS_INV_SEQ_ERROR	BIT(0)
+/* Reset sequence config register */
+#define CDNS_XSPI_RST_SEQ_CFG_0			0x0400
+#define CDNS_XSPI_RST_SEQ_P1_CMD1_VAL		GENMASK(15, 8)
+
+/* Erase sequence config registers */
+#define CDNS_XSPI_ERSS_SEQ_CFG_0			0x0410
+#define CDNS_XSPI_ERSS_SEQ_P1_CMD_VAL		GENMASK(7, 0)
+#define CDNS_XSPI_ERSS_SEQ_P1_CMD_IOS		GENMASK(9, 8)
+#define CDNS_XSPI_ERSS_SEQ_P1_CMD_EDGE		BIT(11)
+#define CDNS_XSPI_ERSS_SEQ_P1_ADDR_CNT		GENMASK(14, 12)
+#define CDNS_XSPI_ERSS_SEQ_P1_CMD_EXT_EN	BIT(15)
+#define CDNS_XSPI_ERSS_SEQ_P1_CMD_EXT_VAL	GENMASK(23, 16)
+#define CDNS_XSPI_ERSS_SEQ_P1_ADDR_IOS		GENMASK(25, 24)
+#define CDNS_XSPI_ERSS_SEQ_P1_ADDR_EDGE		BIT(28)
+
+#define CDNS_XSPI_ERSS_SEQ_CFG_1			0x0414
+#define CDNS_XSPI_ERSS_SEQ_P1_SECT_SIZE		GENMASK(4, 0)
+
+#define CDNS_XSPI_ERSS_SEQ_CFG_2			0x0418
+
+/* Program sequence config registers */
+#define CDNS_XSPI_PROG_SEQ_CFG_0			0x0420
+#define CDNS_XSPI_PROG_SEQ_P1_CMD_VAL		GENMASK(7, 0)
+#define CDNS_XSPI_PROG_SEQ_P1_CMD_IOS		GENMASK(9, 8)
+#define CDNS_XSPI_PROG_SEQ_P1_CMD_EDGE		BIT(11)
+#define CDNS_XSPI_PROG_SEQ_P1_ADDR_CNT		GENMASK(14, 12)
+#define CDNS_XSPI_PROG_SEQ_P1_ADDR_IOS		GENMASK(17, 16)
+#define CDNS_XSPI_PROG_SEQ_P1_ADDR_EDGE		BIT(19)
+#define CDNS_XSPI_PROG_SEQ_P1_DATA_IOS		GENMASK(21, 20)
+#define CDNS_XSPI_PROG_SEQ_P1_DATA_EDGE		BIT(23)
+#define CDNS_XSPI_PROG_SEQ_P1_DUMMY_CNT		GENMASK(29, 24)
+
+#define CDNS_XSPI_PROG_SEQ_CFG_1			0x0424
+#define CDNS_XSPI_PROG_SEQ_P1_CMD_EXT_EN		BIT(0)
+#define CDNS_XSPI_PROG_SEQ_P1_CMD_EXT_VAL		GENMASK(15, 8)
+
+#define CDNS_XSPI_PROG_SEQ_CFG_2			0x0428
+
+/* Read sequence config registers */
+#define CDNS_XSPI_READ_SEQ_CFG_0			0x0430
+#define CDNS_XSPI_READ_SEQ_P1_CMD_VAL		GENMASK(7, 0)
+#define CDNS_XSPI_READ_SEQ_P1_CMD_IOS		GENMASK(9, 8)
+#define CDNS_XSPI_READ_SEQ_P1_CMD_EDGE		BIT(11)
+#define CDNS_XSPI_READ_SEQ_P1_ADDR_CNT		GENMASK(14, 12)
+#define CDNS_XSPI_READ_SEQ_P1_ADDR_IOS		GENMASK(17, 16)
+#define CDNS_XSPI_READ_SEQ_P1_ADDR_EDGE		BIT(19)
+#define CDNS_XSPI_READ_SEQ_P1_DATA_IOS		GENMASK(21, 20)
+#define CDNS_XSPI_READ_SEQ_P1_DATA_EDGE		BIT(23)
+#define CDNS_XSPI_READ_SEQ_P1_DUMMY_CNT		GENMASK(29, 24)
+
+#define CDNS_XSPI_READ_SEQ_CFG_1			0x0434
+#define CDNS_XSPI_READ_SEQ_P1_CMD_EXT_EN		BIT(0)
+#define CDNS_XSPI_READ_SEQ_P1_CACHE_RANDOM_READ_EN	BIT(4)
+#define CDNS_XSPI_READ_SEQ_CFG_2			0x0438
+
+/* Write enable sequence config register */
+#define CDNS_XSPI_WE_SEQ_CFG_0			0x0440
+#define CDNS_XSPI_WE_SEQ_P1_CMD_VAL		GENMASK(7, 0)
+#define CDNS_XSPI_WE_SEQ_P1_CMD_IOS		GENMASK(9, 8)
+#define CDNS_XSPI_WE_SEQ_P1_CMD_EDGE		BIT(11)
+#define CDNS_XSPI_WE_SEQ_P1_EN			BIT(24)
+
+/* Status sequence config registers */
+#define CDNS_XSPI_STAT_SEQ_CFG_0			0x0450
+#define CDNS_XSPI_STAT_SEQ_P1_ADDR_CNT			GENMASK(9, 8)
+#define CDNS_XSPI_STAT_SEQ_CFG_1			0x0454
+#define CDNS_XSPI_P1_DEV_RDY_ADDR_EN		BIT(6)
+#define CDNS_XSPI_P1_PROG_FAIL_ADDR_EN		BIT(22)
+#define CDNS_XSPI_P1_ERS_FAIL_ADDR_EN		BIT(30)
+
+#define CDNS_XSPI_STAT_SEQ_CFG_2			0x0458
+#define CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_CMD_VAL	GENMASK(7, 0)
+#define CDNS_XSPI_STAT_SEQ_P1_ERS_FAIL_CMD_VAL	GENMASK(15, 8)
+#define CDNS_XSPI_STAT_SEQ_P1_PROG_FAIL_CMD_VAL	GENMASK(31, 24)
+
+#define CDNS_XSPI_STAT_SEQ_CFG_3			0x045c
+#define CDNS_XSPI_STAT_SEQ_CFG_4			0x0460
+#define CDNS_XSPI_STAT_SEQ_CFG_5			0x0464
+#define CDNS_XSPI_STAT_SEQ_DEV_RDY_IDX		GENMASK(3, 0)
+#define CDNS_XSPI_STAT_SEQ_DEV_RDY_EN		BIT(6)
+#define CDNS_XSPI_STAT_SEQ_ERS_FAIL_IDX		GENMASK(11, 8)
+#define CDNS_XSPI_STAT_SEQ_ERS_FAIL_VAL		BIT(12)
+#define CDNS_XSPI_STAT_SEQ_ERS_FAIL_EN		BIT(14)
+#define CDNS_XSPI_STAT_SEQ_PROG_FAIL_IDX		GENMASK(27, 24)
+#define CDNS_XSPI_STAT_SEQ_PROG_FAIL_VAL		BIT(28)
+#define CDNS_XSPI_STAT_SEQ_PROG_FAIL_EN			BIT(30)
+
+#define CDNS_XSPI_STAT_SEQ_CFG_7			0x046c
+#define CDNS_XSPI_STAT_SEQ_CFG_8			0x0470
+#define CDNS_XSPI_STAT_SEQ_CFG_9			0x0474
+
+#define CDNS_XSPI_STAT_SEQ_CFG_10		0x0478
+#define CDNS_XSPI_STAT_SEQ_ECC_FAIL_EN		BIT(31)
+#define CDNS_XSPI_STAT_SEQ_CRDY_VAL		BIT(27)
+#define CDNS_XSPI_STAT_SEQ_CRDY_IDX		GENMASK(26, 24)
+#define CDNS_XSPI_STAT_SEQ_ECC_CORR_VAL		GENMASK(23, 16)
+#define CDNS_XSPI_STAT_SEQ_ECC_FAIL_VAL		GENMASK(15, 8)
+#define CDNS_XSPI_STAT_SEQ_ECC_FAIL_MASK		GENMASK(7, 0)
 
 #define CDNS_XSPI_STIG_DONE_FLAG		BIT(0)
 #define CDNS_XSPI_TRD_STATUS			0x0104
 
+#define CDNS_XSPI_FLASH_TYPE_NOR		0
+#define CDNS_XSPI_FLASH_TYPE_NAND		1
+
+#define CDNS_XSPI_GLOBAL_SEQ_CFG			0x0390
+#define CDNS_XSPI_SEQ_TYPE			GENMASK(24, 23)
+#define CDNS_XSPI_SEQ_PAGE_SIZE_PGM		GENMASK(7, 4)
+#define CDNS_XSPI_SEQ_PAGE_SIZE_RD		GENMASK(3, 0)
+#define CDNS_XSPI_SEQ_SPI_NAND			3
+
+#define CDNS_XSPI_GLOBAL_SEQ_CFG_1		0x0394
+#define CDNS_XSPI_SEQ_PLANE_CNT			GENMASK(29, 28)
+#define CDNS_XSPI_SEQ_PAGE_PER_BLOCK		GENMASK(26, 24)
+#define CDNS_XSPI_SEQ_PAGE_CA_SIZE		BIT(16)
+#define CDNS_XSPI_SEQ_PAGE_SIZE_EXT		GENMASK(8, 0)
+
+#define CDNS_XSPI_XIP_MODE_CFG			0x0388
+#define CDNS_XSPI_XIP_EN			BIT(0)
+
 #define MODE_NO_OF_BYTES			GENMASK(25, 24)
 #define MODEBYTES_COUNT			1
 
@@ -305,6 +450,7 @@ enum cdns_xspi_stig_cmd_dir {
 
 struct cdns_xspi_driver_data {
 	bool mrvl_hw_overlay;
+	bool use_acmd;
 	u32 dll_phy_ctrl;
 	u32 ctb_rfile_phy_ctrl;
 	u32 rfile_phy_tsel;
@@ -313,12 +459,26 @@ struct cdns_xspi_driver_data {
 	u32 rfile_phy_gate_lpbk_ctrl;
 	u32 rfile_phy_dll_master_ctrl;
 	u32 rfile_phy_dll_slave_ctrl;
+	u8 flash_type;
 };
 
 static struct cdns_xspi_driver_data cdns_driver_data = {
 	.mrvl_hw_overlay = false,
 };
 
+static struct cdns_xspi_driver_data cdns_nand_driver_data = {
+	.mrvl_hw_overlay = false,
+	.use_acmd = true,
+	.flash_type = CDNS_XSPI_FLASH_TYPE_NAND,
+};
+
+struct cdns_xspi_acmd_info {
+	u64 row_addr;
+	u64 column_addr;
+	size_t data_nbytes;
+	bool row_addr_valid;
+	bool initialized;
+};
 struct cdns_xspi_dev {
 	struct platform_device *pdev;
 	struct spi_controller *host;
@@ -351,6 +511,13 @@ struct cdns_xspi_dev {
 
 	bool xfer_in_progress;
 	int current_xfer_qword;
+	u32 work_mode;
+	u8 flash_type;
+
+	struct cdns_xspi_acmd_info acmd_info;
+	void *dma_buf;
+	dma_addr_t dma_addr;
+	u32 dma_buf_len;
 };
 
 static int cdns_xspi_wait_for_controller_idle(struct cdns_xspi_dev *cdns_xspi)
@@ -425,6 +592,268 @@ static void cdns_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
 	writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
 }
 
+static void cdns_xspi_nand_cfg_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					struct spinand_device *spinand)
+{
+	u32 seq_cfg, seq_cfg1;
+
+	seq_cfg = readl(cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG);
+	seq_cfg1 = readl(cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG_1);
+	seq_cfg = u32_replace_bits(seq_cfg, CDNS_XSPI_SEQ_SPI_NAND,
+				   CDNS_XSPI_SEQ_TYPE);
+	seq_cfg = u32_replace_bits(seq_cfg,
+				   ilog2(spinand->base.memorg.pagesize),
+				   CDNS_XSPI_SEQ_PAGE_SIZE_PGM);
+	seq_cfg = u32_replace_bits(seq_cfg,
+				   ilog2(spinand->base.memorg.pagesize),
+				   CDNS_XSPI_SEQ_PAGE_SIZE_RD);
+	seq_cfg1 = u32_replace_bits(seq_cfg1,
+				    ilog2(spinand->base.memorg.planes_per_lun),
+				    CDNS_XSPI_SEQ_PLANE_CNT);
+	seq_cfg1 = u32_replace_bits(seq_cfg1,
+				    ilog2(spinand->base.memorg.pages_per_eraseblock),
+				    CDNS_XSPI_SEQ_PAGE_PER_BLOCK);
+	seq_cfg1 = u32_replace_bits(seq_cfg1,
+				    !!(spinand->base.memorg.pagesize & BIT(12)),
+				    CDNS_XSPI_SEQ_PAGE_CA_SIZE);
+	seq_cfg1 = u32_replace_bits(seq_cfg1, spinand->base.memorg.oobsize,
+				    CDNS_XSPI_SEQ_PAGE_SIZE_EXT);
+
+	writel(seq_cfg, cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG);
+	writel(seq_cfg1, cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG_1);
+}
+
+static void cdns_xspi_nand_read_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					 struct spinand_device *spinand)
+{
+	const struct spi_mem_op *op = spinand->op_templates->read_cache;
+	u32 dummy_cycles = 0;
+	u32 read_seq_cfg0;
+	u32 read_seq_cfg1;
+
+	if (op->dummy.nbytes && op->dummy.buswidth)
+		dummy_cycles = op->dummy.nbytes * BITS_PER_BYTE /
+			       (op->dummy.buswidth * (op->dummy.dtr + 1));
+
+	read_seq_cfg0 =
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_VAL, op->cmd.opcode) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_IOS,
+			   ilog2(op->cmd.buswidth)) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_ADDR_CNT,
+			   op->addr.nbytes) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_ADDR_IOS,
+			   ilog2(op->addr.buswidth)) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_ADDR_EDGE, op->addr.dtr) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_DATA_IOS,
+			   ilog2(op->data.buswidth)) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_DATA_EDGE, op->data.dtr) |
+		FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_DUMMY_CNT, dummy_cycles);
+	read_seq_cfg1 = FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_EXT_EN,
+				   op->cmd.nbytes > 1) |
+		CDNS_XSPI_READ_SEQ_P1_CACHE_RANDOM_READ_EN;
+
+	writel(read_seq_cfg0, cdns_xspi->iobase + CDNS_XSPI_READ_SEQ_CFG_0);
+	writel(read_seq_cfg1, cdns_xspi->iobase + CDNS_XSPI_READ_SEQ_CFG_1);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_READ_SEQ_CFG_2);
+
+	dev_dbg(cdns_xspi->dev,
+		"ACMD read: op=%02x %u-%u-%u%s addr=%u dummy=%u cfg=%08x/%08x\n",
+		op->cmd.opcode, op->cmd.buswidth, op->addr.buswidth,
+		op->data.buswidth, op->data.dtr ? " DTR" : " SDR",
+		op->addr.nbytes, dummy_cycles, read_seq_cfg0, read_seq_cfg1);
+}
+
+static void cdns_xspi_nand_write_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					  struct spinand_device *spinand)
+{
+	const struct spi_mem_op *op = spinand->op_templates->write_cache;
+	u32 dummy_cycles = 0;
+	u32 write_seq_cfg0;
+	u32 write_seq_cfg1;
+
+	if (op->dummy.nbytes && op->dummy.buswidth)
+		dummy_cycles = op->dummy.nbytes * BITS_PER_BYTE /
+			       (op->dummy.buswidth * (op->dummy.dtr + 1));
+
+	write_seq_cfg0 =
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_VAL, op->cmd.opcode) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_IOS,
+			   ilog2(op->cmd.buswidth)) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_ADDR_CNT,
+			   op->addr.nbytes) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_ADDR_IOS,
+			   ilog2(op->addr.buswidth)) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_ADDR_EDGE, op->addr.dtr) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_DATA_IOS,
+			   ilog2(op->data.buswidth)) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_DATA_EDGE, op->data.dtr) |
+		FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_DUMMY_CNT, dummy_cycles);
+	write_seq_cfg1 = FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_EXT_EN,
+				    op->cmd.nbytes > 1);
+
+	writel(write_seq_cfg0, cdns_xspi->iobase + CDNS_XSPI_PROG_SEQ_CFG_0);
+	writel(write_seq_cfg1, cdns_xspi->iobase + CDNS_XSPI_PROG_SEQ_CFG_1);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_PROG_SEQ_CFG_2);
+
+	dev_dbg(cdns_xspi->dev,
+		"ACMD program: op=%02x %u-%u-%u%s addr=%u dummy=%u cfg=%08x/%08x\n",
+		op->cmd.opcode, op->cmd.buswidth, op->addr.buswidth,
+		op->data.buswidth, op->data.dtr ? " DTR" : " SDR",
+		op->addr.nbytes, dummy_cycles, write_seq_cfg0, write_seq_cfg1);
+}
+
+static void cdns_xspi_nand_erase_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					  struct spinand_device *spinand)
+{
+	u32 erase_seq_cfg0;
+	u32 erase_seq_cfg1;
+
+	/* SPI-NAND block erase is always D8h with a 3-byte row address. */
+	erase_seq_cfg0 =
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_VAL, 0xd8) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_IOS, 0) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_EDGE, 0) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_ADDR_CNT, 3) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_EXT_EN, 0) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_EXT_VAL, 0) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_ADDR_IOS, 0) |
+		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_ADDR_EDGE, 0);
+	erase_seq_cfg1 = FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_SECT_SIZE,
+				    ilog2(nanddev_eraseblock_size(&spinand->base)));
+
+	writel(erase_seq_cfg0,
+	       cdns_xspi->iobase + CDNS_XSPI_ERSS_SEQ_CFG_0);
+	writel(erase_seq_cfg1,
+	       cdns_xspi->iobase + CDNS_XSPI_ERSS_SEQ_CFG_1);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_ERSS_SEQ_CFG_2);
+
+	dev_dbg(cdns_xspi->dev,
+		"ACMD erase sequence: cfg0=%08x cfg1=%08x\n",
+		erase_seq_cfg0, erase_seq_cfg1);
+}
+
+static void cdns_xspi_nand_status_seq_init(struct cdns_xspi_dev *cdns_xspi)
+{
+	u32 stat_seq_cfg1;
+	u32 stat_seq_cfg2;
+	u32 stat_seq_cfg5;
+
+	writel(FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_ADDR_CNT, 0),
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_0);
+
+	stat_seq_cfg1 = CDNS_XSPI_P1_DEV_RDY_ADDR_EN |
+			CDNS_XSPI_P1_PROG_FAIL_ADDR_EN |
+			CDNS_XSPI_P1_ERS_FAIL_ADDR_EN;
+	writel(stat_seq_cfg1, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_1);
+
+	stat_seq_cfg2 = FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_CMD_VAL,
+				   CDNS_XSPI_NAND_OP_GET_FEATURE) |
+		FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_PROG_FAIL_CMD_VAL,
+			   CDNS_XSPI_NAND_OP_GET_FEATURE) |
+		FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_ERS_FAIL_CMD_VAL,
+			   CDNS_XSPI_NAND_OP_GET_FEATURE);
+	writel(stat_seq_cfg2, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_2);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_3);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_4);
+
+	stat_seq_cfg5 =
+		FIELD_PREP(CDNS_XSPI_STAT_SEQ_DEV_RDY_IDX,
+			   __ffs(STATUS_BUSY)) |
+		CDNS_XSPI_STAT_SEQ_DEV_RDY_EN |
+		FIELD_PREP(CDNS_XSPI_STAT_SEQ_ERS_FAIL_IDX,
+			   __ffs(STATUS_ERASE_FAILED)) |
+		CDNS_XSPI_STAT_SEQ_ERS_FAIL_VAL |
+		CDNS_XSPI_STAT_SEQ_ERS_FAIL_EN |
+		FIELD_PREP(CDNS_XSPI_STAT_SEQ_PROG_FAIL_IDX,
+			   __ffs(STATUS_PROG_FAILED)) |
+		CDNS_XSPI_STAT_SEQ_PROG_FAIL_VAL |
+		CDNS_XSPI_STAT_SEQ_PROG_FAIL_EN;
+	writel(stat_seq_cfg5, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_5);
+	writel(CDNS_XSPI_NAND_STATUS_REG,
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_7);
+	writel(CDNS_XSPI_NAND_STATUS_REG,
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_8);
+	writel(CDNS_XSPI_NAND_STATUS_REG,
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_9);
+
+	writel(CDNS_XSPI_STAT_SEQ_ECC_FAIL_EN |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_CRDY_IDX, __ffs(STATUS_BUSY)) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_CRDY_VAL, 0) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_ECC_CORR_VAL,
+			  STATUS_ECC_HAS_BITFLIPS) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_ECC_FAIL_VAL,
+			  STATUS_ECC_UNCOR_ERROR) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_ECC_FAIL_MASK,
+			  STATUS_ECC_MASK),
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_10);
+}
+
+static void cdns_xspi_nand_write_enable_seq_init(struct cdns_xspi_dev *cdns_xspi)
+{
+	u32 cfg;
+
+	cfg = readl(cdns_xspi->iobase + CDNS_XSPI_WE_SEQ_CFG_0);
+	cfg = u32_replace_bits(cfg, 1, CDNS_XSPI_WE_SEQ_P1_EN);
+	cfg = u32_replace_bits(cfg, CDNS_XSPI_NAND_OP_WRITE_ENABLE,
+			       CDNS_XSPI_WE_SEQ_P1_CMD_VAL);
+	cfg = u32_replace_bits(cfg, 0, CDNS_XSPI_WE_SEQ_P1_CMD_IOS);
+	cfg = u32_replace_bits(cfg, 0, CDNS_XSPI_WE_SEQ_P1_CMD_EDGE);
+	writel(cfg, cdns_xspi->iobase + CDNS_XSPI_WE_SEQ_CFG_0);
+}
+
+static void cdns_xspi_set_mode_acmd(struct cdns_xspi_dev *cdns_xspi)
+{
+	u32 reg_val;
+
+	reg_val = readl(cdns_xspi->iobase + CDNS_XSPI_CTRL_CONFIG_REG);
+	reg_val = u32_replace_bits(reg_val, CDNS_XSPI_WORK_MODE_ACMD,
+				   CDNS_XSPI_CTRL_WORK_MODE);
+	writel(reg_val, cdns_xspi->iobase + CDNS_XSPI_CTRL_CONFIG_REG);
+}
+
+static void cdns_xspi_nand_reset_seq_init(struct cdns_xspi_dev *cdns_xspi)
+{
+	u32 cfg;
+
+	cfg = FIELD_PREP(CDNS_XSPI_RST_SEQ_P1_CMD1_VAL,
+			 CDNS_XSPI_NAND_OP_RESET);
+	writel(cfg, cdns_xspi->iobase + CDNS_XSPI_RST_SEQ_CFG_0);
+}
+
+static int cdns_xspi_nand_init(struct cdns_xspi_dev *cdns_xspi,
+			       struct spinand_device *spinand)
+{
+	u32 reg_val;
+
+	cdns_xspi_nand_cfg_seq_init(cdns_xspi, spinand);
+	cdns_xspi_nand_read_seq_init(cdns_xspi, spinand);
+	cdns_xspi_nand_reset_seq_init(cdns_xspi);
+	cdns_xspi_nand_write_seq_init(cdns_xspi, spinand);
+	cdns_xspi_nand_write_enable_seq_init(cdns_xspi);
+	cdns_xspi_nand_status_seq_init(cdns_xspi);
+	cdns_xspi_nand_erase_seq_init(cdns_xspi, spinand);
+
+	reg_val = readl(cdns_xspi->iobase + CDNS_XSPI_XIP_MODE_CFG);
+	if (reg_val & CDNS_XSPI_XIP_EN) {
+		reg_val &= ~CDNS_XSPI_XIP_EN;
+		writel(reg_val, cdns_xspi->iobase + CDNS_XSPI_XIP_MODE_CFG);
+	}
+	cdns_xspi->dma_buf_len = spinand->base.memorg.pagesize +
+				 spinand->base.memorg.oobsize;
+	cdns_xspi->dma_buf = dmam_alloc_coherent(cdns_xspi->dev,
+						 cdns_xspi->dma_buf_len,
+						 &cdns_xspi->dma_addr,
+						 GFP_KERNEL);
+	if (!cdns_xspi->dma_buf)
+		return -ENOMEM;
+
+	cdns_xspi->acmd_info.initialized = true;
+
+	return 0;
+}
+
 static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi)
 {
 	u32 ctrl_ver;
@@ -580,6 +1009,349 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
 	return 0;
 }
 
+static int cdns_xspi_acmd_get_thread_status(struct cdns_xspi_dev *cdns_xspi,
+					    u32 thread)
+{
+	writel(thread, cdns_xspi->iobase + CDNS_XSPI_CMD_STATUS_PTR_REG);
+
+	return cdns_xspi_check_command_status(cdns_xspi);
+}
+
+static int cdns_xspi_acmd_run(struct cdns_xspi_dev *cdns_xspi, u32 cmd_regs[6],
+			      u32 thread)
+{
+	unsigned long timeout;
+	int ret;
+
+	cdns_xspi_set_mode_acmd(cdns_xspi);
+	reinit_completion(&cdns_xspi->auto_cmd_complete);
+	cdns_xspi_set_interrupts(cdns_xspi, true);
+	cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
+
+	timeout = msecs_to_jiffies(CDNS_XSPI_ACMD_TIMEOUT_MS);
+	if (!wait_for_completion_timeout(&cdns_xspi->auto_cmd_complete,
+					 timeout)) {
+		dev_err(cdns_xspi->dev, "ACMD command timed out\n");
+		ret = -ETIMEDOUT;
+	} else {
+		ret = cdns_xspi_acmd_get_thread_status(cdns_xspi, thread);
+	}
+
+	cdns_xspi_set_interrupts(cdns_xspi, false);
+
+	return ret;
+}
+
+static int cdns_xspi_nand_addr(struct spinand_device *spinand, u64 row,
+			       u64 column, u64 *xspi_addr)
+{
+	const struct nand_memory_organization *memorg = &spinand->base.memorg;
+	u64 column_limit, page_stride;
+
+	page_stride = roundup_pow_of_two(memorg->pagesize + memorg->oobsize);
+	if (check_mul_overflow(page_stride, memorg->planes_per_lun,
+			       &column_limit) ||
+	    column >= column_limit ||
+	    check_mul_overflow(row, page_stride, xspi_addr) ||
+	    check_add_overflow(*xspi_addr, column, xspi_addr))
+		return -ERANGE;
+
+	return 0;
+}
+
+static u32 cdns_xspi_acmd_cmd(struct cdns_xspi_dev *cdns_xspi, u32 type,
+			      u32 thread, bool use_dma)
+{
+	return FIELD_PREP(CDNS_XSPI_ACMD_MODE, CDNS_XSPI_ACMD_MODE_PIO) |
+	       FIELD_PREP(CDNS_XSPI_ACMD_TRD_NUM, thread) |
+	       FIELD_PREP(CDNS_XSPI_ACMD_BANK_NUM, cdns_xspi->cur_cs) |
+	       (use_dma ? CDNS_XSPI_ACMD_DMA_SEL : 0) |
+	       CDNS_XSPI_ACMD_INT_EN |
+	       FIELD_PREP(CDNS_XSPI_ACMD_CMD_TYPE, type);
+}
+
+static int cdns_xspi_pio_mdma_erase(struct cdns_xspi_dev *cdns_xspi,
+				    struct spinand_device *spinand,
+				    const struct spi_mem_op *op)
+{
+	u32 cmd_regs[6] = { 0 };
+	u64 xspi_addr;
+	int ret;
+
+	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
+	if (ret) {
+		dev_err(cdns_xspi->dev,
+			"controller did not become idle before erase\n");
+		return ret;
+	}
+
+	ret = cdns_xspi_nand_addr(spinand, op->addr.val, 0, &xspi_addr);
+	if (ret)
+		return ret;
+
+	if (upper_32_bits(xspi_addr)) {
+		dev_err(cdns_xspi->dev,
+			"erase address 0x%llx exceeds the ACMD PIO range\n",
+			xspi_addr);
+		return -ERANGE;
+	}
+
+	cmd_regs[1] = lower_32_bits(xspi_addr);
+	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
+					 CDNS_XSPI_ACMD_ERASE_OP,
+					 CDNS_XSPI_ACMD_ERASE_THREAD, false);
+
+	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
+				 CDNS_XSPI_ACMD_ERASE_THREAD);
+	if (ret)
+		dev_err(cdns_xspi->dev, "ACMD erase failed: %d\n", ret);
+
+	return ret;
+}
+
+static int cdns_xspi_pio_reset(struct cdns_xspi_dev *cdns_xspi)
+{
+	u32 cmd_regs[6] = { 0 };
+	int ret;
+
+	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
+	if (ret) {
+		dev_err(cdns_xspi->dev,
+			"controller did not become idle before reset\n");
+		return ret;
+	}
+	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
+					 CDNS_XSPI_ACMD_RESET_OP,
+					 CDNS_XSPI_ACMD_DATA_THREAD, false);
+	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
+				 CDNS_XSPI_ACMD_DATA_THREAD);
+	if (ret)
+		return ret;
+
+	cdns_xspi->acmd_info.row_addr_valid = false;
+	cdns_xspi->acmd_info.column_addr = 0;
+	cdns_xspi->acmd_info.data_nbytes = 0;
+	cdns_xspi->out_buffer = NULL;
+
+	return 0;
+}
+
+static int cdns_xspi_pio_mdma_program(struct cdns_xspi_dev *cdns_xspi,
+				      struct spinand_device *spinand,
+				      const struct spi_mem_op *op)
+{
+	u32 cmd_regs[6] = { 0 };
+	u64 xspi_addr;
+	int ret;
+
+	if (!cdns_xspi->out_buffer || !cdns_xspi->acmd_info.data_nbytes) {
+		dev_err(cdns_xspi->dev, "missing program-load data\n");
+		ret = -EINVAL;
+		goto out_clear_program_state;
+	}
+
+	if (cdns_xspi->acmd_info.data_nbytes > cdns_xspi->dma_buf_len) {
+		ret = -EMSGSIZE;
+		goto out_clear_program_state;
+	}
+
+	memcpy(cdns_xspi->dma_buf, cdns_xspi->out_buffer,
+	       cdns_xspi->acmd_info.data_nbytes);
+
+	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
+	if (ret) {
+		dev_err(cdns_xspi->dev,
+			"controller did not become idle before program\n");
+		goto out_clear_program_state;
+	}
+
+	ret = cdns_xspi_nand_addr(spinand, op->addr.val,
+				  cdns_xspi->acmd_info.column_addr,
+				  &xspi_addr);
+	if (ret)
+		goto out_clear_program_state;
+
+	if (upper_32_bits(xspi_addr)) {
+		dev_err(cdns_xspi->dev,
+			"program address 0x%llx exceeds the ACMD PIO range\n",
+			xspi_addr);
+		ret = -ERANGE;
+		goto out_clear_program_state;
+	}
+
+	cmd_regs[1] = lower_32_bits(xspi_addr);
+	cmd_regs[2] = lower_32_bits(cdns_xspi->dma_addr);
+	cmd_regs[3] = upper_32_bits(cdns_xspi->dma_addr);
+	cmd_regs[4] = cdns_xspi->acmd_info.data_nbytes - 1;
+	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
+					 CDNS_XSPI_ACMD_PROG_OP,
+					 CDNS_XSPI_ACMD_DATA_THREAD, true);
+
+	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
+				 CDNS_XSPI_ACMD_DATA_THREAD);
+
+out_clear_program_state:
+	cdns_xspi->acmd_info.column_addr = 0;
+	cdns_xspi->acmd_info.data_nbytes = 0;
+	cdns_xspi->out_buffer = NULL;
+	return ret;
+}
+
+static int cdns_xspi_pio_mdma_read(struct cdns_xspi_dev *cdns_xspi,
+				   struct spinand_device *spinand,
+				   const struct spi_mem_op *op)
+{
+	u32 cmd_regs[6] = { 0 };
+	u64 xspi_addr;
+	int ret;
+
+	if (!cdns_xspi->acmd_info.row_addr_valid) {
+		dev_err(cdns_xspi->dev,
+			"read cache command without a pending page read\n");
+		return -EINVAL;
+	}
+
+	if (op->data.dir != SPI_MEM_DATA_IN || !op->data.nbytes ||
+	    !op->data.buf.in) {
+		ret = -EINVAL;
+		goto out_clear_read_state;
+	}
+
+	if (op->data.nbytes > cdns_xspi->dma_buf_len) {
+		ret = -EMSGSIZE;
+		goto out_clear_read_state;
+	}
+
+	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
+	if (ret) {
+		dev_err(cdns_xspi->dev,
+			"controller did not become idle before read\n");
+		goto out_clear_read_state;
+	}
+
+	ret = cdns_xspi_nand_addr(spinand,
+				  cdns_xspi->acmd_info.row_addr,
+				  op->addr.val, &xspi_addr);
+	if (ret)
+		goto out_clear_read_state;
+
+	if (upper_32_bits(xspi_addr)) {
+		dev_err(cdns_xspi->dev,
+			"read address 0x%llx exceeds the ACMD PIO range\n",
+			xspi_addr);
+		ret = -ERANGE;
+		goto out_clear_read_state;
+	}
+
+	cmd_regs[1] = lower_32_bits(xspi_addr);
+	cmd_regs[2] = lower_32_bits(cdns_xspi->dma_addr);
+	cmd_regs[3] = upper_32_bits(cdns_xspi->dma_addr);
+	cmd_regs[4] = op->data.nbytes - 1;
+	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
+					 CDNS_XSPI_ACMD_READ_OP,
+					 CDNS_XSPI_ACMD_DATA_THREAD, true);
+
+	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
+				 CDNS_XSPI_ACMD_DATA_THREAD);
+	if (ret) {
+		dev_err(cdns_xspi->dev, "ACMD read failed: %d\n", ret);
+		goto out_clear_read_state;
+	}
+
+	memcpy(op->data.buf.in, cdns_xspi->dma_buf, op->data.nbytes);
+
+out_clear_read_state:
+	cdns_xspi->acmd_info.row_addr_valid = false;
+	cdns_xspi->acmd_info.row_addr = 0;
+	return ret;
+}
+
+static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
+				      struct spi_mem *mem,
+				      const struct spi_mem_op *op)
+{
+	struct spinand_device *spinand;
+	const struct spi_mem_op *read_cache;
+	const struct spi_mem_op *write_cache;
+	const struct spi_mem_op *update_cache;
+	int ret;
+
+	if (cdns_xspi->flash_type != CDNS_XSPI_FLASH_TYPE_NAND)
+		goto use_stig;
+
+	spinand = spi_mem_get_drvdata(mem);
+	if (!spinand || !spinand->op_templates ||
+	    !spinand->op_templates->read_cache ||
+	    !spinand->op_templates->write_cache ||
+	    !spinand->op_templates->update_cache)
+		goto use_stig;
+
+	if (!cdns_xspi->acmd_info.initialized) {
+		ret = cdns_xspi_nand_init(cdns_xspi, spinand);
+		if (ret) {
+			dev_err(cdns_xspi->dev,
+				"failed to initialize ACMD: %d\n", ret);
+			return ret;
+		}
+	}
+
+	read_cache = spinand->op_templates->read_cache;
+	write_cache = spinand->op_templates->write_cache;
+	update_cache = spinand->op_templates->update_cache;
+
+	if ((write_cache && op->cmd.opcode == write_cache->cmd.opcode) ||
+	    (update_cache && op->cmd.opcode == update_cache->cmd.opcode)) {
+		if (op->data.dir != SPI_MEM_DATA_OUT || !op->data.nbytes ||
+		    !op->data.buf.out)
+			return -EINVAL;
+
+		cdns_xspi_nand_write_enable_seq_init(cdns_xspi);
+		cdns_xspi->out_buffer = op->data.buf.out;
+		cdns_xspi->acmd_info.column_addr = op->addr.val;
+		cdns_xspi->acmd_info.data_nbytes = op->data.nbytes;
+		return 0;
+	}
+
+	switch (op->cmd.opcode) {
+	case CDNS_XSPI_NAND_OP_PAGE_READ:
+		cdns_xspi->acmd_info.row_addr = op->addr.val;
+		cdns_xspi->acmd_info.row_addr_valid = true;
+		return 0;
+
+	case CDNS_XSPI_NAND_OP_GET_FEATURE:
+		if (op->addr.val != CDNS_XSPI_NAND_STATUS_REG ||
+		    !cdns_xspi->acmd_info.row_addr_valid)
+			break;
+
+		if (op->data.dir != SPI_MEM_DATA_IN || !op->data.nbytes ||
+		    !op->data.buf.in)
+			return -EINVAL;
+
+		memset(op->data.buf.in, 0, op->data.nbytes);
+		return 0;
+
+	case CDNS_XSPI_NAND_OP_RESET:
+		return cdns_xspi_pio_reset(cdns_xspi);
+
+	case CDNS_XSPI_NAND_OP_BLOCK_ERASE:
+		return cdns_xspi_pio_mdma_erase(cdns_xspi, spinand, op);
+
+	case CDNS_XSPI_NAND_OP_PROGRAM_EXECUTE:
+		return cdns_xspi_pio_mdma_program(cdns_xspi, spinand, op);
+
+	default:
+		break;
+	}
+
+	if (read_cache && op->cmd.opcode == read_cache->cmd.opcode)
+		return cdns_xspi_pio_mdma_read(cdns_xspi, spinand, op);
+
+use_stig:
+	/* ACMD does not consume this operation; execute it through STIG. */
+	return cdns_xspi_send_stig_command(cdns_xspi, op,
+					   op->data.dir != SPI_MEM_NO_DATA);
+}
+
 static int cdns_xspi_mem_op(struct cdns_xspi_dev *cdns_xspi,
 			    struct spi_mem *mem,
 			    const struct spi_mem_op *op)
@@ -589,6 +1361,9 @@ static int cdns_xspi_mem_op(struct cdns_xspi_dev *cdns_xspi,
 	if (cdns_xspi->cur_cs != spi_get_chipselect(mem->spi, 0))
 		cdns_xspi->cur_cs = spi_get_chipselect(mem->spi, 0);
 
+	if (cdns_xspi->work_mode == CDNS_XSPI_WORK_MODE_ACMD)
+		return cdns_xspi_send_pio_command(cdns_xspi, mem, op);
+
 	return cdns_xspi_send_stig_command(cdns_xspi, op,
 					   (dir != SPI_MEM_NO_DATA));
 }
@@ -1217,6 +1992,9 @@ static int cdns_xspi_probe(struct platform_device *pdev)
 	cdns_xspi->host = host;
 	cdns_xspi->dev = &pdev->dev;
 	cdns_xspi->cur_cs = 0;
+	cdns_xspi->flash_type = cdns_xspi->driver_data->flash_type;
+	cdns_xspi->work_mode = cdns_xspi->driver_data->use_acmd ?
+		CDNS_XSPI_WORK_MODE_ACMD : CDNS_XSPI_WORK_MODE_STIG;
 
 	init_completion(&cdns_xspi->cmd_complete);
 	init_completion(&cdns_xspi->auto_cmd_complete);
@@ -1344,6 +2122,10 @@ static const struct of_device_id cdns_xspi_of_match[] = {
 		.data = &marvell_driver_data,
 	},
 #endif
+	{
+		.compatible = "cdns,xspi-nand",
+		.data = &cdns_nand_driver_data,
+	},
 	{ /* end of table */}
 };
 MODULE_DEVICE_TABLE(of, cdns_xspi_of_match);
-- 
2.34.1

Re: [RFC PATCH 2/4] spi: cadence-xspi: add ACMD support for SPI NAND

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 3943 bytes --]

On Mon, Sep 21, 2026 at 05:36:59PM +0800, Fei Xie wrote:

> Add PIO master-DMA support for the controller auto-command mode.
> Configure the SPI NAND read, program, erase, status and reset
> sequences, retaining STIG for operations not consumed by ACMD.

> +struct cdns_xspi_acmd_info {
> +	u64 row_addr;
> +	u64 column_addr;
> +	size_t data_nbytes;
> +	bool row_addr_valid;
> +	bool initialized;
> +};
>  struct cdns_xspi_dev {

Missing blank line.

> +static void cdns_xspi_nand_erase_seq_init(struct cdns_xspi_dev *cdns_xspi,
> +					  struct spinand_device *spinand)
> +{
> +	u32 erase_seq_cfg0;
> +	u32 erase_seq_cfg1;
> +
> +	/* SPI-NAND block erase is always D8h with a 3-byte row address. */
> +	erase_seq_cfg0 =
> +		FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_VAL, 0xd8) |

CDNS_XSPI_NAND_OP_BLOCK_ERASE.

> +static int cdns_xspi_nand_init(struct cdns_xspi_dev *cdns_xspi,
> +			       struct spinand_device *spinand)
> +{

How much of this initialisation is parameters based on the specific
flash passed in - what if there are two different flashes attached to
the same controller for some reason?

> +	cdns_xspi->dma_buf_len = spinand->base.memorg.pagesize +
> +				 spinand->base.memorg.oobsize;
> +	cdns_xspi->dma_buf = dmam_alloc_coherent(cdns_xspi->dev,
> +						 cdns_xspi->dma_buf_len,
> +						 &cdns_xspi->dma_addr,
> +						 GFP_KERNEL);
> +	if (!cdns_xspi->dma_buf)
> +		return -ENOMEM;

Why do we need the DMA buffer, I'd expect whatever reaches the driver to
already be DMA safe?

> +static int cdns_xspi_acmd_run(struct cdns_xspi_dev *cdns_xspi, u32 cmd_regs[6],
> +			      u32 thread)
> +{
> +	unsigned long timeout;
> +	int ret;
> +
> +	cdns_xspi_set_mode_acmd(cdns_xspi);
> +	reinit_completion(&cdns_xspi->auto_cmd_complete);
> +	cdns_xspi_set_interrupts(cdns_xspi, true);

We have the set_interrupts() operation.

> +	cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
> +
> +	timeout = msecs_to_jiffies(CDNS_XSPI_ACMD_TIMEOUT_MS);
> +	if (!wait_for_completion_timeout(&cdns_xspi->auto_cmd_complete,
> +					 timeout)) {
> +		dev_err(cdns_xspi->dev, "ACMD command timed out\n");
> +		ret = -ETIMEDOUT;

Don't we need to clean up the hardware if this times out?

> +	} else {
> +		ret = cdns_xspi_acmd_get_thread_status(cdns_xspi, thread);
> +	}

Does this need any updates to cover more error types?

> +static int cdns_xspi_pio_mdma_read(struct cdns_xspi_dev *cdns_xspi,
> +				   struct spinand_device *spinand,
> +				   const struct spi_mem_op *op)
> +{

> +	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
> +				 CDNS_XSPI_ACMD_DATA_THREAD);
> +	if (ret) {
> +		dev_err(cdns_xspi->dev, "ACMD read failed: %d\n", ret);
> +		goto out_clear_read_state;
> +	}
> +
> +	memcpy(op->data.buf.in, cdns_xspi->dma_buf, op->data.nbytes);
> +
> +out_clear_read_state:
> +	cdns_xspi->acmd_info.row_addr_valid = false;
> +	cdns_xspi->acmd_info.row_addr = 0;
> +	return ret;
> +}

Does this do the right thing for short reads?

> +static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
> +				      struct spi_mem *mem,
> +				      const struct spi_mem_op *op)
> +{
> +	struct spinand_device *spinand;
> +	const struct spi_mem_op *read_cache;
> +	const struct spi_mem_op *write_cache;
> +	const struct spi_mem_op *update_cache;
> +	int ret;
> +
> +	if (cdns_xspi->flash_type != CDNS_XSPI_FLASH_TYPE_NAND)
> +		goto use_stig;
> +
> +	spinand = spi_mem_get_drvdata(mem);

This appears to be peering into the child's driver data without any
checking that the child is what we expect, this could go horribly wrong.

> +	case CDNS_XSPI_NAND_OP_GET_FEATURE:
> +		if (op->addr.val != CDNS_XSPI_NAND_STATUS_REG ||
> +		    !cdns_xspi->acmd_info.row_addr_valid)
> +			break;
> +
> +		if (op->data.dir != SPI_MEM_DATA_IN || !op->data.nbytes ||
> +		    !op->data.buf.in)
> +			return -EINVAL;
> +
> +		memset(op->data.buf.in, 0, op->data.nbytes);
> +		return 0;

Are you sure the zeroing makes sense here, for example with ECC?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[RFC PATCH 3/4] spi: cadence-xspi: factor out reusable ACMD helpers

[Fei Xie]

From: "fei.xie" <fei.xie@horizon.auto>

Move DMA buffer allocation, XIP setup and PIO command execution into
common helpers.

Keep SPI NAND address conversion and operation state handling in
NAND-specific wrappers. This is a behavior-neutral preparation for
sharing the ACMD PIO and MDMA path with other flash types.

Signed-off-by: fei.xie <fei.xie@horizon.auto>
---
 drivers/spi/spi-cadence-xspi.c | 223 ++++++++++++++++++++-------------
 1 file changed, 136 insertions(+), 87 deletions(-)

diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c
index 0fedf6cd6a590..114c1e0fd79fb 100644
--- a/drivers/spi/spi-cadence-xspi.c
+++ b/drivers/spi/spi-cadence-xspi.c
@@ -822,10 +822,35 @@ static void cdns_xspi_nand_reset_seq_init(struct cdns_xspi_dev *cdns_xspi)
 	writel(cfg, cdns_xspi->iobase + CDNS_XSPI_RST_SEQ_CFG_0);
 }
 
+static int cdns_xspi_acmd_alloc_dma_buf(struct cdns_xspi_dev *cdns_xspi,
+					size_t len)
+{
+	cdns_xspi->dma_buf = dmam_alloc_coherent(cdns_xspi->dev, len,
+						 &cdns_xspi->dma_addr,
+						 GFP_KERNEL);
+	if (!cdns_xspi->dma_buf)
+		return -ENOMEM;
+
+	cdns_xspi->dma_buf_len = len;
+
+	return 0;
+}
+
+static void cdns_xspi_acmd_disable_xip(struct cdns_xspi_dev *cdns_xspi)
+{
+	u32 reg_val;
+
+	reg_val = readl(cdns_xspi->iobase + CDNS_XSPI_XIP_MODE_CFG);
+	if (reg_val & CDNS_XSPI_XIP_EN) {
+		reg_val &= ~CDNS_XSPI_XIP_EN;
+		writel(reg_val, cdns_xspi->iobase + CDNS_XSPI_XIP_MODE_CFG);
+	}
+}
+
 static int cdns_xspi_nand_init(struct cdns_xspi_dev *cdns_xspi,
 			       struct spinand_device *spinand)
 {
-	u32 reg_val;
+	int ret;
 
 	cdns_xspi_nand_cfg_seq_init(cdns_xspi, spinand);
 	cdns_xspi_nand_read_seq_init(cdns_xspi, spinand);
@@ -835,19 +860,12 @@ static int cdns_xspi_nand_init(struct cdns_xspi_dev *cdns_xspi,
 	cdns_xspi_nand_status_seq_init(cdns_xspi);
 	cdns_xspi_nand_erase_seq_init(cdns_xspi, spinand);
 
-	reg_val = readl(cdns_xspi->iobase + CDNS_XSPI_XIP_MODE_CFG);
-	if (reg_val & CDNS_XSPI_XIP_EN) {
-		reg_val &= ~CDNS_XSPI_XIP_EN;
-		writel(reg_val, cdns_xspi->iobase + CDNS_XSPI_XIP_MODE_CFG);
-	}
-	cdns_xspi->dma_buf_len = spinand->base.memorg.pagesize +
-				 spinand->base.memorg.oobsize;
-	cdns_xspi->dma_buf = dmam_alloc_coherent(cdns_xspi->dev,
-						 cdns_xspi->dma_buf_len,
-						 &cdns_xspi->dma_addr,
-						 GFP_KERNEL);
-	if (!cdns_xspi->dma_buf)
-		return -ENOMEM;
+	cdns_xspi_acmd_disable_xip(cdns_xspi);
+	ret = cdns_xspi_acmd_alloc_dma_buf(cdns_xspi,
+					   spinand->base.memorg.pagesize +
+					   spinand->base.memorg.oobsize);
+	if (ret)
+		return ret;
 
 	cdns_xspi->acmd_info.initialized = true;
 
@@ -1070,12 +1088,10 @@ static u32 cdns_xspi_acmd_cmd(struct cdns_xspi_dev *cdns_xspi, u32 type,
 	       FIELD_PREP(CDNS_XSPI_ACMD_CMD_TYPE, type);
 }
 
-static int cdns_xspi_pio_mdma_erase(struct cdns_xspi_dev *cdns_xspi,
-				    struct spinand_device *spinand,
-				    const struct spi_mem_op *op)
+static int cdns_xspi_pio_erase(struct cdns_xspi_dev *cdns_xspi,
+			       u64 xspi_addr)
 {
 	u32 cmd_regs[6] = { 0 };
-	u64 xspi_addr;
 	int ret;
 
 	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
@@ -1085,10 +1101,6 @@ static int cdns_xspi_pio_mdma_erase(struct cdns_xspi_dev *cdns_xspi,
 		return ret;
 	}
 
-	ret = cdns_xspi_nand_addr(spinand, op->addr.val, 0, &xspi_addr);
-	if (ret)
-		return ret;
-
 	if (upper_32_bits(xspi_addr)) {
 		dev_err(cdns_xspi->dev,
 			"erase address 0x%llx exceeds the ACMD PIO range\n",
@@ -1137,71 +1149,141 @@ static int cdns_xspi_pio_reset(struct cdns_xspi_dev *cdns_xspi)
 }
 
 static int cdns_xspi_pio_mdma_program(struct cdns_xspi_dev *cdns_xspi,
-				      struct spinand_device *spinand,
-				      const struct spi_mem_op *op)
+				      u64 xspi_addr, const void *buf,
+				      size_t nbytes)
 {
 	u32 cmd_regs[6] = { 0 };
-	u64 xspi_addr;
 	int ret;
 
-	if (!cdns_xspi->out_buffer || !cdns_xspi->acmd_info.data_nbytes) {
-		dev_err(cdns_xspi->dev, "missing program-load data\n");
-		ret = -EINVAL;
-		goto out_clear_program_state;
-	}
+	if (!buf || !nbytes)
+		return -EINVAL;
 
-	if (cdns_xspi->acmd_info.data_nbytes > cdns_xspi->dma_buf_len) {
-		ret = -EMSGSIZE;
-		goto out_clear_program_state;
+	if (nbytes > cdns_xspi->dma_buf_len)
+		return -EMSGSIZE;
+
+	if (upper_32_bits(xspi_addr)) {
+		dev_err(cdns_xspi->dev,
+			"program address 0x%llx exceeds the ACMD PIO range\n",
+			xspi_addr);
+		return -ERANGE;
 	}
 
-	memcpy(cdns_xspi->dma_buf, cdns_xspi->out_buffer,
-	       cdns_xspi->acmd_info.data_nbytes);
+	memcpy(cdns_xspi->dma_buf, buf, nbytes);
 
 	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
 	if (ret) {
 		dev_err(cdns_xspi->dev,
 			"controller did not become idle before program\n");
-		goto out_clear_program_state;
+		return ret;
 	}
 
-	ret = cdns_xspi_nand_addr(spinand, op->addr.val,
-				  cdns_xspi->acmd_info.column_addr,
-				  &xspi_addr);
-	if (ret)
-		goto out_clear_program_state;
+	cmd_regs[1] = lower_32_bits(xspi_addr);
+	cmd_regs[2] = lower_32_bits(cdns_xspi->dma_addr);
+	cmd_regs[3] = upper_32_bits(cdns_xspi->dma_addr);
+	cmd_regs[4] = nbytes - 1;
+	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
+					 CDNS_XSPI_ACMD_PROG_OP,
+					 CDNS_XSPI_ACMD_DATA_THREAD, true);
+
+	return cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
+				  CDNS_XSPI_ACMD_DATA_THREAD);
+}
+
+static int cdns_xspi_pio_mdma_read(struct cdns_xspi_dev *cdns_xspi,
+				   u64 xspi_addr, void *buf, size_t nbytes)
+{
+	u32 cmd_regs[6] = { 0 };
+	int ret;
+
+	if (!buf || !nbytes)
+		return -EINVAL;
+
+	if (nbytes > cdns_xspi->dma_buf_len)
+		return -EMSGSIZE;
+
+	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
+	if (ret) {
+		dev_err(cdns_xspi->dev,
+			"controller did not become idle before read\n");
+		return ret;
+	}
 
 	if (upper_32_bits(xspi_addr)) {
 		dev_err(cdns_xspi->dev,
-			"program address 0x%llx exceeds the ACMD PIO range\n",
+			"read address 0x%llx exceeds the ACMD PIO range\n",
 			xspi_addr);
-		ret = -ERANGE;
-		goto out_clear_program_state;
+		return -ERANGE;
 	}
 
 	cmd_regs[1] = lower_32_bits(xspi_addr);
 	cmd_regs[2] = lower_32_bits(cdns_xspi->dma_addr);
 	cmd_regs[3] = upper_32_bits(cdns_xspi->dma_addr);
-	cmd_regs[4] = cdns_xspi->acmd_info.data_nbytes - 1;
+	cmd_regs[4] = nbytes - 1;
 	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
-					 CDNS_XSPI_ACMD_PROG_OP,
+					 CDNS_XSPI_ACMD_READ_OP,
 					 CDNS_XSPI_ACMD_DATA_THREAD, true);
 
 	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
 				 CDNS_XSPI_ACMD_DATA_THREAD);
+	if (ret) {
+		dev_err(cdns_xspi->dev, "ACMD read failed: %d\n", ret);
+		return ret;
+	}
+
+	memcpy(buf, cdns_xspi->dma_buf, nbytes);
+
+	return 0;
+}
+
+static int cdns_xspi_nand_pio_erase(struct cdns_xspi_dev *cdns_xspi,
+				    struct spinand_device *spinand,
+				    const struct spi_mem_op *op)
+{
+	u64 xspi_addr;
+	int ret;
+
+	ret = cdns_xspi_nand_addr(spinand, op->addr.val, 0, &xspi_addr);
+	if (ret)
+		return ret;
+
+	return cdns_xspi_pio_erase(cdns_xspi, xspi_addr);
+}
+
+static int cdns_xspi_nand_pio_program(struct cdns_xspi_dev *cdns_xspi,
+				      struct spinand_device *spinand,
+				      const struct spi_mem_op *op)
+{
+	u64 xspi_addr;
+	int ret;
+
+	if (!cdns_xspi->out_buffer || !cdns_xspi->acmd_info.data_nbytes) {
+		dev_err(cdns_xspi->dev, "missing program-load data\n");
+		ret = -EINVAL;
+		goto out_clear_program_state;
+	}
+
+	ret = cdns_xspi_nand_addr(spinand, op->addr.val,
+				  cdns_xspi->acmd_info.column_addr,
+				  &xspi_addr);
+	if (ret)
+		goto out_clear_program_state;
+
+	ret = cdns_xspi_pio_mdma_program(cdns_xspi, xspi_addr,
+					 cdns_xspi->out_buffer,
+					 cdns_xspi->acmd_info.data_nbytes);
 
 out_clear_program_state:
 	cdns_xspi->acmd_info.column_addr = 0;
 	cdns_xspi->acmd_info.data_nbytes = 0;
 	cdns_xspi->out_buffer = NULL;
+
 	return ret;
 }
 
-static int cdns_xspi_pio_mdma_read(struct cdns_xspi_dev *cdns_xspi,
+static int cdns_xspi_nand_pio_read(struct cdns_xspi_dev *cdns_xspi,
 				   struct spinand_device *spinand,
 				   const struct spi_mem_op *op)
 {
-	u32 cmd_regs[6] = { 0 };
 	u64 xspi_addr;
 	int ret;
 
@@ -1217,52 +1299,19 @@ static int cdns_xspi_pio_mdma_read(struct cdns_xspi_dev *cdns_xspi,
 		goto out_clear_read_state;
 	}
 
-	if (op->data.nbytes > cdns_xspi->dma_buf_len) {
-		ret = -EMSGSIZE;
-		goto out_clear_read_state;
-	}
-
-	ret = cdns_xspi_wait_for_controller_idle(cdns_xspi);
-	if (ret) {
-		dev_err(cdns_xspi->dev,
-			"controller did not become idle before read\n");
-		goto out_clear_read_state;
-	}
-
 	ret = cdns_xspi_nand_addr(spinand,
 				  cdns_xspi->acmd_info.row_addr,
 				  op->addr.val, &xspi_addr);
 	if (ret)
 		goto out_clear_read_state;
 
-	if (upper_32_bits(xspi_addr)) {
-		dev_err(cdns_xspi->dev,
-			"read address 0x%llx exceeds the ACMD PIO range\n",
-			xspi_addr);
-		ret = -ERANGE;
-		goto out_clear_read_state;
-	}
-
-	cmd_regs[1] = lower_32_bits(xspi_addr);
-	cmd_regs[2] = lower_32_bits(cdns_xspi->dma_addr);
-	cmd_regs[3] = upper_32_bits(cdns_xspi->dma_addr);
-	cmd_regs[4] = op->data.nbytes - 1;
-	cmd_regs[0] = cdns_xspi_acmd_cmd(cdns_xspi,
-					 CDNS_XSPI_ACMD_READ_OP,
-					 CDNS_XSPI_ACMD_DATA_THREAD, true);
-
-	ret = cdns_xspi_acmd_run(cdns_xspi, cmd_regs,
-				 CDNS_XSPI_ACMD_DATA_THREAD);
-	if (ret) {
-		dev_err(cdns_xspi->dev, "ACMD read failed: %d\n", ret);
-		goto out_clear_read_state;
-	}
-
-	memcpy(op->data.buf.in, cdns_xspi->dma_buf, op->data.nbytes);
+	ret = cdns_xspi_pio_mdma_read(cdns_xspi, xspi_addr,
+				      op->data.buf.in, op->data.nbytes);
 
 out_clear_read_state:
 	cdns_xspi->acmd_info.row_addr_valid = false;
 	cdns_xspi->acmd_info.row_addr = 0;
+
 	return ret;
 }
 
@@ -1334,17 +1383,17 @@ static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
 		return cdns_xspi_pio_reset(cdns_xspi);
 
 	case CDNS_XSPI_NAND_OP_BLOCK_ERASE:
-		return cdns_xspi_pio_mdma_erase(cdns_xspi, spinand, op);
+		return cdns_xspi_nand_pio_erase(cdns_xspi, spinand, op);
 
 	case CDNS_XSPI_NAND_OP_PROGRAM_EXECUTE:
-		return cdns_xspi_pio_mdma_program(cdns_xspi, spinand, op);
+		return cdns_xspi_nand_pio_program(cdns_xspi, spinand, op);
 
 	default:
 		break;
 	}
 
 	if (read_cache && op->cmd.opcode == read_cache->cmd.opcode)
-		return cdns_xspi_pio_mdma_read(cdns_xspi, spinand, op);
+		return cdns_xspi_nand_pio_read(cdns_xspi, spinand, op);
 
 use_stig:
 	/* ACMD does not consume this operation; execute it through STIG. */
-- 
2.34.1

[RFC PATCH 4/4] spi: cadence-xspi: add ACMD support for SPI NOR

[Fei Xie]

From: "fei.xie" <fei.xie@horizon.auto>

Program the automatic command sequences from the SPI NOR operation
templates.

Use the PIO command path with MDMA data transfers for reads, page
programs and erases. Unsupported command layouts continue to use STIG
mode.

Signed-off-by: fei.xie <fei.xie@horizon.auto>
---
 drivers/spi/spi-cadence-xspi.c | 440 ++++++++++++++++++++++++++++++++-
 1 file changed, 433 insertions(+), 7 deletions(-)

diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c
index 114c1e0fd79fb..5900d4cbffb5c 100644
--- a/drivers/spi/spi-cadence-xspi.c
+++ b/drivers/spi/spi-cadence-xspi.c
@@ -12,11 +12,13 @@
 #include <linux/iopoll.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/mtd/spi-nor.h>
 #include <linux/mtd/spinand.h>
 #include <linux/overflow.h>
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
 #include <linux/property.h>
+#include <linux/sizes.h>
 #include <linux/spi/spi.h>
 #include <linux/spi/spi-mem.h>
 #include <linux/bitfield.h>
@@ -73,6 +75,7 @@
 #define CDNS_XSPI_ACMD_ERASE_THREAD		5
 #define CDNS_XSPI_ACMD_TIMEOUT_MS		1000
 #define CDNS_XSPI_ACMD_MODE_PIO			1
+#define CDNS_XSPI_NOR_DMA_BUF_SIZE		SZ_32K
 
 #define CDNS_XSPI_NAND_OP_GET_FEATURE		0x0f
 #define CDNS_XSPI_NAND_OP_WRITE_ENABLE		0x06
@@ -225,6 +228,7 @@
 #define CDNS_XSPI_READ_SEQ_CFG_1			0x0434
 #define CDNS_XSPI_READ_SEQ_P1_CMD_EXT_EN		BIT(0)
 #define CDNS_XSPI_READ_SEQ_P1_CACHE_RANDOM_READ_EN	BIT(4)
+#define CDNS_XSPI_READ_SEQ_P1_CMD_EXT_VAL	GENMASK(15, 8)
 #define CDNS_XSPI_READ_SEQ_CFG_2			0x0438
 
 /* Write enable sequence config register */
@@ -232,12 +236,22 @@
 #define CDNS_XSPI_WE_SEQ_P1_CMD_VAL		GENMASK(7, 0)
 #define CDNS_XSPI_WE_SEQ_P1_CMD_IOS		GENMASK(9, 8)
 #define CDNS_XSPI_WE_SEQ_P1_CMD_EDGE		BIT(11)
+#define CDNS_XSPI_WE_SEQ_P1_CMD_EXT_EN		BIT(15)
+#define CDNS_XSPI_WE_SEQ_P1_CMD_EXT_VAL		GENMASK(23, 16)
 #define CDNS_XSPI_WE_SEQ_P1_EN			BIT(24)
 
 /* Status sequence config registers */
 #define CDNS_XSPI_STAT_SEQ_CFG_0			0x0450
+#define CDNS_XSPI_STAT_SEQ_P1_CMD_IOS		GENMASK(1, 0)
+#define CDNS_XSPI_STAT_SEQ_P1_CMD_EDGE		BIT(4)
+#define CDNS_XSPI_STAT_SEQ_P1_CMD_EXT_EN	BIT(5)
 #define CDNS_XSPI_STAT_SEQ_P1_ADDR_CNT			GENMASK(9, 8)
+#define CDNS_XSPI_STAT_SEQ_P1_ADDR_IOS		GENMASK(11, 10)
+#define CDNS_XSPI_STAT_SEQ_P1_ADDR_EDGE		BIT(12)
+#define CDNS_XSPI_STAT_SEQ_P1_DATA_IOS		GENMASK(21, 20)
+#define CDNS_XSPI_STAT_SEQ_P1_DATA_EDGE		BIT(22)
 #define CDNS_XSPI_STAT_SEQ_CFG_1			0x0454
+#define CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_DUMMY_CNT	GENMASK(5, 0)
 #define CDNS_XSPI_P1_DEV_RDY_ADDR_EN		BIT(6)
 #define CDNS_XSPI_P1_PROG_FAIL_ADDR_EN		BIT(22)
 #define CDNS_XSPI_P1_ERS_FAIL_ADDR_EN		BIT(30)
@@ -248,9 +262,11 @@
 #define CDNS_XSPI_STAT_SEQ_P1_PROG_FAIL_CMD_VAL	GENMASK(31, 24)
 
 #define CDNS_XSPI_STAT_SEQ_CFG_3			0x045c
+#define CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_CMD_EXT_VAL	GENMASK(7, 0)
 #define CDNS_XSPI_STAT_SEQ_CFG_4			0x0460
 #define CDNS_XSPI_STAT_SEQ_CFG_5			0x0464
 #define CDNS_XSPI_STAT_SEQ_DEV_RDY_IDX		GENMASK(3, 0)
+#define CDNS_XSPI_STAT_SEQ_DEV_RDY_SIZE		BIT(5)
 #define CDNS_XSPI_STAT_SEQ_DEV_RDY_EN		BIT(6)
 #define CDNS_XSPI_STAT_SEQ_ERS_FAIL_IDX		GENMASK(11, 8)
 #define CDNS_XSPI_STAT_SEQ_ERS_FAIL_VAL		BIT(12)
@@ -288,6 +304,7 @@
 #define CDNS_XSPI_SEQ_PAGE_PER_BLOCK		GENMASK(26, 24)
 #define CDNS_XSPI_SEQ_PAGE_CA_SIZE		BIT(16)
 #define CDNS_XSPI_SEQ_PAGE_SIZE_EXT		GENMASK(8, 0)
+#define CDNS_XSPI_SEQ_SPI_NOR			0
 
 #define CDNS_XSPI_XIP_MODE_CFG			0x0388
 #define CDNS_XSPI_XIP_EN			BIT(0)
@@ -476,7 +493,13 @@ struct cdns_xspi_acmd_info {
 	u64 row_addr;
 	u64 column_addr;
 	size_t data_nbytes;
+	u8 nor_status_buswidth;
+	u8 nor_write_enable_buswidth;
 	bool row_addr_valid;
+	bool nor_status_dtr;
+	bool nor_write_enable_dtr;
+	bool nor_status_initialized;
+	bool nor_write_enable_initialized;
 	bool initialized;
 };
 struct cdns_xspi_dev {
@@ -592,6 +615,31 @@ static void cdns_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi,
 	writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG);
 }
 
+static u8 cdns_xspi_get_opcode(const struct spi_mem_op *op)
+{
+	if (op->cmd.nbytes == 2)
+		return op->cmd.opcode >> 8;
+
+	return op->cmd.opcode;
+}
+
+static u8 cdns_xspi_get_opcode_extension(const struct spi_mem_op *op)
+{
+	if (op->cmd.nbytes == 2)
+		return op->cmd.opcode;
+
+	return 0;
+}
+
+static u32 cdns_xspi_get_dummy_cycles(const struct spi_mem_op *op)
+{
+	if (!op->dummy.nbytes || !op->dummy.buswidth)
+		return 0;
+
+	return op->dummy.nbytes * BITS_PER_BYTE /
+	       (op->dummy.buswidth * (op->dummy.dtr + 1));
+}
+
 static void cdns_xspi_nand_cfg_seq_init(struct cdns_xspi_dev *cdns_xspi,
 					struct spinand_device *spinand)
 {
@@ -872,6 +920,268 @@ static int cdns_xspi_nand_init(struct cdns_xspi_dev *cdns_xspi,
 	return 0;
 }
 
+static int cdns_xspi_nor_cfg_seq_init(struct cdns_xspi_dev *cdns_xspi,
+				      struct spi_nor *nor)
+{
+	unsigned int page_size;
+	u32 seq_cfg;
+
+	if (!is_power_of_2(nor->mtd.writebufsize))
+		return -EOPNOTSUPP;
+
+	page_size = ilog2(nor->mtd.writebufsize);
+	if (!FIELD_FIT(CDNS_XSPI_SEQ_PAGE_SIZE_PGM, page_size))
+		return -EOPNOTSUPP;
+
+	seq_cfg = readl(cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG);
+	seq_cfg = u32_replace_bits(seq_cfg, CDNS_XSPI_SEQ_SPI_NOR,
+				   CDNS_XSPI_SEQ_TYPE);
+	seq_cfg = u32_replace_bits(seq_cfg, page_size,
+				   CDNS_XSPI_SEQ_PAGE_SIZE_PGM);
+	seq_cfg = u32_replace_bits(seq_cfg,
+				   ilog2(CDNS_XSPI_NOR_DMA_BUF_SIZE),
+				   CDNS_XSPI_SEQ_PAGE_SIZE_RD);
+	writel(seq_cfg, cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_GLOBAL_SEQ_CFG_1);
+
+	return 0;
+}
+
+static int cdns_xspi_nor_read_seq_init(struct cdns_xspi_dev *cdns_xspi,
+				       const struct spi_mem_op *op)
+{
+	u32 dummy_cycles = cdns_xspi_get_dummy_cycles(op);
+	u32 cfg0;
+	u32 cfg1;
+
+	if (!FIELD_FIT(CDNS_XSPI_READ_SEQ_P1_ADDR_CNT, op->addr.nbytes) ||
+	    !FIELD_FIT(CDNS_XSPI_READ_SEQ_P1_DUMMY_CNT, dummy_cycles))
+		return -EOPNOTSUPP;
+
+	cfg0 = FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_VAL,
+			  cdns_xspi_get_opcode(op)) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_IOS,
+			  ilog2(op->cmd.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_ADDR_CNT, op->addr.nbytes) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_ADDR_IOS,
+			  ilog2(op->addr.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_ADDR_EDGE, op->addr.dtr) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_DATA_IOS,
+			  ilog2(op->data.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_DATA_EDGE, op->data.dtr) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_DUMMY_CNT, dummy_cycles);
+	cfg1 = FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_EXT_EN,
+			  op->cmd.nbytes == 2) |
+	       FIELD_PREP(CDNS_XSPI_READ_SEQ_P1_CMD_EXT_VAL,
+			  cdns_xspi_get_opcode_extension(op));
+
+	writel(cfg0, cdns_xspi->iobase + CDNS_XSPI_READ_SEQ_CFG_0);
+	writel(cfg1, cdns_xspi->iobase + CDNS_XSPI_READ_SEQ_CFG_1);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_READ_SEQ_CFG_2);
+
+	return 0;
+}
+
+static int cdns_xspi_nor_program_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					  const struct spi_mem_op *op)
+{
+	u32 dummy_cycles = cdns_xspi_get_dummy_cycles(op);
+	u32 cfg0;
+	u32 cfg1;
+
+	if (!FIELD_FIT(CDNS_XSPI_PROG_SEQ_P1_ADDR_CNT, op->addr.nbytes) ||
+	    !FIELD_FIT(CDNS_XSPI_PROG_SEQ_P1_DUMMY_CNT, dummy_cycles))
+		return -EOPNOTSUPP;
+
+	cfg0 = FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_VAL,
+			  cdns_xspi_get_opcode(op)) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_IOS,
+			  ilog2(op->cmd.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_ADDR_CNT, op->addr.nbytes) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_ADDR_IOS,
+			  ilog2(op->addr.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_ADDR_EDGE, op->addr.dtr) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_DATA_IOS,
+			  ilog2(op->data.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_DATA_EDGE, op->data.dtr) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_DUMMY_CNT, dummy_cycles);
+	cfg1 = FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_EXT_EN,
+			  op->cmd.nbytes == 2) |
+	       FIELD_PREP(CDNS_XSPI_PROG_SEQ_P1_CMD_EXT_VAL,
+			  cdns_xspi_get_opcode_extension(op));
+
+	writel(cfg0, cdns_xspi->iobase + CDNS_XSPI_PROG_SEQ_CFG_0);
+	writel(cfg1, cdns_xspi->iobase + CDNS_XSPI_PROG_SEQ_CFG_1);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_PROG_SEQ_CFG_2);
+
+	return 0;
+}
+
+static int cdns_xspi_nor_erase_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					struct spi_nor *nor,
+					const struct spi_mem_op *op)
+{
+	unsigned int erase_size;
+	u32 cfg0;
+
+	if (!is_power_of_2(nor->mtd.erasesize))
+		return -EOPNOTSUPP;
+
+	erase_size = ilog2(nor->mtd.erasesize);
+	if (!FIELD_FIT(CDNS_XSPI_ERSS_SEQ_P1_ADDR_CNT, op->addr.nbytes) ||
+	    !FIELD_FIT(CDNS_XSPI_ERSS_SEQ_P1_SECT_SIZE, erase_size))
+		return -EOPNOTSUPP;
+
+	cfg0 = FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_VAL,
+			  cdns_xspi_get_opcode(op)) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_IOS,
+			  ilog2(op->cmd.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_ADDR_CNT, op->addr.nbytes) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_EXT_EN,
+			  op->cmd.nbytes == 2) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_CMD_EXT_VAL,
+			  cdns_xspi_get_opcode_extension(op)) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_ADDR_IOS,
+			  ilog2(op->addr.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_ADDR_EDGE, op->addr.dtr);
+
+	writel(cfg0, cdns_xspi->iobase + CDNS_XSPI_ERSS_SEQ_CFG_0);
+	writel(FIELD_PREP(CDNS_XSPI_ERSS_SEQ_P1_SECT_SIZE, erase_size),
+	       cdns_xspi->iobase + CDNS_XSPI_ERSS_SEQ_CFG_1);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_ERSS_SEQ_CFG_2);
+
+	return 0;
+}
+
+static int cdns_xspi_nor_status_seq_init(struct cdns_xspi_dev *cdns_xspi,
+					 const struct spi_mem_op *op)
+{
+	u32 dummy_cycles = cdns_xspi_get_dummy_cycles(op);
+	u32 addr_count = op->addr.nbytes ? op->addr.nbytes - 1 : 0;
+	u32 cfg0;
+	u32 cfg1;
+	u32 cfg5;
+
+	if (op->data.dir != SPI_MEM_DATA_IN || !op->data.nbytes ||
+	    !FIELD_FIT(CDNS_XSPI_STAT_SEQ_P1_ADDR_CNT, addr_count) ||
+	    !FIELD_FIT(CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_DUMMY_CNT,
+		       dummy_cycles) || upper_32_bits(op->addr.val))
+		return -EOPNOTSUPP;
+
+	cfg0 = FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_CMD_IOS,
+			  ilog2(op->cmd.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_CMD_EXT_EN,
+			  op->cmd.nbytes == 2) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_ADDR_CNT, addr_count) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_ADDR_IOS,
+			  op->addr.buswidth ? ilog2(op->addr.buswidth) : 0) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_ADDR_EDGE, op->addr.dtr) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_DATA_IOS,
+			  ilog2(op->data.buswidth)) |
+	       FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_DATA_EDGE, op->data.dtr);
+	cfg1 = FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_DUMMY_CNT,
+			  dummy_cycles) |
+	       (op->addr.nbytes ? CDNS_XSPI_P1_DEV_RDY_ADDR_EN : 0);
+	cfg5 = FIELD_PREP(CDNS_XSPI_STAT_SEQ_DEV_RDY_IDX, __ffs(SR_WIP)) |
+	       (op->data.dtr ? CDNS_XSPI_STAT_SEQ_DEV_RDY_SIZE : 0) |
+	       CDNS_XSPI_STAT_SEQ_DEV_RDY_EN;
+
+	writel(cfg0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_0);
+	writel(cfg1, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_1);
+	writel(FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_CMD_VAL,
+			  cdns_xspi_get_opcode(op)),
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_2);
+	writel(FIELD_PREP(CDNS_XSPI_STAT_SEQ_P1_DEV_RDY_CMD_EXT_VAL,
+			  cdns_xspi_get_opcode_extension(op)),
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_3);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_4);
+	writel(cfg5, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_5);
+	writel(lower_32_bits(op->addr.val),
+	       cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_7);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_8);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_9);
+	writel(0, cdns_xspi->iobase + CDNS_XSPI_STAT_SEQ_CFG_10);
+
+	cdns_xspi->acmd_info.nor_status_initialized = true;
+	cdns_xspi->acmd_info.nor_status_buswidth = op->cmd.buswidth;
+	cdns_xspi->acmd_info.nor_status_dtr = op->cmd.dtr;
+
+	return 0;
+}
+
+static void
+cdns_xspi_nor_write_enable_seq_init(struct cdns_xspi_dev *cdns_xspi,
+				    const struct spi_mem_op *op)
+{
+	u32 cfg;
+
+	cfg = FIELD_PREP(CDNS_XSPI_WE_SEQ_P1_CMD_VAL,
+			 cdns_xspi_get_opcode(op)) |
+	      FIELD_PREP(CDNS_XSPI_WE_SEQ_P1_CMD_IOS,
+			 ilog2(op->cmd.buswidth)) |
+	      FIELD_PREP(CDNS_XSPI_WE_SEQ_P1_CMD_EDGE, op->cmd.dtr) |
+	      FIELD_PREP(CDNS_XSPI_WE_SEQ_P1_CMD_EXT_EN,
+			 op->cmd.nbytes == 2) |
+	      FIELD_PREP(CDNS_XSPI_WE_SEQ_P1_CMD_EXT_VAL,
+			 cdns_xspi_get_opcode_extension(op)) |
+	      CDNS_XSPI_WE_SEQ_P1_EN;
+	writel(cfg, cdns_xspi->iobase + CDNS_XSPI_WE_SEQ_CFG_0);
+	cdns_xspi->acmd_info.nor_write_enable_initialized = true;
+	cdns_xspi->acmd_info.nor_write_enable_buswidth = op->cmd.buswidth;
+	cdns_xspi->acmd_info.nor_write_enable_dtr = op->cmd.dtr;
+}
+
+static int cdns_xspi_nor_init(struct cdns_xspi_dev *cdns_xspi,
+			      struct spi_nor *nor)
+{
+	struct spi_mem_op status_op =
+		SPI_MEM_OP(SPI_MEM_OP_CMD(SPINOR_OP_RDSR, 1),
+			   SPI_MEM_OP_NO_ADDR,
+			   SPI_MEM_OP_NO_DUMMY,
+			   SPI_MEM_OP_DATA_IN(1, NULL, 1));
+	int ret;
+
+	ret = cdns_xspi_nor_cfg_seq_init(cdns_xspi, nor);
+	if (ret)
+		return ret;
+
+	cdns_xspi_acmd_disable_xip(cdns_xspi);
+	ret = cdns_xspi_acmd_alloc_dma_buf(cdns_xspi,
+					   CDNS_XSPI_NOR_DMA_BUF_SIZE);
+	if (ret)
+		return ret;
+
+	if (!spi_nor_protocol_is_dtr(nor->reg_proto) &&
+	    !cdns_xspi->acmd_info.nor_status_initialized) {
+		ret = cdns_xspi_nor_status_seq_init(cdns_xspi, &status_op);
+		if (ret)
+			return ret;
+	}
+
+	cdns_xspi->acmd_info.initialized = true;
+
+	return 0;
+}
+
+static bool
+cdns_xspi_nor_helper_sequences_ready(const struct cdns_xspi_dev *cdns_xspi,
+				     const struct spi_nor *nor)
+{
+	u8 buswidth = spi_nor_get_protocol_inst_nbits(nor->reg_proto);
+	bool dtr = spi_nor_protocol_is_dtr(nor->reg_proto);
+
+	return cdns_xspi->acmd_info.nor_status_initialized &&
+	       cdns_xspi->acmd_info.nor_status_buswidth == buswidth &&
+	       cdns_xspi->acmd_info.nor_status_dtr == dtr &&
+	       cdns_xspi->acmd_info.nor_write_enable_initialized &&
+	       cdns_xspi->acmd_info.nor_write_enable_buswidth == buswidth &&
+	       cdns_xspi->acmd_info.nor_write_enable_dtr == dtr;
+}
+
 static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi)
 {
 	u32 ctrl_ver;
@@ -1315,9 +1625,9 @@ static int cdns_xspi_nand_pio_read(struct cdns_xspi_dev *cdns_xspi,
 	return ret;
 }
 
-static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
-				      struct spi_mem *mem,
-				      const struct spi_mem_op *op)
+static int cdns_xspi_send_nand_pio_command(struct cdns_xspi_dev *cdns_xspi,
+					   struct spi_mem *mem,
+					   const struct spi_mem_op *op)
 {
 	struct spinand_device *spinand;
 	const struct spi_mem_op *read_cache;
@@ -1325,9 +1635,6 @@ static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
 	const struct spi_mem_op *update_cache;
 	int ret;
 
-	if (cdns_xspi->flash_type != CDNS_XSPI_FLASH_TYPE_NAND)
-		goto use_stig;
-
 	spinand = spi_mem_get_drvdata(mem);
 	if (!spinand || !spinand->op_templates ||
 	    !spinand->op_templates->read_cache ||
@@ -1401,6 +1708,115 @@ static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
 					   op->data.dir != SPI_MEM_NO_DATA);
 }
 
+static int cdns_xspi_send_nor_pio_command(struct cdns_xspi_dev *cdns_xspi,
+					  struct spi_mem *mem,
+					  const struct spi_mem_op *op)
+{
+	struct spi_nor *nor = spi_mem_get_drvdata(mem);
+	u8 opcode = cdns_xspi_get_opcode(op);
+	int ret;
+
+	if (!nor)
+		goto use_stig;
+
+	if (opcode == SPINOR_OP_WREN) {
+		cdns_xspi_nor_write_enable_seq_init(cdns_xspi, op);
+		goto use_stig;
+	}
+
+	if (opcode == SPINOR_OP_RDSR && op->data.dir == SPI_MEM_DATA_IN) {
+		cdns_xspi->acmd_info.nor_status_initialized = false;
+		ret = cdns_xspi_nor_status_seq_init(cdns_xspi, op);
+		if (ret)
+			dev_dbg(cdns_xspi->dev,
+				"cannot use RDSR operation for ACMD status: %d\n",
+				ret);
+		goto use_stig;
+	}
+
+	if (!nor->mtd.size || !nor->mtd.writebufsize)
+		goto use_stig;
+
+	if (opcode != nor->read_opcode && opcode != nor->program_opcode &&
+	    opcode != nor->erase_opcode)
+		goto use_stig;
+	if (upper_32_bits(op->addr.val))
+		goto use_stig;
+
+	if (!cdns_xspi->acmd_info.initialized) {
+		ret = cdns_xspi_nor_init(cdns_xspi, nor);
+		if (ret == -EOPNOTSUPP)
+			goto use_stig;
+		if (ret) {
+			dev_err(cdns_xspi->dev,
+				"failed to initialize NOR ACMD: %d\n", ret);
+			return ret;
+		}
+	}
+
+	if (opcode == nor->read_opcode && op->data.dir == SPI_MEM_DATA_IN) {
+		ret = cdns_xspi_nor_read_seq_init(cdns_xspi, op);
+		if (ret == -EOPNOTSUPP)
+			goto use_stig;
+		if (ret)
+			return ret;
+
+		return cdns_xspi_pio_mdma_read(cdns_xspi, op->addr.val,
+					      op->data.buf.in,
+					      op->data.nbytes);
+	}
+
+	if (opcode == nor->program_opcode &&
+	    op->data.dir == SPI_MEM_DATA_OUT) {
+		if (!cdns_xspi_nor_helper_sequences_ready(cdns_xspi, nor))
+			goto use_stig;
+
+		ret = cdns_xspi_nor_program_seq_init(cdns_xspi, op);
+		if (ret == -EOPNOTSUPP)
+			goto use_stig;
+		if (ret)
+			return ret;
+
+		return cdns_xspi_pio_mdma_program(cdns_xspi, op->addr.val,
+						 op->data.buf.out,
+						 op->data.nbytes);
+	}
+
+	if (opcode == nor->erase_opcode && op->data.dir == SPI_MEM_NO_DATA) {
+		if (nor->mtd.numeraseregions ||
+		    !cdns_xspi_nor_helper_sequences_ready(cdns_xspi, nor))
+			goto use_stig;
+
+		ret = cdns_xspi_nor_erase_seq_init(cdns_xspi, nor, op);
+		if (ret == -EOPNOTSUPP)
+			goto use_stig;
+		if (ret)
+			return ret;
+
+		return cdns_xspi_pio_erase(cdns_xspi, op->addr.val);
+	}
+
+use_stig:
+	return cdns_xspi_send_stig_command(cdns_xspi, op,
+					   op->data.dir != SPI_MEM_NO_DATA);
+}
+
+static int cdns_xspi_send_pio_command(struct cdns_xspi_dev *cdns_xspi,
+				      struct spi_mem *mem,
+				      const struct spi_mem_op *op)
+{
+	switch (cdns_xspi->flash_type) {
+	case CDNS_XSPI_FLASH_TYPE_NAND:
+		return cdns_xspi_send_nand_pio_command(cdns_xspi, mem, op);
+
+	case CDNS_XSPI_FLASH_TYPE_NOR:
+		return cdns_xspi_send_nor_pio_command(cdns_xspi, mem, op);
+
+	default:
+		return -EOPNOTSUPP;
+	}
+}
+
 static int cdns_xspi_mem_op(struct cdns_xspi_dev *cdns_xspi,
 			    struct spi_mem *mem,
 			    const struct spi_mem_op *op)
@@ -1484,8 +1900,18 @@ static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op *
 {
 	struct cdns_xspi_dev *cdns_xspi =
 		spi_controller_get_devdata(mem->spi->controller);
+	size_t max_len = cdns_xspi->sdmasize;
+
+	if (cdns_xspi->work_mode == CDNS_XSPI_WORK_MODE_ACMD) {
+		if (cdns_xspi->acmd_info.initialized)
+			max_len = min_t(size_t, max_len,
+					cdns_xspi->dma_buf_len);
+		else if (cdns_xspi->flash_type == CDNS_XSPI_FLASH_TYPE_NOR)
+			max_len = min_t(size_t, max_len,
+					CDNS_XSPI_NOR_DMA_BUF_SIZE);
+	}
 
-	op->data.nbytes = clamp_val(op->data.nbytes, 0, cdns_xspi->sdmasize);
+	op->data.nbytes = min_t(size_t, op->data.nbytes, max_len);
 
 	return 0;
 }
-- 
2.34.1