[PATCH 0/3] mtd: rawnand: sunxi: support the Allwinner randomized OOB format

[PATCH 0/3] mtd: rawnand: sunxi: support the Allwinner randomized OOB format

[James Hilliard]

The sunxi NAND controller randomizes normal pages, including their OOB
data. The mainline driver compensates the bad block marker so that it
remains plain on flash. On H6 and H616 it also fills the OOB space left
after ECC with protected user data. These choices produce a different
physical OOB format from the one used by Allwinner NAND firmware.

Add allwinner,randomized-oob to select the Allwinner randomized OOB
format. The property enables page randomization unconditionally, keeps
the bad block marker in the randomizer data stream, and selects the
controller-specific protected user-data placement. H6 and H616 use four
protected bytes per 1 KiB ECC step, capped at 16 bytes, with the complete
protected region placed before the first ECC step. Older controllers
retain their existing fixed user-data placement.

When the property is absent, retain the existing mainline marker handling
and H6/H616 maximized OOB layout.

The series first documents the opt-in format, then implements randomized
marker handling for all supported controllers and finally selects the
packed H6/H616 protected user-data layout.

Tested on an H616 board with a Toshiba TC58NVG1S3H NAND using matching
Linux and U-Boot implementations. Linux initialized the NAND, SPL and
U-Boot cold-booted from it, U-Boot loaded the FIT from UBI, and Linux
attached the same UBI with 2004 good PEBs, two bad PEBs and no corrupted
PEBs. A target-timed 64 MiB raw-MTD read completed without ECC, timeout
or I/O errors.

Validation completed with strict checkpatch, dt_binding_check and an
arm64 W=1 object build.

Assisted-by: OpenAI Codex (gpt-5.6-sol, max)
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
James Hilliard (3):
      dt-bindings: mtd: sunxi: Add randomized OOB flag
      mtd: rawnand: sunxi: support randomized OOB formats
      mtd: rawnand: sunxi: select the packed H6/H616 OOB layout

 .../bindings/mtd/allwinner,sun4i-a10-nand.yaml     |  9 +++++
 drivers/mtd/nand/raw/sunxi_nand.c                  | 46 ++++++++++++++++++----
 2 files changed, 47 insertions(+), 8 deletions(-)
---
base-commit: 15a3cbce32994141252bb4ecfe3ff3a5d22d0b4f
change-id: 20260810-submit-sunxi-nand-vendor-oob-layout-v1-e3114d10cc9c

Best regards,
--  
James Hilliard <james.hilliard1@gmail.com>

[PATCH 1/3] dt-bindings: mtd: sunxi: Add randomized OOB flag

[James Hilliard]

Allwinner NAND firmware stores the bad block marker through the
controller randomizer. The mainline driver instead compensates the first
two randomized bytes so that the marker remains plain on flash.

On H6 and H616, the firmware also uses a fixed protected user-data
placement while mainline fills the OOB space left after ECC. Add an
opt-in property for NAND using the randomized normal-page OOB format.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
 .../devicetree/bindings/mtd/allwinner,sun4i-a10-nand.yaml        | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Documentation/devicetree/bindings/mtd/allwinner,sun4i-a10-nand.yaml b/Documentation/devicetree/bindings/mtd/allwinner,sun4i-a10-nand.yaml
index 9d061e2216cb..7eda1275135d 100644
--- a/Documentation/devicetree/bindings/mtd/allwinner,sun4i-a10-nand.yaml
+++ b/Documentation/devicetree/bindings/mtd/allwinner,sun4i-a10-nand.yaml
@@ -79,6 +79,15 @@ patternProperties:
           minimum: 0
           maximum: 1
 
+      allwinner,randomized-oob:
+        type: boolean
+        description:
+          Use the Allwinner randomized normal-page OOB format. This enables
+          the controller page randomizer, stores the bad block marker through
+          it instead of compensating the marker so that it remains plain on
+          flash, and selects the controller-specific protected user-data
+          placement used by Allwinner NAND firmware.
+
     unevaluatedProperties: false
 
 required:

-- 
2.53.0

[PATCH 2/3] mtd: rawnand: sunxi: support randomized OOB formats

[James Hilliard]

The controller randomizer covers the bad block marker along with the
rest of the OOB data. The driver currently compensates the marker bytes
before writes and after reads so that they remain plain on flash.

Allwinner NAND firmware instead stores the marker through the
randomizer. Media using that format appears to contain bad blocks unless
the randomizer is enabled while reading the marker.

Honor the allwinner,randomized-oob property by unconditionally enabling
page scrambling and leaving the marker in the randomizer data stream.
Keep the existing plain marker behavior when the property is absent.
This changes marker handling on all supported controllers; the H6/H616
protected user-data placement is handled separately.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
 drivers/mtd/nand/raw/sunxi_nand.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/nand/raw/sunxi_nand.c b/drivers/mtd/nand/raw/sunxi_nand.c
index 45ccbce91551..92b26371a6d1 100644
--- a/drivers/mtd/nand/raw/sunxi_nand.c
+++ b/drivers/mtd/nand/raw/sunxi_nand.c
@@ -264,6 +264,7 @@ struct sunxi_nfc_timings {
  * @clk_rate: clk_rate required for this NAND chip
  * @timing_cfg: TIMING_CFG register value for this NAND chip
  * @timing_ctl: TIMING_CTL register value for this NAND chip
+ * @randomized_oob: use the randomized normal-page OOB format
  * @nsels: number of CS lines required by the NAND chip
  * @sels: array of CS lines descriptions
  * @user_data_bytes: array of user data lengths for all ECC steps
@@ -276,6 +277,7 @@ struct sunxi_nand_chip {
 	u32 timing_cfg;
 	u32 timing_ctl;
 	u8 *user_data_bytes;
+	bool randomized_oob;
 	int nsels;
 	struct sunxi_nand_chip_sel sels[] __counted_by(nsels);
 };
@@ -945,7 +947,8 @@ static void sunxi_nfc_hw_ecc_get_prot_oob_bytes(struct nand_chip *nand, u8 *oob,
 	}
 
 	/* De-randomize the Bad Block Marker. */
-	if (bbm && (nand->options & NAND_NEED_SCRAMBLING))
+	if (bbm && (nand->options & NAND_NEED_SCRAMBLING) &&
+	    !sunxi_nand->randomized_oob)
 		sunxi_nfc_randomize_bbm(nand, page, oob);
 }
 
@@ -1006,7 +1009,8 @@ static void sunxi_nfc_hw_ecc_set_prot_oob_bytes(struct nand_chip *nand,
 	u8 *user_data = NULL;
 
 	/* Randomize the Bad Block Marker. */
-	if (bbm && (nand->options & NAND_NEED_SCRAMBLING)) {
+	if (bbm && (nand->options & NAND_NEED_SCRAMBLING) &&
+	    !sunxi_nand->randomized_oob) {
 		user_data = kmalloc(user_data_sz, GFP_KERNEL);
 		memcpy(user_data, oob, user_data_sz);
 		sunxi_nfc_randomize_bbm(nand, page, user_data);
@@ -2219,6 +2223,7 @@ static int sunxi_nand_hw_ecc_ctrl_init(struct nand_chip *nand,
 
 static int sunxi_nand_attach_chip(struct nand_chip *nand)
 {
+	struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
 	const struct nand_ecc_props *requirements =
 		nanddev_get_ecc_requirements(&nand->base);
 	struct nand_ecc_ctrl *ecc = &nand->ecc;
@@ -2228,6 +2233,9 @@ static int sunxi_nand_attach_chip(struct nand_chip *nand)
 	if (nand->bbt_options & NAND_BBT_USE_FLASH)
 		nand->bbt_options |= NAND_BBT_NO_OOB;
 
+	if (sunxi_nand->randomized_oob)
+		nand->options |= NAND_NEED_SCRAMBLING;
+
 	if (nand->options & NAND_NEED_SCRAMBLING)
 		nand->options |= NAND_NO_SUBPAGE_WRITE;
 
@@ -2457,6 +2465,9 @@ static int sunxi_nand_chip_init(struct device *dev, struct sunxi_nfc *nfc,
 	if (!sunxi_nand)
 		return -ENOMEM;
 
+	sunxi_nand->randomized_oob =
+		of_property_read_bool(np, "allwinner,randomized-oob");
+
 	sunxi_nand->nsels = nsels;
 
 	for (i = 0; i < nsels; i++) {

-- 
2.53.0

[PATCH 3/3] mtd: rawnand: sunxi: select the packed H6/H616 OOB layout

[James Hilliard]

The H6/H616 controller can configure a separate protected user-data
length for every ECC step. Mainline fills all space left after ECC with
user data. This changes the physical ECC offsets from those used by
Allwinner NAND firmware, so pages written by one layout cannot be
decoded with the other.

The allwinner,randomized-oob property selects the randomized normal-page
OOB format. When it is present on H6/H616, use four protected user-data
bytes per 1 KiB ECC step, cap the total at 16 bytes, and assign the
entire total to ECC step zero. Reserve the same total before maximizing
ECC strength so the selected strength leaves enough room.

Keep the existing maximized mainline user-data layout when the property
is absent. Older controllers have fixed four-byte user-data registers
and need no additional placement change.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
 drivers/mtd/nand/raw/sunxi_nand.c | 31 +++++++++++++++++++++++++------
 1 file changed, 25 insertions(+), 6 deletions(-)

diff --git a/drivers/mtd/nand/raw/sunxi_nand.c b/drivers/mtd/nand/raw/sunxi_nand.c
index 92b26371a6d1..2388cc361685 100644
--- a/drivers/mtd/nand/raw/sunxi_nand.c
+++ b/drivers/mtd/nand/raw/sunxi_nand.c
@@ -223,6 +223,9 @@
  */
 #define USER_DATA_SZ 4
 
+/* The randomized H6/H616 layout packs at most 16 bytes before ECC step 0. */
+#define SUNXI_NFC_H6_MAX_USER_DATA_SZ 16
+
 /**
  * struct sunxi_nand_chip_sel - stores information related to NAND Chip Select
  *
@@ -2036,8 +2039,14 @@ static void sunxi_nand_detach_chip(struct nand_chip *nand)
 	sunxi_nand->user_data_bytes = NULL;
 }
 
-static int sunxi_nfc_maximize_user_data(struct nand_chip *nand, uint32_t oobsize,
-					int ecc_bytes, int nsectors)
+static unsigned int sunxi_nfc_h6_user_data_sz(int nsectors)
+{
+	return min(nsectors * USER_DATA_SZ,
+		   SUNXI_NFC_H6_MAX_USER_DATA_SZ);
+}
+
+static int sunxi_nfc_init_user_data(struct nand_chip *nand, uint32_t oobsize,
+				    int ecc_bytes, int nsectors)
 {
 	struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
 	struct sunxi_nfc *nfc = to_sunxi_nfc(nand->controller);
@@ -2050,6 +2059,12 @@ static int sunxi_nfc_maximize_user_data(struct nand_chip *nand, uint32_t oobsize
 	if (!sunxi_nand->user_data_bytes)
 		return -ENOMEM;
 
+	if (sunxi_nand->randomized_oob) {
+		sunxi_nand->user_data_bytes[0] =
+			sunxi_nfc_h6_user_data_sz(nsectors);
+		return 0;
+	}
+
 	for (step = 0; (step < nsectors) && (remaining_bytes > 0); step++) {
 		for (i = 0; i < c->nuser_data_tab; i++) {
 			if (c->user_data_len_tab[i] > remaining_bytes)
@@ -2104,6 +2119,10 @@ static int sunxi_nand_hw_ecc_ctrl_init(struct nand_chip *nand,
 				bytes -= 2;
 
 			bytes -= total_user_data_sz;
+		} else if (sunxi_nand->randomized_oob) {
+			total_user_data_sz =
+				sunxi_nfc_h6_user_data_sz(nsectors);
+			bytes -= total_user_data_sz;
 		} else {
 			/*
 			 * remove at least the BBM size before computing the
@@ -2170,12 +2189,12 @@ static int sunxi_nand_hw_ecc_ctrl_init(struct nand_chip *nand,
 	nsectors = mtd->writesize / ecc->size;
 
 	/*
-	 * The rationale for variable data length is to prioritize maximum ECC
-	 * strength, and then use the remaining space for user data.
+	 * The default variable-length layout prioritizes maximum ECC strength,
+	 * then uses the remaining space for user data.
 	 */
 	if (nfc->caps->reg_user_data_len)
-		sunxi_nfc_maximize_user_data(nand, mtd->oobsize, ecc->bytes,
-					     nsectors);
+		sunxi_nfc_init_user_data(nand, mtd->oobsize, ecc->bytes,
+					 nsectors);
 
 	if (total_user_data_sz == 0)
 		for (i = 0; i < nsectors; i++)

-- 
2.53.0

Re: [PATCH 2/3] mtd: rawnand: sunxi: support randomized OOB formats

[Miquel Raynal]

On 10/08/2026 at 17:04:50 -06, James Hilliard <james.hilliard1@gmail.com> wrote:

> The controller randomizer covers the bad block marker along with the
> rest of the OOB data. The driver currently compensates the marker bytes
> before writes and after reads so that they remain plain on flash.
>
> Allwinner NAND firmware instead stores the marker through the
> randomizer. Media using that format appears to contain bad blocks unless
> the randomizer is enabled while reading the marker.
>
> Honor the allwinner,randomized-oob property by unconditionally enabling
> page scrambling and leaving the marker in the randomizer data stream.
> Keep the existing plain marker behavior when the property is absent.
> This changes marker handling on all supported controllers; the H6/H616
> protected user-data placement is handled separately.

But I think the reason why this has never been done is because it just
breaks raw accesses, including the BBT reads. I don't think this is
handled, is it?

Thanks,
Miquèl

Re: [PATCH 2/3] mtd: rawnand: sunxi: support randomized OOB formats

[James Hilliard]

On Fri, Sep 4, 2026 at 8:42 AM Miquel Raynal <miquel.raynal@bootlin.com> wrote:
>
> On 10/08/2026 at 17:04:50 -06, James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > The controller randomizer covers the bad block marker along with the
> > rest of the OOB data. The driver currently compensates the marker bytes
> > before writes and after reads so that they remain plain on flash.
> >
> > Allwinner NAND firmware instead stores the marker through the
> > randomizer. Media using that format appears to contain bad blocks unless
> > the randomizer is enabled while reading the marker.
> >
> > Honor the allwinner,randomized-oob property by unconditionally enabling
> > page scrambling and leaving the marker in the randomizer data stream.
> > Keep the existing plain marker behavior when the property is absent.
> > This changes marker handling on all supported controllers; the H6/H616
> > protected user-data placement is handled separately.
>
> But I think the reason why this has never been done is because it just
> breaks raw accesses, including the BBT reads. I don't think this is
> handled, is it?

From my understanding raw access bypasses randomization entirely.

When writing the SPL(which does not use the same layout) we use a
pre-randomized image.

>
> Thanks,
> Miquèl

Re: [PATCH 2/3] mtd: rawnand: sunxi: support randomized OOB formats

[Miquel Raynal]

On 04/09/2026 at 11:17:46 -04, James Hilliard <james.hilliard1@gmail.com> wrote:

> On Fri, Sep 4, 2026 at 8:42 AM Miquel Raynal <miquel.raynal@bootlin.com> wrote:
>>
>> On 10/08/2026 at 17:04:50 -06, James Hilliard <james.hilliard1@gmail.com> wrote:
>>
>> > The controller randomizer covers the bad block marker along with the
>> > rest of the OOB data. The driver currently compensates the marker bytes
>> > before writes and after reads so that they remain plain on flash.
>> >
>> > Allwinner NAND firmware instead stores the marker through the
>> > randomizer. Media using that format appears to contain bad blocks unless
>> > the randomizer is enabled while reading the marker.
>> >
>> > Honor the allwinner,randomized-oob property by unconditionally enabling
>> > page scrambling and leaving the marker in the randomizer data stream.
>> > Keep the existing plain marker behavior when the property is absent.
>> > This changes marker handling on all supported controllers; the H6/H616
>> > protected user-data placement is handled separately.
>>
>> But I think the reason why this has never been done is because it just
>> breaks raw accesses, including the BBT reads. I don't think this is
>> handled, is it?
>
> From my understanding raw access bypasses randomization entirely.

yes, so the implementation you propose is invalid, raw reads shall
show the BBM. And also, when accessing a BBT, it is made in raw mode,
without randomization, and hence totally fail. This cannot fly!

Re: [PATCH 2/3] mtd: rawnand: sunxi: support randomized OOB formats

[James Hilliard]

On Fri, Sep 4, 2026 at 9:27 AM Miquel Raynal <miquel.raynal@bootlin.com> wrote:
>
> On 04/09/2026 at 11:17:46 -04, James Hilliard <james.hilliard1@gmail.com> wrote:
>
> > On Fri, Sep 4, 2026 at 8:42 AM Miquel Raynal <miquel.raynal@bootlin.com> wrote:
> >>
> >> On 10/08/2026 at 17:04:50 -06, James Hilliard <james.hilliard1@gmail.com> wrote:
> >>
> >> > The controller randomizer covers the bad block marker along with the
> >> > rest of the OOB data. The driver currently compensates the marker bytes
> >> > before writes and after reads so that they remain plain on flash.
> >> >
> >> > Allwinner NAND firmware instead stores the marker through the
> >> > randomizer. Media using that format appears to contain bad blocks unless
> >> > the randomizer is enabled while reading the marker.
> >> >
> >> > Honor the allwinner,randomized-oob property by unconditionally enabling
> >> > page scrambling and leaving the marker in the randomizer data stream.
> >> > Keep the existing plain marker behavior when the property is absent.
> >> > This changes marker handling on all supported controllers; the H6/H616
> >> > protected user-data placement is handled separately.
> >>
> >> But I think the reason why this has never been done is because it just
> >> breaks raw accesses, including the BBT reads. I don't think this is
> >> handled, is it?
> >
> > From my understanding raw access bypasses randomization entirely.
>
> yes, so the implementation you propose is invalid, raw reads shall
> show the BBM. And also, when accessing a BBT, it is made in raw mode,
> without randomization, and hence totally fail. This cannot fly!

I may be misunderstanding which raw and BBT access paths are at issue,
so I would like to clarify the expected semantics.

The allwinner,randomized-oob property does not alter the per-operation
rule controlling the randomizer. MTD_OPS_RAW still leaves the randomizer
disabled, while normal ECC operations enable it. The property does set
NAND_NEED_SCRAMBLING, ensuring that normal accesses use the randomizer
when the NAND does not already set that flag.

The property instead selects the Allwinner firmware's physical
normal-page OOB representation. It leaves the BBM in the randomizer data
stream and, on H6/H616, selects the firmware's protected-user-data
placement. It does not change the randomizer algorithm or seeds, nor
does it enable the randomizer during raw accesses.

With NAND_NEED_SCRAMBLING set, I understand the current mainline sunxi
behavior as follows:

ops.mode   fd mode  ECC  rand  main data       OOB with -o      nanddump
---------  -------  ---  ----  --------------  ---------------  --------
PLACE_OOB  NORMAL   on   on    corrected and   de-randomized    -o
                                de-randomized   logical OOB
PLACE_OOB  RAW      on   on    corrected and   de-randomized    unavailable
                                de-randomized   logical OOB
RAW        NORMAL   off  off   physical bytes  physical OOB     unavailable
RAW        RAW      off  off   physical bytes  physical OOB     -n -o

OOB data is returned only when the caller supplies an OOB buffer. For
nanddump, this is selected by -o. For MEMREAD/MEMWRITE, ops.mode
overrides the file mode. Stock nanddump exposes only the two diagonal
cases:

nanddump --bb=dumpbad -o /dev/mtdX
nanddump -n --bb=dumpbad -o /dev/mtdX

The first uses the normal ECC/randomizer path and returns corrected,
de-randomized main data followed by de-randomized OOB. The second sets
MTD_FILE_MODE_RAW and reaches the raw callbacks with both ECC and the
randomizer disabled, returning the physical main and OOB bytes.

In current mainline, that physical OOB contains a plain BBM because the
normal write path compensates the marker before hardware randomization.
The normal read path reverses that compensation. With
allwinner,randomized-oob, the compensation is omitted, so a raw read
remains physical but returns the BBM in its randomized on-flash form.
A normal read returns the logical BBM with either representation.

I do not see the current BBT access paths selecting MTD_OPS_RAW:

- nand_block_bad() calls chip->ecc.read_oob();
- scan_block_fast() and scan_read_oob() use MTD_OPS_PLACE_OOB; and
- read_bbt() uses an ordinary mtd_read().

For sunxi, ecc.read_oob calls ecc.read_page, which enables both ECC and
the randomizer when NAND_NEED_SCRAMBLING is set. The internal BBM and
BBT paths therefore appear to receive the decoded marker rather than
the physical randomized bytes.

Could you clarify which BBT path performs an MTD_OPS_RAW access, or
whether I am overlooking another path?

Also, when you say that raw reads must show the BBM, does that mean the
BBM must always be stored plain on flash, or may the raw interface
normalize just the BBM while otherwise returning physical data? If raw
access must remain entirely physical and the BBM must simultaneously be
directly recognizable, does that mean the Allwinner firmware's
randomized-BBM representation cannot be supported through the existing
raw NAND interface?

>