[PATCH v6 0/2] Add pixel formats used in Synatpics SoC

[PATCH v6 0/2] Add pixel formats used in Synatpics SoC

[Hsia-Jun Li]

From: "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>

Those pixel formats are used in Synaptics's VideoSmart series SoCs,
likes VS640, VS680. I just disclose the pixel formats used in the video
codecs and display pipeline this time. Actually any device connected to
the MTR module could support those tiling and compressed pixel formats.

https://synaptics.com/products/multimedia-solutions

Changelog:
v6:
Refresh and fix warnings in its document.
v5:
Moving back the document and rewriting the description.
v4:
Removed the patches for V4L2, V4L2 would use the drm_fourcc.h .
Moving the documents to the mesa project.
v3:
There was a mistake in format macro.
Correcting the description of 64L4 variant modifiers.
v2:
The DRM modifiers in the first draft is too simple, it can't tell
the tiles in group attribute in memory layout.
Removing the v4l2 fourcc. Adding a document for the future v4l2 extended
fmt.
v1:
first draft of DRM modifiers
Try to put basic tile formats into v4l2 fourcc

Hsia-Jun(Randy) Li (1):
  drm/fourcc: Add Synaptics VideoSmart tiled modifiers

Randy Li (1):
  Documentation/gpu: Add Synaptics tiling formats documentation

 Documentation/gpu/synaptics.rst | 81 +++++++++++++++++++++++++++++++++
 include/uapi/drm/drm_fourcc.h   | 75 ++++++++++++++++++++++++++++++
 2 files changed, 156 insertions(+)
 create mode 100644 Documentation/gpu/synaptics.rst

-- 
2.17.1

[PATCH v6 1/2] drm/fourcc: Add Synaptics VideoSmart tiled modifiers

[Hsia-Jun Li]

From: "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>

Those modifiers only record the parameters would effort pixel
layout or memory layout. Whether physical memory page mapping
is used is not a part of format.

Signed-off-by: Hsia-Jun(Randy) Li <randy.li@synaptics.com>
---
 include/uapi/drm/drm_fourcc.h | 75 +++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index de703c6be969..ee13250f06f4 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -419,6 +419,7 @@ extern "C" {
 #define DRM_FORMAT_MOD_VENDOR_ARM     0x08
 #define DRM_FORMAT_MOD_VENDOR_ALLWINNER 0x09
 #define DRM_FORMAT_MOD_VENDOR_AMLOGIC 0x0a
+#define DRM_FORMAT_MOD_VENDOR_SYNAPTICS 0x0b
 
 /* add more to the end as needed */
 
@@ -1519,6 +1520,80 @@ drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier)
 #define AMD_FMT_MOD_CLEAR(field) \
 	(~((__u64)AMD_FMT_MOD_##field##_MASK << AMD_FMT_MOD_##field##_SHIFT))
 
+/*
+ * Synaptics VideoSmart modifiers
+ *
+ * Tiles could be arranged in Groups of Tiles (GOTs), it is a small tile
+ * within a tile. GOT size and layout varies based on platform and
+ * performance concern. When the compression is applied, it is possible
+ * that we would have two tile type in the GOT, these parameters can't
+ * tell the secondary tile type.
+ *
+ * Besides, an 8 size 4 bytes arrary (32 bytes) would be need to store
+ * some compression parameters for a compression meta data plane.
+ *
+ *       Macro
+ * Bits  Param Description
+ * ----  ----- -----------------------------------------------------------------
+ *
+ *  7:0  f     Scan direction description.
+ *
+ *               0 = Invalid
+ *               1 = V4, the scan would always start from vertical for 4 pixel
+ *                   then move back to the start pixel of the next horizontal
+ *                   direction.
+ *               2 = Reserved for future use.
+ *
+ * 15:8  m     The times of pattern repeat in the right angle direction from
+ *             the first scan direction.
+ *
+ * 19:16 p     The padding bits after the whole scan, could be zero.
+ *
+ * 20:20 g     GOT packing flag.
+ *
+ * 23:21 -     Reserved for future use.  Must be zero.
+ *
+ * 27:24 h     log2(horizontal) of bytes, in GOTs.
+ *
+ * 31:28 v     log2(vertical) of bytes, in GOTs.
+ *
+ * 35:32 -     Reserved for future use.  Must be zero.
+ *
+ * 36:36 c     Compression flag.
+ *
+ * 55:37 -     Reserved for future use.  Must be zero.
+ *
+ */
+
+#define DRM_FORMAT_MOD_SYNA_V4_TILED		fourcc_mod_code(SYNAPTICS, 1)
+
+#define DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(f, m, p, g, h, v, c) \
+	fourcc_mod_code(SYNAPTICS, ((__u64)((f) & 0xff) | \
+				 ((__u64)((m) & 0xff) << 8) | \
+				 ((__u64)((p) & 0xf) << 16) | \
+				 ((__u64)((g) & 0x1) << 20) | \
+				 ((__u64)((h) & 0xf) << 24) | \
+				 ((__u64)((v) & 0xf) << 28) | \
+				 ((__u64)((c) & 0x1) << 36)))
+
+#define DRM_FORMAT_MOD_SYNA_V4H1 \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 1, 0, 0, 0, 0, 0)
+
+#define DRM_FORMAT_MOD_SYNA_V4H3P8 \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 3, 8, 0, 0, 0, 0)
+
+#define DRM_FORMAT_MOD_SYNA_V4H1_64L4_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 1, 0, 1, 6, 2, 1)
+
+#define DRM_FORMAT_MOD_SYNA_V4H3P8_64L4_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 3, 8, 1, 6, 2, 1)
+
+#define DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 1, 0, 1, 7, 7, 1)
+
+#define DRM_FORMAT_MOD_SYNA_V4H3P8_128L128_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 3, 8, 1, 7, 7, 1)
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.17.1

[PATCH v6 2/2] Documentation/gpu: Add Synaptics tiling formats documentation

[Hsia-Jun Li]

From: Randy Li <ayaka@soulik.info>

Signed-off-by: Randy Li <ayaka@soulik.info>
Signed-off-by: Hsia-Jun(Randy) Li <randy.li@synaptics.com>
---
 Documentation/gpu/synaptics.rst | 81 +++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 Documentation/gpu/synaptics.rst

diff --git a/Documentation/gpu/synaptics.rst b/Documentation/gpu/synaptics.rst
new file mode 100644
index 000000000000..4185ca536bf1
--- /dev/null
+++ b/Documentation/gpu/synaptics.rst
@@ -0,0 +1,81 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+================
+Synaptics Tiling
+================
+
+The tiling pixel formats in Synpatics Video Smart platform have
+many variants. Tiles could form the group of tiles, pixels within
+the group (nearest) width and height are stored into tile.
+Meanwhile, the tile in a group may not follow dimension layout,
+tile could form a small group of tiles, then that (sub)group
+of tiles would form a bigger group. We won't describe the dimension
+layout inside the group of tiles here. The layout of the group
+of tiles is fixed with the group width and height parameters
+in the same generation of the platform.
+
+Compression
+===========
+The proprietary lossless image compression protocol in Synaptics
+could minimizes the amount of data transferred (less memory bandwidth
+consumption) between devices. It would usually apply to the tiling
+pixel format.
+
+Each component would request an extra page aligned length buffer
+for storing the compression meta data. Also a 32 bytes parameters
+set would come with a compression meta data buffer.
+
+The component here corresponds to a signal type (i.e. Luma, chroma).
+They could be encoded into one or multiple metadata planes, but
+their compression parameters still would be individual.
+
+Pixel format modifiers
+======================
+Addition alignment requirement for stride and size of a memory plane
+could apply beyond what has been mentioned below. Remember always
+negotiating with all the devices in pipeline before allocation.
+
+.. flat-table:: Synpatics Image Format Modifiers
+
+	* - Identifier
+	  - Fourcc
+	  - Details
+
+	* - DRM_FORMAT_MOD_SYNA_V4H1
+	  - DRM_FORMAT_NV12
+	  - The plain uncompressed 8 bits tile format. It sounds similar to
+	Intel's Y-tile. but it won't take any pixel from the next X direction
+	in a tile group. The line stride and image height must be aligned to
+	a multiple of 16. The height of chrominance plane would plus 8.
+
+	* - DRM_FORMAT_MOD_SYNA_V4H3P8
+	  - DRM_FORMAT_NV15
+	  - The plain uncompressed 10 bits tile format. It stores pixel in 2D
+	3x4 tiles with a 8bits padding to each of tile. Then a tile is in a
+	128 bits cache line.
+
+	* - DRM_FORMAT_MOD_SYNA_V4H1_64L4_COMPRESSED
+	  - DRM_FORMAT_NV12
+	  - Group of tiles and compressed variant of ``DRM_FORMAT_MOD_SYNA_V4H1``.
+    A group of tiles would contain 64x4 pixels, where a tile has 1x4
+    pixel.
+
+	* - DRM_FORMAT_MOD_SYNA_V4H3P8_64L4_COMPRESSED
+	  - DRM_FORMAT_NV15
+	  - Group of tiles and compressed variant of ``DRM_FORMAT_MOD_SYNA_V4H3P8``.
+     A group of tiles would contains 48x4 pixels, where a tile has 3x4 pixels
+     and a 8 bits padding in the end of a tile. A group of tiles would
+     be 256 bytes.
+
+	* - ``DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED``
+	  - DRM_FORMAT_NV12
+	  - Group of tiles and compressed variant of ``DRM_FORMAT_MOD_SYNA_V4H1``.
+    A group of tiles would contain 128x32 pixels, where a tile has 1x4
+    pixel.
+
+	* - ``DRM_FORMAT_MOD_SYNA_V4H3P8_128L128_COMPRESSED``
+	  - DRM_FORMAT_NV15
+	  - Group of tiles and compressed variant of ``DRM_FORMAT_MOD_SYNA_V4H3P8``.
+     A group of tiles would contains 96x128 pixels, where a tile has 3x4 pixels
+     and a 8 bits padding in the end of a tile. A group of tiles would
+     be 16 KiB.
-- 
2.17.1

Re: [PATCH v6 2/2] Documentation/gpu: Add Synaptics tiling formats documentation

[kernel test robot]

Hi Hsia-Jun,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on drm-misc/drm-misc-next]
[also build test WARNING on drm/drm-next drm-exynos/exynos-drm-next drm-intel/for-linux-next drm-intel/for-linux-next-fixes drm-tip/drm-tip linus/master v6.3-rc3 next-20230322]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Hsia-Jun-Li/drm-fourcc-Add-Synaptics-VideoSmart-tiled-modifiers/20230322-163252
base:   git://anongit.freedesktop.org/drm/drm-misc drm-misc-next
patch link:    https://lore.kernel.org/r/20230322082910.115371-3-randy.li%40synaptics.com
patch subject: [PATCH v6 2/2] Documentation/gpu: Add Synaptics tiling formats documentation
reproduce:
        # https://github.com/intel-lab-lkp/linux/commit/51642395567738204b07b9c48e27d4a5298f1ca9
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Hsia-Jun-Li/drm-fourcc-Add-Synaptics-VideoSmart-tiled-modifiers/20230322-163252
        git checkout 51642395567738204b07b9c48e27d4a5298f1ca9
        make menuconfig
        # enable CONFIG_COMPILE_TEST, CONFIG_WARN_MISSING_DOCUMENTS, CONFIG_WARN_ABI_ERRORS
        make htmldocs

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202303230311.mRoFe6oK-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> Documentation/gpu/synaptics.rst:47: WARNING: Bullet list ends without a blank line; unexpected unindent.
>> Documentation/gpu/synaptics.rst:60: WARNING: Block quote ends without a blank line; unexpected unindent.
>> Documentation/gpu/synaptics.rst:38: WARNING: Error parsing content block for the "flat-table" directive: exactly one bullet list expected.

vim +47 Documentation/gpu/synaptics.rst

    37	
  > 38	.. flat-table:: Synpatics Image Format Modifiers
    39	
    40		* - Identifier
    41		  - Fourcc
    42		  - Details
    43	
    44		* - DRM_FORMAT_MOD_SYNA_V4H1
    45		  - DRM_FORMAT_NV12
    46		  - The plain uncompressed 8 bits tile format. It sounds similar to
  > 47		Intel's Y-tile. but it won't take any pixel from the next X direction
    48		in a tile group. The line stride and image height must be aligned to
    49		a multiple of 16. The height of chrominance plane would plus 8.
    50	
    51		* - DRM_FORMAT_MOD_SYNA_V4H3P8
    52		  - DRM_FORMAT_NV15
    53		  - The plain uncompressed 10 bits tile format. It stores pixel in 2D
    54		3x4 tiles with a 8bits padding to each of tile. Then a tile is in a
    55		128 bits cache line.
    56	
    57		* - DRM_FORMAT_MOD_SYNA_V4H1_64L4_COMPRESSED
    58		  - DRM_FORMAT_NV12
    59		  - Group of tiles and compressed variant of ``DRM_FORMAT_MOD_SYNA_V4H1``.
  > 60	    A group of tiles would contain 64x4 pixels, where a tile has 1x4

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests