mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Douglas Anderson <dianders@chromium.org>
To: Devicetree-compiler@vger.kernel.org,
	Rob Herring <robh@kernel.org>,
	Krzysztof Kozlowski <krzk+dt@kernel.org>,
	Conor Dooley <conor+dt@kernel.org>,
	Konrad Dybcio <konradybcio@kernel.org>
Cc: Devicetree@vger.kernel.org,
	"Tudor Ambarus" <tudor.ambarus@linaro.org>,
	"Chen-Yu Tsai" <wenst@chromium.org>,
	"Saravana Kannan" <saravanak@kernel.org>,
	"David Gibson" <david@gibson.dropbear.id.au>,
	"André Draszik" <andre.draszik@linaro.org>,
	"Simon Glass" <sjg@chromium.org>,
	linux-kernel@vger.kernel.org,
	"Brian Norris" <briannorris@chromium.org>,
	"Peter Griffin" <peter.griffin@linaro.org>,
	"Julius Werner" <jwerner@chromium.org>,
	"Douglas Anderson" <dianders@chromium.org>
Subject: [PATCH v4 06/14] pylibfdt: Speed up FdtRo.properties() with a C helper
Date: Mon, 16 Nov 2026 12:53:03 -0800	[thread overview]
Message-ID: <20261116205320.1621386-7-dianders@chromium.org> (raw)
In-Reply-To: <20261116205320.1621386-1-dianders@chromium.org>

Iterating over a node's properties via `FdtRo.properties()` currently
crosses the Python/SWIG boundary `1 + 2 * num_props` times per node
(`first_property_offset()`, `get_property_by_offset()`, and
`next_property_offset()`, along with their Python `check_err()` and
`check_err_null()` wrappers). When walking every property across large
device trees from Python, those per-property SWIG round-trips dominate
runtime.

Add a C helper `fdt_node_properties_stub()` in `pylibfdt/libfdt.i`
backing `FdtRo.properties()` so all `Property` objects for a node are
collected in a single C call per node, interning property name strings
via `PyUnicode_InternFromString()`. Also add an invalid-offset exception
check to `testPropertiesIterator` in `tests/pylibfdt_tests.py`.

On one stress test (walking all ~573k nodes and ~2.06M properties across
438 arm64 Qualcomm DTBs) this reduced `FdtRo.properties()` time by
~1.8x.

Assisted-by: Gemini:gemini-pro-prerelease
Signed-off-by: Douglas Anderson <dianders@chromium.org>
---
Found this speedup while benchmarking fdtfactor. It's not critical to
functionality, but seems like a nice win.

Changes in v4:
- New patch speeding up FdtRo.properties() with a C helper

 pylibfdt/libfdt.i       | 64 ++++++++++++++++++++++++++++++++++++++---
 tests/pylibfdt_tests.py |  3 ++
 2 files changed, 63 insertions(+), 4 deletions(-)

diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
index e4a9357d278a..333a25f27946 100644
--- a/pylibfdt/libfdt.i
+++ b/pylibfdt/libfdt.i
@@ -24,6 +24,61 @@ static int fdt_property_stub(void *fdt, const char *name, const void *val,
     return fdt_property(fdt, name, val, len);
 }
 
+static PyObject *fdt_node_properties_stub(const void *fdt, int nodeoffset,
+                                          PyObject *prop_cls)
+{
+    PyObject *list = PyList_New(0);
+    int poff;
+
+    if (!list)
+        return NULL;
+
+    poff = fdt_first_property_offset(fdt, nodeoffset);
+    while (poff >= 0) {
+        const char *pname;
+        int len;
+        const void *val = fdt_getprop_by_offset(fdt, poff, &pname, &len);
+        PyObject *name_obj, *val_obj, *prop_obj;
+        int rc;
+
+        if (!val) {
+            Py_DECREF(list);
+            return PyLong_FromLong(len);
+        }
+        name_obj = PyUnicode_InternFromString(pname);
+        if (!name_obj) {
+            Py_DECREF(list);
+            return NULL;
+        }
+        val_obj = PyBytes_FromStringAndSize((const char *)val, len);
+        if (!val_obj) {
+            Py_DECREF(name_obj);
+            Py_DECREF(list);
+            return NULL;
+        }
+        prop_obj = PyObject_CallFunctionObjArgs(prop_cls, name_obj, val_obj,
+                                                NULL);
+        Py_DECREF(name_obj);
+        Py_DECREF(val_obj);
+        if (!prop_obj) {
+            Py_DECREF(list);
+            return NULL;
+        }
+        rc = PyList_Append(list, prop_obj);
+        Py_DECREF(prop_obj);
+        if (rc < 0) {
+            Py_DECREF(list);
+            return NULL;
+        }
+        poff = fdt_next_property_offset(fdt, poff);
+    }
+    if (poff != -FDT_ERR_NOTFOUND) {
+        Py_DECREF(list);
+        return PyLong_FromLong(poff);
+    }
+    return list;
+}
+
 %}
 
 %pythoncode %{
@@ -477,10 +532,9 @@ class FdtRo(object):
         Raises:
             FdtException on error (e.g. invalid nodeoffset)
         """
-        poff = self.first_property_offset(nodeoffset, quiet=QUIET_NOTFOUND)
-        while poff >= 0:
-            yield self.get_property_by_offset(poff)
-            poff = self.next_property_offset(poff, quiet=QUIET_NOTFOUND)
+        return iter(check_err(
+            fdt_node_properties_stub(self._fdt, nodeoffset, Property)
+        ))
 
     def getprop(self, nodeoffset, prop_name, quiet=()):
         """Get a property from a node
@@ -1420,5 +1474,7 @@ int fdt_property_cell(void *fdt, const char *name, uint32_t val);
   * function and a struct, which confuses SWIG.
  */
 int fdt_property_stub(void *fdt, const char *name, const void *val, int len);
+PyObject *fdt_node_properties_stub(const void *fdt, int nodeoffset,
+                                   PyObject *prop_cls);
 
 %include <libfdt.h>
diff --git a/tests/pylibfdt_tests.py b/tests/pylibfdt_tests.py
index e2f9c6d5afe4..9c0104bfbabb 100644
--- a/tests/pylibfdt_tests.py
+++ b/tests/pylibfdt_tests.py
@@ -270,6 +270,9 @@ class PyLibfdtBasicTests(unittest.TestCase):
         self.assertEqual(props[0], b'subnode1\0')
         leaf = self.fdt.path_offset('/subnode@1/ss1')
         self.assertEqual(list(self.fdt.properties(leaf)), [])
+        with self.assertRaises(FdtException) as e:
+            list(self.fdt.properties(-1))
+        self.assertEqual(e.exception.err, -libfdt.BADOFFSET)
 
     def testFirstNextSubnodeOffsetExceptions(self):
         """Check except handling for first/next subnode functions"""
-- 
2.56.0.679.gc527c65306-goog


  parent reply	other threads:[~2026-11-16 20:53 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 20:52 [PATCH v4 00/14] fdtfactor: Introduce fdtfactor to auto-factor overlays Douglas Anderson
2026-11-16 20:52 ` [PATCH v4 01/14] README: Add overview blurbs for dtc and ancillary tools Douglas Anderson
2026-11-16 20:52 ` [PATCH v4 02/14] pylibfdt: Make Property a subclass of bytes instead of bytearray Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 03/14] pylibfdt: Trim Fdt bytearray to totalsize() in FdtSw.as_fdt() Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 04/14] pylibfdt: Add Property.from_*() creation methods Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 05/14] pylibfdt: Unpack Property.as_list() in a single struct.unpack() call Douglas Anderson
2026-11-16 20:53 ` Douglas Anderson [this message]
2026-11-16 20:53 ` [PATCH v4 07/14] libpydt: Add DeviceTree live tree representation Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 08/14] libfdt: Speed up fdt_find_string_len_() with memchr() Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 09/14] libfdt: overlay: Update base /__local_fixups__ on apply Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 10/14] fdtfactor: Add tool to split DTBs into a base DTB and overlays Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 11/14] fdtfactor: Add -S/--strip-local-fixups option Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 12/14] fdtfactor: Support compressed DTB inputs and outputs Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 13/14] fdtfactor: Add compiled FIT factoring support Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 14/14] fdtfactor: Parallelize blob compression and decompression Douglas Anderson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116205320.1621386-7-dianders@chromium.org \
    --to=dianders@chromium.org \
    --cc=Devicetree-compiler@vger.kernel.org \
    --cc=Devicetree@vger.kernel.org \
    --cc=andre.draszik@linaro.org \
    --cc=briannorris@chromium.org \
    --cc=conor+dt@kernel.org \
    --cc=david@gibson.dropbear.id.au \
    --cc=jwerner@chromium.org \
    --cc=konradybcio@kernel.org \
    --cc=krzk+dt@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peter.griffin@linaro.org \
    --cc=robh@kernel.org \
    --cc=saravanak@kernel.org \
    --cc=sjg@chromium.org \
    --cc=tudor.ambarus@linaro.org \
    --cc=wenst@chromium.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®