From: Douglas Anderson <dianders@chromium.org>
To: Devicetree-compiler@vger.kernel.org,
Rob Herring <robh@kernel.org>,
Krzysztof Kozlowski <krzk+dt@kernel.org>,
Conor Dooley <conor+dt@kernel.org>,
Konrad Dybcio <konradybcio@kernel.org>
Cc: Devicetree@vger.kernel.org,
"Tudor Ambarus" <tudor.ambarus@linaro.org>,
"Chen-Yu Tsai" <wenst@chromium.org>,
"Saravana Kannan" <saravanak@kernel.org>,
"David Gibson" <david@gibson.dropbear.id.au>,
"André Draszik" <andre.draszik@linaro.org>,
"Simon Glass" <sjg@chromium.org>,
linux-kernel@vger.kernel.org,
"Brian Norris" <briannorris@chromium.org>,
"Peter Griffin" <peter.griffin@linaro.org>,
"Julius Werner" <jwerner@chromium.org>,
"Douglas Anderson" <dianders@chromium.org>
Subject: [PATCH v4 06/14] pylibfdt: Speed up FdtRo.properties() with a C helper
Date: Mon, 16 Nov 2026 12:53:03 -0800 [thread overview]
Message-ID: <20261116205320.1621386-7-dianders@chromium.org> (raw)
In-Reply-To: <20261116205320.1621386-1-dianders@chromium.org>
Iterating over a node's properties via `FdtRo.properties()` currently
crosses the Python/SWIG boundary `1 + 2 * num_props` times per node
(`first_property_offset()`, `get_property_by_offset()`, and
`next_property_offset()`, along with their Python `check_err()` and
`check_err_null()` wrappers). When walking every property across large
device trees from Python, those per-property SWIG round-trips dominate
runtime.
Add a C helper `fdt_node_properties_stub()` in `pylibfdt/libfdt.i`
backing `FdtRo.properties()` so all `Property` objects for a node are
collected in a single C call per node, interning property name strings
via `PyUnicode_InternFromString()`. Also add an invalid-offset exception
check to `testPropertiesIterator` in `tests/pylibfdt_tests.py`.
On one stress test (walking all ~573k nodes and ~2.06M properties across
438 arm64 Qualcomm DTBs) this reduced `FdtRo.properties()` time by
~1.8x.
Assisted-by: Gemini:gemini-pro-prerelease
Signed-off-by: Douglas Anderson <dianders@chromium.org>
---
Found this speedup while benchmarking fdtfactor. It's not critical to
functionality, but seems like a nice win.
Changes in v4:
- New patch speeding up FdtRo.properties() with a C helper
pylibfdt/libfdt.i | 64 ++++++++++++++++++++++++++++++++++++++---
tests/pylibfdt_tests.py | 3 ++
2 files changed, 63 insertions(+), 4 deletions(-)
diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
index e4a9357d278a..333a25f27946 100644
--- a/pylibfdt/libfdt.i
+++ b/pylibfdt/libfdt.i
@@ -24,6 +24,61 @@ static int fdt_property_stub(void *fdt, const char *name, const void *val,
return fdt_property(fdt, name, val, len);
}
+static PyObject *fdt_node_properties_stub(const void *fdt, int nodeoffset,
+ PyObject *prop_cls)
+{
+ PyObject *list = PyList_New(0);
+ int poff;
+
+ if (!list)
+ return NULL;
+
+ poff = fdt_first_property_offset(fdt, nodeoffset);
+ while (poff >= 0) {
+ const char *pname;
+ int len;
+ const void *val = fdt_getprop_by_offset(fdt, poff, &pname, &len);
+ PyObject *name_obj, *val_obj, *prop_obj;
+ int rc;
+
+ if (!val) {
+ Py_DECREF(list);
+ return PyLong_FromLong(len);
+ }
+ name_obj = PyUnicode_InternFromString(pname);
+ if (!name_obj) {
+ Py_DECREF(list);
+ return NULL;
+ }
+ val_obj = PyBytes_FromStringAndSize((const char *)val, len);
+ if (!val_obj) {
+ Py_DECREF(name_obj);
+ Py_DECREF(list);
+ return NULL;
+ }
+ prop_obj = PyObject_CallFunctionObjArgs(prop_cls, name_obj, val_obj,
+ NULL);
+ Py_DECREF(name_obj);
+ Py_DECREF(val_obj);
+ if (!prop_obj) {
+ Py_DECREF(list);
+ return NULL;
+ }
+ rc = PyList_Append(list, prop_obj);
+ Py_DECREF(prop_obj);
+ if (rc < 0) {
+ Py_DECREF(list);
+ return NULL;
+ }
+ poff = fdt_next_property_offset(fdt, poff);
+ }
+ if (poff != -FDT_ERR_NOTFOUND) {
+ Py_DECREF(list);
+ return PyLong_FromLong(poff);
+ }
+ return list;
+}
+
%}
%pythoncode %{
@@ -477,10 +532,9 @@ class FdtRo(object):
Raises:
FdtException on error (e.g. invalid nodeoffset)
"""
- poff = self.first_property_offset(nodeoffset, quiet=QUIET_NOTFOUND)
- while poff >= 0:
- yield self.get_property_by_offset(poff)
- poff = self.next_property_offset(poff, quiet=QUIET_NOTFOUND)
+ return iter(check_err(
+ fdt_node_properties_stub(self._fdt, nodeoffset, Property)
+ ))
def getprop(self, nodeoffset, prop_name, quiet=()):
"""Get a property from a node
@@ -1420,5 +1474,7 @@ int fdt_property_cell(void *fdt, const char *name, uint32_t val);
* function and a struct, which confuses SWIG.
*/
int fdt_property_stub(void *fdt, const char *name, const void *val, int len);
+PyObject *fdt_node_properties_stub(const void *fdt, int nodeoffset,
+ PyObject *prop_cls);
%include <libfdt.h>
diff --git a/tests/pylibfdt_tests.py b/tests/pylibfdt_tests.py
index e2f9c6d5afe4..9c0104bfbabb 100644
--- a/tests/pylibfdt_tests.py
+++ b/tests/pylibfdt_tests.py
@@ -270,6 +270,9 @@ class PyLibfdtBasicTests(unittest.TestCase):
self.assertEqual(props[0], b'subnode1\0')
leaf = self.fdt.path_offset('/subnode@1/ss1')
self.assertEqual(list(self.fdt.properties(leaf)), [])
+ with self.assertRaises(FdtException) as e:
+ list(self.fdt.properties(-1))
+ self.assertEqual(e.exception.err, -libfdt.BADOFFSET)
def testFirstNextSubnodeOffsetExceptions(self):
"""Check except handling for first/next subnode functions"""
--
2.56.0.679.gc527c65306-goog
next prev parent reply other threads:[~2026-11-16 20:53 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 20:52 [PATCH v4 00/14] fdtfactor: Introduce fdtfactor to auto-factor overlays Douglas Anderson
2026-11-16 20:52 ` [PATCH v4 01/14] README: Add overview blurbs for dtc and ancillary tools Douglas Anderson
2026-11-16 20:52 ` [PATCH v4 02/14] pylibfdt: Make Property a subclass of bytes instead of bytearray Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 03/14] pylibfdt: Trim Fdt bytearray to totalsize() in FdtSw.as_fdt() Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 04/14] pylibfdt: Add Property.from_*() creation methods Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 05/14] pylibfdt: Unpack Property.as_list() in a single struct.unpack() call Douglas Anderson
2026-11-16 20:53 ` Douglas Anderson [this message]
2026-11-16 20:53 ` [PATCH v4 07/14] libpydt: Add DeviceTree live tree representation Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 08/14] libfdt: Speed up fdt_find_string_len_() with memchr() Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 09/14] libfdt: overlay: Update base /__local_fixups__ on apply Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 10/14] fdtfactor: Add tool to split DTBs into a base DTB and overlays Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 11/14] fdtfactor: Add -S/--strip-local-fixups option Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 12/14] fdtfactor: Support compressed DTB inputs and outputs Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 13/14] fdtfactor: Add compiled FIT factoring support Douglas Anderson
2026-11-16 20:53 ` [PATCH v4 14/14] fdtfactor: Parallelize blob compression and decompression Douglas Anderson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116205320.1621386-7-dianders@chromium.org \
--to=dianders@chromium.org \
--cc=Devicetree-compiler@vger.kernel.org \
--cc=Devicetree@vger.kernel.org \
--cc=andre.draszik@linaro.org \
--cc=briannorris@chromium.org \
--cc=conor+dt@kernel.org \
--cc=david@gibson.dropbear.id.au \
--cc=jwerner@chromium.org \
--cc=konradybcio@kernel.org \
--cc=krzk+dt@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peter.griffin@linaro.org \
--cc=robh@kernel.org \
--cc=saravanak@kernel.org \
--cc=sjg@chromium.org \
--cc=tudor.ambarus@linaro.org \
--cc=wenst@chromium.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®