From: Jiri Pirko <jiri@resnulli.us>
To: Vadim Fedorenko <vadim.fedorenko@linux.dev>
Cc: Ivan Vecera <ivecera@redhat.com>,
Jakub Kicinski <kuba@kernel.org>,
Arkadiusz Kubalewski <arkadiusz.kubalewski@intel.com>,
netdev@vger.kernel.org, Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Randy Dunlap <rdunlap@infradead.org>,
Min Li <min.li@microchip.com>,
Tony Nguyen <anthony.l.nguyen@intel.com>,
Przemek Kitszel <przemyslaw.kitszel@intel.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Paolo Abeni <pabeni@redhat.com>,
Richard Cochran <richardcochran@gmail.com>,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
intel-wired-lan@lists.osuosl.org,
Grzegorz Nitka <grzegorz.nitka@intel.com>,
Karol Kolacinski <karol.kolacinski@intel.com>,
Saeed Mahameed <saeedm@nvidia.com>,
Tariq Toukan <tariqt@nvidia.com>, Petr Oros <poros@redhat.com>
Subject: Re: [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock
Date: Sun, 22 Nov 2026 16:54:05 +0100 [thread overview]
Message-ID: <awMP9Gj3Xhia0ZeI@FV6GYCPJ69> (raw)
In-Reply-To: <ff8b63b7-b943-444a-bd37-b6fb493b5d9f@linux.dev>
Fri, Nov 20, 2026 at 09:00:27PM +0100, vadim.fedorenko@linux.dev wrote:
>On 20/11/2026 07:39, Ivan Vecera wrote:
>> On 11/19/26 22:38, Jakub Kicinski wrote:
>> > Well, I'm not convinced. The relations between objects in DPLL are
>> > already quite hard to follow, and objects from one driver routinely
>> > "plug" into objects from another. The explanation of the locking
>> > rules in patch 9 does not look clean or clear. The driver facing
>> > API needs to be much more "dummy-proof".
>>
>> Fair enough. The complicated part of the rules comes exactly from
>> that: a pin registered by another driver, like the zl3073x pins that
>> ice registers with its TXC DPLL or uses as parents of its RCLK pins.
>> A message about such a pin runs callbacks of both drivers. I have
>> prototypes of the first four ways below to lock this, each with KUnit
>> tests, and would like your opinion before going further.
>>
>> In all of them a domain is the set of objects created with the same
>> module and clock_id, dpll_lock becomes an rwsem for the topology and
>> lifetime, and a mutex per domain serializes the callbacks.
>>
>> A) Pin's domain (this series)
>>
>> Callbacks of a pin run under the domain of the pin, whichever
>> driver registered them.
>>
>> Pros:
>> * one domain lock at a time, so no lock order
>> * a request or a message about a pin is atomic
>> Cons:
>> * the callbacks of a foreign pin are serialized with the driver
>> that created the pin, not with the other callbacks of the driver
>> that registered it, so that driver has to protect the shared
>> state itself, and lockdep cannot check it
>> * restrictions for measured_freq_get, reference sync pairs and the
>> private data of a parent pin, which patch 9 has to explain
>>
>> B) Registrant's domain, deferred notifications
>>
>> Each callback runs under the domain of the driver that registered
>> it. A message about a pin takes the domain locks one after another,
>> and notifications from callbacks are queued and sent once the
>> domain lock is dropped.
>>
>> Pros:
>> * callbacks registered by one driver instance are always
>> serialized with each other
>> * one domain lock at a time, so no lock order
>> Cons:
>> * a request or a message about a pin registered across domains is
>> not atomic
>> * the most complex core: a notification queue per domain, a lock
>> per object to keep its messages in order, and pin setters split
>> into validation and apply parts
>>
>> C) Registrant's domain nested under the pin's domain
>>
>> Pros:
>> * callbacks registered by one driver instance are serialized with
>> each other and with the driver that created the pin
>> * a request or a message about a pin is atomic
>> Cons:
>> * the lock order comes from the topology, so a pin of A registered
>> with a DPLL of B together with a pin of B registered with a DPLL
>> of A deadlocks, and lockdep sees it only with a lockdep class per
>> domain
>> * a stuck foreign registrant stops the domain of the pin
>>
>> D) Merged domains (your suggestion)
>>
>> Registering a pin with a DPLL or a parent pin of another domain
>> merges both domains. They are not split again.
>>
>> Pros:
>> * the rules for drivers stay as they are with the global lock
>> today, and lockdep checks them; the only new one is that state
>> shared between chips not connected by a pin needs a driver lock
>> * a request or a message is atomic
>> Cons:
>> * only hardware not connected by pins runs in parallel; on the
>> GNR-D system the zl3073x and all ice PFs end up in one domain
>> * a callback that waits for the hardware, like the planned resync
>> for up to 1 s, or for firmware still stalls all DPLL devices in
>> the merged domain
>>
>> E) Global dpll_lock, slow callbacks asynchronous
>>
>> Pros:
>> * no change in the core
>> Cons:
>> * the resync problems from my previous mail, and a callback that
>> waits for firmware still stalls all DPLL devices in the system
>>
>> I wrote before that stretching the domains would not improve
>> anything on that system. That holds for the ToD stall I measured,
>> but the zl3073x does not need to hold its channel lock across the
>> edge waits. Without that, the remaining waits are the planned resync
>> and output state callbacks, and with D they stall only the devices
>> wired to that chip, which depend on it anyway.
>>
>> So I lean towards D, as it gives drivers nothing new to get right.
>> Would that address your concern, or do you see a better way?
>>
>> It would be good to hear from the other DPLL maintainers as well.
>> Jiri, Vadim, Arkadiusz, what do you think?
>
>I totally agree with Jakub. Current implementation of DPLL subsystem is
>over complicated, and all these complications came just to completely
>model what was designed for Intel NICs, basically ice driver. But now we
>are again back to the discussion how to complicate things further just
>to fit what was implemented in the new generation of the same hardware.
>
>Maybe it's time to step back and think if we really need all this, or
>find simpler way to express/model things?
Agreed.
prev parent reply other threads:[~2026-11-22 15:54 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 16:40 Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 1/9] ice: dpll: notify TSPLL lock status from the periodic work only Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 2/9] dpll: add domain object Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 3/9] dpll: take domain locks around driver callbacks Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 4/9] dpll: convert dpll_lock to rwsem Ivan Vecera
2026-11-16 8:21 ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 5/9] dpll: add KUnit tests for core locking Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 6/9] dpll: reject measured_freq_get for foreign and on-pin registrations Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 7/9] dpll: zl3073x: update locking comments Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 8/9] ice: dpll: " Ivan Vecera
2026-11-16 8:22 ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 9/9] dpll: document locking rules Ivan Vecera
2026-11-16 8:23 ` Ivan Vecera
2026-11-16 8:24 ` [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock Ivan Vecera
2026-11-17 2:17 ` Jakub Kicinski
2026-11-18 15:30 ` Ivan Vecera
2026-11-17 2:16 ` Jakub Kicinski
2026-11-18 15:29 ` Ivan Vecera
2026-11-19 1:44 ` Jakub Kicinski
2026-11-19 12:29 ` Ivan Vecera
2026-11-19 21:38 ` Jakub Kicinski
2026-11-20 7:39 ` Ivan Vecera
2026-11-20 20:00 ` Vadim Fedorenko
2026-11-22 15:54 ` Jiri Pirko [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=awMP9Gj3Xhia0ZeI@FV6GYCPJ69 \
--to=jiri@resnulli.us \
--cc=andrew+netdev@lunn.ch \
--cc=anthony.l.nguyen@intel.com \
--cc=arkadiusz.kubalewski@intel.com \
--cc=corbet@lwn.net \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=grzegorz.nitka@intel.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=ivecera@redhat.com \
--cc=karol.kolacinski@intel.com \
--cc=kuba@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=min.li@microchip.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=poros@redhat.com \
--cc=przemyslaw.kitszel@intel.com \
--cc=rdunlap@infradead.org \
--cc=richardcochran@gmail.com \
--cc=saeedm@nvidia.com \
--cc=skhan@linuxfoundation.org \
--cc=tariqt@nvidia.com \
--cc=vadim.fedorenko@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®