mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jiri Pirko <jiri@resnulli.us>
To: Vadim Fedorenko <vadim.fedorenko@linux.dev>
Cc: Ivan Vecera <ivecera@redhat.com>,
	Jakub Kicinski <kuba@kernel.org>,
	 Arkadiusz Kubalewski <arkadiusz.kubalewski@intel.com>,
	netdev@vger.kernel.org, Jonathan Corbet <corbet@lwn.net>,
	 Shuah Khan <skhan@linuxfoundation.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	 Min Li <min.li@microchip.com>,
	Tony Nguyen <anthony.l.nguyen@intel.com>,
	 Przemek Kitszel <przemyslaw.kitszel@intel.com>,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	 "David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	 Paolo Abeni <pabeni@redhat.com>,
	Richard Cochran <richardcochran@gmail.com>,
	 linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	intel-wired-lan@lists.osuosl.org,
	 Grzegorz Nitka <grzegorz.nitka@intel.com>,
	Karol Kolacinski <karol.kolacinski@intel.com>,
	 Saeed Mahameed <saeedm@nvidia.com>,
	Tariq Toukan <tariqt@nvidia.com>, Petr Oros <poros@redhat.com>
Subject: Re: [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock
Date: Sun, 22 Nov 2026 16:54:05 +0100	[thread overview]
Message-ID: <awMP9Gj3Xhia0ZeI@FV6GYCPJ69> (raw)
In-Reply-To: <ff8b63b7-b943-444a-bd37-b6fb493b5d9f@linux.dev>

Fri, Nov 20, 2026 at 09:00:27PM +0100, vadim.fedorenko@linux.dev wrote:
>On 20/11/2026 07:39, Ivan Vecera wrote:
>> On 11/19/26 22:38, Jakub Kicinski wrote:
>> > Well, I'm not convinced. The relations between objects in DPLL are
>> > already quite hard to follow, and objects from one driver routinely
>> > "plug" into objects from another. The explanation of the locking
>> > rules in patch 9 does not look clean or clear. The driver facing
>> > API needs to be much more "dummy-proof".
>> 
>> Fair enough. The complicated part of the rules comes exactly from
>> that: a pin registered by another driver, like the zl3073x pins that
>> ice registers with its TXC DPLL or uses as parents of its RCLK pins.
>> A message about such a pin runs callbacks of both drivers. I have
>> prototypes of the first four ways below to lock this, each with KUnit
>> tests, and would like your opinion before going further.
>> 
>> In all of them a domain is the set of objects created with the same
>> module and clock_id, dpll_lock becomes an rwsem for the topology and
>> lifetime, and a mutex per domain serializes the callbacks.
>> 
>> A) Pin's domain (this series)
>> 
>>     Callbacks of a pin run under the domain of the pin, whichever
>>     driver registered them.
>> 
>>     Pros:
>>     * one domain lock at a time, so no lock order
>>     * a request or a message about a pin is atomic
>>     Cons:
>>     * the callbacks of a foreign pin are serialized with the driver
>>       that created the pin, not with the other callbacks of the driver
>>       that registered it, so that driver has to protect the shared
>>       state itself, and lockdep cannot check it
>>     * restrictions for measured_freq_get, reference sync pairs and the
>>       private data of a parent pin, which patch 9 has to explain
>> 
>> B) Registrant's domain, deferred notifications
>> 
>>     Each callback runs under the domain of the driver that registered
>>     it. A message about a pin takes the domain locks one after another,
>>     and notifications from callbacks are queued and sent once the
>>     domain lock is dropped.
>> 
>>     Pros:
>>     * callbacks registered by one driver instance are always
>>       serialized with each other
>>     * one domain lock at a time, so no lock order
>>     Cons:
>>     * a request or a message about a pin registered across domains is
>>       not atomic
>>     * the most complex core: a notification queue per domain, a lock
>>       per object to keep its messages in order, and pin setters split
>>       into validation and apply parts
>> 
>> C) Registrant's domain nested under the pin's domain
>> 
>>     Pros:
>>     * callbacks registered by one driver instance are serialized with
>>       each other and with the driver that created the pin
>>     * a request or a message about a pin is atomic
>>     Cons:
>>     * the lock order comes from the topology, so a pin of A registered
>>       with a DPLL of B together with a pin of B registered with a DPLL
>>       of A deadlocks, and lockdep sees it only with a lockdep class per
>>       domain
>>     * a stuck foreign registrant stops the domain of the pin
>> 
>> D) Merged domains (your suggestion)
>> 
>>     Registering a pin with a DPLL or a parent pin of another domain
>>     merges both domains. They are not split again.
>> 
>>     Pros:
>>     * the rules for drivers stay as they are with the global lock
>>       today, and lockdep checks them; the only new one is that state
>>       shared between chips not connected by a pin needs a driver lock
>>     * a request or a message is atomic
>>     Cons:
>>     * only hardware not connected by pins runs in parallel; on the
>>       GNR-D system the zl3073x and all ice PFs end up in one domain
>>     * a callback that waits for the hardware, like the planned resync
>>       for up to 1 s, or for firmware still stalls all DPLL devices in
>>       the merged domain
>> 
>> E) Global dpll_lock, slow callbacks asynchronous
>> 
>>     Pros:
>>     * no change in the core
>>     Cons:
>>     * the resync problems from my previous mail, and a callback that
>>       waits for firmware still stalls all DPLL devices in the system
>> 
>> I wrote before that stretching the domains would not improve
>> anything on that system. That holds for the ToD stall I measured,
>> but the zl3073x does not need to hold its channel lock across the
>> edge waits. Without that, the remaining waits are the planned resync
>> and output state callbacks, and with D they stall only the devices
>> wired to that chip, which depend on it anyway.
>> 
>> So I lean towards D, as it gives drivers nothing new to get right.
>> Would that address your concern, or do you see a better way?
>> 
>> It would be good to hear from the other DPLL maintainers as well.
>> Jiri, Vadim, Arkadiusz, what do you think?
>
>I totally agree with Jakub. Current implementation of DPLL subsystem is
>over complicated, and all these complications came just to completely
>model what was designed for Intel NICs, basically ice driver. But now we
>are again back to the discussion how to complicate things further just
>to fit what was implemented in the new generation of the same hardware.
>
>Maybe it's time to step back and think if we really need all this, or
>find simpler way to express/model things?

Agreed.


      reply	other threads:[~2026-11-22 15:54 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11 16:40 Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 1/9] ice: dpll: notify TSPLL lock status from the periodic work only Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 2/9] dpll: add domain object Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 3/9] dpll: take domain locks around driver callbacks Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 4/9] dpll: convert dpll_lock to rwsem Ivan Vecera
2026-11-16  8:21   ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 5/9] dpll: add KUnit tests for core locking Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 6/9] dpll: reject measured_freq_get for foreign and on-pin registrations Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 7/9] dpll: zl3073x: update locking comments Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 8/9] ice: dpll: " Ivan Vecera
2026-11-16  8:22   ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 9/9] dpll: document locking rules Ivan Vecera
2026-11-16  8:23   ` Ivan Vecera
2026-11-16  8:24 ` [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock Ivan Vecera
2026-11-17  2:17   ` Jakub Kicinski
2026-11-18 15:30     ` Ivan Vecera
2026-11-17  2:16 ` Jakub Kicinski
2026-11-18 15:29   ` Ivan Vecera
2026-11-19  1:44     ` Jakub Kicinski
2026-11-19 12:29       ` Ivan Vecera
2026-11-19 21:38         ` Jakub Kicinski
2026-11-20  7:39           ` Ivan Vecera
2026-11-20 20:00             ` Vadim Fedorenko
2026-11-22 15:54               ` Jiri Pirko [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=awMP9Gj3Xhia0ZeI@FV6GYCPJ69 \
    --to=jiri@resnulli.us \
    --cc=andrew+netdev@lunn.ch \
    --cc=anthony.l.nguyen@intel.com \
    --cc=arkadiusz.kubalewski@intel.com \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=grzegorz.nitka@intel.com \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=ivecera@redhat.com \
    --cc=karol.kolacinski@intel.com \
    --cc=kuba@kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=min.li@microchip.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=poros@redhat.com \
    --cc=przemyslaw.kitszel@intel.com \
    --cc=rdunlap@infradead.org \
    --cc=richardcochran@gmail.com \
    --cc=saeedm@nvidia.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tariqt@nvidia.com \
    --cc=vadim.fedorenko@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®