mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ivan Vecera <ivecera@redhat.com>
To: Jakub Kicinski <kuba@kernel.org>,
	Vadim Fedorenko <vadim.fedorenko@linux.dev>,
	Jiri Pirko <jiri@resnulli.us>,
	Arkadiusz Kubalewski <arkadiusz.kubalewski@intel.com>
Cc: netdev@vger.kernel.org, Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Min Li <min.li@microchip.com>,
	Tony Nguyen <anthony.l.nguyen@intel.com>,
	Przemek Kitszel <przemyslaw.kitszel@intel.com>,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	Paolo Abeni <pabeni@redhat.com>,
	Richard Cochran <richardcochran@gmail.com>,
	linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	intel-wired-lan@lists.osuosl.org,
	Grzegorz Nitka <grzegorz.nitka@intel.com>,
	Karol Kolacinski <karol.kolacinski@intel.com>,
	Saeed Mahameed <saeedm@nvidia.com>,
	Tariq Toukan <tariqt@nvidia.com>, Petr Oros <poros@redhat.com>
Subject: Re: [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock
Date: Fri, 20 Nov 2026 08:39:01 +0100	[thread overview]
Message-ID: <bdfa88fe-13bb-4050-b750-962586b1d846@redhat.com> (raw)
In-Reply-To: <20261119133851.3f25074c@kernel.org>

On 11/19/26 22:38, Jakub Kicinski wrote:
> Well, I'm not convinced. The relations between objects in DPLL are
> already quite hard to follow, and objects from one driver routinely
> "plug" into objects from another. The explanation of the locking
> rules in patch 9 does not look clean or clear. The driver facing
> API needs to be much more "dummy-proof".

Fair enough. The complicated part of the rules comes exactly from
that: a pin registered by another driver, like the zl3073x pins that
ice registers with its TXC DPLL or uses as parents of its RCLK pins.
A message about such a pin runs callbacks of both drivers. I have
prototypes of the first four ways below to lock this, each with KUnit
tests, and would like your opinion before going further.

In all of them a domain is the set of objects created with the same
module and clock_id, dpll_lock becomes an rwsem for the topology and
lifetime, and a mutex per domain serializes the callbacks.

A) Pin's domain (this series)

    Callbacks of a pin run under the domain of the pin, whichever
    driver registered them.

    Pros:
    * one domain lock at a time, so no lock order
    * a request or a message about a pin is atomic
    Cons:
    * the callbacks of a foreign pin are serialized with the driver
      that created the pin, not with the other callbacks of the driver
      that registered it, so that driver has to protect the shared
      state itself, and lockdep cannot check it
    * restrictions for measured_freq_get, reference sync pairs and the
      private data of a parent pin, which patch 9 has to explain

B) Registrant's domain, deferred notifications

    Each callback runs under the domain of the driver that registered
    it. A message about a pin takes the domain locks one after another,
    and notifications from callbacks are queued and sent once the
    domain lock is dropped.

    Pros:
    * callbacks registered by one driver instance are always
      serialized with each other
    * one domain lock at a time, so no lock order
    Cons:
    * a request or a message about a pin registered across domains is
      not atomic
    * the most complex core: a notification queue per domain, a lock
      per object to keep its messages in order, and pin setters split
      into validation and apply parts

C) Registrant's domain nested under the pin's domain

    Pros:
    * callbacks registered by one driver instance are serialized with
      each other and with the driver that created the pin
    * a request or a message about a pin is atomic
    Cons:
    * the lock order comes from the topology, so a pin of A registered
      with a DPLL of B together with a pin of B registered with a DPLL
      of A deadlocks, and lockdep sees it only with a lockdep class per
      domain
    * a stuck foreign registrant stops the domain of the pin

D) Merged domains (your suggestion)

    Registering a pin with a DPLL or a parent pin of another domain
    merges both domains. They are not split again.

    Pros:
    * the rules for drivers stay as they are with the global lock
      today, and lockdep checks them; the only new one is that state
      shared between chips not connected by a pin needs a driver lock
    * a request or a message is atomic
    Cons:
    * only hardware not connected by pins runs in parallel; on the
      GNR-D system the zl3073x and all ice PFs end up in one domain
    * a callback that waits for the hardware, like the planned resync
      for up to 1 s, or for firmware still stalls all DPLL devices in
      the merged domain

E) Global dpll_lock, slow callbacks asynchronous

    Pros:
    * no change in the core
    Cons:
    * the resync problems from my previous mail, and a callback that
      waits for firmware still stalls all DPLL devices in the system

I wrote before that stretching the domains would not improve
anything on that system. That holds for the ToD stall I measured,
but the zl3073x does not need to hold its channel lock across the
edge waits. Without that, the remaining waits are the planned resync
and output state callbacks, and with D they stall only the devices
wired to that chip, which depend on it anyway.

So I lean towards D, as it gives drivers nothing new to get right.
Would that address your concern, or do you see a better way?

It would be good to hear from the other DPLL maintainers as well.
Jiri, Vadim, Arkadiusz, what do you think?

Thanks,
Ivan


      reply	other threads:[~2026-11-20  7:39 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11 16:40 Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 1/9] ice: dpll: notify TSPLL lock status from the periodic work only Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 2/9] dpll: add domain object Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 3/9] dpll: take domain locks around driver callbacks Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 4/9] dpll: convert dpll_lock to rwsem Ivan Vecera
2026-11-16  8:21   ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 5/9] dpll: add KUnit tests for core locking Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 6/9] dpll: reject measured_freq_get for foreign and on-pin registrations Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 7/9] dpll: zl3073x: update locking comments Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 8/9] ice: dpll: " Ivan Vecera
2026-11-16  8:22   ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 9/9] dpll: document locking rules Ivan Vecera
2026-11-16  8:23   ` Ivan Vecera
2026-11-16  8:24 ` [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock Ivan Vecera
2026-11-17  2:17   ` Jakub Kicinski
2026-11-18 15:30     ` Ivan Vecera
2026-11-17  2:16 ` Jakub Kicinski
2026-11-18 15:29   ` Ivan Vecera
2026-11-19  1:44     ` Jakub Kicinski
2026-11-19 12:29       ` Ivan Vecera
2026-11-19 21:38         ` Jakub Kicinski
2026-11-20  7:39           ` Ivan Vecera [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bdfa88fe-13bb-4050-b750-962586b1d846@redhat.com \
    --to=ivecera@redhat.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=anthony.l.nguyen@intel.com \
    --cc=arkadiusz.kubalewski@intel.com \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=grzegorz.nitka@intel.com \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=jiri@resnulli.us \
    --cc=karol.kolacinski@intel.com \
    --cc=kuba@kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=min.li@microchip.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=poros@redhat.com \
    --cc=przemyslaw.kitszel@intel.com \
    --cc=rdunlap@infradead.org \
    --cc=richardcochran@gmail.com \
    --cc=saeedm@nvidia.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tariqt@nvidia.com \
    --cc=vadim.fedorenko@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®