mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Anup Patel <anup.patel@oss.qualcomm.com>
To: Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Walmsley <pjw@kernel.org>,
	Rahul Pathak <rahul@summations.net>,
	Stephen Boyd <sboyd@kernel.org>,
	Brian Masney <bmasney+clk@redhat.com>,
	Jerome Brunet <jbrunet+clk@baylibre.com>,
	Thomas Gleixner <tglx@kernel.org>, Radu Rendec <radu@rendec.net>,
	Jassi Brar <jassisinghbrar@gmail.com>
Cc: Sunil V L <sunilvl@oss.qualcomm.com>,
	Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>,
	Atish Patra <atish.patra@linux.dev>,
	Anup Patel <anup@brainfault.org>,
	Amirreza Zarrabi <amirreza.zarrabi@oss.qualcomm.com>,
	linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
	linux-clk@vger.kernel.org,
	Anup Patel <anup.patel@oss.qualcomm.com>
Subject: [PATCH v2 4/5] mailbox: document direct synchronous transmit
Date: Mon, 16 Nov 2026 09:30:48 +0530	[thread overview]
Message-ID: <20261116040049.1795610-5-anup.patel@oss.qualcomm.com> (raw)
In-Reply-To: <20261116040049.1795610-1-anup.patel@oss.qualcomm.com>

From: Jassi Brar <jassisinghbrar@gmail.com>

Document the three transmit models supported by the mailbox
framework: non-blocking queued, blocking queued, and direct
synchronous.

Describe the controller-side capability advertisement and
operation contracts, including atomic versus sleepable
operations, per-CPU and controller-wide resource protection,
registration validation, and the client-side context,
lifetime, and free/unbind rules.  Add the normative interaction
table for the queued APIs on a direct channel and examples for
atomic-safe and per-CPU controllers.

Co-developed-by: Anup Patel <anup.patel@oss.qualcomm.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Signed-off-by: Jassi Brar <jassisinghbrar@gmail.com>
---
 Documentation/driver-api/mailbox.rst | 350 +++++++++++++++++++++++++--
 1 file changed, 329 insertions(+), 21 deletions(-)

diff --git a/Documentation/driver-api/mailbox.rst b/Documentation/driver-api/mailbox.rst
index d46c60bd5df6..81d471027539 100644
--- a/Documentation/driver-api/mailbox.rst
+++ b/Documentation/driver-api/mailbox.rst
@@ -20,14 +20,28 @@ ever deal at client's protocol level.
 Some of the choices made during implementation are the result of this
 peculiarity of this "common" framework.
 
+The framework supports three transmit models:
+
+* non-blocking queued: ``mbox_send_message()`` returns a queue token and
+  completion is reported later through ``tx_done``;
+* blocking queued: ``tx_block`` is set and the caller waits on the
+  channel completion;
+* direct synchronous: the controller completes the transfer in the
+  caller's context and the client receives the per-call result directly.
+
+The first two models share the per-channel queue and TX-done state
+machine.  The direct model is opt-in per channel and bypasses that state
+machine entirely.
 
 
 Controller Driver (See include/linux/mailbox_controller.h)
 ==========================================================
 
-
 Allocate mbox_controller and the array of mbox_chan.
-Populate mbox_chan_ops, except flush() and peek_data() all are mandatory.
+Populate mbox_chan_ops.  ``send_data`` is mandatory for queued channels;
+``flush``, ``startup``, ``shutdown``, ``last_tx_done``, and ``peek_data``
+are optional depending on the controller's capabilities and the selected
+TX-done method.
 The controller driver might know a message has been consumed
 by the remote by getting an IRQ or polling some hardware flag
 or it can never know (the client knows by way of the protocol).
@@ -35,14 +49,197 @@ The method in order of preference is IRQ -> Poll -> None, which
 the controller driver should set via 'txdone_irq' or 'txdone_poll'
 or neither.
 
+Queued transmit
+---------------
+
+``send_data()`` is called by ``msg_submit()`` with ``chan->lock`` held and
+local IRQs disabled.  It must not sleep.  It should return 0 if the message
+was accepted, or a negative errno (for example ``-EBUSY``) if it could not
+be accepted.  Completion is reported later with ``mbox_chan_txdone()``
+(IRQ mode) or by the polling timer calling ``last_tx_done()``.  A client
+using the ``knows_txdone`` retrofit calls ``mbox_client_txdone()`` itself.
+
+The queue and TX-done state machine assume a single outstanding request
+per channel.  All senders on a queued channel are therefore serialized by
+the framework, and ``tx_block`` callers must serialize themselves.
+
+Direct synchronous transmit
+---------------------------
+
+Some controllers complete a transfer synchronously in the calling context
+and can return the result immediately.  Such a controller may advertise a
+direct channel by setting ``struct mbox_chan.sync_caps`` before calling
+``mbox_controller_register()``:
+
+.. code-block:: c
+
+    #define MBOX_CHAN_CAP_SYNC_ATOMIC	BIT(0)
+    #define MBOX_CHAN_CAP_SYNC_SLEEPABLE	BIT(1)
+
+A channel with either bit set is a *direct channel*.  ``sync_caps`` is
+immutable after registration; zero means the channel keeps the existing
+queued behavior.  Controllers must therefore initialize it to zero for
+queued channels; ``kzalloc()`` and static storage already do so.
+
+The controller implements one or both of these operations:
+
+.. code-block:: c
+
+    int (*send_data_sync)(struct mbox_chan *chan, void *data);
+    int (*send_data_sync_sleepable)(struct mbox_chan *chan, void *data);
+
+``send_data_sync()``
+    Atomic-safe direct transmit.  It must not sleep, must complete the
+    transfer (or return a per-call error) before returning, and must not
+    use a later TX-done interrupt, polling timer, or ``mbox_chan_txdone()``.
+    It must not call the txdone APIs or any client callback recursively.
+    It must not retain ``data`` after returning.
+
+``send_data_sync_sleepable()``
+    Sleepable direct transmit.  It has the same per-call, no-retained-buffer
+    contract but may use mutexes, completions, sleeping bus access, and other
+    sleepable primitives.  It must only be called from sleepable context.
+
+Both operations are called in the client's context with no framework lock
+held and with the caller's IRQ and preemption state.  The framework calls
+the operation exactly once per client call and returns its result to that
+same caller.
+
+A controller that implements a direct operation but does not set the
+matching capability bit is valid; the operation is then simply unused.
+Registration rejects unknown capability bits, a capability bit without the
+matching operation, and a queued channel without ``send_data``.
+
+Concurrency and resource ownership
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The framework does not serialize direct callers.  Multiple CPUs may call
+the direct API on the same channel concurrently.  This is deliberate: a
+per-channel framework lock would violate the concurrency contract and would
+still not protect controller-wide or per-CPU resources.
+
+The controller owns protection of everything its direct operation touches:
+
+* controller-wide registers and shared FIFOs;
+* per-channel state, which may live in ``chan->con_priv``;
+* per-CPU scratch memory and other per-CPU state;
+* resources shared with non-mailbox users of the same controller helpers.
+
+The controller may take its own lock to serialize, or may return ``-EBUSY``
+to a caller it cannot serve concurrently.  It must not assume that the
+framework disabled IRQs for it.
+
+If the operation uses per-CPU state, it must also prevent same-CPU IRQ
+re-entry.  ``local_irq_save()`` is the portable hardirq-safe choice.
+``local_lock_irqsave()`` gives lockdep coverage on non-RT kernels, but on
+PREEMPT_RT it maps to a per-CPU sleeping spinlock and must not be used from
+hardirq context.  A per-CPU ``raw_spinlock_t`` taken with IRQs disabled is
+another option.  A plain ``get_cpu()``/``put_cpu()`` pair only disables
+preemption and does not stop an IRQ on the same CPU from re-entering.
+
+Example: atomic-safe controller
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: c
+
+    struct my_chan {
+        void __iomem *tx_reg;
+        spinlock_t lock;	/* protects this controller's registers */
+        bool busy;
+    };
+
+    static int my_send_data_sync(struct mbox_chan *chan, void *data)
+    {
+        struct my_chan *c = chan->con_priv;
+        unsigned long flags;
+        u32 status;
+        int ret = 0;
+
+        /*
+         * The framework does not serialize direct callers and does not
+         * disable IRQs.  This controller shares one register block across
+         * all CPUs, so it uses its own IRQ-safe lock.
+         */
+        spin_lock_irqsave(&c->lock, flags);
+        if (c->busy) {
+            ret = -EBUSY;
+            goto out;
+        }
+
+        c->busy = true;
+        writel(*(u32 *)data, c->tx_reg);
+        if (readl_poll_timeout_atomic(c->tx_reg + MY_STATUS, status,
+                                      status & MY_DONE, 1, 1000))
+            ret = -ETIMEDOUT;
+        c->busy = false;
+    out:
+        spin_unlock_irqrestore(&c->lock, flags);
+        return ret;
+    }
+
+    static const struct mbox_chan_ops my_ops = {
+        .send_data_sync = my_send_data_sync,
+        /* .send_data may be NULL if every channel is direct-only. */
+    };
+
+    static int my_probe(struct platform_device *pdev)
+    {
+        ...
+        for (i = 0; i < MY_NUM_CHANS; i++)
+            mbox->chans[i].sync_caps = MBOX_CHAN_CAP_SYNC_ATOMIC;
+        ...
+        return devm_mbox_controller_register(dev, &mbox->controller);
+    }
+
+Example: per-CPU shared-memory controller
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A controller that hands a per-CPU buffer to firmware must guard the whole
+copy/firmware-call/copy sequence, not just the copy:
+
+.. code-block:: c
+
+    static int my_send_data_sync(struct mbox_chan *chan, void *data)
+    {
+        struct my_msg *msg = data;
+        int ret;
+
+        /*
+         * The firmware uses per-CPU scratch memory.  A hardirq on this CPU
+         * could otherwise re-enter and clobber it between the copy and the
+         * firmware call.
+         */
+        guard(irqsave)();
+        memcpy(this_cpu_ptr(&my_buf)->data, msg->tx, msg->len);
+        ret = my_firmware_call(this_cpu_ptr(&my_buf)->phys, msg->len);
+        if (!ret)
+            memcpy(msg->rx, this_cpu_ptr(&my_buf)->data, msg->rx_len);
+        return ret;
+    }
+
+Every user of the per-CPU buffer must take the same guard.  Attribute
+queries, notification fetches, MSI programming, and non-mailbox users that
+share the buffer are all part of the same critical section.  Delivering RX
+through ``mbox_chan_received_data()`` while holding that guard can deadlock
+if the client sends from ``rx_callback``; release the guard first or use a
+separate RX lock.
+
 
 Client Driver (See include/linux/mailbox_client.h)
 ==================================================
 
+The client might want to operate in one of three modes:
 
-The client might want to operate in blocking mode (synchronously
-send a message through before returning) or non-blocking/async mode (submit
-a message and a callback function to the API and return immediately).
+* non-blocking queued: submit a message and a callback, return immediately;
+* blocking queued: wait until the queued transfer completes or times out;
+* direct synchronous: call the controller operation and receive its result.
+
+The first two use ``mbox_send_message()`` and the queued state machine.
+The third uses ``mbox_send_message_sync()`` or
+``mbox_send_message_sync_sleepable()`` and must be used on a direct channel.
+
+Blocking and non-blocking queued modes
+--------------------------------------
 
 ::
 
@@ -82,12 +279,12 @@ a message and a callback function to the API and return immediately).
 
 	static void client_demo(struct platform_device *pdev)
 	{
-		struct demo_client *dc_sync, *dc_async;
-		/* The controller already knows async_pkt and sync_pkt */
+		struct demo_client *dc_blocking, *dc_async;
+		/* The controller already knows async_pkt and blocking_pkt */
 		struct async_pkt ap;
-		struct sync_pkt sp;
+		struct blocking_pkt bp;
 
-		dc_sync = kzalloc_obj(*dc_sync);
+		dc_blocking = kzalloc_obj(*dc_blocking);
 		dc_async = kzalloc_obj(*dc_async);
 
 		/* Populate non-blocking mode client */
@@ -101,13 +298,13 @@ a message and a callback function to the API and return immediately).
 		init_completion(&dc_async->c);
 
 		/* Populate blocking mode client */
-		dc_sync->cl.dev = &pdev->dev;
-		dc_sync->cl.rx_callback = message_from_remote;
-		dc_sync->cl.tx_done = NULL; /* operate in blocking mode */
-		dc_sync->cl.tx_block = true;
-		dc_sync->cl.tx_tout = 500; /* by half a second */
-		dc_sync->cl.knows_txdone = false; /* depending upon protocol */
-		dc_sync->async = false;
+		dc_blocking->cl.dev = &pdev->dev;
+		dc_blocking->cl.rx_callback = message_from_remote;
+		dc_blocking->cl.tx_done = NULL; /* operate in blocking mode */
+		dc_blocking->cl.tx_block = true;
+		dc_blocking->cl.tx_tout = 500; /* by half a second */
+		dc_blocking->cl.knows_txdone = false; /* depending upon protocol */
+		dc_blocking->async = false;
 
 		/* ASync mailbox is listed second in 'mboxes' property */
 		dc_async->mbox = mbox_request_channel(&dc_async->cl, 1);
@@ -116,14 +313,125 @@ a message and a callback function to the API and return immediately).
 		/* Send async message to remote */
 		mbox_send_message(dc_async->mbox, &ap);
 
-		/* Sync mailbox is listed first in 'mboxes' property */
-		dc_sync->mbox = mbox_request_channel(&dc_sync->cl, 0);
+		/* Blocking mailbox is listed first in 'mboxes' property */
+		dc_blocking->mbox = mbox_request_channel(&dc_blocking->cl, 0);
 		/* Populate data packet */
-		/* sp.abc = 123; etc */
+		/* bp.abc = 123; etc */
 		/* Send message to remote in blocking mode */
-		mbox_send_message(dc_sync->mbox, &sp);
-		/* At this point 'sp' has been sent */
+		mbox_send_message(dc_blocking->mbox, &bp);
+		/* At this point 'bp' has been sent */
 
 		/* Now wait for async chan to be done */
 		wait_for_completion(&dc_async->c);
 	}
+
+The documentation uses "blocking queued" rather than "sync" for
+``tx_block=true`` so that it is not confused with direct synchronous
+transmit.  ``tx_block`` callers must serialize themselves on a channel.
+
+Direct synchronous mode
+-----------------------
+
+.. code-block:: c
+
+    int __must_check mbox_send_message_sync(struct mbox_chan *chan,
+                                            void *mssg);
+    int __must_check mbox_send_message_sync_sleepable(struct mbox_chan *chan,
+                                                      void *mssg);
+
+``mbox_send_message_sync()``
+    Calls ``send_data_sync()``.  It never sleeps and may be called from any
+    context in which the controller operation is safe, including hardirq.
+    It does not fall back to a sleepable operation.
+
+``mbox_send_message_sync_sleepable()``
+    Calls ``send_data_sync_sleepable()`` when the channel advertises it.
+    If only ``MBOX_CHAN_CAP_SYNC_ATOMIC`` is advertised, it calls the
+    stronger ``send_data_sync()`` instead.  It calls ``might_sleep()`` and
+    may therefore be used only from sleepable context.
+
+Both functions return 0 on success, ``-EINVAL`` for invalid arguments,
+``-EOPNOTSUPP`` when the channel does not support the requested direct
+mode, or another negative errno from the controller.  Positive values are
+not used by the direct API; controllers must return 0 or a negative errno.
+The declarations are ``__must_check`` because the per-call result is the
+whole point of the API.
+
+The caller must keep the message buffer valid until the call returns.  The
+controller must not retain the pointer afterwards.  The caller is also
+responsible for serializing the call against ``mbox_free_channel()`` and
+driver removal; the framework does not wait for or refcount in-flight direct
+calls.  The ``chan->cl`` check in the wrapper is only a best-effort sanity
+check.
+
+The capability probe is the call itself: a client may attempt the direct
+API and handle ``-EOPNOTSUPP``.  There is no capability query helper in the
+minimal API.
+
+Example:
+
+.. code-block:: c
+
+    ret = mbox_send_message_sync(client->chan, msg);
+    if (ret)
+        /* Handle the per-call error. */
+
+The direct path calls neither ``tx_prepare`` nor ``tx_done``.  The client
+prepares its buffer before the call, and the return value is the completion
+notification.  ``tx_block``, ``tx_tout``, and ``knows_txdone`` are ignored
+on a direct channel because ``mbox_send_message()`` and
+``mbox_client_txdone()`` are rejected there.
+
+Interaction with the existing API
+=================================
+
+The following table is normative for a direct channel
+(``sync_caps != 0``):
+
+.. list-table:: Interaction with the existing API on a direct channel
+   :header-rows: 1
+   :widths: 30 70
+
+   * - API / callback
+     - Behavior on a direct channel
+   * - ``mbox_send_message()``
+     - Returns ``-EOPNOTSUPP``.  Mixing queued and direct paths would
+       reintroduce the single-request state machine.
+   * - ``mbox_flush()``
+     - Returns ``-EOPNOTSUPP``.  There is no framework transmission to
+       flush.
+   * - ``tx_prepare``
+     - Never called.  The client prepares its own buffer.
+   * - ``tx_done``
+     - Never called.  The direct return value is the notification.
+   * - ``mbox_chan_txdone()``
+     - ``WARN_ON_ONCE()`` and no-op.  The controller must return the result
+       from the direct operation.
+   * - ``mbox_client_txdone()``
+     - ``WARN_ON_ONCE()`` and no-op.  Same reason.
+   * - ``mbox_chan_tx_slots_available()``
+     - Returns 0.  A direct channel has no queue.
+   * - ``tx_block`` / ``tx_tout``
+     - Ignored.  ``mbox_send_message()`` is rejected.
+   * - ``knows_txdone``
+     - Ignored.  ``mbox_client_txdone()`` is rejected.
+   * - ``startup`` / ``shutdown``
+     - Unchanged.  Called on bind/unbind and may sleep.
+   * - ``rx_callback`` / ``peek_data`` / ``mbox_chan_received_data()``
+     - Unchanged.  Receive delivery is independent of the transmit mode.
+   * - ``mbox_free_channel()``
+     - Unchanged.  The client must serialize it against in-flight direct
+       calls.
+   * - debugfs mailbox summary
+     - Unchanged; it only reports channel ownership.
+
+Debugging aids provided by the framework are
+``lockdep_assert_not_held(&chan->lock)`` in both direct wrappers,
+``might_sleep()`` in the sleepable wrapper, txdone misuse warnings, and
+registration-time capability validation.  ``CONFIG_DEBUG_ATOMIC_SLEEP`` and
+``CONFIG_PROVE_LOCKING`` catch controller bugs that the framework cannot
+detect statically.
+
+The KUnit suite in ``drivers/mailbox/mailbox-sync-test.c`` exercises the
+capability matrix, registration validation, per-call results, multi-CPU
+concurrency, hardirq context, queued compatibility, and free/rebind.
-- 
2.43.0


  parent reply	other threads:[~2026-11-16  4:01 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  4:00 [PATCH v2 0/5] Parallel message send using SBI MPXY Anup Patel
2026-11-16  4:00 ` [PATCH v2 1/5] RISC-V: Move common MPXY helper routines to arch/riscv Anup Patel
2026-11-16  4:00 ` [PATCH v2 2/5] RISC-V: Use irqsave to prevent interrupt when accessing MPXY shared memory Anup Patel
2026-11-16  4:00 ` [PATCH v2 3/5] mailbox: add direct synchronous transmit API Anup Patel
2026-11-16  4:00 ` Anup Patel [this message]
2026-11-16  4:00 ` [PATCH v2 5/5] mailbox: convert RISC-V SBI MPXY to direct transmit Anup Patel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116040049.1795610-5-anup.patel@oss.qualcomm.com \
    --to=anup.patel@oss.qualcomm.com \
    --cc=amirreza.zarrabi@oss.qualcomm.com \
    --cc=anup@brainfault.org \
    --cc=atish.patra@linux.dev \
    --cc=bmasney+clk@redhat.com \
    --cc=himanshu.chauhan@oss.qualcomm.com \
    --cc=jassisinghbrar@gmail.com \
    --cc=jbrunet+clk@baylibre.com \
    --cc=linux-clk@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=palmer@dabbelt.com \
    --cc=pjw@kernel.org \
    --cc=radu@rendec.net \
    --cc=rahul@summations.net \
    --cc=sboyd@kernel.org \
    --cc=sunilvl@oss.qualcomm.com \
    --cc=tglx@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®