[PATCH v2 0/4] staging: fsl-mc: MC command serialization

[PATCH v2 0/4] staging: fsl-mc: MC command serialization

[J. German Rivera]

This patch series depends on the patch series posted
at http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg999081.html

This patch series addresses the following item from the TODO list
for the MC bus driver to exit staging:

* Management Complex (MC) command serialization. Locking mechanisms
  are needed by drivers to serialize commands sent to the MC, including
  from atomic context.

CHANGE HISTORY

Changes in v2:
- Updated link to patch series this patch series depends on.
- Added missing Signed-off-by entries.

Patch 1: Changed timeout units for MC cmd completion
Patch 2: Refactored mc_send_command()
Patch 3: Added support for atomic portals
Patch 4: Added serialization to mc_send_command()

[PATCH v2 1/4] staging: fsl-mc: changed timeout units for MC cmd completion

[J. German Rivera]

Changed units for the timeout to wait for completion
of MC command, from jiffies to milliseconds.

Signed-off-by: J. German Rivera <German.Rivera@freescale.com>
---
Changes in v2:
- Added missing Signed-off-by entry

 drivers/staging/fsl-mc/bus/mc-sys.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index 07848a0..2c5f109 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -42,9 +42,9 @@
 #include "dpmcp.h"

 /**
- * Timeout in jiffies to wait for the completion of an MC command
+ * Timeout in milliseconds to wait for the completion of an MC command
  */
-#define MC_CMD_COMPLETION_TIMEOUT_JIFFIES   (HZ / 2)	/* 500 ms */
+#define MC_CMD_COMPLETION_TIMEOUT_MS	500

 /*
  * usleep_range() min and max values used to throttle down polling
@@ -305,7 +305,7 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	enum mc_cmd_status status;
 	unsigned long jiffies_until_timeout =
-	    jiffies + MC_CMD_COMPLETION_TIMEOUT_JIFFIES;
+		jiffies + msecs_to_jiffies(MC_CMD_COMPLETION_TIMEOUT_MS);

 	/*
 	 * Send command to the MC hardware:
--
2.3.3

[PATCH v2 2/4] staging: fsl-mc: refactored mc_send_command()

[J. German Rivera]

Moved wait logic in mc_send_command() to its own function

Signed-off-by: J. German Rivera <German.Rivera@freescale.com>
---
Changes in v2:
- Added missing Signed-off-by entry

 drivers/staging/fsl-mc/bus/mc-sys.c | 50 ++++++++++++++++++++++++++++---------
 1 file changed, 38 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index 2c5f109..c4f5bdd 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -292,27 +292,22 @@ static inline enum mc_cmd_status mc_read_response(struct mc_command __iomem *
 }

 /**
- * Sends an command to the MC device using the given MC I/O object
+ * Waits for the completion of an MC command doing preemptible polling.
+ * uslepp_range() is called between polling iterations.
  *
  * @mc_io: MC I/O object to be used
- * @cmd: command to be sent
- *
- * Returns '0' on Success; Error code otherwise.
- *
- * NOTE: This function cannot be invoked from from atomic contexts.
+ * @cmd: command buffer to receive MC response
+ * @mc_status: MC command completion status
  */
-int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
+static int mc_polling_wait_preemptible(struct fsl_mc_io *mc_io,
+				       struct mc_command *cmd,
+				       enum mc_cmd_status *mc_status)
 {
 	enum mc_cmd_status status;
 	unsigned long jiffies_until_timeout =
 		jiffies + msecs_to_jiffies(MC_CMD_COMPLETION_TIMEOUT_MS);

 	/*
-	 * Send command to the MC hardware:
-	 */
-	mc_write_command(mc_io->portal_virt_addr, cmd);
-
-	/*
 	 * Wait for response from the MC hardware:
 	 */
 	for (;;) {
@@ -339,6 +334,37 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 		}
 	}

+	*mc_status = status;
+	return 0;
+}
+
+/**
+ * Sends a command to the MC device using the given MC I/O object
+ *
+ * @mc_io: MC I/O object to be used
+ * @cmd: command to be sent
+ *
+ * Returns '0' on Success; Error code otherwise.
+ *
+ * NOTE: This function cannot be invoked from from atomic contexts.
+ */
+int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
+{
+	int error;
+	enum mc_cmd_status status;
+
+	/*
+	 * Send command to the MC hardware:
+	 */
+	mc_write_command(mc_io->portal_virt_addr, cmd);
+
+	/*
+	 * Wait for response from the MC hardware:
+	 */
+	error = mc_polling_wait_preemptible(mc_io, cmd, &status);
+	if (error < 0)
+		return error;
+
 	if (status != MC_CMD_STATUS_OK) {
 		pr_debug("MC command failed: portal: %#llx, obj handle: %#x, command: %#x, status: %s (%#x)\n",
 			 mc_io->portal_phys_addr,
--
2.3.3

[PATCH v2 3/4] staging: fsl-mc:Added support for atomic portals

[J. German Rivera]

Refactored mc_send_command() to support two flavors of polling:
- preemptible (for non-atomic portals), which was already supported.
  It calls usleep_range() between polling iterations.
- non-preemptible (for atomic portals), which is needed when
  mc_send_command() is called with interrupts disabled.
  It calls udelay() between polling iterations.

Signed-off-by: J. German Rivera <German.Rivera@freescale.com>
---
Changes in v2: none

 drivers/staging/fsl-mc/bus/mc-sys.c     | 53 +++++++++++++++++++++++++++++++--
 drivers/staging/fsl-mc/include/mc-sys.h |  5 ++++
 2 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index c4f5bdd..6eeb9fa 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -339,20 +339,63 @@ static int mc_polling_wait_preemptible(struct fsl_mc_io *mc_io,
 }

 /**
+ * Waits for the completion of an MC command doing atomic polling.
+ * udelay() is called between polling iterations.
+ *
+ * @mc_io: MC I/O object to be used
+ * @cmd: command buffer to receive MC response
+ * @mc_status: MC command completion status
+ */
+static int mc_polling_wait_atomic(struct fsl_mc_io *mc_io,
+				  struct mc_command *cmd,
+				  enum mc_cmd_status *mc_status)
+{
+	enum mc_cmd_status status;
+	unsigned long timeout_usecs = MC_CMD_COMPLETION_TIMEOUT_MS * 1000;
+
+	BUILD_BUG_ON((MC_CMD_COMPLETION_TIMEOUT_MS * 1000) %
+		     MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS != 0);
+
+	for (;;) {
+		status = mc_read_response(mc_io->portal_virt_addr, cmd);
+		if (status != MC_CMD_STATUS_READY)
+			break;
+
+		udelay(MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS);
+		timeout_usecs -= MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS;
+		if (timeout_usecs == 0) {
+			pr_debug("MC command timed out (portal: %#llx, obj handle: %#x, command: %#x)\n",
+				 mc_io->portal_phys_addr,
+				 (unsigned int)
+					MC_CMD_HDR_READ_TOKEN(cmd->header),
+				 (unsigned int)
+					MC_CMD_HDR_READ_CMDID(cmd->header));
+
+			return -ETIMEDOUT;
+		}
+	}
+
+	*mc_status = status;
+	return 0;
+}
+
+/**
  * Sends a command to the MC device using the given MC I/O object
  *
  * @mc_io: MC I/O object to be used
  * @cmd: command to be sent
  *
  * Returns '0' on Success; Error code otherwise.
- *
- * NOTE: This function cannot be invoked from from atomic contexts.
  */
 int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	int error;
 	enum mc_cmd_status status;

+	if (WARN_ON(in_irq() &&
+		    !(mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)))
+		return -EINVAL;
+
 	/*
 	 * Send command to the MC hardware:
 	 */
@@ -361,7 +404,11 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 	/*
 	 * Wait for response from the MC hardware:
 	 */
-	error = mc_polling_wait_preemptible(mc_io, cmd, &status);
+	if (!(mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL))
+		error = mc_polling_wait_preemptible(mc_io, cmd, &status);
+	else
+		error = mc_polling_wait_atomic(mc_io, cmd, &status);
+
 	if (error < 0)
 		return error;

diff --git a/drivers/staging/fsl-mc/include/mc-sys.h b/drivers/staging/fsl-mc/include/mc-sys.h
index 7d44d8c..15e19af 100644
--- a/drivers/staging/fsl-mc/include/mc-sys.h
+++ b/drivers/staging/fsl-mc/include/mc-sys.h
@@ -40,6 +40,11 @@
 #include <linux/io.h>
 #include <linux/dma-mapping.h>

+/**
+ * Bit masks for a MC I/O object (struct fsl_mc_io) flags
+ */
+#define FSL_MC_IO_ATOMIC_CONTEXT_PORTAL	0x0001
+
 struct fsl_mc_resource;
 struct mc_command;

--
2.3.3

[PATCH v2 4/4] staging: fsl-mc: Added serialization to mc_send_command()

[J. German Rivera]

When the same portal is used to call mc_send_command() from two
different threads or a thread and an interrupt handler, serialization
is required, as the MC only supports one outstanding command per MC
portal. Thus, a new command should not be sent to the MC until the
last command sent has been responded by the MC.

Signed-off-by: J. German Rivera <German.Rivera@freescale.com>
---
Changes in v2:
- Added missing Signed-off-by entry

 drivers/staging/fsl-mc/bus/mc-sys.c     | 25 ++++++++++++++++++++++---
 drivers/staging/fsl-mc/include/mc-sys.h | 29 +++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index 6eeb9fa..6e14892 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -88,6 +88,11 @@ int __must_check fsl_create_mc_io(struct device *dev,
 	mc_io->flags = flags;
 	mc_io->portal_phys_addr = mc_portal_phys_addr;
 	mc_io->portal_size = mc_portal_size;
+	if (flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)
+		spin_lock_init(&mc_io->spinlock);
+	else
+		mutex_init(&mc_io->mutex);
+
 	res = devm_request_mem_region(dev,
 				      mc_portal_phys_addr,
 				      mc_portal_size,
@@ -391,11 +396,17 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	int error;
 	enum mc_cmd_status status;
+	unsigned long irq_flags = 0;

 	if (WARN_ON(in_irq() &&
 		    !(mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)))
 		return -EINVAL;

+	if (mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)
+		spin_lock_irqsave(&mc_io->spinlock, irq_flags);
+	else
+		mutex_lock(&mc_io->mutex);
+
 	/*
 	 * Send command to the MC hardware:
 	 */
@@ -410,7 +421,7 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 		error = mc_polling_wait_atomic(mc_io, cmd, &status);

 	if (error < 0)
-		return error;
+		goto common_exit;

 	if (status != MC_CMD_STATUS_OK) {
 		pr_debug("MC command failed: portal: %#llx, obj handle: %#x, command: %#x, status: %s (%#x)\n",
@@ -420,9 +431,17 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 			 mc_status_to_string(status),
 			 (unsigned int)status);

-		return mc_status_to_error(status);
+		error = mc_status_to_error(status);
+		goto common_exit;
 	}

-	return 0;
+	error = 0;
+common_exit:
+	if (mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)
+		spin_unlock_irqrestore(&mc_io->spinlock, irq_flags);
+	else
+		mutex_unlock(&mc_io->mutex);
+
+	return error;
 }
 EXPORT_SYMBOL(mc_send_command);
diff --git a/drivers/staging/fsl-mc/include/mc-sys.h b/drivers/staging/fsl-mc/include/mc-sys.h
index 15e19af..c5038cc 100644
--- a/drivers/staging/fsl-mc/include/mc-sys.h
+++ b/drivers/staging/fsl-mc/include/mc-sys.h
@@ -39,6 +39,8 @@
 #include <linux/errno.h>
 #include <linux/io.h>
 #include <linux/dma-mapping.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>

 /**
  * Bit masks for a MC I/O object (struct fsl_mc_io) flags
@@ -56,6 +58,20 @@ struct mc_command;
  * @portal_phys_addr: MC command portal physical address
  * @portal_virt_addr: MC command portal virtual address
  * @dpmcp_dev: pointer to the DPMCP device associated with the MC portal.
+ *
+ * Fields are only meaningful if the FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is not
+ * set:
+ * @mutex: Mutex to serialize mc_send_command() calls that use the same MC
+ * portal, if the fsl_mc_io object was created with the
+ * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag off. mc_send_command() calls for this
+ * fsl_mc_io object must be made only from non-atomic context.
+ *
+ * Fields are only meaningful if the FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is
+ * set:
+ * @spinlock: Spinlock to serialize mc_send_command() calls that use the same MC
+ * portal, if the fsl_mc_io object was created with the
+ * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag on. mc_send_command() calls for this
+ * fsl_mc_io object can be made from atomic or non-atomic context.
  */
 struct fsl_mc_io {
 	struct device *dev;
@@ -64,6 +80,19 @@ struct fsl_mc_io {
 	phys_addr_t portal_phys_addr;
 	void __iomem *portal_virt_addr;
 	struct fsl_mc_device *dpmcp_dev;
+	union {
+		/*
+		 * This field is only meaningful if the
+		 * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is not set
+		 */
+		struct mutex mutex; /* serializes mc_send_command() */
+
+		/*
+		 * This field is only meaningful if the
+		 * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is set
+		 */
+		spinlock_t spinlock;	/* serializes mc_send_command() */
+	};
 };

 int __must_check fsl_create_mc_io(struct device *dev,
--
2.3.3