[PATCH 0/4] staging: fsl-mc: MC command serialization

[PATCH 0/4] staging: fsl-mc: MC command serialization

[J. German Rivera]

This patch series depends on the patch series posted
at http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg996780.html

This patch series addresses the following item from the TODO list
for the MC bus driver to exit staging:

* Management Complex (MC) command serialization. Locking mechanisms
  are needed by drivers to serialize commands sent to the MC, including
  from atomic context.

Patch 1: Changed timeout units for MC cmd completion
Patch 2: Refactored mc_send_command()
Patch 3: Added support for atomic portals
Patch 4: Added serialization to mc_send_command()

[PATCH 1/4] staging: fsl-mc: changed timeout units for MC cmd completion

[J. German Rivera]

Changed units for the timeout to wait for completion
of MC command, from jiffies to milliseconds.
---
 drivers/staging/fsl-mc/bus/mc-sys.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index 07848a0..2c5f109 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -42,9 +42,9 @@
 #include "dpmcp.h"
 
 /**
- * Timeout in jiffies to wait for the completion of an MC command
+ * Timeout in milliseconds to wait for the completion of an MC command
  */
-#define MC_CMD_COMPLETION_TIMEOUT_JIFFIES   (HZ / 2)	/* 500 ms */
+#define MC_CMD_COMPLETION_TIMEOUT_MS	500
 
 /*
  * usleep_range() min and max values used to throttle down polling
@@ -305,7 +305,7 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	enum mc_cmd_status status;
 	unsigned long jiffies_until_timeout =
-	    jiffies + MC_CMD_COMPLETION_TIMEOUT_JIFFIES;
+		jiffies + msecs_to_jiffies(MC_CMD_COMPLETION_TIMEOUT_MS);
 
 	/*
 	 * Send command to the MC hardware:
-- 
2.3.3

[PATCH 2/4] staging: fsl-mc: refactored mc_send_command()

[J. German Rivera]

Moved wait logic in mc_send_command() to its own function
---
 drivers/staging/fsl-mc/bus/mc-sys.c | 50 ++++++++++++++++++++++++++++---------
 1 file changed, 38 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index 2c5f109..c4f5bdd 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -292,27 +292,22 @@ static inline enum mc_cmd_status mc_read_response(struct mc_command __iomem *
 }
 
 /**
- * Sends an command to the MC device using the given MC I/O object
+ * Waits for the completion of an MC command doing preemptible polling.
+ * uslepp_range() is called between polling iterations.
  *
  * @mc_io: MC I/O object to be used
- * @cmd: command to be sent
- *
- * Returns '0' on Success; Error code otherwise.
- *
- * NOTE: This function cannot be invoked from from atomic contexts.
+ * @cmd: command buffer to receive MC response
+ * @mc_status: MC command completion status
  */
-int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
+static int mc_polling_wait_preemptible(struct fsl_mc_io *mc_io,
+				       struct mc_command *cmd,
+				       enum mc_cmd_status *mc_status)
 {
 	enum mc_cmd_status status;
 	unsigned long jiffies_until_timeout =
 		jiffies + msecs_to_jiffies(MC_CMD_COMPLETION_TIMEOUT_MS);
 
 	/*
-	 * Send command to the MC hardware:
-	 */
-	mc_write_command(mc_io->portal_virt_addr, cmd);
-
-	/*
 	 * Wait for response from the MC hardware:
 	 */
 	for (;;) {
@@ -339,6 +334,37 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 		}
 	}
 
+	*mc_status = status;
+	return 0;
+}
+
+/**
+ * Sends a command to the MC device using the given MC I/O object
+ *
+ * @mc_io: MC I/O object to be used
+ * @cmd: command to be sent
+ *
+ * Returns '0' on Success; Error code otherwise.
+ *
+ * NOTE: This function cannot be invoked from from atomic contexts.
+ */
+int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
+{
+	int error;
+	enum mc_cmd_status status;
+
+	/*
+	 * Send command to the MC hardware:
+	 */
+	mc_write_command(mc_io->portal_virt_addr, cmd);
+
+	/*
+	 * Wait for response from the MC hardware:
+	 */
+	error = mc_polling_wait_preemptible(mc_io, cmd, &status);
+	if (error < 0)
+		return error;
+
 	if (status != MC_CMD_STATUS_OK) {
 		pr_debug("MC command failed: portal: %#llx, obj handle: %#x, command: %#x, status: %s (%#x)\n",
 			 mc_io->portal_phys_addr,
-- 
2.3.3

[PATCH 3/4] staging: fsl-mc:Added support for atomic portals

[J. German Rivera]

Refactored mc_send_command() to support two flavors of polling:
- preemptible (for non-atomic portals), which was already supported.
  It calls usleep_range() between polling iterations.
- non-preemptible (for atomic portals), which is needed when
  mc_send_command() is called with interrupts disabled.
  It calls udelay() between polling iterations.

Signed-off-by: J. German Rivera <German.Rivera@freescale.com>
---
 drivers/staging/fsl-mc/bus/mc-sys.c     | 53 +++++++++++++++++++++++++++++++--
 drivers/staging/fsl-mc/include/mc-sys.h |  5 ++++
 2 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index c4f5bdd..6eeb9fa 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -339,20 +339,63 @@ static int mc_polling_wait_preemptible(struct fsl_mc_io *mc_io,
 }
 
 /**
+ * Waits for the completion of an MC command doing atomic polling.
+ * udelay() is called between polling iterations.
+ *
+ * @mc_io: MC I/O object to be used
+ * @cmd: command buffer to receive MC response
+ * @mc_status: MC command completion status
+ */
+static int mc_polling_wait_atomic(struct fsl_mc_io *mc_io,
+				  struct mc_command *cmd,
+				  enum mc_cmd_status *mc_status)
+{
+	enum mc_cmd_status status;
+	unsigned long timeout_usecs = MC_CMD_COMPLETION_TIMEOUT_MS * 1000;
+
+	BUILD_BUG_ON((MC_CMD_COMPLETION_TIMEOUT_MS * 1000) %
+		     MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS != 0);
+
+	for (;;) {
+		status = mc_read_response(mc_io->portal_virt_addr, cmd);
+		if (status != MC_CMD_STATUS_READY)
+			break;
+
+		udelay(MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS);
+		timeout_usecs -= MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS;
+		if (timeout_usecs == 0) {
+			pr_debug("MC command timed out (portal: %#llx, obj handle: %#x, command: %#x)\n",
+				 mc_io->portal_phys_addr,
+				 (unsigned int)
+					MC_CMD_HDR_READ_TOKEN(cmd->header),
+				 (unsigned int)
+					MC_CMD_HDR_READ_CMDID(cmd->header));
+
+			return -ETIMEDOUT;
+		}
+	}
+
+	*mc_status = status;
+	return 0;
+}
+
+/**
  * Sends a command to the MC device using the given MC I/O object
  *
  * @mc_io: MC I/O object to be used
  * @cmd: command to be sent
  *
  * Returns '0' on Success; Error code otherwise.
- *
- * NOTE: This function cannot be invoked from from atomic contexts.
  */
 int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	int error;
 	enum mc_cmd_status status;
 
+	if (WARN_ON(in_irq() &&
+		    !(mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)))
+		return -EINVAL;
+
 	/*
 	 * Send command to the MC hardware:
 	 */
@@ -361,7 +404,11 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 	/*
 	 * Wait for response from the MC hardware:
 	 */
-	error = mc_polling_wait_preemptible(mc_io, cmd, &status);
+	if (!(mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL))
+		error = mc_polling_wait_preemptible(mc_io, cmd, &status);
+	else
+		error = mc_polling_wait_atomic(mc_io, cmd, &status);
+
 	if (error < 0)
 		return error;
 
diff --git a/drivers/staging/fsl-mc/include/mc-sys.h b/drivers/staging/fsl-mc/include/mc-sys.h
index 7d44d8c..15e19af 100644
--- a/drivers/staging/fsl-mc/include/mc-sys.h
+++ b/drivers/staging/fsl-mc/include/mc-sys.h
@@ -40,6 +40,11 @@
 #include <linux/io.h>
 #include <linux/dma-mapping.h>
 
+/**
+ * Bit masks for a MC I/O object (struct fsl_mc_io) flags
+ */
+#define FSL_MC_IO_ATOMIC_CONTEXT_PORTAL	0x0001
+
 struct fsl_mc_resource;
 struct mc_command;
 
-- 
2.3.3

[PATCH 4/4] staging: fsl-mc: Added serialization to mc_send_command()

[J. German Rivera]

When the same portal is used to call mc_send_command() from two
different threads or a thread and an interrupt handler, serialization
is required, as the MC only supports one outstanding command per MC
portal. Thus, a new command should not be sent to the MC until the
last command sent has been responded by the MC.

Signed-off-by: J. German Rivera <German.Rivera@freescale.com>
---
 drivers/staging/fsl-mc/bus/mc-sys.c     | 25 ++++++++++++++++++++++---
 drivers/staging/fsl-mc/include/mc-sys.h | 29 +++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/fsl-mc/bus/mc-sys.c b/drivers/staging/fsl-mc/bus/mc-sys.c
index 6eeb9fa..6e14892 100644
--- a/drivers/staging/fsl-mc/bus/mc-sys.c
+++ b/drivers/staging/fsl-mc/bus/mc-sys.c
@@ -88,6 +88,11 @@ int __must_check fsl_create_mc_io(struct device *dev,
 	mc_io->flags = flags;
 	mc_io->portal_phys_addr = mc_portal_phys_addr;
 	mc_io->portal_size = mc_portal_size;
+	if (flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)
+		spin_lock_init(&mc_io->spinlock);
+	else
+		mutex_init(&mc_io->mutex);
+
 	res = devm_request_mem_region(dev,
 				      mc_portal_phys_addr,
 				      mc_portal_size,
@@ -391,11 +396,17 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	int error;
 	enum mc_cmd_status status;
+	unsigned long irq_flags = 0;
 
 	if (WARN_ON(in_irq() &&
 		    !(mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)))
 		return -EINVAL;
 
+	if (mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)
+		spin_lock_irqsave(&mc_io->spinlock, irq_flags);
+	else
+		mutex_lock(&mc_io->mutex);
+
 	/*
 	 * Send command to the MC hardware:
 	 */
@@ -410,7 +421,7 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 		error = mc_polling_wait_atomic(mc_io, cmd, &status);
 
 	if (error < 0)
-		return error;
+		goto common_exit;
 
 	if (status != MC_CMD_STATUS_OK) {
 		pr_debug("MC command failed: portal: %#llx, obj handle: %#x, command: %#x, status: %s (%#x)\n",
@@ -420,9 +431,17 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 			 mc_status_to_string(status),
 			 (unsigned int)status);
 
-		return mc_status_to_error(status);
+		error = mc_status_to_error(status);
+		goto common_exit;
 	}
 
-	return 0;
+	error = 0;
+common_exit:
+	if (mc_io->flags & FSL_MC_IO_ATOMIC_CONTEXT_PORTAL)
+		spin_unlock_irqrestore(&mc_io->spinlock, irq_flags);
+	else
+		mutex_unlock(&mc_io->mutex);
+
+	return error;
 }
 EXPORT_SYMBOL(mc_send_command);
diff --git a/drivers/staging/fsl-mc/include/mc-sys.h b/drivers/staging/fsl-mc/include/mc-sys.h
index 15e19af..c5038cc 100644
--- a/drivers/staging/fsl-mc/include/mc-sys.h
+++ b/drivers/staging/fsl-mc/include/mc-sys.h
@@ -39,6 +39,8 @@
 #include <linux/errno.h>
 #include <linux/io.h>
 #include <linux/dma-mapping.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
 
 /**
  * Bit masks for a MC I/O object (struct fsl_mc_io) flags
@@ -56,6 +58,20 @@ struct mc_command;
  * @portal_phys_addr: MC command portal physical address
  * @portal_virt_addr: MC command portal virtual address
  * @dpmcp_dev: pointer to the DPMCP device associated with the MC portal.
+ *
+ * Fields are only meaningful if the FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is not
+ * set:
+ * @mutex: Mutex to serialize mc_send_command() calls that use the same MC
+ * portal, if the fsl_mc_io object was created with the
+ * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag off. mc_send_command() calls for this
+ * fsl_mc_io object must be made only from non-atomic context.
+ *
+ * Fields are only meaningful if the FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is
+ * set:
+ * @spinlock: Spinlock to serialize mc_send_command() calls that use the same MC
+ * portal, if the fsl_mc_io object was created with the
+ * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag on. mc_send_command() calls for this
+ * fsl_mc_io object can be made from atomic or non-atomic context.
  */
 struct fsl_mc_io {
 	struct device *dev;
@@ -64,6 +80,19 @@ struct fsl_mc_io {
 	phys_addr_t portal_phys_addr;
 	void __iomem *portal_virt_addr;
 	struct fsl_mc_device *dpmcp_dev;
+	union {
+		/*
+		 * This field is only meaningful if the
+		 * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is not set
+		 */
+		struct mutex mutex; /* serializes mc_send_command() */
+
+		/*
+		 * This field is only meaningful if the
+		 * FSL_MC_IO_ATOMIC_CONTEXT_PORTAL flag is set
+		 */
+		spinlock_t spinlock;	/* serializes mc_send_command() */
+	};
 };
 
 int __must_check fsl_create_mc_io(struct device *dev,
-- 
2.3.3

Re: [PATCH 0/4] staging: fsl-mc: MC command serialization

[Greg KH]

On Wed, Oct 14, 2015 at 03:11:41PM -0500, J. German Rivera wrote:
> This patch series depends on the patch series posted
> at http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg996780.html

I didn't take that series for obvious reasons :(

RE: [PATCH 0/4] staging: fsl-mc: MC command serialization

[Jose Rivera]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Saturday, October 17, 2015 1:16 AM
> To: Rivera Jose-B46482
> Cc: arnd@arndb.de; devel@driverdev.osuosl.org; linux-
> kernel@vger.kernel.org; Hamciuc Bogdan-BHAMCIU1; Sharma Bhupesh-B45370;
> agraf@suse.de; Yoder Stuart-B08248; Erez Nir-RM30794; katz Itai-RM05202;
> Wood Scott-B07421; Pan Lijun-B44306; Li Yang-Leo-R58472; Marginean
> Alexandru-R89243; dan.carpenter@oracle.com; Schmitt Richard-B43082
> Subject: Re: [PATCH 0/4] staging: fsl-mc: MC command serialization
> 
> On Wed, Oct 14, 2015 at 03:11:41PM -0500, J. German Rivera wrote:
> > This patch series depends on the patch series posted at
> > http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg996780.htm
> > l
> 
> I didn't take that series for obvious reasons :(

Greg,

Do I need to respin this series to include the updated link to the patch
series this series depends on
(http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg999081.html)?

Thanks,

German

Re: [PATCH 0/4] staging: fsl-mc: MC command serialization

[Greg KH]

On Sat, Oct 17, 2015 at 04:36:44PM +0000, Jose Rivera wrote:
> > -----Original Message-----
> > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > Sent: Saturday, October 17, 2015 1:16 AM
> > To: Rivera Jose-B46482
> > Cc: arnd@arndb.de; devel@driverdev.osuosl.org; linux-
> > kernel@vger.kernel.org; Hamciuc Bogdan-BHAMCIU1; Sharma Bhupesh-B45370;
> > agraf@suse.de; Yoder Stuart-B08248; Erez Nir-RM30794; katz Itai-RM05202;
> > Wood Scott-B07421; Pan Lijun-B44306; Li Yang-Leo-R58472; Marginean
> > Alexandru-R89243; dan.carpenter@oracle.com; Schmitt Richard-B43082
> > Subject: Re: [PATCH 0/4] staging: fsl-mc: MC command serialization
> > 
> > On Wed, Oct 14, 2015 at 03:11:41PM -0500, J. German Rivera wrote:
> > > This patch series depends on the patch series posted at
> > > http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg996780.htm
> > > l
> > 
> > I didn't take that series for obvious reasons :(
> 
> Greg,
> 
> Do I need to respin this series to include the updated link to the patch
> series this series depends on
> (http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg999081.html)?

You need to resend it as it is out of my to-apply queue and lost to the
archives somewhere.

thanks,

greg k-h