[PATCH 0/3] rapidio: changes to enumeration/discovery

[PATCH 0/3] rapidio: changes to enumeration/discovery

[Alexandre Bounine]

Systems that use RapidIO fabric may need to implement their own enumeration
and discovery methods which are better suitable for needs of a target
application.

The following set of patches is intended to simplify process of introduction of
new RapidIO fabric enumeration/discovery methods.

The first patch offers ability to add new RapidIO enumeration/discovery methods
using kernel configuration options or loadable modules. The new configuration
option mechanism allows to select built-in or modular enumeration/discovery
method from the list of existing methods or use external module(s).

This patch also updates the currently existing enumeration/discovery code to be
used as built-in or modular method. The corresponding configuration option is
named "Basic enumeration/discovery" method. This is the only one built-in
configuration option available today but new methods are expected to be
introduced after adoption of provided patches.

The second patch address a long time complaint of RapidIO subsystem users
regarding fabric enumeration/discovery start sequence. Existing implementation
offers only a boot-time enumeration/discovery start which requires synchronized
boot of all endpoints in RapidIO network. While it works for small closed
configurations with limited number of endpoints, using this approach in systems
with large number of endpoints is quite challenging.

To eliminate requirement for synchronized start the second patch introduces
RapidIO enumeration/discovery start from user space.

For compatibility with the existing RapidIO subsystem implementation, automatic
boot time enumeration/discovery start can be configured in by selecting
CONFIG_RAPIDIO_ENUM_AUTO option.

Cc: Matt Porter <mporter@kernel.crashing.org>
Cc: Li Yang <leoli@freescale.com>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Andre van Herk <andre.van.herk@Prodrive.nl>
Cc: Micha Nelissen <micha.nelissen@Prodrive.nl>

Alexandre Bounine (3):
  rapidio: make enumeration/discovery configurable
  rapidio: add enumeration/discovery start from user space
  rapidio: documentation update for enumeration changes

 Documentation/rapidio/rapidio.txt |  137 ++++++++++++++++++++--
 Documentation/rapidio/sysfs.txt   |   17 +++
 drivers/rapidio/Kconfig           |   28 +++++
 drivers/rapidio/Makefile          |    3 +-
 drivers/rapidio/rio-driver.c      |    8 ++
 drivers/rapidio/rio-scan.c        |  195 ++++++++++--------------------
 drivers/rapidio/rio-sysfs.c       |   45 +++++++
 drivers/rapidio/rio.c             |  234 ++++++++++++++++++++++++++++++++++++-
 drivers/rapidio/rio.h             |   12 ++-
 include/linux/rio.h               |   17 +++
 include/linux/rio_drv.h           |    1 +
 11 files changed, 548 insertions(+), 149 deletions(-)

-- 
1.7.8.4

[PATCH 1/3] rapidio: make enumeration/discovery configurable

[Alexandre Bounine]

Rework to implement RapidIO enumeration/discovery method selection
combined with ability to use enumeration/discovery as a kernel module.

This patch adds ability to introduce new RapidIO enumeration/discovery methods
using kernel configuration options or loadable modules. Configuration option
mechanism allows to select built-in or modular enumeration/discovery method from
the list of existing methods or use external modules.
If a modular enumeration/discovery is selected each RapidIO mport device can
have its own method attached to it.

The currently existing enumeration/discovery code was updated to be used
as built-in or modular method. This configuration option is named "Basic
enumeration/discovery" method.

Several common routines have been moved from rio-scan.c to make them available
to other enumeration methods and reduce number of exported symbols.

Signed-off-by: Alexandre Bounine <alexandre.bounine@idt.com>
Cc: Matt Porter <mporter@kernel.crashing.org>
Cc: Li Yang <leoli@freescale.com>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Andre van Herk <andre.van.herk@Prodrive.nl>
Cc: Micha Nelissen <micha.nelissen@Prodrive.nl>
---
 drivers/rapidio/Kconfig      |   19 ++++
 drivers/rapidio/Makefile     |    3 +-
 drivers/rapidio/rio-driver.c |    7 ++
 drivers/rapidio/rio-scan.c   |  171 +++++++++-------------------------
 drivers/rapidio/rio.c        |  212 +++++++++++++++++++++++++++++++++++++++++-
 drivers/rapidio/rio.h        |   10 ++-
 include/linux/rio.h          |   12 +++
 include/linux/rio_drv.h      |    1 +
 8 files changed, 300 insertions(+), 135 deletions(-)

diff --git a/drivers/rapidio/Kconfig b/drivers/rapidio/Kconfig
index 6194d35..e392cab 100644
--- a/drivers/rapidio/Kconfig
+++ b/drivers/rapidio/Kconfig
@@ -47,4 +47,23 @@ config RAPIDIO_DEBUG
 
 	  If you are unsure about this, say N here.
 
+choice
+	prompt "Enumeration method"
+	depends on RAPIDIO
+	help
+	  There are different enumeration and discovery mechanisms offered
+	  for RapidIO subsystem. You may select single built-in method or
+	  or any number of methods to be built as modules.
+	  Selecting a built-in method disables use of loadable methods.
+
+	  If unsure, select Basic built-in.
+
+config RAPIDIO_ENUM_BASIC
+	tristate "Basic"
+	help
+	  This option includes basic RapidIO fabric enumeration and discovery
+	  mechanism similar to one described in RapidIO specification Annex 1.
+
+endchoice
+
 source "drivers/rapidio/switches/Kconfig"
diff --git a/drivers/rapidio/Makefile b/drivers/rapidio/Makefile
index ec3fb81..3036702 100644
--- a/drivers/rapidio/Makefile
+++ b/drivers/rapidio/Makefile
@@ -1,7 +1,8 @@
 #
 # Makefile for RapidIO interconnect services
 #
-obj-y += rio.o rio-access.o rio-driver.o rio-scan.o rio-sysfs.o
+obj-y += rio.o rio-access.o rio-driver.o rio-sysfs.o
+obj-$(CONFIG_RAPIDIO_ENUM_BASIC) += rio-scan.o
 
 obj-$(CONFIG_RAPIDIO)		+= switches/
 obj-$(CONFIG_RAPIDIO)		+= devices/
diff --git a/drivers/rapidio/rio-driver.c b/drivers/rapidio/rio-driver.c
index 0f4a53b..55850bb 100644
--- a/drivers/rapidio/rio-driver.c
+++ b/drivers/rapidio/rio-driver.c
@@ -164,6 +164,13 @@ void rio_unregister_driver(struct rio_driver *rdrv)
 	driver_unregister(&rdrv->driver);
 }
 
+void rio_attach_device(struct rio_dev *rdev)
+{
+	rdev->dev.bus = &rio_bus_type;
+	rdev->dev.parent = &rio_bus;
+}
+EXPORT_SYMBOL_GPL(rio_attach_device);
+
 /**
  *  rio_match_bus - Tell if a RIO device structure has a matching RIO driver device id structure
  *  @dev: the standard device structure to match against
diff --git a/drivers/rapidio/rio-scan.c b/drivers/rapidio/rio-scan.c
index a965acd..33fc332 100644
--- a/drivers/rapidio/rio-scan.c
+++ b/drivers/rapidio/rio-scan.c
@@ -37,12 +37,8 @@
 
 #include "rio.h"
 
-LIST_HEAD(rio_devices);
-
 static void rio_init_em(struct rio_dev *rdev);
 
-DEFINE_SPINLOCK(rio_global_list_lock);
-
 static int next_destid = 0;
 static int next_comptag = 1;
 
@@ -327,127 +323,6 @@ static int rio_is_switch(struct rio_dev *rdev)
 }
 
 /**
- * rio_switch_init - Sets switch operations for a particular vendor switch
- * @rdev: RIO device
- * @do_enum: Enumeration/Discovery mode flag
- *
- * Searches the RIO switch ops table for known switch types. If the vid
- * and did match a switch table entry, then call switch initialization
- * routine to setup switch-specific routines.
- */
-static void rio_switch_init(struct rio_dev *rdev, int do_enum)
-{
-	struct rio_switch_ops *cur = __start_rio_switch_ops;
-	struct rio_switch_ops *end = __end_rio_switch_ops;
-
-	while (cur < end) {
-		if ((cur->vid == rdev->vid) && (cur->did == rdev->did)) {
-			pr_debug("RIO: calling init routine for %s\n",
-				 rio_name(rdev));
-			cur->init_hook(rdev, do_enum);
-			break;
-		}
-		cur++;
-	}
-
-	if ((cur >= end) && (rdev->pef & RIO_PEF_STD_RT)) {
-		pr_debug("RIO: adding STD routing ops for %s\n",
-			rio_name(rdev));
-		rdev->rswitch->add_entry = rio_std_route_add_entry;
-		rdev->rswitch->get_entry = rio_std_route_get_entry;
-		rdev->rswitch->clr_table = rio_std_route_clr_table;
-	}
-
-	if (!rdev->rswitch->add_entry || !rdev->rswitch->get_entry)
-		printk(KERN_ERR "RIO: missing routing ops for %s\n",
-		       rio_name(rdev));
-}
-
-/**
- * rio_add_device- Adds a RIO device to the device model
- * @rdev: RIO device
- *
- * Adds the RIO device to the global device list and adds the RIO
- * device to the RIO device list.  Creates the generic sysfs nodes
- * for an RIO device.
- */
-static int rio_add_device(struct rio_dev *rdev)
-{
-	int err;
-
-	err = device_add(&rdev->dev);
-	if (err)
-		return err;
-
-	spin_lock(&rio_global_list_lock);
-	list_add_tail(&rdev->global_list, &rio_devices);
-	spin_unlock(&rio_global_list_lock);
-
-	rio_create_sysfs_dev_files(rdev);
-
-	return 0;
-}
-
-/**
- * rio_enable_rx_tx_port - enable input receiver and output transmitter of
- * given port
- * @port: Master port associated with the RIO network
- * @local: local=1 select local port otherwise a far device is reached
- * @destid: Destination ID of the device to check host bit
- * @hopcount: Number of hops to reach the target
- * @port_num: Port (-number on switch) to enable on a far end device
- *
- * Returns 0 or 1 from on General Control Command and Status Register
- * (EXT_PTR+0x3C)
- */
-inline int rio_enable_rx_tx_port(struct rio_mport *port,
-				 int local, u16 destid,
-				 u8 hopcount, u8 port_num) {
-#ifdef CONFIG_RAPIDIO_ENABLE_RX_TX_PORTS
-	u32 regval;
-	u32 ext_ftr_ptr;
-
-	/*
-	* enable rx input tx output port
-	*/
-	pr_debug("rio_enable_rx_tx_port(local = %d, destid = %d, hopcount = "
-		 "%d, port_num = %d)\n", local, destid, hopcount, port_num);
-
-	ext_ftr_ptr = rio_mport_get_physefb(port, local, destid, hopcount);
-
-	if (local) {
-		rio_local_read_config_32(port, ext_ftr_ptr +
-				RIO_PORT_N_CTL_CSR(0),
-				&regval);
-	} else {
-		if (rio_mport_read_config_32(port, destid, hopcount,
-		ext_ftr_ptr + RIO_PORT_N_CTL_CSR(port_num), &regval) < 0)
-			return -EIO;
-	}
-
-	if (regval & RIO_PORT_N_CTL_P_TYP_SER) {
-		/* serial */
-		regval = regval | RIO_PORT_N_CTL_EN_RX_SER
-				| RIO_PORT_N_CTL_EN_TX_SER;
-	} else {
-		/* parallel */
-		regval = regval | RIO_PORT_N_CTL_EN_RX_PAR
-				| RIO_PORT_N_CTL_EN_TX_PAR;
-	}
-
-	if (local) {
-		rio_local_write_config_32(port, ext_ftr_ptr +
-					  RIO_PORT_N_CTL_CSR(0), regval);
-	} else {
-		if (rio_mport_write_config_32(port, destid, hopcount,
-		    ext_ftr_ptr + RIO_PORT_N_CTL_CSR(port_num), regval) < 0)
-			return -EIO;
-	}
-#endif
-	return 0;
-}
-
-/**
  * rio_setup_device- Allocates and sets up a RIO device
  * @net: RIO network
  * @port: Master port to send transactions
@@ -587,8 +462,7 @@ static struct rio_dev *rio_setup_device(struct rio_net *net,
 			     rdev->destid);
 	}
 
-	rdev->dev.bus = &rio_bus_type;
-	rdev->dev.parent = &rio_bus;
+	rio_attach_device(rdev);
 
 	device_initialize(&rdev->dev);
 	rdev->dev.release = rio_release_dev;
@@ -1421,3 +1295,46 @@ enum_done:
 bail:
 	return -EBUSY;
 }
+
+struct rio_scan rio_scan_ops = {
+	.enumerate = rio_enum_mport,
+	.discover = rio_disc_mport,
+};
+
+
+#ifdef MODULE
+
+static bool scan;
+module_param(scan, bool, 0);
+MODULE_PARM_DESC(scan, "Start RapidIO network enumeration/discovery "
+			"(default = 1)");
+
+/**
+ * rio_basic_attach:
+ *
+ * When this enumeration/discovery method is loaded as a module this function
+ * registers its specific enumeration and discover routines for all available
+ * RapidIO mport devices. The "scan" command line parameter controls ability of
+ * the module to start RapidIO enumeration/discovery automatically.
+ *
+ * Returns 0 for success or -EIO if unable to register itself.
+ *
+ * This enumeration/discovery method cannot be unloaded and therefore does not
+ * provide a matching cleanup_module routine.
+ */
+
+int __init rio_basic_attach(void)
+{
+	if (rio_register_scan(RIO_MPORT_ANY, &rio_scan_ops))
+		return -EIO;
+	if (scan)
+		rio_init_mports();
+	return 0;
+}
+
+module_init(rio_basic_attach);
+
+MODULE_DESCRIPTION("Basic RapidIO enumeration/discovery");
+MODULE_LICENSE("GPL");
+
+#endif /* MODULE */
diff --git a/drivers/rapidio/rio.c b/drivers/rapidio/rio.c
index d553b5d..e36628a 100644
--- a/drivers/rapidio/rio.c
+++ b/drivers/rapidio/rio.c
@@ -31,6 +31,9 @@
 
 #include "rio.h"
 
+LIST_HEAD(rio_devices);
+DEFINE_SPINLOCK(rio_global_list_lock);
+
 static LIST_HEAD(rio_mports);
 static unsigned char next_portid;
 static DEFINE_SPINLOCK(rio_mmap_lock);
@@ -53,6 +56,32 @@ u16 rio_local_get_device_id(struct rio_mport *port)
 }
 
 /**
+ * rio_add_device- Adds a RIO device to the device model
+ * @rdev: RIO device
+ *
+ * Adds the RIO device to the global device list and adds the RIO
+ * device to the RIO device list.  Creates the generic sysfs nodes
+ * for an RIO device.
+ */
+int rio_add_device(struct rio_dev *rdev)
+{
+	int err;
+
+	err = device_add(&rdev->dev);
+	if (err)
+		return err;
+
+	spin_lock(&rio_global_list_lock);
+	list_add_tail(&rdev->global_list, &rio_devices);
+	spin_unlock(&rio_global_list_lock);
+
+	rio_create_sysfs_dev_files(rdev);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(rio_add_device);
+
+/**
  * rio_request_inb_mbox - request inbound mailbox service
  * @mport: RIO master port from which to allocate the mailbox resource
  * @dev_id: Device specific pointer to pass on event
@@ -489,6 +518,7 @@ rio_mport_get_physefb(struct rio_mport *port, int local,
 
 	return ext_ftr_ptr;
 }
+EXPORT_SYMBOL_GPL(rio_mport_get_physefb);
 
 /**
  * rio_get_comptag - Begin or continue searching for a RIO device by component tag
@@ -521,6 +551,7 @@ exit:
 	spin_unlock(&rio_global_list_lock);
 	return rdev;
 }
+EXPORT_SYMBOL_GPL(rio_get_comptag);
 
 /**
  * rio_set_port_lockout - Sets/clears LOCKOUT bit (RIO EM 1.3) for a switch port.
@@ -545,6 +576,107 @@ int rio_set_port_lockout(struct rio_dev *rdev, u32 pnum, int lock)
 				  regval);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(rio_set_port_lockout);
+
+/**
+ * rio_switch_init - Sets switch operations for a particular vendor switch
+ * @rdev: RIO device
+ * @do_enum: Enumeration/Discovery mode flag
+ *
+ * Searches the RIO switch ops table for known switch types. If the vid
+ * and did match a switch table entry, then call switch initialization
+ * routine to setup switch-specific routines.
+ */
+void rio_switch_init(struct rio_dev *rdev, int do_enum)
+{
+	struct rio_switch_ops *cur = __start_rio_switch_ops;
+	struct rio_switch_ops *end = __end_rio_switch_ops;
+
+	while (cur < end) {
+		if ((cur->vid == rdev->vid) && (cur->did == rdev->did)) {
+			pr_debug("RIO: calling init routine for %s\n",
+				 rio_name(rdev));
+			cur->init_hook(rdev, do_enum);
+			break;
+		}
+		cur++;
+	}
+
+	if ((cur >= end) && (rdev->pef & RIO_PEF_STD_RT)) {
+		pr_debug("RIO: adding STD routing ops for %s\n",
+			rio_name(rdev));
+		rdev->rswitch->add_entry = rio_std_route_add_entry;
+		rdev->rswitch->get_entry = rio_std_route_get_entry;
+		rdev->rswitch->clr_table = rio_std_route_clr_table;
+	}
+
+	if (!rdev->rswitch->add_entry || !rdev->rswitch->get_entry)
+		printk(KERN_ERR "RIO: missing routing ops for %s\n",
+		       rio_name(rdev));
+}
+EXPORT_SYMBOL_GPL(rio_switch_init);
+
+/**
+ * rio_enable_rx_tx_port - enable input receiver and output transmitter of
+ * given port
+ * @port: Master port associated with the RIO network
+ * @local: local=1 select local port otherwise a far device is reached
+ * @destid: Destination ID of the device to check host bit
+ * @hopcount: Number of hops to reach the target
+ * @port_num: Port (-number on switch) to enable on a far end device
+ *
+ * Returns 0 or 1 from on General Control Command and Status Register
+ * (EXT_PTR+0x3C)
+ */
+int rio_enable_rx_tx_port(struct rio_mport *port,
+			  int local, u16 destid,
+			  u8 hopcount, u8 port_num)
+{
+#ifdef CONFIG_RAPIDIO_ENABLE_RX_TX_PORTS
+	u32 regval;
+	u32 ext_ftr_ptr;
+
+	/*
+	* enable rx input tx output port
+	*/
+	pr_debug("rio_enable_rx_tx_port(local = %d, destid = %d, hopcount = "
+		 "%d, port_num = %d)\n", local, destid, hopcount, port_num);
+
+	ext_ftr_ptr = rio_mport_get_physefb(port, local, destid, hopcount);
+
+	if (local) {
+		rio_local_read_config_32(port, ext_ftr_ptr +
+				RIO_PORT_N_CTL_CSR(0),
+				&regval);
+	} else {
+		if (rio_mport_read_config_32(port, destid, hopcount,
+		ext_ftr_ptr + RIO_PORT_N_CTL_CSR(port_num), &regval) < 0)
+			return -EIO;
+	}
+
+	if (regval & RIO_PORT_N_CTL_P_TYP_SER) {
+		/* serial */
+		regval = regval | RIO_PORT_N_CTL_EN_RX_SER
+				| RIO_PORT_N_CTL_EN_TX_SER;
+	} else {
+		/* parallel */
+		regval = regval | RIO_PORT_N_CTL_EN_RX_PAR
+				| RIO_PORT_N_CTL_EN_TX_PAR;
+	}
+
+	if (local) {
+		rio_local_write_config_32(port, ext_ftr_ptr +
+					  RIO_PORT_N_CTL_CSR(0), regval);
+	} else {
+		if (rio_mport_write_config_32(port, destid, hopcount,
+		    ext_ftr_ptr + RIO_PORT_N_CTL_CSR(port_num), regval) < 0)
+			return -EIO;
+	}
+#endif
+	return 0;
+}
+EXPORT_SYMBOL_GPL(rio_enable_rx_tx_port);
+
 
 /**
  * rio_chk_dev_route - Validate route to the specified device.
@@ -610,6 +742,7 @@ rio_mport_chk_dev_access(struct rio_mport *mport, u16 destid, u8 hopcount)
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(rio_mport_chk_dev_access);
 
 /**
  * rio_chk_dev_access - Validate access to the specified device.
@@ -941,6 +1074,7 @@ rio_mport_get_efb(struct rio_mport *port, int local, u16 destid,
 		return RIO_GET_BLOCK_ID(reg_val);
 	}
 }
+EXPORT_SYMBOL_GPL(rio_mport_get_efb);
 
 /**
  * rio_mport_get_feature - query for devices' extended features
@@ -997,6 +1131,7 @@ rio_mport_get_feature(struct rio_mport * port, int local, u16 destid,
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(rio_mport_get_feature);
 
 /**
  * rio_get_asm - Begin or continue searching for a RIO device by vid/did/asm_vid/asm_did
@@ -1246,6 +1381,67 @@ EXPORT_SYMBOL_GPL(rio_dma_prep_slave_sg);
 
 #endif /* CONFIG_RAPIDIO_DMA_ENGINE */
 
+/**
+ * rio_register_scan - enumeration/discovery method registration interface
+ * @mport_id: mport device ID for which fabric scan routine has to be set
+ *            (RIO_MPORT_ANY = set for all available mports)
+ * @scan_ops: enumeration/discovery control structure
+ *
+ * Assigns enumeration or discovery method to the specified mport device (or all
+ * available mports if RIO_MPORT_ANY is specified).
+ * Returns error if the mport already has an enumerator attached to it.
+ * In case of RIO_MPORT_ANY ignores ports with valid scan routines and returns
+ * an error if was unable to find at least one available mport.
+ */
+int rio_register_scan(int mport_id, struct rio_scan *scan_ops)
+{
+	struct rio_mport *port;
+	int rc = -EBUSY;
+
+	list_for_each_entry(port, &rio_mports, node) {
+		if (port->id == mport_id || mport_id == RIO_MPORT_ANY) {
+			if (port->nscan && mport_id == RIO_MPORT_ANY)
+				continue;
+			else if (port->nscan)
+				break;
+
+			port->nscan = scan_ops;
+			rc = 0;
+
+			if (mport_id != RIO_MPORT_ANY)
+				break;
+		}
+	}
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(rio_register_scan);
+
+/**
+ * rio_unregister_scan - removes enumeration/discovery method from mport
+ * @mport_id: mport device ID for which fabric scan routine has to be
+ *            unregistered (RIO_MPORT_ANY = set for all available mports)
+ *
+ * Removes enumeration or discovery method assigned to the specified mport
+ * device (or all available mports if RIO_MPORT_ANY is specified).
+ */
+int rio_unregister_scan(int mport_id)
+{
+	struct rio_mport *port;
+
+	list_for_each_entry(port, &rio_mports, node) {
+		if (port->id == mport_id || mport_id == RIO_MPORT_ANY) {
+			if (port->nscan)
+				port->nscan = NULL;
+			if (mport_id != RIO_MPORT_ANY)
+				break;
+		}
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(rio_unregister_scan);
+
 static void rio_fixup_device(struct rio_dev *dev)
 {
 }
@@ -1274,7 +1470,7 @@ static void disc_work_handler(struct work_struct *_work)
 	work = container_of(_work, struct rio_disc_work, work);
 	pr_debug("RIO: discovery work for mport %d %s\n",
 		 work->mport->id, work->mport->name);
-	rio_disc_mport(work->mport);
+	work->mport->nscan->discover(work->mport);
 }
 
 int rio_init_mports(void)
@@ -1291,9 +1487,10 @@ int rio_init_mports(void)
 	 * on any of the registered mports.
 	 */
 	list_for_each_entry(port, &rio_mports, node) {
-		if (port->host_deviceid >= 0)
-			rio_enum_mport(port);
-		else
+		if (port->host_deviceid >= 0) {
+			if (port->nscan)
+				port->nscan->enumerate(port);
+		} else
 			n++;
 	}
 
@@ -1323,7 +1520,7 @@ int rio_init_mports(void)
 
 	n = 0;
 	list_for_each_entry(port, &rio_mports, node) {
-		if (port->host_deviceid < 0) {
+		if (port->host_deviceid < 0 && port->nscan) {
 			work[n].mport = port;
 			INIT_WORK(&work[n].work, disc_work_handler);
 			queue_work(rio_wq, &work[n].work);
@@ -1362,6 +1559,9 @@ static int rio_hdid_setup(char *str)
 
 __setup("riohdid=", rio_hdid_setup);
 
+/* Enumerator structure that links built-in enumerator */
+extern struct rio_scan __weak rio_scan_ops;
+
 int rio_register_mport(struct rio_mport *port)
 {
 	if (next_portid >= RIO_MAX_MPORTS) {
@@ -1371,6 +1571,7 @@ int rio_register_mport(struct rio_mport *port)
 
 	port->id = next_portid++;
 	port->host_deviceid = rio_get_hdid(port->id);
+	port->nscan = &rio_scan_ops;
 	list_add_tail(&port->node, &rio_mports);
 	return 0;
 }
@@ -1386,3 +1587,4 @@ EXPORT_SYMBOL_GPL(rio_request_inb_mbox);
 EXPORT_SYMBOL_GPL(rio_release_inb_mbox);
 EXPORT_SYMBOL_GPL(rio_request_outb_mbox);
 EXPORT_SYMBOL_GPL(rio_release_outb_mbox);
+EXPORT_SYMBOL_GPL(rio_init_mports);
diff --git a/drivers/rapidio/rio.h b/drivers/rapidio/rio.h
index b1af414..d44e7e6 100644
--- a/drivers/rapidio/rio.h
+++ b/drivers/rapidio/rio.h
@@ -15,6 +15,7 @@
 #include <linux/rio.h>
 
 #define RIO_MAX_CHK_RETRY	3
+#define RIO_MPORT_ANY		(-1)
 
 /* Functions internal to the RIO core code */
 
@@ -27,8 +28,6 @@ extern u32 rio_mport_get_efb(struct rio_mport *port, int local, u16 destid,
 extern int rio_mport_chk_dev_access(struct rio_mport *mport, u16 destid,
 				    u8 hopcount);
 extern int rio_create_sysfs_dev_files(struct rio_dev *rdev);
-extern int rio_enum_mport(struct rio_mport *mport);
-extern int rio_disc_mport(struct rio_mport *mport);
 extern int rio_std_route_add_entry(struct rio_mport *mport, u16 destid,
 				   u8 hopcount, u16 table, u16 route_destid,
 				   u8 route_port);
@@ -39,6 +38,13 @@ extern int rio_std_route_clr_table(struct rio_mport *mport, u16 destid,
 				   u8 hopcount, u16 table);
 extern int rio_set_port_lockout(struct rio_dev *rdev, u32 pnum, int lock);
 extern struct rio_dev *rio_get_comptag(u32 comp_tag, struct rio_dev *from);
+extern int rio_add_device(struct rio_dev *rdev);
+extern void rio_switch_init(struct rio_dev *rdev, int do_enum);
+extern int rio_enable_rx_tx_port(struct rio_mport *port, int local, u16 destid,
+				 u8 hopcount, u8 port_num);
+extern int rio_register_scan(int mport_id, struct rio_scan *scan_ops);
+extern int rio_unregister_scan(int mport_id);
+extern void rio_attach_device(struct rio_dev *rdev);
 
 /* Structures internal to the RIO core code */
 extern struct device_attribute rio_dev_attrs[];
diff --git a/include/linux/rio.h b/include/linux/rio.h
index a3e7842..30a48a8 100644
--- a/include/linux/rio.h
+++ b/include/linux/rio.h
@@ -237,6 +237,7 @@ enum rio_phy_type {
  * @name: Port name string
  * @priv: Master port private data
  * @dma: DMA device associated with mport
+ * @nscan: RapidIO network enumeration/discovery operations
  */
 struct rio_mport {
 	struct list_head dbells;	/* list of doorbell events */
@@ -262,6 +263,7 @@ struct rio_mport {
 #ifdef CONFIG_RAPIDIO_DMA_ENGINE
 	struct dma_device	dma;
 #endif
+	struct rio_scan *nscan;
 };
 
 struct rio_id_table {
@@ -460,6 +462,16 @@ static inline struct rio_mport *dma_to_mport(struct dma_device *ddev)
 }
 #endif /* CONFIG_RAPIDIO_DMA_ENGINE */
 
+/**
+ * struct rio_scan - RIO enumeration and discovery operations
+ * @enumerate: Callback to perform RapidIO fabric enumeration.
+ * @discover: Callback to perform RapidIO fabric discovery.
+ */
+struct rio_scan {
+	int (*enumerate)(struct rio_mport *mport);
+	int (*discover)(struct rio_mport *mport);
+};
+
 /* Architecture and hardware-specific functions */
 extern int rio_register_mport(struct rio_mport *);
 extern int rio_open_inb_mbox(struct rio_mport *, void *, int, int);
diff --git a/include/linux/rio_drv.h b/include/linux/rio_drv.h
index b75c059..5059994 100644
--- a/include/linux/rio_drv.h
+++ b/include/linux/rio_drv.h
@@ -433,5 +433,6 @@ extern u16 rio_local_get_device_id(struct rio_mport *port);
 extern struct rio_dev *rio_get_device(u16 vid, u16 did, struct rio_dev *from);
 extern struct rio_dev *rio_get_asm(u16 vid, u16 did, u16 asm_vid, u16 asm_did,
 				   struct rio_dev *from);
+extern int rio_init_mports(void);
 
 #endif				/* LINUX_RIO_DRV_H */
-- 
1.7.8.4

Re: [PATCH 1/3] rapidio: make enumeration/discovery configurable

[Andrew Morton]

On Wed, 24 Apr 2013 10:31:57 -0400 Alexandre Bounine <alexandre.bounine@idt.com> wrote:

> Rework to implement RapidIO enumeration/discovery method selection
> combined with ability to use enumeration/discovery as a kernel module.
> 
> This patch adds ability to introduce new RapidIO enumeration/discovery methods
> using kernel configuration options or loadable modules. Configuration option
> mechanism allows to select built-in or modular enumeration/discovery method from
> the list of existing methods or use external modules.
> If a modular enumeration/discovery is selected each RapidIO mport device can
> have its own method attached to it.
> 
> The currently existing enumeration/discovery code was updated to be used
> as built-in or modular method. This configuration option is named "Basic
> enumeration/discovery" method.
> 
> Several common routines have been moved from rio-scan.c to make them available
> to other enumeration methods and reduce number of exported symbols.
> 
> ...
>
> @@ -1421,3 +1295,46 @@ enum_done:
>  bail:
>  	return -EBUSY;
>  }
> +
> +struct rio_scan rio_scan_ops = {
> +	.enumerate = rio_enum_mport,
> +	.discover = rio_disc_mport,
> +};
> +
> +
> +#ifdef MODULE

Why the `ifdef MODULE'?  The module parameters are still accessible if
the driver is statically linked and we do want the driver to behave in
the same way regardless of how it was linked and loaded.

> +static bool scan;
> +module_param(scan, bool, 0);
> +MODULE_PARM_DESC(scan, "Start RapidIO network enumeration/discovery "
> +			"(default = 1)");
> +
> +/**
> + * rio_basic_attach:
> + *
> + * When this enumeration/discovery method is loaded as a module this function
> + * registers its specific enumeration and discover routines for all available
> + * RapidIO mport devices. The "scan" command line parameter controls ability of
> + * the module to start RapidIO enumeration/discovery automatically.
> + *
> + * Returns 0 for success or -EIO if unable to register itself.
> + *
> + * This enumeration/discovery method cannot be unloaded and therefore does not
> + * provide a matching cleanup_module routine.
> + */
> +
> +int __init rio_basic_attach(void)

static

> +{
> +	if (rio_register_scan(RIO_MPORT_ANY, &rio_scan_ops))
> +		return -EIO;
> +	if (scan)
> +		rio_init_mports();
> +	return 0;
> +}
> +
> +module_init(rio_basic_attach);
> +
> +MODULE_DESCRIPTION("Basic RapidIO enumeration/discovery");
> +MODULE_LICENSE("GPL");
> +
> +#endif /* MODULE */
> diff --git a/drivers/rapidio/rio.c b/drivers/rapidio/rio.c
> index d553b5d..e36628a 100644
> --- a/drivers/rapidio/rio.c
> +++ b/drivers/rapidio/rio.c
> @@ -31,6 +31,9 @@
>  
>  #include "rio.h"
>  
> +LIST_HEAD(rio_devices);

static?

> +DEFINE_SPINLOCK(rio_global_list_lock);

static?

> +
>  static LIST_HEAD(rio_mports);
>  static unsigned char next_portid;
>  static DEFINE_SPINLOCK(rio_mmap_lock);
> 
> ...
>
> +/**
> + * rio_switch_init - Sets switch operations for a particular vendor switch
> + * @rdev: RIO device
> + * @do_enum: Enumeration/Discovery mode flag
> + *
> + * Searches the RIO switch ops table for known switch types. If the vid
> + * and did match a switch table entry, then call switch initialization
> + * routine to setup switch-specific routines.
> + */
> +void rio_switch_init(struct rio_dev *rdev, int do_enum)
> +{
> +	struct rio_switch_ops *cur = __start_rio_switch_ops;
> +	struct rio_switch_ops *end = __end_rio_switch_ops;

huh, I hadn't noticed that RIO has its very own vmlinux section.  How
peculair.

> +	while (cur < end) {
> +		if ((cur->vid == rdev->vid) && (cur->did == rdev->did)) {
> +			pr_debug("RIO: calling init routine for %s\n",
> +				 rio_name(rdev));
> +			cur->init_hook(rdev, do_enum);
> +			break;
> +		}
> +		cur++;
> +	}
> +
> +	if ((cur >= end) && (rdev->pef & RIO_PEF_STD_RT)) {
> +		pr_debug("RIO: adding STD routing ops for %s\n",
> +			rio_name(rdev));
> +		rdev->rswitch->add_entry = rio_std_route_add_entry;
> +		rdev->rswitch->get_entry = rio_std_route_get_entry;
> +		rdev->rswitch->clr_table = rio_std_route_clr_table;
> +	}
> +
> +	if (!rdev->rswitch->add_entry || !rdev->rswitch->get_entry)
> +		printk(KERN_ERR "RIO: missing routing ops for %s\n",
> +		       rio_name(rdev));
> +}
> +EXPORT_SYMBOL_GPL(rio_switch_init);
> 
> ...
>
> +int rio_register_scan(int mport_id, struct rio_scan *scan_ops)
> +{
> +	struct rio_mport *port;
> +	int rc = -EBUSY;
> +
> +	list_for_each_entry(port, &rio_mports, node) {

How come the driver has no locking for rio_mports?  If a bugfix isn't
needed here then a code comment is!

> +		if (port->id == mport_id || mport_id == RIO_MPORT_ANY) {
> +			if (port->nscan && mport_id == RIO_MPORT_ANY)
> +				continue;
> +			else if (port->nscan)
> +				break;
> +
> +			port->nscan = scan_ops;
> +			rc = 0;
> +
> +			if (mport_id != RIO_MPORT_ANY)
> +				break;
> +		}
> +	}
> +
> +	return rc;
> +}
> 
> ...
>

RE: [PATCH 1/3] rapidio: make enumeration/discovery configurable

[Bounine, Alexandre]

On Wednesday, April 24, 2013 5:37 PM Andrew Morton wrote:
> 
> On Wed, 24 Apr 2013 10:31:57 -0400 Alexandre Bounine
> <alexandre.bounine@idt.com> wrote:
> 
> > Rework to implement RapidIO enumeration/discovery method selection
> > combined with ability to use enumeration/discovery as a kernel module.
> >
> > ...
> >
> > @@ -1421,3 +1295,46 @@ enum_done:
> >  bail:
> >  	return -EBUSY;
> >  }
> > +
> > +struct rio_scan rio_scan_ops = {
> > +	.enumerate = rio_enum_mport,
> > +	.discover = rio_disc_mport,
> > +};
> > +
> > +
> > +#ifdef MODULE
> 
> Why the `ifdef MODULE'?  The module parameters are still accessible if
> the driver is statically linked and we do want the driver to behave in
> the same way regardless of how it was linked and loaded.

Marked for review.

> > +static bool scan;
> > +module_param(scan, bool, 0);
> > +MODULE_PARM_DESC(scan, "Start RapidIO network enumeration/discovery
> "
> > +			"(default = 1)");
> > +
> > +/**
> > + * rio_basic_attach:
> > + *
> > + * When this enumeration/discovery method is loaded as a module this function
> > + * registers its specific enumeration and discover routines for all available
> > + * RapidIO mport devices. The "scan" command line parameter controls ability of
> > + * the module to start RapidIO enumeration/discovery automatically.
> > + *
> > + * Returns 0 for success or -EIO if unable to register itself.
> > + *
> > + * This enumeration/discovery method cannot be unloaded and therefore does not
> > + * provide a matching cleanup_module routine.
> > + */
> > +
> > +int __init rio_basic_attach(void)
> 
> static

My miss. Will fix.

> > +{
> > +	if (rio_register_scan(RIO_MPORT_ANY, &rio_scan_ops))
> > +		return -EIO;
> > +	if (scan)
> > +		rio_init_mports();
> > +	return 0;
> > +}
> > +
> > +module_init(rio_basic_attach);
> > +
> > +MODULE_DESCRIPTION("Basic RapidIO enumeration/discovery");
> > +MODULE_LICENSE("GPL");
> > +
> > +#endif /* MODULE */
> > diff --git a/drivers/rapidio/rio.c b/drivers/rapidio/rio.c
> > index d553b5d..e36628a 100644
> > --- a/drivers/rapidio/rio.c
> > +++ b/drivers/rapidio/rio.c
> > @@ -31,6 +31,9 @@
> >
> >  #include "rio.h"
> >
> > +LIST_HEAD(rio_devices);
> 
> static?
> 
> > +DEFINE_SPINLOCK(rio_global_list_lock);
> 
> static?

These two have been very tempting for me to make them static.
But because a goal was simple code move and they have been visible
from very beginning of RapidIO code, I decided to keep the scope "as is".
While I am against using the device list directly, I am not sure that this
patch is a good place to change the existing scope.

Because keeping them global prompted your comment I will happily make them
static and see if anyone will complain about it.
 
> > +
> >  static LIST_HEAD(rio_mports);
> >  static unsigned char next_portid;
> >  static DEFINE_SPINLOCK(rio_mmap_lock);
> >
> > ...
> >
> > +/**
> > + * rio_switch_init - Sets switch operations for a particular vendor switch
> > + * @rdev: RIO device
> > + * @do_enum: Enumeration/Discovery mode flag
> > + *
> > + * Searches the RIO switch ops table for known switch types. If the vid
> > + * and did match a switch table entry, then call switch initialization
> > + * routine to setup switch-specific routines.
> > + */
> > +void rio_switch_init(struct rio_dev *rdev, int do_enum)
> > +{
> > +	struct rio_switch_ops *cur = __start_rio_switch_ops;
> > +	struct rio_switch_ops *end = __end_rio_switch_ops;
> 
> huh, I hadn't noticed that RIO has its very own vmlinux section.  How
> peculair.

Yes it is there (since 2.6.15). We will address it at some point later.
At this moment just moving the code from one file to another.
 
> > +	while (cur < end) {
> > +		if ((cur->vid == rdev->vid) && (cur->did == rdev->did)) {
> > +			pr_debug("RIO: calling init routine for %s\n",
> > +				 rio_name(rdev));
> > +			cur->init_hook(rdev, do_enum);
> > +			break;
> > +		}
> > +		cur++;
> > +	}
> > +
> > +	if ((cur >= end) && (rdev->pef & RIO_PEF_STD_RT)) {
> > +		pr_debug("RIO: adding STD routing ops for %s\n",
> > +			rio_name(rdev));
> > +		rdev->rswitch->add_entry = rio_std_route_add_entry;
> > +		rdev->rswitch->get_entry = rio_std_route_get_entry;
> > +		rdev->rswitch->clr_table = rio_std_route_clr_table;
> > +	}
> > +
> > +	if (!rdev->rswitch->add_entry || !rdev->rswitch->get_entry)
> > +		printk(KERN_ERR "RIO: missing routing ops for %s\n",
> > +		       rio_name(rdev));
> > +}
> > +EXPORT_SYMBOL_GPL(rio_switch_init);
> >
> > ...
> >
> > +int rio_register_scan(int mport_id, struct rio_scan *scan_ops)
> > +{
> > +	struct rio_mport *port;
> > +	int rc = -EBUSY;
> > +
> > +	list_for_each_entry(port, &rio_mports, node) {
> 
> How come the driver has no locking for rio_mports?  If a bugfix isn't
> needed here then a code comment is!

Locking is not needed at this moment, but has to be added sooner or later anyway.
I will add it now to avoid fixing it later. 
 
> > +		if (port->id == mport_id || mport_id == RIO_MPORT_ANY) {
> > +			if (port->nscan && mport_id == RIO_MPORT_ANY)
> > +				continue;
> > +			else if (port->nscan)
> > +				break;
> > +
> > +			port->nscan = scan_ops;
> > +			rc = 0;
> > +
> > +			if (mport_id != RIO_MPORT_ANY)
> > +				break;
> > +		}
> > +	}
> > +
> > +	return rc;
> > +}
> >
> > ...

Re: [PATCH 1/3] rapidio: make enumeration/discovery configurable

[Andrew Morton]

On Wed, 24 Apr 2013 10:31:57 -0400 Alexandre Bounine <alexandre.bounine@idt.com> wrote:

> Rework to implement RapidIO enumeration/discovery method selection
> combined with ability to use enumeration/discovery as a kernel module.
> 
> This patch adds ability to introduce new RapidIO enumeration/discovery methods
> using kernel configuration options or loadable modules. Configuration option
> mechanism allows to select built-in or modular enumeration/discovery method from
> the list of existing methods or use external modules.
> If a modular enumeration/discovery is selected each RapidIO mport device can
> have its own method attached to it.
> 
> The currently existing enumeration/discovery code was updated to be used
> as built-in or modular method. This configuration option is named "Basic
> enumeration/discovery" method.
> 
> Several common routines have been moved from rio-scan.c to make them available
> to other enumeration methods and reduce number of exported symbols.
> 
> ...
>
> --- a/drivers/rapidio/Kconfig
> +++ b/drivers/rapidio/Kconfig
> @@ -47,4 +47,23 @@ config RAPIDIO_DEBUG
>  
>  	  If you are unsure about this, say N here.
>  
> +choice
> +	prompt "Enumeration method"
> +	depends on RAPIDIO
> +	help
> +	  There are different enumeration and discovery mechanisms offered
> +	  for RapidIO subsystem. You may select single built-in method or
> +	  or any number of methods to be built as modules.
> +	  Selecting a built-in method disables use of loadable methods.
> +
> +	  If unsure, select Basic built-in.
> +
> +config RAPIDIO_ENUM_BASIC
> +	tristate "Basic"
> +	help
> +	  This option includes basic RapidIO fabric enumeration and discovery
> +	  mechanism similar to one described in RapidIO specification Annex 1.
> +
> +endchoice

This Kconfig change makes my kbuild do Weird Things.

make mrproper ; yes "" | make allmodconfig ; make 2>/tmp/x

: scripts/kconfig/conf --silentoldconfig Kconfig
: *
: * Restart config...
: *
: *
: * Bus options (PCI etc.)
: *
: PCI support (PCI) [Y/n/?] y
:   Support mmconfig PCI config space access (PCI_MMCONFIG) [Y/n] y
:   Read CNB20LE Host Bridge Windows (PCI_CNB20LE_QUIRK) [Y/n/?] y
:   PCI Express support (PCIEPORTBUS) [Y/n/?] y
:     PCI Express Hotplug driver (HOTPLUG_PCI_PCIE) [M/n/?] m
:     Root Port Advanced Error Reporting support (PCIEAER) [Y/n/?] y
:       PCI Express ECRC settings control (PCIE_ECRC) [Y/n/?] y
:       PCIe AER error injector support (PCIEAER_INJECT) [M/n/y/?] m
:     PCI Express ASPM control (PCIEASPM) [Y/n/?] y
:       Debug PCI Express ASPM (PCIEASPM_DEBUG) [Y/n/?] y
:       Default ASPM policy
:       > 1. BIOS default (PCIEASPM_DEFAULT)
:         2. Powersave (PCIEASPM_POWERSAVE)
:         3. Performance (PCIEASPM_PERFORMANCE)
:       choice[1-3]: 1
: Message Signaled Interrupts (MSI and MSI-X) (PCI_MSI) [Y/?] y
: PCI Debugging (PCI_DEBUG) [Y/n/?] y
: Enable PCI resource re-allocation detection (PCI_REALLOC_ENABLE_AUTO) [Y/n/?] y
: PCI Stub driver (PCI_STUB) [M/n/y/?] m
: Xen PCI Frontend (XEN_PCIDEV_FRONTEND) [M/n/y/?] m
: Interrupts on hypertransport devices (HT_IRQ) [Y/n/?] y
: PCI IOV support (PCI_IOV) [Y/n/?] y
: PCI PRI support (PCI_PRI) [Y/?] y
: PCI PASID support (PCI_PASID) [Y/?] y
: PCI IO-APIC hotplug support (PCI_IOAPIC) [M/n/y] m
: ISA-style DMA support (ISA_DMA_API) [Y/n/?] y
: RapidIO support (RAPIDIO) [Y/n/?] y
:   IDT Tsi721 PCI Express SRIO Controller support (RAPIDIO_TSI721) [Y/n/?] y
:   Discovery timeout duration (seconds) (RAPIDIO_DISC_TIMEOUT) [30] 30
:   Enable RapidIO Input/Output Ports (RAPIDIO_ENABLE_RX_TX_PORTS) [Y/n/?] y
:   DMA Engine support for RapidIO (RAPIDIO_DMA_ENGINE) [Y/n/?] y
:   RapidIO subsystem debug messages (RAPIDIO_DEBUG) [Y/n/?] y
:   Enumeration method [M/y/?] (NEW) aborted!
: 
: Console input/output is redirected. Run 'make oldconfig' to update configuration.
: 
:   SYSHDR  arch/x86/syscalls/../include/generated/uapi/asm/unistd_32.h
:   SYSHDR  arch/x86/syscalls/../include/generated/uapi/asm/unistd_64.h
:   SYSHDR  arch/x86/syscalls/../include/generated/uapi/asm/unistd_x32.h


See the "Enumeration method [M/y/?] (NEW) aborted!"

Note that this only happens when make's stderr is redirected.

I've no idea what's going on here.  This appears to fix things:

--- a/drivers/rapidio/Kconfig~rapidio-make-enumeration-discovery-configurable-fix
+++ a/drivers/rapidio/Kconfig
@@ -59,7 +59,7 @@ choice
 	  If unsure, select Basic built-in.
 
 config RAPIDIO_ENUM_BASIC
-	tristate "Basic"
+	bool "Basic"
 	help
 	  This option includes basic RapidIO fabric enumeration and discovery
 	  mechanism similar to one described in RapidIO specification Annex 1.

but doesn't appear to be what you intended.

RE: [PATCH 1/3] rapidio: make enumeration/discovery configurable

[Bounine, Alexandre]

On Fri, 26 Apr 2013 6:53 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> Subject: Re: [PATCH 1/3] rapidio: make enumeration/discovery
> configurable
> 
> This Kconfig change makes my kbuild do Weird Things.
> 
> make mrproper ; yes "" | make allmodconfig ; make 2>/tmp/x
... skip ... 
> :   DMA Engine support for RapidIO (RAPIDIO_DMA_ENGINE) [Y/n/?] y
> :   RapidIO subsystem debug messages (RAPIDIO_DEBUG) [Y/n/?] y
> :   Enumeration method [M/y/?] (NEW) aborted!
> :
> : Console input/output is redirected. Run 'make oldconfig' to update configuration.
> :
> :   SYSHDR  arch/x86/syscalls/../include/generated/uapi/asm/unistd_32.h
> :   SYSHDR  arch/x86/syscalls/../include/generated/uapi/asm/unistd_64.h
> :   SYSHDR
> arch/x86/syscalls/../include/generated/uapi/asm/unistd_x32.h
> 
> 
> See the "Enumeration method [M/y/?] (NEW) aborted!"
> 
> Note that this only happens when make's stderr is redirected.
> 
> I've no idea what's going on here.  This appears to fix things:
> 
> --- a/drivers/rapidio/Kconfig~rapidio-make-enumeration-discovery-
> configurable-fix
> +++ a/drivers/rapidio/Kconfig
> @@ -59,7 +59,7 @@ choice
>  	  If unsure, select Basic built-in.
> 
>  config RAPIDIO_ENUM_BASIC
> -	tristate "Basic"
> +	bool "Basic"
>  	help
>  	  This option includes basic RapidIO fabric enumeration and
> discovery
>  	  mechanism similar to one described in RapidIO specification
> Annex 1.
> 
> but doesn't appear to be what you intended.

Goal is to build enumerator as a module or built-in.
I will retest v2 of patches for this issue.
There are some changes to Kconfig (removed AUTO_ENUM option).

[PATCH 2/3] rapidio: add enumeration/discovery start from user space

[Alexandre Bounine]

Add RapidIO enumeration/discovery start from user space.
User space start allows to defer RapidIO fabric scan until the moment when all
participating endpoints are initialized avoiding mandatory synchronized start
of all endpoints (which may be challenging in systems with large number of
RapidIO endpoints).

For compatibility with the existing RapidIO subsystem implementation, automatic
boot time enumeration/discovery start can be configured in by selecting
CONFIG_RAPIDIO_ENUM_AUTO option.

Signed-off-by: Alexandre Bounine <alexandre.bounine@idt.com>
Cc: Matt Porter <mporter@kernel.crashing.org>
Cc: Li Yang <leoli@freescale.com>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Andre van Herk <andre.van.herk@Prodrive.nl>
Cc: Micha Nelissen <micha.nelissen@Prodrive.nl>
---
 drivers/rapidio/Kconfig      |    9 ++++++++
 drivers/rapidio/rio-driver.c |    1 +
 drivers/rapidio/rio-scan.c   |   24 +++++++++++++++++++--
 drivers/rapidio/rio-sysfs.c  |   45 ++++++++++++++++++++++++++++++++++++++++++
 drivers/rapidio/rio.c        |   26 ++++++++++++++++++++++-
 drivers/rapidio/rio.h        |    2 +
 include/linux/rio.h          |    9 ++++++-
 7 files changed, 109 insertions(+), 7 deletions(-)

diff --git a/drivers/rapidio/Kconfig b/drivers/rapidio/Kconfig
index e392cab..8b7f92d 100644
--- a/drivers/rapidio/Kconfig
+++ b/drivers/rapidio/Kconfig
@@ -66,4 +66,13 @@ config RAPIDIO_ENUM_BASIC
 
 endchoice
 
+config RAPIDIO_ENUM_AUTO
+	bool "Automatic RapidIO enumeration/discovery start"
+	depends on RAPIDIO && RAPIDIO_ENUM_BASIC = y
+	help
+	  Say Y if you want RapidIO subsystem to start fabric enumeration and
+	  discovery automatically during kernel initialization time.
+	  If you are unsure, say N here. You will be able to start RapidIO
+	  fabric enumeration or discovery later by a user request.
+
 source "drivers/rapidio/switches/Kconfig"
diff --git a/drivers/rapidio/rio-driver.c b/drivers/rapidio/rio-driver.c
index 55850bb..a0c8755 100644
--- a/drivers/rapidio/rio-driver.c
+++ b/drivers/rapidio/rio-driver.c
@@ -207,6 +207,7 @@ struct bus_type rio_bus_type = {
 	.name = "rapidio",
 	.match = rio_match_bus,
 	.dev_attrs = rio_dev_attrs,
+	.bus_attrs = rio_bus_attrs,
 	.probe = rio_device_probe,
 	.remove = rio_device_remove,
 };
diff --git a/drivers/rapidio/rio-scan.c b/drivers/rapidio/rio-scan.c
index 33fc332..3e371b1 100644
--- a/drivers/rapidio/rio-scan.c
+++ b/drivers/rapidio/rio-scan.c
@@ -1134,19 +1134,30 @@ static void rio_pw_enable(struct rio_mport *port, int enable)
 /**
  * rio_enum_mport- Start enumeration through a master port
  * @mport: Master port to send transactions
+ * @flags: Enumeration control flags
  *
  * Starts the enumeration process. If somebody has enumerated our
  * master port device, then give up. If not and we have an active
  * link, then start recursive peer enumeration. Returns %0 if
  * enumeration succeeds or %-EBUSY if enumeration fails.
  */
-int rio_enum_mport(struct rio_mport *mport)
+int rio_enum_mport(struct rio_mport *mport, u32 flags)
 {
 	struct rio_net *net = NULL;
 	int rc = 0;
 
 	printk(KERN_INFO "RIO: enumerate master port %d, %s\n", mport->id,
 	       mport->name);
+
+	/*
+	 * To avoid multiple start requests (repeat enumeration is not supported
+	 * by this method) check if enumeration/discovery was performed for this
+	 * mport: if mport was added into the list of mports for a net exit
+	 * with error.
+	 */
+	if (mport->nnode.next || mport->nnode.prev)
+		return -EBUSY;
+
 	/* If somebody else enumerated our master port device, bail. */
 	if (rio_enum_host(mport) < 0) {
 		printk(KERN_INFO
@@ -1236,14 +1247,16 @@ static void rio_build_route_tables(struct rio_net *net)
 /**
  * rio_disc_mport- Start discovery through a master port
  * @mport: Master port to send transactions
+ * @flags: discovery control flags
  *
  * Starts the discovery process. If we have an active link,
- * then wait for the signal that enumeration is complete.
+ * then wait for the signal that enumeration is complete (if wait
+ * is allowed).
  * When enumeration completion is signaled, start recursive
  * peer discovery. Returns %0 if discovery succeeds or %-EBUSY
  * on failure.
  */
-int rio_disc_mport(struct rio_mport *mport)
+int rio_disc_mport(struct rio_mport *mport, u32 flags)
 {
 	struct rio_net *net = NULL;
 	unsigned long to_end;
@@ -1253,6 +1266,11 @@ int rio_disc_mport(struct rio_mport *mport)
 
 	/* If master port has an active link, allocate net and discover peers */
 	if (rio_mport_is_active(mport)) {
+		if (rio_enum_complete(mport))
+			goto enum_done;
+		else if (flags & RIO_SCAN_ENUM_NO_WAIT)
+			return -EAGAIN;
+
 		pr_debug("RIO: wait for enumeration to complete...\n");
 
 		to_end = jiffies + CONFIG_RAPIDIO_DISC_TIMEOUT * HZ;
diff --git a/drivers/rapidio/rio-sysfs.c b/drivers/rapidio/rio-sysfs.c
index 4dbe360..66d4acd 100644
--- a/drivers/rapidio/rio-sysfs.c
+++ b/drivers/rapidio/rio-sysfs.c
@@ -285,3 +285,48 @@ void rio_remove_sysfs_dev_files(struct rio_dev *rdev)
 			rdev->rswitch->sw_sysfs(rdev, RIO_SW_SYSFS_REMOVE);
 	}
 }
+
+static ssize_t bus_scan_store(struct bus_type *bus, const char *buf,
+				size_t count)
+{
+	long val;
+	struct rio_mport *port = NULL;
+	int rc;
+
+	if (kstrtol(buf, 0, &val) < 0)
+		return -EINVAL;
+
+	if (val == RIO_MPORT_ANY) {
+		rc = rio_init_mports();
+		goto exit;
+	}
+
+	if (val < 0 || val >= RIO_MAX_MPORTS)
+		return -EINVAL;
+
+	port = rio_find_mport((int)val);
+
+	if (!port) {
+		pr_debug("RIO: %s: mport_%d not available\n",
+			 __func__, (int)val);
+		return -EINVAL;
+	}
+
+	if (!port->nscan)
+		return -EINVAL;
+
+	if (port->host_deviceid >= 0)
+		rc = port->nscan->enumerate(port, 0);
+	else
+		rc = port->nscan->discover(port, RIO_SCAN_ENUM_NO_WAIT);
+exit:
+	if (!rc)
+		rc = count;
+
+	return rc;
+}
+
+struct bus_attribute rio_bus_attrs[] = {
+	__ATTR(scan, (S_IWUSR|S_IWGRP), NULL, bus_scan_store),
+	__ATTR_NULL
+};
diff --git a/drivers/rapidio/rio.c b/drivers/rapidio/rio.c
index e36628a..3bfbcec 100644
--- a/drivers/rapidio/rio.c
+++ b/drivers/rapidio/rio.c
@@ -1382,6 +1382,26 @@ EXPORT_SYMBOL_GPL(rio_dma_prep_slave_sg);
 #endif /* CONFIG_RAPIDIO_DMA_ENGINE */
 
 /**
+ * rio_find_mport - find RIO mport by its ID
+ * @mport_id: number (ID) of mport device
+ *
+ * Given a RIO mport number, the desired mport is located
+ * in the global list of mports. If the mport is found, a pointer to its
+ * data structure is returned.  If no mport is found, %NULL is returned.
+ */
+struct rio_mport *rio_find_mport(int mport_id)
+{
+	struct rio_mport *port = NULL;
+
+	list_for_each_entry(port, &rio_mports, node) {
+		if (port->id == mport_id)
+			return port;
+	}
+
+	return NULL;
+}
+
+/**
  * rio_register_scan - enumeration/discovery method registration interface
  * @mport_id: mport device ID for which fabric scan routine has to be set
  *            (RIO_MPORT_ANY = set for all available mports)
@@ -1470,7 +1490,7 @@ static void disc_work_handler(struct work_struct *_work)
 	work = container_of(_work, struct rio_disc_work, work);
 	pr_debug("RIO: discovery work for mport %d %s\n",
 		 work->mport->id, work->mport->name);
-	work->mport->nscan->discover(work->mport);
+	work->mport->nscan->discover(work->mport, 0);
 }
 
 int rio_init_mports(void)
@@ -1489,7 +1509,7 @@ int rio_init_mports(void)
 	list_for_each_entry(port, &rio_mports, node) {
 		if (port->host_deviceid >= 0) {
 			if (port->nscan)
-				port->nscan->enumerate(port);
+				port->nscan->enumerate(port, 0);
 		} else
 			n++;
 	}
@@ -1539,7 +1559,9 @@ no_disc:
 	return 0;
 }
 
+#ifdef CONFIG_RAPIDIO_ENUM_AUTO
 device_initcall_sync(rio_init_mports);
+#endif
 
 static int hdids[RIO_MAX_MPORTS + 1];
 
diff --git a/drivers/rapidio/rio.h b/drivers/rapidio/rio.h
index d44e7e6..bdd339e 100644
--- a/drivers/rapidio/rio.h
+++ b/drivers/rapidio/rio.h
@@ -45,9 +45,11 @@ extern int rio_enable_rx_tx_port(struct rio_mport *port, int local, u16 destid,
 extern int rio_register_scan(int mport_id, struct rio_scan *scan_ops);
 extern int rio_unregister_scan(int mport_id);
 extern void rio_attach_device(struct rio_dev *rdev);
+extern struct rio_mport *rio_find_mport(int mport_id);
 
 /* Structures internal to the RIO core code */
 extern struct device_attribute rio_dev_attrs[];
+extern struct bus_attribute rio_bus_attrs[];
 extern spinlock_t rio_global_list_lock;
 
 extern struct rio_switch_ops __start_rio_switch_ops[];
diff --git a/include/linux/rio.h b/include/linux/rio.h
index 30a48a8..a29a35b 100644
--- a/include/linux/rio.h
+++ b/include/linux/rio.h
@@ -266,6 +266,11 @@ struct rio_mport {
 	struct rio_scan *nscan;
 };
 
+/*
+ * Enumeration/discovery control flags
+ */
+#define RIO_SCAN_ENUM_NO_WAIT	0x00000001 /* Do not wait for enum completed */
+
 struct rio_id_table {
 	u16 start;	/* logical minimal id */
 	u32 max;	/* max number of IDs in table */
@@ -468,8 +473,8 @@ static inline struct rio_mport *dma_to_mport(struct dma_device *ddev)
  * @discover: Callback to perform RapidIO fabric discovery.
  */
 struct rio_scan {
-	int (*enumerate)(struct rio_mport *mport);
-	int (*discover)(struct rio_mport *mport);
+	int (*enumerate)(struct rio_mport *mport, u32 flags);
+	int (*discover)(struct rio_mport *mport, u32 flags);
 };
 
 /* Architecture and hardware-specific functions */
-- 
1.7.8.4

[PATCH 3/3] rapidio: documentation update for enumeration changes

[Alexandre Bounine]

Update RapidIO documentation to reflect changes made to enumeration/discovery
build configuration and user space triggering mechanism.

Signed-off-by: Alexandre Bounine <alexandre.bounine@idt.com>
Cc: Matt Porter <mporter@kernel.crashing.org>
Cc: Li Yang <leoli@freescale.com>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Andre van Herk <andre.van.herk@Prodrive.nl>
Cc: Micha Nelissen <micha.nelissen@Prodrive.nl>
---
 Documentation/rapidio/rapidio.txt |  137 ++++++++++++++++++++++++++++++++++---
 Documentation/rapidio/sysfs.txt   |   17 +++++
 2 files changed, 143 insertions(+), 11 deletions(-)

diff --git a/Documentation/rapidio/rapidio.txt b/Documentation/rapidio/rapidio.txt
index c75694b..d97576f 100644
--- a/Documentation/rapidio/rapidio.txt
+++ b/Documentation/rapidio/rapidio.txt
@@ -79,20 +79,64 @@ master port that is used to communicate with devices within the network.
 In order to initialize the RapidIO subsystem, a platform must initialize and
 register at least one master port within the RapidIO network. To register mport
 within the subsystem controller driver initialization code calls function
-rio_register_mport() for each available master port. After all active master
-ports are registered with a RapidIO subsystem, the rio_init_mports() routine
-is called to perform enumeration and discovery.
+rio_register_mport() for each available master port.
 
-In the current PowerPC-based implementation a subsys_initcall() is specified to
-perform controller initialization and mport registration. At the end it directly
-calls rio_init_mports() to execute RapidIO enumeration and discovery.
+RapidIO subsystem uses subsys_initcall() or device_initcall() to perform
+controller initialization (depending on controller device type).
+
+After all active master ports are registered with a RapidIO subsystem,
+an enumeration and/or discovery routine may be called automatically or
+by user-space command.
 
 4. Enumeration and Discovery
 ----------------------------
 
-When rio_init_mports() is called it scans a list of registered master ports and
-calls an enumeration or discovery routine depending on the configured role of a
-master port: host or agent.
+4.1 Overview
+------------
+
+RapidIO subsystem configuration options allow users to specify enumeration and
+discovery methods as built-in components or loadable modules.
+For built-in options only one method can be selected for all mports in a system.
+Selecting a modular build option for enumeration/discovery allows to use
+different enumerators attached to available mport device individually.
+
+Depending on selected enumeration/discovery build configuration, there are
+several methods to initiate enumeration and/or discovery process:
+
+  (a) Built-in enumeration and discovery process can be started automatically
+  during kernel initialization time. This is the original method used since
+  introduction of RapidIO subsystem. This method relies	on kernel initcall that
+  calls rio_init_mports() routine after all available mports have been
+  registered. When automatic start of enumeration/discovery is used a user has
+  to ensure that all discovering endpoints are started before the enumerating
+  endpoint and are waiting for enumeration to be completed.
+  Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
+  endpoint waits for enumeration to be completed. If the specified timeout
+  expires the discovery process is terminated without obtaining RapidIO network
+  information. NOTE: a timed out discovery process may be restarted later using
+  a user-space command as it is described later if the given endpoint was
+  enumerated successfully.
+
+  (b) Built-in enumeration and discovery process can be started by a command
+  from user space. This initiation method provides more freedom for system
+  startup compared to the option (a) above. After all participating endpoints
+  have been successfully booted, an enumeration process shall be started first
+  by issuing a user-space command, as soon as enumeration is completed
+  a discovery process can be started on all remaining endpoints.
+  Configuration option CONFIG_RAPIDIO_ENUM_AUTO defines which method will be
+  used to start a built-in enumeration and discovery process.
+
+  (c) Modular enumeration and discovery process can be started by a command from
+  user space. After an enumeration/discovery module is loaded, a network scan
+  process can be started by issuing a user-space command.
+  Similar to the option (b) above, an enumerator has to be started first.
+
+  (d) Modular enumeration and discovery process can be started by a module
+  initialization routine. As in the cases above an enumerating module shall be
+  loaded first.
+
+When a network scan process is started it calls an enumeration or discovery
+routine depending on the configured role of a master port: host or agent.
 
 Enumeration is performed by a master port if it is configured as a host port by
 assigning a host device ID greater than or equal to zero. A host device ID is
@@ -104,8 +148,57 @@ for it.
 The enumeration and discovery routines use RapidIO maintenance transactions
 to access the configuration space of devices.
 
-The enumeration process is implemented according to the enumeration algorithm
-outlined in the RapidIO Interconnect Specification: Annex I [1].
+4.2 Automatic Start of Enumeration and Discovery
+------------------------------------------------
+
+Automatic enumeration/discovery start method is applicable only to built-in
+enumeration/discovery RapidIO configuration selection. To enable automatic
+enumeration/discovery start set CONFIG_RAPIDIO_ENUM_AUTO=y.
+
+This configuration requires synchronized start of all RapidIO endpoints that
+form a network which will be enumerated/discovered. Discovering endpoints have
+to be started before an enumeration starts to ensure that all RapidIO
+controllers have been initialized and are ready to be discovered. Configuration
+parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
+a discovering endpoint will wait for enumeration to be completed.
+
+When automatic enumeration/discovery start method is selected RapidIO subsystem
+core uses device_initcall_sync() to invoke rio_init_mports() routine to perform
+enumeration or discovery for all known mport devices.
+
+Depending on RapidIO network size and configuration this automatic
+enumeration/discovery start method may be difficult to use due to the
+requirement for synchronized start of all endpoints.
+
+4.3 User-space Start of Enumeration and Discovery
+-------------------------------------------------
+
+User-space start of enumeration and discovery can be used with built-in and
+modular build configurations. For user-space controlled start RapidIO subsystem
+creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
+an enumeration or discovery process on specific mport device, a user needs to
+write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
+sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
+registration. For example for machine with single RapidIO controller, mport_ID
+for that controller always will be 0.
+
+To initiate RapidIO enumeration/discovery on all available mports a user may
+write '-1' (or RIO_MPORT_ANY) into the scan attribute file.
+
+4.4 Basic Enumeration Method
+----------------------------
+
+This is an original enumeration/discovery method which is available since
+first release of RapidIO subsystem code. The enumeration process is
+implemented according to the enumeration algorithm outlined in the RapidIO
+Interconnect Specification: Annex I [1].
+
+This method can be configured as built-in or loadable module. When used as a
+kernel module this method offers parameter "scan" witch allows to trigger the
+enumeration/discovery process from module initialization routine.
+
+This enumeration/discovery method can be started only once and does not support
+unloading if it is built as a module.
 
 The enumeration process traverses the network using a recursive depth-first
 algorithm. When a new device is found, the enumerator takes ownership of that
@@ -160,6 +253,28 @@ time period. If this wait time period expires before enumeration is completed,
 an agent skips RapidIO discovery and continues with remaining kernel
 initialization.
 
+4.5 Adding New Enumeration/Discovery Method
+-------------------------------------------
+
+RapidIO subsystem code organization allows addition of new enumeration/discovery
+methods as new configuration options without significant impact to to the core
+RapidIO code.
+
+RapidIO developers can add new enumeration/discovery methods as built-in code or
+loadable kernel modules. In case of built-in methods only one method may be
+configured in, having multiple built-in enumeration/discovery methods is not
+supported by mport role assignment mechanism.
+
+To be registered with RapidIO subsystem a new built-in method must provide
+global data structure named "rio_scan_ops" (see definition of 'struct rio_scan'
+in include/linux/rio.h file). Example of this structure is available in the
+existing implementation of basic RapidIO enumeration method (see rio-scan.c).
+
+If a new enumeration/discovery method is implemented as a module, its module
+initialization routine has to call rio_register_scan() routine to attach
+a loadable enumerator to a specified mport device. The basic enumerator
+implementation demonstrates this process as well.
+
 5. References
 -------------
 
diff --git a/Documentation/rapidio/sysfs.txt b/Documentation/rapidio/sysfs.txt
index 97f71ce..c3b7d18 100644
--- a/Documentation/rapidio/sysfs.txt
+++ b/Documentation/rapidio/sysfs.txt
@@ -88,3 +88,20 @@ that exports additional attributes.
 
 IDT_GEN2:
  errlog - reads contents of device error log until it is empty.
+
+
+5. RapidIO Bus Attributes
+-------------------------
+
+RapidIO bus subdirectory /sys/bus/rapidio implements the following bus-specific
+attribute:
+
+  scan - allows to trigger enumeration discovery process from user space. This
+	 is write-only attribute. To initiate an enumeration or discovery
+	 process on specific mport device, a user needs to write mport_ID (not
+	 RapidIO destination ID) into this file. The mport_ID is a sequential
+	 number (0 ... RIO_MAX_MPORTS) assigned to mport device. For example for
+	 machine with single RapidIO controller, mport_ID for that controller
+	 always will be 0. To initiate RapidIO enumeration/discovery on all
+	 available mports a user must write '-1' (or RIO_MPORT_ANY) into this
+	 attribute file.
-- 
1.7.8.4