[PATCH 0/2] rcar-csi2: make use V4L2_ASYNC_MATCH_CUSTOM to do fwnode matching

[PATCH 0/2] rcar-csi2: make use V4L2_ASYNC_MATCH_CUSTOM to do fwnode matching

[Lad Prabhakar]

Hi All,

This patch series adds support for fwnode matching to be handled by rcar-csi2 driver.

Thanks,
Prabhakar

Lad Prabhakar (2):
  media: v4l2-async: Pass pointer to struct v4l2_subdev in match_custom
    callback
  media: rcar-csi2: Let the driver handle fwnode matching using
    match_custom callback

 drivers/media/platform/rcar-vin/rcar-csi2.c | 46 +++++++++++++++++++--
 drivers/media/v4l2-core/v4l2-async.c        |  2 +-
 include/media/v4l2-async.h                  |  4 +-
 3 files changed, 46 insertions(+), 6 deletions(-)

-- 
2.20.1

[PATCH 1/2] media: v4l2-async: Pass pointer to struct v4l2_subdev in match_custom callback

[Lad Prabhakar]

Passing a pointer to struct device for the match_custom callback is of no
use as in the bridge driver to match the fwnode, so instead pass the
struct v4l2_subdev pointer so that the bridge driver has enough
information to match against the subdevices.

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
 drivers/media/v4l2-core/v4l2-async.c | 2 +-
 include/media/v4l2-async.h           | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
index 8bde33c21ce4..f897d4025f97 100644
--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -80,7 +80,7 @@ static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
 		/* Match always */
 		return true;
 
-	return asd->match.custom.match(sd->dev, asd);
+	return asd->match.custom.match(sd, asd);
 }
 
 static LIST_HEAD(subdev_list);
diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h
index 8319284c93cb..8c014e3bbd6c 100644
--- a/include/media/v4l2-async.h
+++ b/include/media/v4l2-async.h
@@ -86,8 +86,8 @@ struct v4l2_async_subdev {
 			unsigned short address;
 		} i2c;
 		struct {
-			bool (*match)(struct device *dev,
-				      struct v4l2_async_subdev *sd);
+			bool (*match)(struct v4l2_subdev *sd,
+				      struct v4l2_async_subdev *asd);
 			void *priv;
 		} custom;
 	} match;
-- 
2.20.1

Re: [PATCH 1/2] media: v4l2-async: Pass pointer to struct v4l2_subdev in match_custom callback

[Laurent Pinchart]

Hi Prabhakar,

Thank you for the patch.

On Sun, Mar 15, 2020 at 10:27:23AM +0000, Lad Prabhakar wrote:
> Passing a pointer to struct device for the match_custom callback is of no
> use as in the bridge driver to match the fwnode, so instead pass the
> struct v4l2_subdev pointer so that the bridge driver has enough
> information to match against the subdevices.

I'm not sure I like this. Conceptually speaking, the driver that
registers the notifier wants to get v4l2_subdev instances corresponding
to devices. A struct device is thus all it should need. Giving the match
function access to the subdev opens the door to all kind of nasty hacks.

In any case, I don't think is is required, see my reply to patch 2/2.

> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
>  drivers/media/v4l2-core/v4l2-async.c | 2 +-
>  include/media/v4l2-async.h           | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
> index 8bde33c21ce4..f897d4025f97 100644
> --- a/drivers/media/v4l2-core/v4l2-async.c
> +++ b/drivers/media/v4l2-core/v4l2-async.c
> @@ -80,7 +80,7 @@ static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
>  		/* Match always */
>  		return true;
>  
> -	return asd->match.custom.match(sd->dev, asd);
> +	return asd->match.custom.match(sd, asd);
>  }
>  
>  static LIST_HEAD(subdev_list);
> diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h
> index 8319284c93cb..8c014e3bbd6c 100644
> --- a/include/media/v4l2-async.h
> +++ b/include/media/v4l2-async.h
> @@ -86,8 +86,8 @@ struct v4l2_async_subdev {
>  			unsigned short address;
>  		} i2c;
>  		struct {
> -			bool (*match)(struct device *dev,
> -				      struct v4l2_async_subdev *sd);
> +			bool (*match)(struct v4l2_subdev *sd,
> +				      struct v4l2_async_subdev *asd);
>  			void *priv;
>  		} custom;
>  	} match;

-- 
Regards,

Laurent Pinchart

[PATCH 2/2] media: rcar-csi2: Let the driver handle fwnode matching using match_custom callback

[Lad Prabhakar]

The rcar-csi2 driver uses the v4l2-async framework to do endpoint matching
instead of node matching. This is needed as it needs to work with the
adv748x driver which register it self in v4l2-async using endpoints
instead of nodes. The reason for this is that from a single DT node it
creates multiple subdevices, one for each endpoint.

But when using subdevs which register itself in v4l2-async using nodes,
the rcar-csi2 driver failed to find the matching endpoint because the
match.fwnode was pointing to remote endpoint instead of remote parent
port.

This commit adds support in rcar-csi2 driver to handle both the cases
where subdev registers in v4l2-async using endpoints/nodes, by using
match_type as V4L2_ASYNC_MATCH_CUSTOM and implementing the match()
callback to compare the fwnode of either remote/parent.

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
 drivers/media/platform/rcar-vin/rcar-csi2.c | 46 +++++++++++++++++++--
 1 file changed, 43 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
index faa9fb23a2e9..1bbf05e9f025 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -808,6 +808,46 @@ static int rcsi2_parse_v4l2(struct rcar_csi2 *priv,
 	return 0;
 }
 
+static bool rcsi2_asd_match(struct v4l2_subdev *sd,
+			    struct v4l2_async_subdev *asd)
+{
+	struct rcar_csi2 *priv = (struct rcar_csi2 *)asd->match.custom.priv;
+	struct fwnode_handle *remote_endpoint;
+	struct fwnode_handle *subdev_endpoint;
+	struct device_node *np;
+	bool matched = false;
+
+	np = of_graph_get_endpoint_by_regs(priv->dev->of_node, 0, 0);
+	if (!np) {
+		dev_err(priv->dev, "Not connected to subdevice\n");
+		return matched;
+	}
+
+	remote_endpoint =
+		fwnode_graph_get_remote_endpoint(of_fwnode_handle(np));
+	if (!remote_endpoint) {
+		dev_err(priv->dev, "Failed to get remote endpoint\n");
+		of_node_put(np);
+		return matched;
+	}
+	of_node_put(np);
+
+	if (sd->fwnode != dev_fwnode(sd->dev)) {
+		if (remote_endpoint == sd->fwnode)
+			matched = true;
+	} else {
+		subdev_endpoint =
+		      fwnode_graph_get_next_endpoint(dev_fwnode(sd->dev), NULL);
+		if (remote_endpoint == subdev_endpoint)
+			matched = true;
+		fwnode_handle_put(subdev_endpoint);
+	}
+
+	fwnode_handle_put(remote_endpoint);
+
+	return matched;
+}
+
 static int rcsi2_parse_dt(struct rcar_csi2 *priv)
 {
 	struct device_node *ep;
@@ -833,9 +873,9 @@ static int rcsi2_parse_dt(struct rcar_csi2 *priv)
 		return ret;
 	}
 
-	priv->asd.match.fwnode =
-		fwnode_graph_get_remote_endpoint(of_fwnode_handle(ep));
-	priv->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
+	priv->asd.match.custom.match = &rcsi2_asd_match;
+	priv->asd.match.custom.priv = priv;
+	priv->asd.match_type = V4L2_ASYNC_MATCH_CUSTOM;
 
 	of_node_put(ep);
 
-- 
2.20.1

Re: [PATCH 2/2] media: rcar-csi2: Let the driver handle fwnode matching using match_custom callback

[Laurent Pinchart]

Hi Prabhakar,

Thank you for the patch.

On Sun, Mar 15, 2020 at 10:27:24AM +0000, Lad Prabhakar wrote:
> The rcar-csi2 driver uses the v4l2-async framework to do endpoint matching
> instead of node matching. This is needed as it needs to work with the
> adv748x driver which register it self in v4l2-async using endpoints
> instead of nodes. The reason for this is that from a single DT node it
> creates multiple subdevices, one for each endpoint.
> 
> But when using subdevs which register itself in v4l2-async using nodes,
> the rcar-csi2 driver failed to find the matching endpoint because the
> match.fwnode was pointing to remote endpoint instead of remote parent
> port.
> 
> This commit adds support in rcar-csi2 driver to handle both the cases
> where subdev registers in v4l2-async using endpoints/nodes, by using
> match_type as V4L2_ASYNC_MATCH_CUSTOM and implementing the match()
> callback to compare the fwnode of either remote/parent.

This is not the way to go. The v4l2-async framework needs to be fixed
instead, so that fwnode match will do the right thing automatically
regardless of whether the node is a device node or and endpoint node.

> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
>  drivers/media/platform/rcar-vin/rcar-csi2.c | 46 +++++++++++++++++++--
>  1 file changed, 43 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
> index faa9fb23a2e9..1bbf05e9f025 100644
> --- a/drivers/media/platform/rcar-vin/rcar-csi2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
> @@ -808,6 +808,46 @@ static int rcsi2_parse_v4l2(struct rcar_csi2 *priv,
>  	return 0;
>  }
>  
> +static bool rcsi2_asd_match(struct v4l2_subdev *sd,
> +			    struct v4l2_async_subdev *asd)
> +{
> +	struct rcar_csi2 *priv = (struct rcar_csi2 *)asd->match.custom.priv;
> +	struct fwnode_handle *remote_endpoint;
> +	struct fwnode_handle *subdev_endpoint;
> +	struct device_node *np;
> +	bool matched = false;
> +
> +	np = of_graph_get_endpoint_by_regs(priv->dev->of_node, 0, 0);
> +	if (!np) {
> +		dev_err(priv->dev, "Not connected to subdevice\n");
> +		return matched;
> +	}
> +
> +	remote_endpoint =
> +		fwnode_graph_get_remote_endpoint(of_fwnode_handle(np));
> +	if (!remote_endpoint) {
> +		dev_err(priv->dev, "Failed to get remote endpoint\n");
> +		of_node_put(np);
> +		return matched;
> +	}
> +	of_node_put(np);
> +
> +	if (sd->fwnode != dev_fwnode(sd->dev)) {
> +		if (remote_endpoint == sd->fwnode)
> +			matched = true;
> +	} else {
> +		subdev_endpoint =
> +		      fwnode_graph_get_next_endpoint(dev_fwnode(sd->dev), NULL);
> +		if (remote_endpoint == subdev_endpoint)
> +			matched = true;
> +		fwnode_handle_put(subdev_endpoint);
> +	}
> +
> +	fwnode_handle_put(remote_endpoint);
> +
> +	return matched;
> +}
> +
>  static int rcsi2_parse_dt(struct rcar_csi2 *priv)
>  {
>  	struct device_node *ep;
> @@ -833,9 +873,9 @@ static int rcsi2_parse_dt(struct rcar_csi2 *priv)
>  		return ret;
>  	}
>  
> -	priv->asd.match.fwnode =
> -		fwnode_graph_get_remote_endpoint(of_fwnode_handle(ep));
> -	priv->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> +	priv->asd.match.custom.match = &rcsi2_asd_match;
> +	priv->asd.match.custom.priv = priv;
> +	priv->asd.match_type = V4L2_ASYNC_MATCH_CUSTOM;
>  
>  	of_node_put(ep);
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 2/2] media: rcar-csi2: Let the driver handle fwnode matching using match_custom callback

[Lad, Prabhakar]

Hi Laurent,

Thank you for the quick review.

On Sun, Mar 15, 2020 at 10:30 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Prabhakar,
>
> Thank you for the patch.
>
> On Sun, Mar 15, 2020 at 10:27:24AM +0000, Lad Prabhakar wrote:
> > The rcar-csi2 driver uses the v4l2-async framework to do endpoint matching
> > instead of node matching. This is needed as it needs to work with the
> > adv748x driver which register it self in v4l2-async using endpoints
> > instead of nodes. The reason for this is that from a single DT node it
> > creates multiple subdevices, one for each endpoint.
> >
> > But when using subdevs which register itself in v4l2-async using nodes,
> > the rcar-csi2 driver failed to find the matching endpoint because the
> > match.fwnode was pointing to remote endpoint instead of remote parent
> > port.
> >
> > This commit adds support in rcar-csi2 driver to handle both the cases
> > where subdev registers in v4l2-async using endpoints/nodes, by using
> > match_type as V4L2_ASYNC_MATCH_CUSTOM and implementing the match()
> > callback to compare the fwnode of either remote/parent.
>
> This is not the way to go. The v4l2-async framework needs to be fixed
> instead, so that fwnode match will do the right thing automatically
> regardless of whether the node is a device node or and endpoint node.
>
OK, so moving forward should the v4l2-async do strictly endpoint
matching only or
both nodes/endpoints. fwnode in all the bridge drivers be replaced to
remote endpoints ?

Looking at the adv7604 its registered as node to v4l2-async which can
have upto 3 endpoints,
adv748x is the single driver which registers itself as endpoint to
v4l2-async, and rest of the
other subdevices have single endpoint and are registered as node to
v4l2-async. How would you
suggest to handle these cases.

Cheers,
--Prabhakar Lad

> > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > ---
> >  drivers/media/platform/rcar-vin/rcar-csi2.c | 46 +++++++++++++++++++--
> >  1 file changed, 43 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
> > index faa9fb23a2e9..1bbf05e9f025 100644
> > --- a/drivers/media/platform/rcar-vin/rcar-csi2.c
> > +++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
> > @@ -808,6 +808,46 @@ static int rcsi2_parse_v4l2(struct rcar_csi2 *priv,
> >       return 0;
> >  }
> >
> > +static bool rcsi2_asd_match(struct v4l2_subdev *sd,
> > +                         struct v4l2_async_subdev *asd)
> > +{
> > +     struct rcar_csi2 *priv = (struct rcar_csi2 *)asd->match.custom.priv;
> > +     struct fwnode_handle *remote_endpoint;
> > +     struct fwnode_handle *subdev_endpoint;
> > +     struct device_node *np;
> > +     bool matched = false;
> > +
> > +     np = of_graph_get_endpoint_by_regs(priv->dev->of_node, 0, 0);
> > +     if (!np) {
> > +             dev_err(priv->dev, "Not connected to subdevice\n");
> > +             return matched;
> > +     }
> > +
> > +     remote_endpoint =
> > +             fwnode_graph_get_remote_endpoint(of_fwnode_handle(np));
> > +     if (!remote_endpoint) {
> > +             dev_err(priv->dev, "Failed to get remote endpoint\n");
> > +             of_node_put(np);
> > +             return matched;
> > +     }
> > +     of_node_put(np);
> > +
> > +     if (sd->fwnode != dev_fwnode(sd->dev)) {
> > +             if (remote_endpoint == sd->fwnode)
> > +                     matched = true;
> > +     } else {
> > +             subdev_endpoint =
> > +                   fwnode_graph_get_next_endpoint(dev_fwnode(sd->dev), NULL);
> > +             if (remote_endpoint == subdev_endpoint)
> > +                     matched = true;
> > +             fwnode_handle_put(subdev_endpoint);
> > +     }
> > +
> > +     fwnode_handle_put(remote_endpoint);
> > +
> > +     return matched;
> > +}
> > +
> >  static int rcsi2_parse_dt(struct rcar_csi2 *priv)
> >  {
> >       struct device_node *ep;
> > @@ -833,9 +873,9 @@ static int rcsi2_parse_dt(struct rcar_csi2 *priv)
> >               return ret;
> >       }
> >
> > -     priv->asd.match.fwnode =
> > -             fwnode_graph_get_remote_endpoint(of_fwnode_handle(ep));
> > -     priv->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > +     priv->asd.match.custom.match = &rcsi2_asd_match;
> > +     priv->asd.match.custom.priv = priv;
> > +     priv->asd.match_type = V4L2_ASYNC_MATCH_CUSTOM;
> >
> >       of_node_put(ep);
> >
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 2/2] media: rcar-csi2: Let the driver handle fwnode matching using match_custom callback

[Laurent Pinchart]

Hi Prabhakar,

On Sun, Mar 15, 2020 at 12:10:14PM +0000, Lad, Prabhakar wrote:
> On Sun, Mar 15, 2020 at 10:30 AM Laurent Pinchart wrote:
> > On Sun, Mar 15, 2020 at 10:27:24AM +0000, Lad Prabhakar wrote:
> > > The rcar-csi2 driver uses the v4l2-async framework to do endpoint matching
> > > instead of node matching. This is needed as it needs to work with the
> > > adv748x driver which register it self in v4l2-async using endpoints
> > > instead of nodes. The reason for this is that from a single DT node it
> > > creates multiple subdevices, one for each endpoint.
> > >
> > > But when using subdevs which register itself in v4l2-async using nodes,
> > > the rcar-csi2 driver failed to find the matching endpoint because the
> > > match.fwnode was pointing to remote endpoint instead of remote parent
> > > port.
> > >
> > > This commit adds support in rcar-csi2 driver to handle both the cases
> > > where subdev registers in v4l2-async using endpoints/nodes, by using
> > > match_type as V4L2_ASYNC_MATCH_CUSTOM and implementing the match()
> > > callback to compare the fwnode of either remote/parent.
> >
> > This is not the way to go. The v4l2-async framework needs to be fixed
> > instead, so that fwnode match will do the right thing automatically
> > regardless of whether the node is a device node or and endpoint node.
>
> OK, so moving forward should the v4l2-async do strictly endpoint
> matching only or both nodes/endpoints. fwnode in all the bridge
> drivers be replaced to remote endpoints ?

Long term I think everything should use endpoint matching, but to get
there we shouldn't transition all drivers in one go. I've submitted a
patch to v4l2-async that I believe will fix your problem and allow for a
smooth transition. Could you give it a try ?

> Looking at the adv7604 its registered as node to v4l2-async which can
> have upto 3 endpoints, adv748x is the single driver which registers
> itself as endpoint to v4l2-async, and rest of the other subdevices
> have single endpoint and are registered as node to v4l2-async. How
> would you suggest to handle these cases.
> 
> > > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> > > ---
> > >  drivers/media/platform/rcar-vin/rcar-csi2.c | 46 +++++++++++++++++++--
> > >  1 file changed, 43 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c b/drivers/media/platform/rcar-vin/rcar-csi2.c
> > > index faa9fb23a2e9..1bbf05e9f025 100644
> > > --- a/drivers/media/platform/rcar-vin/rcar-csi2.c
> > > +++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
> > > @@ -808,6 +808,46 @@ static int rcsi2_parse_v4l2(struct rcar_csi2 *priv,
> > >       return 0;
> > >  }
> > >
> > > +static bool rcsi2_asd_match(struct v4l2_subdev *sd,
> > > +                         struct v4l2_async_subdev *asd)
> > > +{
> > > +     struct rcar_csi2 *priv = (struct rcar_csi2 *)asd->match.custom.priv;
> > > +     struct fwnode_handle *remote_endpoint;
> > > +     struct fwnode_handle *subdev_endpoint;
> > > +     struct device_node *np;
> > > +     bool matched = false;
> > > +
> > > +     np = of_graph_get_endpoint_by_regs(priv->dev->of_node, 0, 0);
> > > +     if (!np) {
> > > +             dev_err(priv->dev, "Not connected to subdevice\n");
> > > +             return matched;
> > > +     }
> > > +
> > > +     remote_endpoint =
> > > +             fwnode_graph_get_remote_endpoint(of_fwnode_handle(np));
> > > +     if (!remote_endpoint) {
> > > +             dev_err(priv->dev, "Failed to get remote endpoint\n");
> > > +             of_node_put(np);
> > > +             return matched;
> > > +     }
> > > +     of_node_put(np);
> > > +
> > > +     if (sd->fwnode != dev_fwnode(sd->dev)) {
> > > +             if (remote_endpoint == sd->fwnode)
> > > +                     matched = true;
> > > +     } else {
> > > +             subdev_endpoint =
> > > +                   fwnode_graph_get_next_endpoint(dev_fwnode(sd->dev), NULL);
> > > +             if (remote_endpoint == subdev_endpoint)
> > > +                     matched = true;
> > > +             fwnode_handle_put(subdev_endpoint);
> > > +     }
> > > +
> > > +     fwnode_handle_put(remote_endpoint);
> > > +
> > > +     return matched;
> > > +}
> > > +
> > >  static int rcsi2_parse_dt(struct rcar_csi2 *priv)
> > >  {
> > >       struct device_node *ep;
> > > @@ -833,9 +873,9 @@ static int rcsi2_parse_dt(struct rcar_csi2 *priv)
> > >               return ret;
> > >       }
> > >
> > > -     priv->asd.match.fwnode =
> > > -             fwnode_graph_get_remote_endpoint(of_fwnode_handle(ep));
> > > -     priv->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > +     priv->asd.match.custom.match = &rcsi2_asd_match;
> > > +     priv->asd.match.custom.priv = priv;
> > > +     priv->asd.match_type = V4L2_ASYNC_MATCH_CUSTOM;
> > >
> > >       of_node_put(ep);
> > >

-- 
Regards,

Laurent Pinchart