[PATCH] usb: chipidea: otg: fix fsm deadlock and B-device gadget registration

[PATCH] usb: chipidea: otg: fix fsm deadlock and B-device gadget registration

[Xu Yang]

From: Xu Yang <xu.yang_2@nxp.com>

Commit dd0c03d68522 ("usb: chipidea: udc: support dynamic gadget
add/remove") introduced two regressions on the OTG FSM path:

1. Self-deadlock on gadget->host role switch.

   otg_statemachine() holds fsm.lock while calling otg_set_state()
   -> otg_set_protocol() -> ci_otg_start_host(fsm, 1), which calls
   ci_role_stop() to tear down the gadget role. Since rdrv->stop is
   now udc_stop() (previously the lightweight udc_id_switch_for_host()),
   this reaches:

     ci_role_stop(ci)
       -> udc_stop(ci)
         -> usb_del_gadget_udc()
           -> gadget_unbind_driver()
             -> usb_gadget_udc_stop_locked()
               -> ci_udc_stop()
                 -> ci_udc_stop_for_otg_fsm()
                   -> mutex_lock(&ci->fsm.lock)  <- deadlock

   The same thread that already holds fsm.lock tries to acquire it
   again. Use mutex_trylock() in ci_udc_stop_for_otg_fsm() instead:
   when the lock is held by this thread, otg_statemachine() is
   already driving the exact same transition and will leave the FSM
   state consistent on its own (otg_set_protocol()/otg_set_state()
   overwrite whatever ci_udc_stop_for_otg_fsm() would set), so it is
   safe to skip the redundant correction rather than deadlock on it.

2. B-device gadget UDC never registered in FSM mode.

   Before dd0c03d68522, ci_hdrc_gadget_init() unconditionally called
   udc_start(), which registers the gadget via usb_add_gadget_udc().
   That commit moved udc_start()/udc_stop() to rdrv->start/stop and
   removed the call from ci_hdrc_gadget_init(), relying on
   ci_role_start() to invoke rdrv->start when a role becomes active.

   However, ci_hdrc_otg_fsm_init() does not go through
   ci_role_start() for the B-device (id=1) case, so the gadget is
   never registered in FSM mode, and no gadget driver can ever bind
   to start the state machine. Call ci_role_start(ci, CI_ROLE_GADGET)
   there to register the gadget, same as the non-FSM mode path does.

Closes: https://lore.kernel.org/linux-usb/8e659c4f-723b-4241-9c2b-8ea42c6c3600@gmail.com
Fixes: dd0c03d68522 ("usb: chipidea: udc: support dynamic gadget add/remove")
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Xu Yang <xu.yang_2@nxp.com>
---
 drivers/usb/chipidea/otg_fsm.c | 2 ++
 drivers/usb/chipidea/udc.c     | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/usb/chipidea/otg_fsm.c b/drivers/usb/chipidea/otg_fsm.c
index 642729893cc2..03bebee17eb1 100644
--- a/drivers/usb/chipidea/otg_fsm.c
+++ b/drivers/usb/chipidea/otg_fsm.c
@@ -840,6 +840,8 @@ int ci_hdrc_otg_fsm_init(struct ci_hdrc *ci)
 			hw_read_otgsc(ci, OTGSC_BSV) ? 1 : 0;
 		/* Enable BSV irq */
 		hw_write_otgsc(ci, OTGSC_BSVIE, OTGSC_BSVIE);
+
+		return ci_role_start(ci, CI_ROLE_GADGET);
 	}
 
 	return 0;
diff --git a/drivers/usb/chipidea/udc.c b/drivers/usb/chipidea/udc.c
index c29b2d0dcc7c..1e0d295f5431 100644
--- a/drivers/usb/chipidea/udc.c
+++ b/drivers/usb/chipidea/udc.c
@@ -2167,7 +2167,8 @@ static void ci_udc_stop_for_otg_fsm(struct ci_hdrc *ci)
 	if (!ci_otg_is_fsm_mode(ci))
 		return;
 
-	mutex_lock(&ci->fsm.lock);
+	if (!mutex_trylock(&ci->fsm.lock))
+		return;
 	if (ci->fsm.otg->state == OTG_STATE_A_PERIPHERAL) {
 		ci->fsm.a_bidl_adis_tmout = 1;
 		ci_hdrc_otg_fsm_start(ci);
-- 
2.34.1