[PATCH rtw-next v2] wifi: rtw88: pci: wake the TX queues when the rings are reset

[PATCH rtw-next v2] wifi: rtw88: pci: wake the TX queues when the rings are reset

[Abdurrahman Karadag]

When a TX ring fills, rtw_pci_tx_write() stops the mac80211 queue and
records that in ring->queue_stopped, which is cleared only by the
completion loop in rtw_pci_tx_isr(). Resetting the rings rewinds the
indices, so nothing that was in flight can complete and run that loop,
and the queue stays stopped over a ring that is now empty.
ieee80211_restart_hw() does not release that stop either, because
ieee80211_reconfig() wakes only IEEE80211_QUEUE_STOP_REASON_SUSPEND.

Release those queues where the reset happens. The stop side is the one
that matters: rtw_fw_recovery() reaches rtw_pci_dma_release() through
rtw_enter_ips() before it calls ieee80211_restart_hw(), so that is
where a queue stopped before the crash is still stopped. By the time
rtw_pci_setup() runs the flag is already clear.

Wake all the queues, but only if at least one ring had filled. This
cannot tell one DRIVER stop from another: rtw_pci_io_err_detected() and
the hardware scan paths in fw.c stop every queue with the same
IEEE80211_QUEUE_STOP_REASON_DRIVER, which mac80211 keeps as a single
non-refcounted slot per queue. Gating on ring->queue_stopped at least
keeps the resets that stopped nothing out of it.

On an RTL8821CE, a BE ring held until rtw_pci_tx_write() stopped its
queue did not come back from the restart. The ring itself was reset -
0x3a8 read 0x00000000 and REG_TXPAUSE was clear again - but BE's
mac80211 stop reason still read 0x1 three minutes later and TX never
resumed. With this patch it reads 0x0 and TX resumes within 2 s. The
ring was filled by writing REG_TXPAUSE 0x0f from debugfs under load,
and rtw_fw_recovery() was called from a debug build.

Signed-off-by: Abdurrahman Karadag <abdurrahmankaradag19@gmail.com>
---
v2:
 - wake with ieee80211_wake_queues() instead of recording and replaying
   per-ring queue mappings (Ping-Ke Shih)
 - drop the Fixes tag: the bug only bites once a recovery path exists,
   which the blamed commit predates (Ping-Ke Shih)
 - cut the commit message down to why and how (Ping-Ke Shih)
 - say which of the two reset call sites fires during recovery, and
   that the measured stall was induced from debugfs
 - patch 2 of the v1 series is dropped rather than resent, while the
   cause of the stall is still open

v1: https://lore.kernel.org/linux-wireless/20261002231013.11792-1-abdurrahmankaradag19@gmail.com/

 drivers/net/wireless/realtek/rtw88/pci.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c
index 66d2e5f..4d3524e 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.c
+++ b/drivers/net/wireless/realtek/rtw88/pci.c
@@ -475,7 +475,29 @@ static void rtw_pci_reset_buf_desc(struct rtw_dev *rtwdev)
 
 static void rtw_pci_reset_trx_ring(struct rtw_dev *rtwdev)
 {
+	struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv;
+	enum rtw_tx_queue_type queue;
+	bool wake = false;
+
 	rtw_pci_reset_buf_desc(rtwdev);
+
+	/*
+	 * The indices are back at zero, so nothing that was in flight can
+	 * complete and reach the wake in rtw_pci_tx_isr(); a queue stopped
+	 * when its ring filled would stay stopped for good. Only wake when
+	 * that happened: the stop reason is shared with the AER and scan
+	 * paths, which wake the queues again themselves.
+	 */
+	for (queue = 0; queue < RTK_MAX_TX_QUEUE_NUM; queue++) {
+		if (!rtwpci->tx_rings[queue].queue_stopped)
+			continue;
+
+		rtwpci->tx_rings[queue].queue_stopped = false;
+		wake = true;
+	}
+
+	if (wake)
+		ieee80211_wake_queues(rtwdev->hw);
 }
 
 static void rtw_pci_enable_interrupt(struct rtw_dev *rtwdev,
-- 
2.55.0

base-commit: 83de3a16c7b37064a59305040ba9b0f93b832794

RE: [PATCH rtw-next v2] wifi: rtw88: pci: wake the TX queues when the rings are reset

[Ping-Ke Shih]

Abdurrahman Karadag <abdurrahmankaradag19@gmail.com> wrote:
> When a TX ring fills, rtw_pci_tx_write() stops the mac80211 queue and
> records that in ring->queue_stopped, which is cleared only by the
> completion loop in rtw_pci_tx_isr(). Resetting the rings rewinds the
> indices, so nothing that was in flight can complete and run that loop,
> and the queue stays stopped over a ring that is now empty.
> ieee80211_restart_hw() does not release that stop either, because
> ieee80211_reconfig() wakes only IEEE80211_QUEUE_STOP_REASON_SUSPEND.

The main reason why we need this patch is ieee80211_restart_hw(), so
please point out this in subject. 

> 
> Release those queues where the reset happens. The stop side is the one
> that matters: rtw_fw_recovery() reaches rtw_pci_dma_release() through
> rtw_enter_ips() before it calls ieee80211_restart_hw(), so that is
> where a queue stopped before the crash is still stopped. By the time
> rtw_pci_setup() runs the flag is already clear.
> 
> Wake all the queues, but only if at least one ring had filled. This
> cannot tell one DRIVER stop from another: rtw_pci_io_err_detected() and
> the hardware scan paths in fw.c stop every queue with the same
> IEEE80211_QUEUE_STOP_REASON_DRIVER, which mac80211 keeps as a single
> non-refcounted slot per queue. Gating on ring->queue_stopped at least
> keeps the resets that stopped nothing out of it.
> 
> On an RTL8821CE, a BE ring held until rtw_pci_tx_write() stopped its
> queue did not come back from the restart. The ring itself was reset -
> 0x3a8 read 0x00000000 and REG_TXPAUSE was clear again - but BE's
> mac80211 stop reason still read 0x1 three minutes later and TX never
> resumed. With this patch it reads 0x0 and TX resumes within 2 s. The
> ring was filled by writing REG_TXPAUSE 0x0f from debugfs under load,
> and rtw_fw_recovery() was called from a debug build.

This isn't related to this patch, but it is still digging, so no need.

As I see, the commit message can be very simple -- wake up queues for
ieee80211_restart_hw() case.

(For the analysis in another mail, I'll read it later.)

> 
> Signed-off-by: Abdurrahman Karadag <abdurrahmankaradag19@gmail.com>
> ---
> v2:
>  - wake with ieee80211_wake_queues() instead of recording and replaying
>    per-ring queue mappings (Ping-Ke Shih)
>  - drop the Fixes tag: the bug only bites once a recovery path exists,
>    which the blamed commit predates (Ping-Ke Shih)
>  - cut the commit message down to why and how (Ping-Ke Shih)
>  - say which of the two reset call sites fires during recovery, and
>    that the measured stall was induced from debugfs
>  - patch 2 of the v1 series is dropped rather than resent, while the
>    cause of the stall is still open
> 
> v1: https://lore.kernel.org/linux-wireless/20261002231013.11792-1-abdurrahmankaradag19@gmail.com/
> 
>  drivers/net/wireless/realtek/rtw88/pci.c | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c
> index 66d2e5f..4d3524e 100644
> --- a/drivers/net/wireless/realtek/rtw88/pci.c
> +++ b/drivers/net/wireless/realtek/rtw88/pci.c
> @@ -475,7 +475,29 @@ static void rtw_pci_reset_buf_desc(struct rtw_dev *rtwdev)
> 
>  static void rtw_pci_reset_trx_ring(struct rtw_dev *rtwdev)

I'm thinking if here is too early to wake up queues. Should we do this in
rtw_reconfig_complete()?

Or we check flag RTW_FLAG_RESTARTING of rtwdev->flags to wake up queues.
This will be clear why we need to wake up. The comment isn't necessary
by the way.

>  {
> +       struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv;
> +       enum rtw_tx_queue_type queue;
> +       bool wake = false;
> +
>         rtw_pci_reset_buf_desc(rtwdev);
> +
> +       /*
> +        * The indices are back at zero, so nothing that was in flight can
> +        * complete and reach the wake in rtw_pci_tx_isr(); a queue stopped
> +        * when its ring filled would stay stopped for good. Only wake when
> +        * that happened: the stop reason is shared with the AER and scan
> +        * paths, which wake the queues again themselves.
> +        */
> +       for (queue = 0; queue < RTK_MAX_TX_QUEUE_NUM; queue++) {
> +               if (!rtwpci->tx_rings[queue].queue_stopped)
> +                       continue;
> +
> +               rtwpci->tx_rings[queue].queue_stopped = false;
> +               wake = true;
> +       }
> +
> +       if (wake)
> +               ieee80211_wake_queues(rtwdev->hw);
>  }
> 
>  static void rtw_pci_enable_interrupt(struct rtw_dev *rtwdev,
> --
> 2.55.0
> 
> base-commit: 83de3a16c7b37064a59305040ba9b0f93b832794