* [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling
@ 2026-11-12 11:25 Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection Kang Yang
` (9 more replies)
0 siblings, 10 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:25 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
Recovery failures can consume significant time before the
driver eventually gives up and enters ATH10K_STATE_WEDGED.
This series fixes several issues in the recovery and WEDGED
failure-handling paths, including recovery detection,
recovery time bounding, waiter release, and PCIe error
handling.
Several patches fix specific bugs and therefore carry Fixes
tags. The remaining patches address closely related issues in
the same recovery path because the fixes alone do not fully
resolve the reported failures.
Taken together, the series ensures that recovery either
succeeds or transitions to ATH10K_STATE_WEDGED within a
bounded amount of time, avoids unnecessary waits after
recovery has failed, and reduces the likelihood of watchdog
timeouts on affected platforms.
Google has reported this class of failures for an extended
period, with more than 4000 observed crashes over the last
month.
---
v3:
- rebase onto the latest ath-next.
- patch 1: fold the old patch 5 ("check the recovery deadline on its own
timeline") into this patch, so the recovery monitor is a self-rearming
delayed work from the start; move recovery_start_jiffies to patch 4.
- patch 4: reword the commit message; recovery_start_jiffies is now
introduced here, where it is first used.
- patch 8 (new): on SDIO, mask the target interrupts and start recovery
when an invalid HTC bundle count is read from the lookahead, to stop
the SDIO IRQ storm.
- patch 9 (new): reject scan requests while the device is not running.
- patch 10 (new): sdio: drop pending async writes and stop the sleep timer
and work across power cycles, so stale requests cannot touch the bus
after power up.
v2:
- patch 3: also wake wow.wakeup_completed, peer_stats_info_complete and
wmi.radar_confirm on WEDGED; reinit peer_delete_done before each
peer delete command.
- patch 5: remove the early-return time_after() branch from the monitor;
in ath10k_reconfig_complete(), cancel the monitor with
cancel_delayed_work_sync() before clearing fail_cont_count and
pending_recovery.
- add SDIO Test-on tag.
To: Jeff Johnson <jjohnson@kernel.org>
To: Loic Poulain <loic.poulain@oss.qualcomm.com>
To: Baochen Qiang <baochen.qiang@oss.qualcomm.com>
Cc: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Cc: linux-wireless@vger.kernel.org
Cc: ath10k@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
Kang Yang (10):
wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection
wifi: ath10k: wake blocked waiters when entering WEDGED state
wifi: ath10k: also wake remaining blocked waiters
wifi: ath10k: bound recovery time before entering WEDGED
wifi: ath10k: hard-block the radio when entering WEDGED state
wifi: ath10k: remove now-unused driver_recovery completion
wifi: ath10k: trigger recovery on PCIe MMIO wake-up failures
wifi: ath10k: trigger recovery on corrupted HTC bundle lookahead
wifi: ath10k: reject scan requests while the device is not running
wifi: ath10k: drop async writes and sleep timer across power cycles
drivers/net/wireless/ath/ath10k/core.c | 132 +++++++++++++++++++++------------
drivers/net/wireless/ath/ath10k/core.h | 13 +++-
drivers/net/wireless/ath/ath10k/mac.c | 23 +++++-
drivers/net/wireless/ath/ath10k/pci.c | 27 +++++++
drivers/net/wireless/ath/ath10k/sdio.c | 85 +++++++++++++++------
drivers/net/wireless/ath/ath10k/snoc.c | 8 +-
6 files changed, 210 insertions(+), 78 deletions(-)
---
base-commit: ff4c7359e310744a7abe53cf66d921fa2969eb19
change-id: 20261111-10k-sdio-crash-39b0faf282bd
Best regards,
--
Kang Yang <kang.yang@oss.qualcomm.com>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-13 9:24 ` Loic Poulain
2026-11-12 11:26 ` [PATCH ath-next v3 02/10] wifi: ath10k: wake blocked waiters when entering WEDGED state Kang Yang
` (8 subsequent siblings)
9 siblings, 1 reply; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
Recovery monitoring runs on workqueue_aux, which is also
used by SDIO RX dispatch and WMI SERVICE_READY handling.
A single firmware crash may generate multiple recovery
triggers. Recovery detection currently handles each one by
waiting for recovery completion on workqueue_aux. On SDIO,
this blocks the same workqueue needed to deliver recovery
traffic, preventing recovery from completing.
As a result, otherwise recoverable firmware crashes fail
with:
ctl_resp never came in (-110)
Treat duplicate triggers as part of the same recovery attempt. Use an
atomic compare-and-exchange on pending_recovery so only the first
trigger starts restart_work.
Convert the recovery monitor to self-rearming delayed work. While
recovery remains pending, the monitor periodically checks its state
without blocking workqueue_aux. Cancel the monitor when recovery
succeeds or the device is stopped.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Fixes: f35a07a4842a ("wifi: ath10k: move recovery check logic into a new work")
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/core.c | 71 +++++++++++++++++++---------------
drivers/net/wireless/ath/ath10k/core.h | 2 +-
drivers/net/wireless/ath/ath10k/mac.c | 13 ++++++-
3 files changed, 53 insertions(+), 33 deletions(-)
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index 7c2939cbde5f..82bb0bb72dc7 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -2493,48 +2493,54 @@ static int ath10k_init_hw_params(struct ath10k *ar)
static void ath10k_core_recovery_check_work(struct work_struct *work)
{
- struct ath10k *ar = container_of(work, struct ath10k, recovery_check_work);
- long time_left;
+ struct ath10k *ar = container_of(work, struct ath10k,
+ recovery_check_work.work);
+ int fail_count;
- /* Sometimes the recovery will fail and then the next all recovery fail,
- * so avoid infinite recovery.
- */
- if (atomic_read(&ar->fail_cont_count) >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
- ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
- atomic_read(&ar->fail_cont_count));
- ar->state = ATH10K_STATE_WEDGED;
+ if (ar->state == ATH10K_STATE_WEDGED)
return;
- }
-
- ath10k_dbg(ar, ATH10K_DBG_BOOT, "total recovery count: %d", ++ar->recovery_count);
- if (atomic_read(&ar->pending_recovery)) {
- /* Sometimes it happened another recovery work before the previous one
- * completed, then the second recovery work will destroy the previous
- * one, thus below is to avoid that.
- */
- time_left = wait_for_completion_timeout(&ar->driver_recovery,
- ATH10K_RECOVERY_TIMEOUT_HZ);
- if (time_left) {
- ath10k_warn(ar, "previous recovery succeeded, skip this!\n");
- return;
- }
+ /*
+ * Pure monitor: only ath10k_core_start_recovery() starts a fresh
+ * cycle. Bail if there is nothing in flight.
+ */
+ if (!atomic_read(&ar->pending_recovery))
+ return;
- /* Record the continuous recovery fail count when recovery failed. */
- atomic_inc(&ar->fail_cont_count);
+ /* Record the continuous recovery fail count when recovery is late. */
+ fail_count = atomic_inc_return(&ar->fail_cont_count);
- /* Avoid having multiple recoveries at the same time. */
+ if (fail_count >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
+ ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
+ fail_count);
+ ar->state = ATH10K_STATE_WEDGED;
return;
}
- atomic_inc(&ar->pending_recovery);
- queue_work(ar->workqueue, &ar->restart_work);
+ /* Recovery still in flight; check again later. */
+ queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
+ ATH10K_RECOVERY_TIMEOUT_HZ);
}
void ath10k_core_start_recovery(struct ath10k *ar)
{
- /* Use workqueue_aux to avoid blocking recovery tracking */
- queue_work(ar->workqueue_aux, &ar->recovery_check_work);
+ if (ar->state == ATH10K_STATE_WEDGED)
+ return;
+
+ /*
+ * SDIO raises several triggers within microseconds of one crash;
+ * only the first winner starts a fresh cycle. Losers do nothing
+ * while the owner runs recovery.
+ */
+ if (atomic_cmpxchg(&ar->pending_recovery, 0, 1) != 0)
+ return;
+ ath10k_dbg(ar, ATH10K_DBG_BOOT, "total recovery count: %d",
+ ++ar->recovery_count);
+ /* Start a fresh failure budget; the last cycle may not have completed. */
+ atomic_set(&ar->fail_cont_count, 0);
+ queue_work(ar->workqueue, &ar->restart_work);
+ queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
+ ATH10K_RECOVERY_TIMEOUT_HZ);
}
EXPORT_SYMBOL(ath10k_core_start_recovery);
@@ -3727,7 +3733,7 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
INIT_WORK(&ar->register_work, ath10k_core_register_work);
INIT_WORK(&ar->restart_work, ath10k_core_restart);
- INIT_WORK(&ar->recovery_check_work, ath10k_core_recovery_check_work);
+ INIT_DELAYED_WORK(&ar->recovery_check_work, ath10k_core_recovery_check_work);
INIT_WORK(&ar->set_coverage_class_work,
ath10k_core_set_coverage_class_work);
@@ -3764,6 +3770,9 @@ EXPORT_SYMBOL(ath10k_core_create);
void ath10k_core_destroy(struct ath10k *ar)
{
+ cancel_work_sync(&ar->restart_work);
+ cancel_delayed_work_sync(&ar->recovery_check_work);
+
destroy_workqueue(ar->workqueue);
destroy_workqueue(ar->workqueue_aux);
diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
index dfee432615eb..e8fcbef38ca6 100644
--- a/drivers/net/wireless/ath/ath10k/core.h
+++ b/drivers/net/wireless/ath/ath10k/core.h
@@ -1207,7 +1207,7 @@ struct ath10k {
struct work_struct register_work;
struct work_struct restart_work;
- struct work_struct recovery_check_work;
+ struct delayed_work recovery_check_work;
struct work_struct bundle_tx_work;
struct work_struct tx_complete_work;
diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 8e604697d6c2..3c45e5483b60 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -5427,7 +5427,15 @@ static void ath10k_stop(struct ieee80211_hw *hw, bool suspend)
cancel_work_sync(&ar->set_coverage_class_work);
cancel_delayed_work_sync(&ar->scan.timeout);
cancel_work_sync(&ar->restart_work);
- cancel_work_sync(&ar->recovery_check_work);
+ cancel_delayed_work_sync(&ar->recovery_check_work);
+
+ /* ath10k_stop() can race with an in-flight recovery. Clear the
+ * recovery state after the works are cancelled so a later
+ * ath10k_start() can start a fresh cycle instead of being blocked
+ * by a stale pending_recovery that nothing will clear.
+ */
+ atomic_set(&ar->fail_cont_count, 0);
+ atomic_set(&ar->pending_recovery, 0);
}
static int ath10k_config_ps(struct ath10k *ar)
@@ -8170,6 +8178,9 @@ static void ath10k_reconfig_complete(struct ieee80211_hw *hw,
ar->state = ATH10K_STATE_ON;
ieee80211_wake_queues(ar->hw);
+ /* Cancel the monitor first so it cannot bump the count after reset. */
+ cancel_delayed_work_sync(&ar->recovery_check_work);
+
/* Clear recovery state. */
complete(&ar->driver_recovery);
atomic_set(&ar->fail_cont_count, 0);
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 02/10] wifi: ath10k: wake blocked waiters when entering WEDGED state
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 03/10] wifi: ath10k: also wake remaining blocked waiters Kang Yang
` (7 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
Commit c256a94d1b1b ("wifi: ath10k: shutdown driver when
hardware is unreliable") added a fail-count based transition
to ATH10K_STATE_WEDGED after repeated recovery failures.
When recovery starts, ath10k_core_restart() sets
ATH10K_FLAG_CRASH_FLUSH and wakes waiters already blocked at
the time the failure is detected. However, additional waits
may be created later while the recovery attempt is still in
progress.
If recovery ultimately fails and the device transitions to
ATH10K_STATE_WEDGED, those later waiters continue sleeping
until their individual timeouts expire even though no further
progress is possible.
On suspend/resume, these extra wait timeouts consume watchdog
budget. Crash logs show cases where one or two such timeouts
are enough to cause the suspend watchdog to fire before the
driver finishes shutting down.
Apply the same CRASH_FLUSH, queue-stop and waiter wakeup
handling when entering ATH10K_STATE_WEDGED so blocked callers
fail immediately once the driver gives up.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Fixes: c256a94d1b1b ("wifi: ath10k: shutdown driver when hardware is unreliable")
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/core.c | 43 +++++++++++++++++++++++-----------
1 file changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index 82bb0bb72dc7..e32bacf5b771 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -2491,6 +2491,32 @@ static int ath10k_init_hw_params(struct ath10k *ar)
return 0;
}
+static void ath10k_core_wake_all_waiters(struct ath10k *ar)
+{
+ complete(&ar->scan.started);
+ complete(&ar->scan.completed);
+ complete(&ar->scan.on_channel);
+ complete(&ar->offchan_tx_completed);
+ complete(&ar->install_key_done);
+ complete(&ar->vdev_setup_done);
+ complete(&ar->vdev_delete_done);
+ complete(&ar->thermal.wmi_sync);
+ complete(&ar->bss_survey_done);
+ wake_up(&ar->htt.empty_tx_wq);
+ wake_up(&ar->wmi.tx_credits_wq);
+ wake_up(&ar->peer_mapping_wq);
+}
+
+static void ath10k_core_enter_wedged(struct ath10k *ar)
+{
+ ar->state = ATH10K_STATE_WEDGED;
+
+ set_bit(ATH10K_FLAG_CRASH_FLUSH, &ar->dev_flags);
+
+ ieee80211_stop_queues(ar->hw);
+ ath10k_core_wake_all_waiters(ar);
+}
+
static void ath10k_core_recovery_check_work(struct work_struct *work)
{
struct ath10k *ar = container_of(work, struct ath10k,
@@ -2513,7 +2539,7 @@ static void ath10k_core_recovery_check_work(struct work_struct *work)
if (fail_count >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
fail_count);
- ar->state = ATH10K_STATE_WEDGED;
+ ath10k_core_enter_wedged(ar);
return;
}
@@ -2585,18 +2611,7 @@ static void ath10k_core_restart(struct work_struct *work)
ieee80211_stop_queues(ar->hw);
ath10k_drain_tx(ar);
- complete(&ar->scan.started);
- complete(&ar->scan.completed);
- complete(&ar->scan.on_channel);
- complete(&ar->offchan_tx_completed);
- complete(&ar->install_key_done);
- complete(&ar->vdev_setup_done);
- complete(&ar->vdev_delete_done);
- complete(&ar->thermal.wmi_sync);
- complete(&ar->bss_survey_done);
- wake_up(&ar->htt.empty_tx_wq);
- wake_up(&ar->wmi.tx_credits_wq);
- wake_up(&ar->peer_mapping_wq);
+ ath10k_core_wake_all_waiters(ar);
/* TODO: We can have one instance of cancelling coverage_class_work by
* moving it to ath10k_halt(), so that both stop() and restart() would
@@ -2624,7 +2639,7 @@ static void ath10k_core_restart(struct work_struct *work)
/* hw restart might be requested from multiple places */
break;
case ATH10K_STATE_RESTARTED:
- ar->state = ATH10K_STATE_WEDGED;
+ ath10k_core_enter_wedged(ar);
fallthrough;
case ATH10K_STATE_WEDGED:
ath10k_warn(ar, "device is wedged, will not restart\n");
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 03/10] wifi: ath10k: also wake remaining blocked waiters
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 02/10] wifi: ath10k: wake blocked waiters when entering WEDGED state Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 04/10] wifi: ath10k: bound recovery time before entering WEDGED Kang Yang
` (6 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
ath10k_core_wake_all_waiters() still misses several completions,
leaving their callers blocked after the driver has given up on the
device.
In particular, callers waiting on peer_delete_done, target_suspend,
wow.wakeup_completed, peer_stats_info_complete or wmi.radar_confirm
continue sleeping until their individual timeouts expire even though
no further progress is possible. The WoW and suspend waits are on the
suspend and resume paths, where these delays consume watchdog budget.
Add the missing completions to ath10k_core_wake_all_waiters() so the
callers are released immediately.
peer_delete_done is not otherwise reinitialized before every wait.
A completion left signaled by recovery would therefore make a later
peer-delete wait return immediately. Reinitialize it before sending
each peer-delete command.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/core.c | 5 +++++
drivers/net/wireless/ath/ath10k/mac.c | 3 +++
2 files changed, 8 insertions(+)
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index e32bacf5b771..d86c4ead06e6 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -2500,7 +2500,12 @@ static void ath10k_core_wake_all_waiters(struct ath10k *ar)
complete(&ar->install_key_done);
complete(&ar->vdev_setup_done);
complete(&ar->vdev_delete_done);
+ complete(&ar->peer_delete_done);
+ complete(&ar->target_suspend);
+ complete(&ar->wow.wakeup_completed);
+ complete(&ar->peer_stats_info_complete);
complete(&ar->thermal.wmi_sync);
+ complete(&ar->wmi.radar_confirm);
complete(&ar->bss_survey_done);
wake_up(&ar->htt.empty_tx_wq);
wake_up(&ar->wmi.tx_credits_wq);
diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 3c45e5483b60..6179a029914d 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -843,6 +843,7 @@ static int ath10k_peer_delete(struct ath10k *ar, u32 vdev_id, const u8 *addr)
lockdep_assert_held(&ar->conf_mutex);
+ reinit_completion(&ar->peer_delete_done);
ret = ath10k_wmi_peer_delete(ar, vdev_id, addr);
if (ret)
return ret;
@@ -5898,6 +5899,7 @@ static int ath10k_add_interface(struct ieee80211_hw *hw,
err_peer_delete:
if (arvif->vdev_type == WMI_VDEV_TYPE_AP ||
arvif->vdev_type == WMI_VDEV_TYPE_IBSS) {
+ reinit_completion(&ar->peer_delete_done);
ath10k_wmi_peer_delete(ar, arvif->vdev_id, vif->addr);
ath10k_wait_for_peer_delete_done(ar, arvif->vdev_id,
vif->addr);
@@ -5960,6 +5962,7 @@ static void ath10k_remove_interface(struct ieee80211_hw *hw,
if (arvif->vdev_type == WMI_VDEV_TYPE_AP ||
arvif->vdev_type == WMI_VDEV_TYPE_IBSS) {
+ reinit_completion(&ar->peer_delete_done);
ret = ath10k_wmi_peer_delete(arvif->ar, arvif->vdev_id,
vif->addr);
if (ret)
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 04/10] wifi: ath10k: bound recovery time before entering WEDGED
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (2 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 03/10] wifi: ath10k: also wake remaining blocked waiters Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 05/10] wifi: ath10k: hard-block the radio when entering WEDGED state Kang Yang
` (5 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
Commit c256a94d1b1b ("wifi: ath10k: shutdown driver when
hardware is unreliable") added fail_cont_count tracking and
declared the device wedged after recovery remained pending for
a fixed number of timeout checks.
The monitor currently relies only on the number of checks and
does not consider how long the recovery attempt has actually
been running. Blocking waits in the recovery path may delay its
completion, so waiting for four timeout checks can consume most
of the suspend watchdog budget before the device enters
ATH10K_STATE_WEDGED.
This leaves insufficient time for blocked callers to exit and
for the driver to complete the WEDGED transition and shut down.
Record when each recovery attempt starts and add a 15-second
deadline based on the elapsed recovery time. Declare the device
wedged when either the deadline expires or fail_cont_count
reaches ATH10K_RECOVERY_MAX_FAIL_COUNT.
Also reduce ATH10K_RECOVERY_MAX_FAIL_COUNT from four to two so
a recovery that remains pending reaches the WEDGED state sooner
when the monitor runs normally.
This prevents a failed recovery from consuming the suspend
watchdog budget before the driver can shut down cleanly.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Fixes: c256a94d1b1b ("wifi: ath10k: shutdown driver when hardware is unreliable")
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/core.c | 10 +++++++---
drivers/net/wireless/ath/ath10k/core.h | 10 +++++++++-
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index d86c4ead06e6..f6b6b81a4896 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -15,6 +15,7 @@
#include <linux/ctype.h>
#include <linux/pm_qos.h>
#include <linux/nvmem-consumer.h>
+#include <linux/jiffies.h>
#include <asm/byteorder.h>
#include "core.h"
@@ -2526,6 +2527,7 @@ static void ath10k_core_recovery_check_work(struct work_struct *work)
{
struct ath10k *ar = container_of(work, struct ath10k,
recovery_check_work.work);
+ unsigned long recovery_start = READ_ONCE(ar->recovery_start_jiffies);
int fail_count;
if (ar->state == ATH10K_STATE_WEDGED)
@@ -2541,9 +2543,10 @@ static void ath10k_core_recovery_check_work(struct work_struct *work)
/* Record the continuous recovery fail count when recovery is late. */
fail_count = atomic_inc_return(&ar->fail_cont_count);
- if (fail_count >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
- ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
- fail_count);
+ if (fail_count >= ATH10K_RECOVERY_MAX_FAIL_COUNT ||
+ time_after(jiffies, recovery_start + ATH10K_RECOVERY_DEADLINE_HZ)) {
+ ath10k_err(ar, "recovery timeout %ums, fail count %d, shutting down driver!",
+ jiffies_to_msecs(jiffies - recovery_start), fail_count);
ath10k_core_enter_wedged(ar);
return;
}
@@ -2569,6 +2572,7 @@ void ath10k_core_start_recovery(struct ath10k *ar)
++ar->recovery_count);
/* Start a fresh failure budget; the last cycle may not have completed. */
atomic_set(&ar->fail_cont_count, 0);
+ WRITE_ONCE(ar->recovery_start_jiffies, jiffies);
queue_work(ar->workqueue, &ar->restart_work);
queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
ATH10K_RECOVERY_TIMEOUT_HZ);
diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
index e8fcbef38ca6..a1ded2cba103 100644
--- a/drivers/net/wireless/ath/ath10k/core.h
+++ b/drivers/net/wireless/ath/ath10k/core.h
@@ -88,7 +88,13 @@
#define ATH10K_ITER_RESUME_FLAGS (IEEE80211_IFACE_ITER_RESUME_ALL |\
IEEE80211_IFACE_SKIP_SDATA_NOT_IN_DRIVER)
#define ATH10K_RECOVERY_TIMEOUT_HZ (5 * HZ)
-#define ATH10K_RECOVERY_MAX_FAIL_COUNT 4
+#define ATH10K_RECOVERY_MAX_FAIL_COUNT 2
+/*
+ * Some platforms configure suspend watchdogs as low as 25s; cap the
+ * total recovery time well under that so the WEDGED transition and
+ * any teardown that follows still have room to run before it fires.
+ */
+#define ATH10K_RECOVERY_DEADLINE_HZ (15 * HZ)
struct ath10k;
@@ -1215,6 +1221,8 @@ struct ath10k {
unsigned int recovery_count;
/* continuous recovery fail count */
atomic_t fail_cont_count;
+ /* jiffies when the current recovery attempt started */
+ unsigned long recovery_start_jiffies;
/* cycle count is reported twice for each visited channel during scan.
* access protected by data_lock
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 05/10] wifi: ath10k: hard-block the radio when entering WEDGED state
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (3 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 04/10] wifi: ath10k: bound recovery time before entering WEDGED Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 06/10] wifi: ath10k: remove now-unused driver_recovery completion Kang Yang
` (4 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
Once ath10k reaches ATH10K_STATE_WEDGED, the device can no
longer recover and remains unusable until the driver is
reloaded or the system is rebooted.
Userspace daemons may continue issuing scan and connect
requests, which the driver can only reject individually.
Mark the radio as hardware-blocked through rfkill when
entering the WEDGED state so userspace stops retrying
operations against a device that can no longer operate.
Both WEDGED transitions already route through
ath10k_core_enter_wedged(), so adding the call there is
enough to cover them consistently.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/core.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index f6b6b81a4896..4eb91efdba41 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -2521,6 +2521,14 @@ static void ath10k_core_enter_wedged(struct ath10k *ar)
ieee80211_stop_queues(ar->hw);
ath10k_core_wake_all_waiters(ar);
+
+ /*
+ * Signal the radio as hard-blocked so userspace (e.g.
+ * wpa_supplicant, NetworkManager) stops looping scan and connect
+ * requests at a device the driver has already given up on. The
+ * state is cleared only when the driver is reloaded.
+ */
+ wiphy_rfkill_set_hw_state(ar->hw->wiphy, true);
}
static void ath10k_core_recovery_check_work(struct work_struct *work)
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 06/10] wifi: ath10k: remove now-unused driver_recovery completion
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (4 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 05/10] wifi: ath10k: hard-block the radio when entering WEDGED state Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 07/10] wifi: ath10k: trigger recovery on PCIe MMIO wake-up failures Kang Yang
` (3 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
The recovery monitor no longer waits on driver_recovery, leaving the
SNOC remove path as its only remaining user.
SNOC removal only needs to ensure that restart_work and
recovery_check_work have stopped before their resources are freed.
cancel_work_sync() and cancel_delayed_work_sync() already provide
that synchronization.
Use the work cancellation helpers in the SNOC remove path, then
remove the unused driver_recovery completion and its initialization
and completion calls.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/core.c | 3 ---
drivers/net/wireless/ath/ath10k/core.h | 1 -
drivers/net/wireless/ath/ath10k/mac.c | 1 -
drivers/net/wireless/ath/ath10k/snoc.c | 8 ++++----
4 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index 4eb91efdba41..3be069724ddd 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -2617,8 +2617,6 @@ static void ath10k_core_restart(struct work_struct *work)
struct ath10k *ar = container_of(work, struct ath10k, restart_work);
int ret;
- reinit_completion(&ar->driver_recovery);
-
set_bit(ATH10K_FLAG_CRASH_FLUSH, &ar->dev_flags);
/* Place a barrier to make sure the compiler doesn't reorder
@@ -3716,7 +3714,6 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
init_completion(&ar->scan.completed);
init_completion(&ar->scan.on_channel);
init_completion(&ar->target_suspend);
- init_completion(&ar->driver_recovery);
init_completion(&ar->wow.wakeup_completed);
init_completion(&ar->install_key_done);
diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
index a1ded2cba103..7c961308d7b9 100644
--- a/drivers/net/wireless/ath/ath10k/core.h
+++ b/drivers/net/wireless/ath/ath10k/core.h
@@ -1067,7 +1067,6 @@ struct ath10k {
} hif;
struct completion target_suspend;
- struct completion driver_recovery;
const struct ath10k_hw_regs *regs;
const struct ath10k_hw_ce_regs *hw_ce_regs;
diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 6179a029914d..50eedc078c12 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -8185,7 +8185,6 @@ static void ath10k_reconfig_complete(struct ieee80211_hw *hw,
cancel_delayed_work_sync(&ar->recovery_check_work);
/* Clear recovery state. */
- complete(&ar->driver_recovery);
atomic_set(&ar->fail_cont_count, 0);
atomic_set(&ar->pending_recovery, 0);
diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c
index 8f171ab6f4de..1ec58ec1029b 100644
--- a/drivers/net/wireless/ath/ath10k/snoc.c
+++ b/drivers/net/wireless/ath/ath10k/snoc.c
@@ -1916,10 +1916,10 @@ static void ath10k_snoc_remove(struct platform_device *pdev)
ath10k_dbg(ar, ATH10K_DBG_SNOC, "snoc remove\n");
- reinit_completion(&ar->driver_recovery);
-
- if (test_bit(ATH10K_SNOC_FLAG_RECOVERY, &ar_snoc->flags))
- wait_for_completion_timeout(&ar->driver_recovery, 3 * HZ);
+ if (test_bit(ATH10K_SNOC_FLAG_RECOVERY, &ar_snoc->flags)) {
+ cancel_work_sync(&ar->restart_work);
+ cancel_delayed_work_sync(&ar->recovery_check_work);
+ }
ath10k_snoc_free_resources(ar);
}
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 07/10] wifi: ath10k: trigger recovery on PCIe MMIO wake-up failures
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (5 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 06/10] wifi: ath10k: remove now-unused driver_recovery completion Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 08/10] wifi: ath10k: trigger recovery on corrupted HTC bundle lookahead Kang Yang
` (2 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
Commit 77258d409ce4 ("ath10k: enable pci soc powersaving")
introduced MMIO wake-up handling through ath10k_pci_wake().
The current code only logs an error when the target cannot be
woken for an MMIO access:
failed to wake target for read32 at 0x00036044: -110
failed to wake target for write32 of 0xfffff81f at 0x00034834: -110
When the target or PCIe link stops responding, an MMIO wake-up
failure may be the earliest indication that the device is no
longer accessible. Since it does not trigger recovery, recovery
may be delayed until another path requests it.
Trigger recovery immediately on MMIO wake-up failures while the
device is running. Also abort PCIe wake, BMI and target-
initialization waits once the device enters ATH10K_STATE_WEDGED,
because no further hardware progress is possible.
This triggers recovery earlier and avoids unnecessary timeout
delays that may lead to suspend watchdog timeouts.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Fixes: 77258d409ce4 ("ath10k: enable pci soc powersaving")
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/pci.c | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/drivers/net/wireless/ath/ath10k/pci.c b/drivers/net/wireless/ath/ath10k/pci.c
index 335bc7c488e4..0c671630cfc9 100644
--- a/drivers/net/wireless/ath/ath10k/pci.c
+++ b/drivers/net/wireless/ath/ath10k/pci.c
@@ -478,6 +478,10 @@ static int ath10k_pci_wake_wait(struct ath10k *ar)
return 0;
}
+ /* The device cannot wake once it is wedged. */
+ if (ar->state == ATH10K_STATE_WEDGED)
+ return -ESHUTDOWN;
+
udelay(curr_delay);
tot_delay += curr_delay;
@@ -642,6 +646,16 @@ static void ath10k_bus_pci_write32(struct ath10k *ar, u32 offset, u32 value)
if (ret) {
ath10k_warn(ar, "failed to wake target for write32 of 0x%08x at 0x%08x: %d\n",
value, offset, ret);
+ /*
+ * wake_wait timed out; the device is unresponsive at the
+ * MMIO level. write32 returns void so the write is
+ * silently dropped -- kick recovery here instead of
+ * letting an upper WMI wait notice it seconds later.
+ * Guards confine this to runtime; cmpxchg dedups.
+ */
+ if (test_bit(ATH10K_FLAG_CORE_REGISTERED, &ar->dev_flags) &&
+ ar->state == ATH10K_STATE_ON)
+ ath10k_core_start_recovery(ar);
return;
}
@@ -665,6 +679,9 @@ static u32 ath10k_bus_pci_read32(struct ath10k *ar, u32 offset)
if (ret) {
ath10k_warn(ar, "failed to wake target for read32 at 0x%08x: %d\n",
offset, ret);
+ if (test_bit(ATH10K_FLAG_CORE_REGISTERED, &ar->dev_flags) &&
+ ar->state == ATH10K_STATE_ON)
+ ath10k_core_start_recovery(ar);
return 0xffffffff;
}
@@ -2248,6 +2265,12 @@ static int ath10k_pci_bmi_wait(struct ath10k *ar,
goto out;
}
+ /* Stop waiting once the device is wedged. */
+ if (ar->state == ATH10K_STATE_WEDGED) {
+ ret = -ESHUTDOWN;
+ goto out;
+ }
+
schedule();
}
@@ -3308,6 +3331,10 @@ int ath10k_pci_wait_for_target_init(struct ath10k *ar)
if (val & FW_IND_INITIALIZED)
break;
+ /* Stop waiting once the device is wedged. */
+ if (ar->state == ATH10K_STATE_WEDGED)
+ break;
+
if (ar_pci->oper_irq_mode == ATH10K_PCI_IRQ_INTX)
/* Fix potential race by repeating CORE_BASE writes */
ath10k_pci_enable_intx_irq(ar);
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 08/10] wifi: ath10k: trigger recovery on corrupted HTC bundle lookahead
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (6 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 07/10] wifi: ath10k: trigger recovery on PCIe MMIO wake-up failures Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 09/10] wifi: ath10k: reject scan requests while the device is not running Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 10/10] wifi: ath10k: drop async writes and sleep timer across power cycles Kang Yang
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
When firmware crashes, the SDIO interrupt handler may read a
corrupted RX lookahead whose header reports more messages than
ar->htc.max_msgs_per_htc_bundle. The driver then prints errors such
as:
HTC bundle length 2322 exceeds maximum 32
failed to allocate a bundle: -12
failed to get pending recv messages: -12
failed to process pending SDIO interrupts: -12
The handler returns -ENOMEM without consuming the mailbox lookahead
or checking the firmware-crash CPU interrupt. The SDIO interrupt
therefore remains asserted and ath10k_sdio_irq_handler() repeatedly
processes the same invalid lookahead.
Stress-test logs show that this interrupt storm can continue for
15 to 20 seconds before the firmware watchdog allows the crash
interrupt to be observed and recovery to start. RX remains stalled
and user traffic times out during this interval.
ath10k_sdio_mbox_rx_alloc() already triggers recovery when the
lookahead payload length exceeds ATH10K_HTC_MBOX_MAX_PAYLOAD_LENGTH.
A bundle count above max_msgs_per_htc_bundle is the same class of
corrupted lookahead.
Mask the target interrupts immediately when the invalid bundle count
is detected, then start recovery so the HIF is reinitialized instead
of waiting for the firmware watchdog.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/sdio.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/net/wireless/ath/ath10k/sdio.c b/drivers/net/wireless/ath/ath10k/sdio.c
index 748ba6749339..fdf19ce4b55b 100644
--- a/drivers/net/wireless/ath/ath10k/sdio.c
+++ b/drivers/net/wireless/ath/ath10k/sdio.c
@@ -497,6 +497,8 @@ static int ath10k_sdio_mbox_rx_process_packets(struct ath10k *ar,
return ret;
}
+static int ath10k_sdio_disable_intrs(struct ath10k *ar);
+
static int ath10k_sdio_mbox_alloc_bundle(struct ath10k *ar,
struct ath10k_sdio_rx_data *rx_pkts,
struct ath10k_htc_hdr *htc_hdr,
@@ -513,6 +515,11 @@ static int ath10k_sdio_mbox_alloc_bundle(struct ath10k *ar,
"HTC bundle length %u exceeds maximum %u\n",
le16_to_cpu(htc_hdr->len),
max_msgs);
+ ath10k_core_start_recovery(ar);
+ /* Recovery is a no-op if already pending, so also mask the
+ * target interrupts to stop the IRQ storm.
+ */
+ ath10k_sdio_disable_intrs(ar);
return -ENOMEM;
}
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 09/10] wifi: ath10k: reject scan requests while the device is not running
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (7 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 08/10] wifi: ath10k: trigger recovery on corrupted HTC bundle lookahead Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 10/10] wifi: ath10k: drop async writes and sleep timer across power cycles Kang Yang
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
ath10k_core_restart() releases ar->conf_mutex after halting the
device, and ath10k_start() later takes it again while powering the
device back up. ath10k_hw_scan() does not check ar->state, so a scan
request may enter between these operations and be sent while the
device is powered down.
On SDIO, ath10k_wmi_cmd_send_nowait() queues the command on
ar_sdio->wr_async_work and returns success. The queued mailbox write
may then run while ath10k_start() is initializing the chip in BMI
mode, resulting in errors such as:
failed to stop wmi scan: -108
failed to start hw scan: -110
failed to read from address 0x868: -110
bmi communication timeout
could not write board data (-110)
The BMI initialization then fails and the driver may enter the
WEDGED state.
Reject scan requests with -EBUSY unless the device is in
ATH10K_STATE_ON or ATH10K_STATE_RESTARTED, matching the state check
already used by ath10k_mac_op_set_antenna().
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/mac.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 50eedc078c12..de466752f988 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -6416,6 +6416,12 @@ static int ath10k_hw_scan(struct ieee80211_hw *hw,
mutex_lock(&ar->conf_mutex);
+ if (ar->state != ATH10K_STATE_ON &&
+ ar->state != ATH10K_STATE_RESTARTED) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
if (ath10k_mac_tdls_vif_stations_count(hw, vif) > 0) {
ret = -EBUSY;
goto exit;
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH ath-next v3 10/10] wifi: ath10k: drop async writes and sleep timer across power cycles
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
` (8 preceding siblings ...)
2026-11-12 11:26 ` [PATCH ath-next v3 09/10] wifi: ath10k: reject scan requests while the device is not running Kang Yang
@ 2026-11-12 11:26 ` Kang Yang
9 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-12 11:26 UTC (permalink / raw)
To: Jeff Johnson, Loic Poulain, Baochen Qiang
Cc: Jeff Johnson, linux-wireless, ath10k, linux-kernel, Kang Yang
ath10k_sdio_hif_tx_sg() queues writes on ar_sdio->wr_asyncq and
schedules wr_async_work. The SDIO sleep timer also schedules this
work, and the work may re-arm the timer.
The current shutdown sequence does not prevent this activity from
surviving an SDIO power cycle:
- ath10k_sdio_hif_stop() drains wr_asyncq once, but new writes may
still be queued before the card is marked disabled.
- ath10k_sdio_hif_power_down() deletes the sleep timer but does not
cancel wr_async_work.
- ath10k_sdio_hif_power_up() does not clear work or requests left
from the previous power cycle.
As a result, stale work may access the SDIO bus while the device is
being reinitialized in BMI mode, causing errors such as:
failed to read from address 0x868: -110
failed to read fifo/chip control register: -110
bmi communication timeout
could not write board data (-110)
Mark the card disabled before power-down begins so new asynchronous
transfers are rejected. Make wr_async_work drop requests that raced
with the state change instead of accessing the bus.
Synchronize the sleep timer and wr_async_work, then drain wr_asyncq
during both power-down and power-up so no request from an earlier
power cycle can run during BMI initialization. Factor the queue
cleanup into a helper shared with ath10k_sdio_hif_stop().
A dropped request is completed the same way as a written one, only
without the bus access. HTC messages go through
ath10k_htc_notify_tx_completion(), which lets the endpoint release the
skb. The IRQ disable request signals its completion and leaves the skb
to the waiter, which frees it. Every bus request is returned to the
free pool, so nothing is leaked.
Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
---
drivers/net/wireless/ath/ath10k/sdio.c | 78 ++++++++++++++++++++++++----------
1 file changed, 56 insertions(+), 22 deletions(-)
diff --git a/drivers/net/wireless/ath/ath10k/sdio.c b/drivers/net/wireless/ath/ath10k/sdio.c
index fdf19ce4b55b..f0e1feacb629 100644
--- a/drivers/net/wireless/ath/ath10k/sdio.c
+++ b/drivers/net/wireless/ath/ath10k/sdio.c
@@ -1461,6 +1461,33 @@ static void ath10k_sdio_sleep_timer_handler(struct timer_list *t)
queue_work(ar_sdio->workqueue, &ar_sdio->wr_async_work);
}
+/* Drop any requests still queued on wr_asyncq without touching the bus.
+ * Caller must ensure wr_async_work is not running concurrently.
+ */
+static void ath10k_sdio_drop_wr_async_reqs(struct ath10k *ar)
+{
+ struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+ struct ath10k_sdio_bus_request *req, *tmp_req;
+
+ spin_lock_bh(&ar_sdio->wr_async_lock);
+
+ list_for_each_entry_safe(req, tmp_req, &ar_sdio->wr_asyncq, list) {
+ struct ath10k_htc_ep *ep;
+
+ list_del(&req->list);
+
+ if (req->htc_msg) {
+ ep = &ar->htc.endpoint[req->eid];
+ ath10k_htc_notify_tx_completion(ep, req->skb);
+ } else if (req->comp) {
+ complete(req->comp);
+ }
+ ath10k_sdio_free_bus_req(ar, req);
+ }
+
+ spin_unlock_bh(&ar_sdio->wr_async_lock);
+}
+
static void ath10k_sdio_write_async_work(struct work_struct *work)
{
struct ath10k_sdio *ar_sdio = container_of(work, struct ath10k_sdio,
@@ -1469,6 +1496,15 @@ static void ath10k_sdio_write_async_work(struct work_struct *work)
struct ath10k_sdio_bus_request *req, *tmp_req;
struct ath10k_mbox_info *mbox_info = &ar_sdio->mbox_info;
+ /* The card is powered down; drop pending requests instead of
+ * touching the bus. Prevents stale writes from racing with a
+ * concurrent BMI power-up on the next recovery cycle.
+ */
+ if (ar_sdio->is_disabled) {
+ ath10k_sdio_drop_wr_async_reqs(ar);
+ return;
+ }
+
spin_lock_bh(&ar_sdio->wr_async_lock);
list_for_each_entry_safe(req, tmp_req, &ar_sdio->wr_asyncq, list) {
@@ -1586,6 +1622,13 @@ static int ath10k_sdio_hif_power_up(struct ath10k *ar,
if (!ar_sdio->is_disabled)
return 0;
+ /* Flush any sleep timer or async work left over from the previous
+ * cycle so they cannot fire during BMI and poke the bus.
+ */
+ timer_delete_sync(&ar_sdio->sleep_timer);
+ cancel_work_sync(&ar_sdio->wr_async_work);
+ ath10k_sdio_drop_wr_async_reqs(ar);
+
ath10k_dbg(ar, ATH10K_DBG_BOOT, "sdio power on\n");
ret = ath10k_sdio_config(ar);
@@ -1629,7 +1672,16 @@ static void ath10k_sdio_hif_power_down(struct ath10k *ar)
ath10k_dbg(ar, ATH10K_DBG_BOOT, "sdio power off\n");
+ /* Reject new async writes from ath10k_sdio_hif_tx_sg() and make
+ * ath10k_sdio_write_async_work() drop any already-queued
+ * requests before they touch the bus.
+ */
+ ar_sdio->is_disabled = true;
+
timer_delete_sync(&ar_sdio->sleep_timer);
+ cancel_work_sync(&ar_sdio->wr_async_work);
+ ath10k_sdio_drop_wr_async_reqs(ar);
+
ath10k_sdio_set_mbox_sleep(ar, true);
/* Disable the card */
@@ -1647,8 +1699,6 @@ static void ath10k_sdio_hif_power_down(struct ath10k *ar)
ath10k_warn(ar, "unable to reset sdio: %d\n", ret);
sdio_release_host(ar_sdio->func);
-
- ar_sdio->is_disabled = true;
}
static int ath10k_sdio_hif_tx_sg(struct ath10k *ar, u8 pipe_id,
@@ -1659,6 +1709,9 @@ static int ath10k_sdio_hif_tx_sg(struct ath10k *ar, u8 pipe_id,
struct sk_buff *skb;
int ret, i;
+ if (ar_sdio->is_disabled)
+ return -ESHUTDOWN;
+
eid = pipe_id_to_eid(pipe_id);
for (i = 0; i < n_items; i++) {
@@ -1972,7 +2025,6 @@ static void ath10k_sdio_irq_disable(struct ath10k *ar)
static void ath10k_sdio_hif_stop(struct ath10k *ar)
{
- struct ath10k_sdio_bus_request *req, *tmp_req;
struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
struct sk_buff *skb;
@@ -1984,25 +2036,7 @@ static void ath10k_sdio_hif_stop(struct ath10k *ar)
dev_kfree_skb_any(skb);
cancel_work_sync(&ar_sdio->wr_async_work);
-
- spin_lock_bh(&ar_sdio->wr_async_lock);
-
- /* Free all bus requests that have not been handled */
- list_for_each_entry_safe(req, tmp_req, &ar_sdio->wr_asyncq, list) {
- struct ath10k_htc_ep *ep;
-
- list_del(&req->list);
-
- if (req->htc_msg) {
- ep = &ar->htc.endpoint[req->eid];
- ath10k_htc_notify_tx_completion(ep, req->skb);
- } else if (req->skb) {
- kfree_skb(req->skb);
- }
- ath10k_sdio_free_bus_req(ar, req);
- }
-
- spin_unlock_bh(&ar_sdio->wr_async_lock);
+ ath10k_sdio_drop_wr_async_reqs(ar);
ath10k_core_napi_sync_disable(ar);
}
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection
2026-11-12 11:26 ` [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection Kang Yang
@ 2026-11-13 9:24 ` Loic Poulain
2026-11-20 6:42 ` Kang Yang
0 siblings, 1 reply; 13+ messages in thread
From: Loic Poulain @ 2026-11-13 9:24 UTC (permalink / raw)
To: Kang Yang
Cc: Jeff Johnson, Baochen Qiang, Jeff Johnson, linux-wireless,
ath10k, linux-kernel
On Thu, Nov 12, 2026 at 12:26 PM Kang Yang <kang.yang@oss.qualcomm.com> wrote:
>
> Recovery monitoring runs on workqueue_aux, which is also
> used by SDIO RX dispatch and WMI SERVICE_READY handling.
>
> A single firmware crash may generate multiple recovery
> triggers. Recovery detection currently handles each one by
> waiting for recovery completion on workqueue_aux. On SDIO,
> this blocks the same workqueue needed to deliver recovery
> traffic, preventing recovery from completing.
>
> As a result, otherwise recoverable firmware crashes fail
> with:
> ctl_resp never came in (-110)
>
> Treat duplicate triggers as part of the same recovery attempt. Use an
> atomic compare-and-exchange on pending_recovery so only the first
> trigger starts restart_work.
>
> Convert the recovery monitor to self-rearming delayed work. While
> recovery remains pending, the monitor periodically checks its state
> without blocking workqueue_aux. Cancel the monitor when recovery
> succeeds or the device is stopped.
>
> Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
> Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
>
> Fixes: f35a07a4842a ("wifi: ath10k: move recovery check logic into a new work")
> Signed-off-by: Kang Yang <kang.yang@oss.qualcomm.com>
> ---
> drivers/net/wireless/ath/ath10k/core.c | 71 +++++++++++++++++++---------------
> drivers/net/wireless/ath/ath10k/core.h | 2 +-
> drivers/net/wireless/ath/ath10k/mac.c | 13 ++++++-
> 3 files changed, 53 insertions(+), 33 deletions(-)
>
> diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
> index 7c2939cbde5f..82bb0bb72dc7 100644
> --- a/drivers/net/wireless/ath/ath10k/core.c
> +++ b/drivers/net/wireless/ath/ath10k/core.c
> @@ -2493,48 +2493,54 @@ static int ath10k_init_hw_params(struct ath10k *ar)
>
> static void ath10k_core_recovery_check_work(struct work_struct *work)
> {
> - struct ath10k *ar = container_of(work, struct ath10k, recovery_check_work);
> - long time_left;
> + struct ath10k *ar = container_of(work, struct ath10k,
> + recovery_check_work.work);
> + int fail_count;
>
> - /* Sometimes the recovery will fail and then the next all recovery fail,
> - * so avoid infinite recovery.
> - */
> - if (atomic_read(&ar->fail_cont_count) >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
> - ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
> - atomic_read(&ar->fail_cont_count));
> - ar->state = ATH10K_STATE_WEDGED;
> + if (ar->state == ATH10K_STATE_WEDGED)
> return;
> - }
> -
> - ath10k_dbg(ar, ATH10K_DBG_BOOT, "total recovery count: %d", ++ar->recovery_count);
>
> - if (atomic_read(&ar->pending_recovery)) {
> - /* Sometimes it happened another recovery work before the previous one
> - * completed, then the second recovery work will destroy the previous
> - * one, thus below is to avoid that.
> - */
> - time_left = wait_for_completion_timeout(&ar->driver_recovery,
> - ATH10K_RECOVERY_TIMEOUT_HZ);
> - if (time_left) {
> - ath10k_warn(ar, "previous recovery succeeded, skip this!\n");
> - return;
> - }
> + /*
> + * Pure monitor: only ath10k_core_start_recovery() starts a fresh
> + * cycle. Bail if there is nothing in flight.
> + */
> + if (!atomic_read(&ar->pending_recovery))
> + return;
>
> - /* Record the continuous recovery fail count when recovery failed. */
> - atomic_inc(&ar->fail_cont_count);
> + /* Record the continuous recovery fail count when recovery is late. */
> + fail_count = atomic_inc_return(&ar->fail_cont_count);
I don't understand why we need so much complication and the need to
count and restart this work again, why not simply scheduling the work
or a timer at the final expected timeout instead, then if we reach the
timeout the device is simply marked as wedged.
>
> - /* Avoid having multiple recoveries at the same time. */
> + if (fail_count >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
> + ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
> + fail_count);
> + ar->state = ATH10K_STATE_WEDGED;
> return;
> }
>
> - atomic_inc(&ar->pending_recovery);
> - queue_work(ar->workqueue, &ar->restart_work);
> + /* Recovery still in flight; check again later. */
> + queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
> + ATH10K_RECOVERY_TIMEOUT_HZ);
> }
>
> void ath10k_core_start_recovery(struct ath10k *ar)
> {
> - /* Use workqueue_aux to avoid blocking recovery tracking */
> - queue_work(ar->workqueue_aux, &ar->recovery_check_work);
> + if (ar->state == ATH10K_STATE_WEDGED)
> + return;
> +
> + /*
> + * SDIO raises several triggers within microseconds of one crash;
> + * only the first winner starts a fresh cycle. Losers do nothing
> + * while the owner runs recovery.
> + */
> + if (atomic_cmpxchg(&ar->pending_recovery, 0, 1) != 0)
> + return;
> + ath10k_dbg(ar, ATH10K_DBG_BOOT, "total recovery count: %d",
> + ++ar->recovery_count);
> + /* Start a fresh failure budget; the last cycle may not have completed. */
> + atomic_set(&ar->fail_cont_count, 0);
> + queue_work(ar->workqueue, &ar->restart_work);
> + queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
> + ATH10K_RECOVERY_TIMEOUT_HZ);
> }
> EXPORT_SYMBOL(ath10k_core_start_recovery);
>
> @@ -3727,7 +3733,7 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
>
> INIT_WORK(&ar->register_work, ath10k_core_register_work);
> INIT_WORK(&ar->restart_work, ath10k_core_restart);
> - INIT_WORK(&ar->recovery_check_work, ath10k_core_recovery_check_work);
> + INIT_DELAYED_WORK(&ar->recovery_check_work, ath10k_core_recovery_check_work);
> INIT_WORK(&ar->set_coverage_class_work,
> ath10k_core_set_coverage_class_work);
>
> @@ -3764,6 +3770,9 @@ EXPORT_SYMBOL(ath10k_core_create);
>
> void ath10k_core_destroy(struct ath10k *ar)
> {
> + cancel_work_sync(&ar->restart_work);
> + cancel_delayed_work_sync(&ar->recovery_check_work);
> +
> destroy_workqueue(ar->workqueue);
>
> destroy_workqueue(ar->workqueue_aux);
> diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
> index dfee432615eb..e8fcbef38ca6 100644
> --- a/drivers/net/wireless/ath/ath10k/core.h
> +++ b/drivers/net/wireless/ath/ath10k/core.h
> @@ -1207,7 +1207,7 @@ struct ath10k {
>
> struct work_struct register_work;
> struct work_struct restart_work;
> - struct work_struct recovery_check_work;
> + struct delayed_work recovery_check_work;
> struct work_struct bundle_tx_work;
> struct work_struct tx_complete_work;
>
> diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
> index 8e604697d6c2..3c45e5483b60 100644
> --- a/drivers/net/wireless/ath/ath10k/mac.c
> +++ b/drivers/net/wireless/ath/ath10k/mac.c
> @@ -5427,7 +5427,15 @@ static void ath10k_stop(struct ieee80211_hw *hw, bool suspend)
> cancel_work_sync(&ar->set_coverage_class_work);
> cancel_delayed_work_sync(&ar->scan.timeout);
> cancel_work_sync(&ar->restart_work);
> - cancel_work_sync(&ar->recovery_check_work);
> + cancel_delayed_work_sync(&ar->recovery_check_work);
> +
> + /* ath10k_stop() can race with an in-flight recovery. Clear the
> + * recovery state after the works are cancelled so a later
> + * ath10k_start() can start a fresh cycle instead of being blocked
> + * by a stale pending_recovery that nothing will clear.
> + */
> + atomic_set(&ar->fail_cont_count, 0);
> + atomic_set(&ar->pending_recovery, 0);
> }
>
> static int ath10k_config_ps(struct ath10k *ar)
> @@ -8170,6 +8178,9 @@ static void ath10k_reconfig_complete(struct ieee80211_hw *hw,
> ar->state = ATH10K_STATE_ON;
> ieee80211_wake_queues(ar->hw);
>
> + /* Cancel the monitor first so it cannot bump the count after reset. */
> + cancel_delayed_work_sync(&ar->recovery_check_work);
> +
> /* Clear recovery state. */
> complete(&ar->driver_recovery);
> atomic_set(&ar->fail_cont_count, 0);
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection
2026-11-13 9:24 ` Loic Poulain
@ 2026-11-20 6:42 ` Kang Yang
0 siblings, 0 replies; 13+ messages in thread
From: Kang Yang @ 2026-11-20 6:42 UTC (permalink / raw)
To: Loic Poulain
Cc: Jeff Johnson, Baochen Qiang, Jeff Johnson, linux-wireless,
ath10k, linux-kernel
On 11/13/2026 5:24 PM, Loic Poulain wrote:
> On Thu, Nov 12, 2026 at 12:26 PM Kang Yang<kang.yang@oss.qualcomm.com> wrote:
>> Recovery monitoring runs on workqueue_aux, which is also
>> used by SDIO RX dispatch and WMI SERVICE_READY handling.
>>
>> A single firmware crash may generate multiple recovery
>> triggers. Recovery detection currently handles each one by
>> waiting for recovery completion on workqueue_aux. On SDIO,
>> this blocks the same workqueue needed to deliver recovery
>> traffic, preventing recovery from completing.
>>
>> As a result, otherwise recoverable firmware crashes fail
>> with:
>> ctl_resp never came in (-110)
>>
>> Treat duplicate triggers as part of the same recovery attempt. Use an
>> atomic compare-and-exchange on pending_recovery so only the first
>> trigger starts restart_work.
>>
>> Convert the recovery monitor to self-rearming delayed work. While
>> recovery remains pending, the monitor periodically checks its state
>> without blocking workqueue_aux. Cancel the monitor when recovery
>> succeeds or the device is stopped.
>>
>> Tested-on: QCA6174 hw3.2 PCI WLAN.RM.4.4.1-00288-QCARMSWPZ-1
>> Tested-on: QCA6174 hw3.2 SDIO WLAN.RMH.4.4.1-00189
>>
>> Fixes: f35a07a4842a ("wifi: ath10k: move recovery check logic into a new work")
>> Signed-off-by: Kang Yang<kang.yang@oss.qualcomm.com>
>> ---
>> drivers/net/wireless/ath/ath10k/core.c | 71 +++++++++++++++++++---------------
>> drivers/net/wireless/ath/ath10k/core.h | 2 +-
>> drivers/net/wireless/ath/ath10k/mac.c | 13 ++++++-
>> 3 files changed, 53 insertions(+), 33 deletions(-)
>>
>> diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
>> index 7c2939cbde5f..82bb0bb72dc7 100644
>> --- a/drivers/net/wireless/ath/ath10k/core.c
>> +++ b/drivers/net/wireless/ath/ath10k/core.c
>> @@ -2493,48 +2493,54 @@ static int ath10k_init_hw_params(struct ath10k *ar)
>>
>> static void ath10k_core_recovery_check_work(struct work_struct *work)
>> {
>> - struct ath10k *ar = container_of(work, struct ath10k, recovery_check_work);
>> - long time_left;
>> + struct ath10k *ar = container_of(work, struct ath10k,
>> + recovery_check_work.work);
>> + int fail_count;
>>
>> - /* Sometimes the recovery will fail and then the next all recovery fail,
>> - * so avoid infinite recovery.
>> - */
>> - if (atomic_read(&ar->fail_cont_count) >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
>> - ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
>> - atomic_read(&ar->fail_cont_count));
>> - ar->state = ATH10K_STATE_WEDGED;
>> + if (ar->state == ATH10K_STATE_WEDGED)
>> return;
>> - }
>> -
>> - ath10k_dbg(ar, ATH10K_DBG_BOOT, "total recovery count: %d", ++ar->recovery_count);
>>
>> - if (atomic_read(&ar->pending_recovery)) {
>> - /* Sometimes it happened another recovery work before the previous one
>> - * completed, then the second recovery work will destroy the previous
>> - * one, thus below is to avoid that.
>> - */
>> - time_left = wait_for_completion_timeout(&ar->driver_recovery,
>> - ATH10K_RECOVERY_TIMEOUT_HZ);
>> - if (time_left) {
>> - ath10k_warn(ar, "previous recovery succeeded, skip this!\n");
>> - return;
>> - }
>> + /*
>> + * Pure monitor: only ath10k_core_start_recovery() starts a fresh
>> + * cycle. Bail if there is nothing in flight.
>> + */
>> + if (!atomic_read(&ar->pending_recovery))
>> + return;
>>
>> - /* Record the continuous recovery fail count when recovery failed. */
>> - atomic_inc(&ar->fail_cont_count);
>> + /* Record the continuous recovery fail count when recovery is late. */
>> + fail_count = atomic_inc_return(&ar->fail_cont_count);
> I don't understand why we need so much complication and the need to
> count and restart this work again, why not simply scheduling the work
> or a timer at the final expected timeout instead, then if we reach the
> timeout the device is simply marked as wedged.
Yeah, you are right. Early this patch-set will re-queue restart work so
we have the fail_cont_count. Now only one restart work will be executed
at one time, so we can delete the fail_cont_count and related code.
>
>
>> - /* Avoid having multiple recoveries at the same time. */
>> + if (fail_count >= ATH10K_RECOVERY_MAX_FAIL_COUNT) {
>> + ath10k_err(ar, "consecutive fail %d times, will shutdown driver!",
>> + fail_count);
>> + ar->state = ATH10K_STATE_WEDGED;
>> return;
>> }
>>
>> - atomic_inc(&ar->pending_recovery);
>> - queue_work(ar->workqueue, &ar->restart_work);
>> + /* Recovery still in flight; check again later. */
>> + queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
>> + ATH10K_RECOVERY_TIMEOUT_HZ);
>> }
>>
>> void ath10k_core_start_recovery(struct ath10k *ar)
>> {
>> - /* Use workqueue_aux to avoid blocking recovery tracking */
>> - queue_work(ar->workqueue_aux, &ar->recovery_check_work);
>> + if (ar->state == ATH10K_STATE_WEDGED)
>> + return;
>> +
>> + /*
>> + * SDIO raises several triggers within microseconds of one crash;
>> + * only the first winner starts a fresh cycle. Losers do nothing
>> + * while the owner runs recovery.
>> + */
>> + if (atomic_cmpxchg(&ar->pending_recovery, 0, 1) != 0)
>> + return;
>> + ath10k_dbg(ar, ATH10K_DBG_BOOT, "total recovery count: %d",
>> + ++ar->recovery_count);
>> + /* Start a fresh failure budget; the last cycle may not have completed. */
>> + atomic_set(&ar->fail_cont_count, 0);
>> + queue_work(ar->workqueue, &ar->restart_work);
>> + queue_delayed_work(ar->workqueue_aux, &ar->recovery_check_work,
>> + ATH10K_RECOVERY_TIMEOUT_HZ);
>> }
>> EXPORT_SYMBOL(ath10k_core_start_recovery);
>>
>> @@ -3727,7 +3733,7 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
>>
>> INIT_WORK(&ar->register_work, ath10k_core_register_work);
>> INIT_WORK(&ar->restart_work, ath10k_core_restart);
>> - INIT_WORK(&ar->recovery_check_work, ath10k_core_recovery_check_work);
>> + INIT_DELAYED_WORK(&ar->recovery_check_work, ath10k_core_recovery_check_work);
>> INIT_WORK(&ar->set_coverage_class_work,
>> ath10k_core_set_coverage_class_work);
>>
>> @@ -3764,6 +3770,9 @@ EXPORT_SYMBOL(ath10k_core_create);
>>
>> void ath10k_core_destroy(struct ath10k *ar)
>> {
>> + cancel_work_sync(&ar->restart_work);
>> + cancel_delayed_work_sync(&ar->recovery_check_work);
>> +
>> destroy_workqueue(ar->workqueue);
>>
>> destroy_workqueue(ar->workqueue_aux);
>> diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
>> index dfee432615eb..e8fcbef38ca6 100644
>> --- a/drivers/net/wireless/ath/ath10k/core.h
>> +++ b/drivers/net/wireless/ath/ath10k/core.h
>> @@ -1207,7 +1207,7 @@ struct ath10k {
>>
>> struct work_struct register_work;
>> struct work_struct restart_work;
>> - struct work_struct recovery_check_work;
>> + struct delayed_work recovery_check_work;
>> struct work_struct bundle_tx_work;
>> struct work_struct tx_complete_work;
>>
>> diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
>> index 8e604697d6c2..3c45e5483b60 100644
>> --- a/drivers/net/wireless/ath/ath10k/mac.c
>> +++ b/drivers/net/wireless/ath/ath10k/mac.c
>> @@ -5427,7 +5427,15 @@ static void ath10k_stop(struct ieee80211_hw *hw, bool suspend)
>> cancel_work_sync(&ar->set_coverage_class_work);
>> cancel_delayed_work_sync(&ar->scan.timeout);
>> cancel_work_sync(&ar->restart_work);
>> - cancel_work_sync(&ar->recovery_check_work);
>> + cancel_delayed_work_sync(&ar->recovery_check_work);
>> +
>> + /* ath10k_stop() can race with an in-flight recovery. Clear the
>> + * recovery state after the works are cancelled so a later
>> + * ath10k_start() can start a fresh cycle instead of being blocked
>> + * by a stale pending_recovery that nothing will clear.
>> + */
>> + atomic_set(&ar->fail_cont_count, 0);
>> + atomic_set(&ar->pending_recovery, 0);
>> }
>>
>> static int ath10k_config_ps(struct ath10k *ar)
>> @@ -8170,6 +8178,9 @@ static void ath10k_reconfig_complete(struct ieee80211_hw *hw,
>> ar->state = ATH10K_STATE_ON;
>> ieee80211_wake_queues(ar->hw);
>>
>> + /* Cancel the monitor first so it cannot bump the count after reset. */
>> + cancel_delayed_work_sync(&ar->recovery_check_work);
>> +
>> /* Clear recovery state. */
>> complete(&ar->driver_recovery);
>> atomic_set(&ar->fail_cont_count, 0);
>>
>> --
>> 2.34.1
>>
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2026-11-20 6:42 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-12 11:25 [PATCH ath-next v3 00/10] wifi: ath10k: fix recovery and WEDGED failure handling Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 01/10] wifi: ath10k: fix SDIO aux workqueue starvation in recovery detection Kang Yang
2026-11-13 9:24 ` Loic Poulain
2026-11-20 6:42 ` Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 02/10] wifi: ath10k: wake blocked waiters when entering WEDGED state Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 03/10] wifi: ath10k: also wake remaining blocked waiters Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 04/10] wifi: ath10k: bound recovery time before entering WEDGED Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 05/10] wifi: ath10k: hard-block the radio when entering WEDGED state Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 06/10] wifi: ath10k: remove now-unused driver_recovery completion Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 07/10] wifi: ath10k: trigger recovery on PCIe MMIO wake-up failures Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 08/10] wifi: ath10k: trigger recovery on corrupted HTC bundle lookahead Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 09/10] wifi: ath10k: reject scan requests while the device is not running Kang Yang
2026-11-12 11:26 ` [PATCH ath-next v3 10/10] wifi: ath10k: drop async writes and sleep timer across power cycles Kang Yang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®