* [PATCH] wifi: b43: Use 32-bit signed integers for N-PHY RX IQ compensation math
@ 2026-11-19 17:04 Ian Rogers
0 siblings, 0 replies; only message in thread
From: Ian Rogers @ 2026-11-19 17:04 UTC (permalink / raw)
To: Johannes Berg
Cc: linux-wireless, b43-dev, linux-kernel, llvm, Elkin Cruz, Ian Rogers
In b43_nphy_calc_rx_iq_comp(), the intermediate variables tmp, a, and b
are declared as u16, even though the fixed-point calibration arithmetic
shifts 32-bit IQ correlation and power estimates up to bit 30 before
dividing:
a = -((iq << (30 - iq_nbits)) + (ii >> (1 + arsh)));
tmp = ii >> arsh;
a /= tmp;
...
b = (qq << (31 - qq_nbits));
tmp = ii >> brsh;
b = int_sqrt(b / tmp - a * a) - (1 << 10);
Truncating these 30/31-bit numerators and ~20-bit denominators to u16
discards their upper bits (frequently collapsing b = qq << (31 -
qq_nbits) to 0 and corrupting the signed division a /= tmp), producing
bogus 10-bit RX IQ calibration coefficients compared to the 32-bit int
implementation in lpphy_calc_rx_iq_comp().
Change tmp, a, and b to s32 so the fixed-point divisions and square-root
input retain full 32-bit signed precision.
Found by enabling Clang's -Wimplicit-int-conversion-on-negation on an
x86_64 allyesconfig build, with AI coding assistant help to audit the
warning and prepare the fix. Verified with a standalone C fixed-point
test and Clang build; not runtime-tested on Broadcom 802.11n hardware.
Fixes: 34a56f2cae86 ("b43: N-PHY: implement calculating RX IQ comp")
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/net/wireless/broadcom/b43/phy_n.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/wireless/broadcom/b43/phy_n.c b/drivers/net/wireless/broadcom/b43/phy_n.c
index 007940f3996e..7b020e0fcd93 100644
--- a/drivers/net/wireless/broadcom/b43/phy_n.c
+++ b/drivers/net/wireless/broadcom/b43/phy_n.c
@@ -4482,7 +4482,7 @@ static void b43_nphy_calc_rx_iq_comp(struct b43_wldev *dev, u8 mask)
u32 qq;
int iq_nbits, qq_nbits;
int arsh, brsh;
- u16 tmp, a, b;
+ s32 tmp, a, b;
struct nphy_iq_est est;
struct b43_phy_n_iq_comp old;
--
2.56.1.708.g77570a6a1d-goog
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2026-11-19 17:04 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-19 17:04 [PATCH] wifi: b43: Use 32-bit signed integers for N-PHY RX IQ compensation math Ian Rogers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®