From: Ian Rogers <irogers@google.com>
To: Johannes Berg <johannes@sipsolutions.net>
Cc: linux-wireless@vger.kernel.org, b43-dev@lists.infradead.org,
linux-kernel@vger.kernel.org, llvm@lists.linux.dev,
Elkin Cruz <elkin@google.com>, Ian Rogers <irogers@google.com>
Subject: [PATCH] wifi: b43: Use 32-bit signed integers for N-PHY RX IQ compensation math
Date: Thu, 19 Nov 2026 09:04:37 -0800 [thread overview]
Message-ID: <20261119170437.1512945-1-irogers@google.com> (raw)
In b43_nphy_calc_rx_iq_comp(), the intermediate variables tmp, a, and b
are declared as u16, even though the fixed-point calibration arithmetic
shifts 32-bit IQ correlation and power estimates up to bit 30 before
dividing:
a = -((iq << (30 - iq_nbits)) + (ii >> (1 + arsh)));
tmp = ii >> arsh;
a /= tmp;
...
b = (qq << (31 - qq_nbits));
tmp = ii >> brsh;
b = int_sqrt(b / tmp - a * a) - (1 << 10);
Truncating these 30/31-bit numerators and ~20-bit denominators to u16
discards their upper bits (frequently collapsing b = qq << (31 -
qq_nbits) to 0 and corrupting the signed division a /= tmp), producing
bogus 10-bit RX IQ calibration coefficients compared to the 32-bit int
implementation in lpphy_calc_rx_iq_comp().
Change tmp, a, and b to s32 so the fixed-point divisions and square-root
input retain full 32-bit signed precision.
Found by enabling Clang's -Wimplicit-int-conversion-on-negation on an
x86_64 allyesconfig build, with AI coding assistant help to audit the
warning and prepare the fix. Verified with a standalone C fixed-point
test and Clang build; not runtime-tested on Broadcom 802.11n hardware.
Fixes: 34a56f2cae86 ("b43: N-PHY: implement calculating RX IQ comp")
Signed-off-by: Ian Rogers <irogers@google.com>
---
drivers/net/wireless/broadcom/b43/phy_n.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/wireless/broadcom/b43/phy_n.c b/drivers/net/wireless/broadcom/b43/phy_n.c
index 007940f3996e..7b020e0fcd93 100644
--- a/drivers/net/wireless/broadcom/b43/phy_n.c
+++ b/drivers/net/wireless/broadcom/b43/phy_n.c
@@ -4482,7 +4482,7 @@ static void b43_nphy_calc_rx_iq_comp(struct b43_wldev *dev, u8 mask)
u32 qq;
int iq_nbits, qq_nbits;
int arsh, brsh;
- u16 tmp, a, b;
+ s32 tmp, a, b;
struct nphy_iq_est est;
struct b43_phy_n_iq_comp old;
--
2.56.1.708.g77570a6a1d-goog
reply other threads:[~2026-11-19 17:04 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119170437.1512945-1-irogers@google.com \
--to=irogers@google.com \
--cc=b43-dev@lists.infradead.org \
--cc=elkin@google.com \
--cc=johannes@sipsolutions.net \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-wireless@vger.kernel.org \
--cc=llvm@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®