From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752056AbdJYO7x (ORCPT ); Wed, 25 Oct 2017 10:59:53 -0400 Received: from mail-sn1nam01on0040.outbound.protection.outlook.com ([104.47.32.40]:27920 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751682AbdJYO7u (ORCPT ); Wed, 25 Oct 2017 10:59:50 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Yuri.Norov@cavium.com; Date: Wed, 25 Oct 2017 17:59:38 +0300 From: Yury Norov To: Clement Courbet Cc: Arnd Bergmann , Rasmus Villemoes , Andrew Morton , Matthew Wilcox , Ingo Molnar , linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2] lib: optimize cpumask_next_and() Message-ID: <20171025145938.wpr7spsujounqudk@yury-thinkpad> References: <20171024105159.18293-1-courbet@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20171024105159.18293-1-courbet@google.com> User-Agent: NeoMutt/20170113 (1.7.2) X-Originating-IP: [95.104.161.7] X-ClientProxiedBy: VI1PR08CA0117.eurprd08.prod.outlook.com (2603:10a6:800:d4::19) To CY4PR0701MB3827.namprd07.prod.outlook.com (2603:10b6:910:94::33) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f12faa1-d36c-41ee-e997-08d51bb9093d X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603199);SRVR:CY4PR0701MB3827; X-Microsoft-Exchange-Diagnostics: 1;CY4PR0701MB3827;3:nIIipXPcHC0AjvMNtucMRUdWP7Lgtp5k40ZqDfE6dapNsL5yAZO58GjWvZUwh+LtoZLZV9FSRrIr5jXTFdZnbgIRyJRAViMberMhPDR2cYtn7NGFS6DGZdXgQC1achN8wNnfYs/OfiOxXigMMTvZDiCh+JsHWMq7fNUmPYD8xAW8/Q4aYP/uMFnF2VZyBFqjU/xUSVc4GVzO1cBq4e5yIwzWzkDdNhHgLVjiBsU44JbSv2HVpCHAvjS6HOTRdGKb;25:RN2e3D89M+Xl0YY82gu7vixiwMuVzLLuv6yil9b//VIz5b9EHFJeiJaRV61V9i1QbbWwT/4Hrs84u9Pvsggp1b1j43/nEwVHTmkeIObeRRyhS983rbEsCnDMkrFxCUQ3dPW5s4UjePT64x9KfFNRz4t34D3D7WuLLEmGAk+vLv2BGVtvksIEI1cQTslLIGtP5amZk45d36XsTrjbnAYqmJm40LBYKiF6d7Km8fzXipaOTaqnKmeNFy3mJfI4TlZZPOw7j9Fzai9MYbt8Pv2pxB94hOJ6EVhkkIimazcNUJ49/5PYRrHl1HePxy+8MTZCtbg6x5NIW6PUTJalW9PUDg==;31:YgT0YDmvRpm5uUTz3ziLXIO2zdtlTDx5lr3GbF83Rr34uSeZ24oyvL8kwRZG8R1ZanuY0rB7D+a6B0IUMRsysF9rs+SwF2a8T6dV0hbZjE/0QMX/4lOM4wpiuB+N4MEMUtaEtnCfjRRNPnqIS6KVm+EMKu8fF8nIVXjxI4sxNNo3510yY9p91cm6RGJRhnnrgqXZZSVK6J8MC6hXYueIeE9WOlsT2xGZJ1IVsHN/V7M= X-MS-TrafficTypeDiagnostic: CY4PR0701MB3827: X-Microsoft-Exchange-Diagnostics: 1;CY4PR0701MB3827;20:2UEKZZM6BaCij5emYovRuQ9lUhdXY1kCCHXiJ7BBdpYNGZelh83ilZKInB49FWahPADXjDakydtuiEYAYjD5OxhwA7EbT01YBjNNv6/vQ7z1PECCRtosovrNQ+q3e54WlF/Wlabp6aSpCDvVV9kEFuSDSfGsreEcw87/5ZgcKW1EMOSlIG2+ow4nVr5Tk/fweJ8dP9tBoG3BH0FXh0ygVbiRu1mrV+QbmMVFLgEaA+KjgufMdg3yN2mApzRMC/yYffluziiiMNZV/tS9qiE0Qk667RPCUbAsX+4xW9yMWfXd4OHQYYosgXObFx/8H8pmlbzOAbc6uOZLtTwyDIMFZgpj84XiesvaLOXW22Ur6JxfOUYp/gbhfubJnjVFopNm6yGa6pTOrJWYObG+joG2+PWNSP014qk1mxpraNMEEYBulDZtBwIT/juufgIwGA6BJXzqeKxQuAQ5OVFj6LrSUsKdO3wWuKvPh8/GDDac5ydFbvaYwjCwRC9NdYMDCE4vsq0ssHxg7jUqDYovrl9UUfeDggOxuaGjr7R3hpAJkGHrNQWqSty4BEF274KSPkow93jSB5MpPgSAqHNUN/+oBspVYSTfWj241FfBcQDYpD8= X-Exchange-Antispam-Report-Test: UriScan:(211936372134217)(153496737603132); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(3002001)(100000703101)(100105400095)(3231020)(10201501046)(920507026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123562025)(20161123564025)(20161123558100)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CY4PR0701MB3827;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CY4PR0701MB3827; X-Microsoft-Exchange-Diagnostics: 1;CY4PR0701MB3827;4:SnwIHNcOvy49zqf2ljwlhyWCwuGJbR+VRT8tGOs7yGX++mRAdUkxR6QB+/j8xW6vgSIZ0lhEx6vmdibNxhmnG29DCs7T9QtryksaEGdnJ/DraVdGfo9TVP794xlMFg7N4JOKRK9B+Ggzg1pvYx6E0AkJoiZJm+BeKKFviaCs+BIMdCaRRD1KS2ltreU5tagoba9R9SBaXoZWs0zrB+I6a3aWcyo2jWUxG2f5o7Sc0NxxXIKnEBhsfQRs01YsbCCYLejgb5q5wVfWx24uKsMoi5Ua9fYRJVJW7FREitQmLAk5e7l/JBXlx2QOWB5ijoEy2DrC7bKn/UqkIgkessjXkw== X-Forefront-PRVS: 0471B73328 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(6069001)(7916004)(376002)(346002)(199003)(24454002)(189002)(105586002)(34040400001)(229853002)(6916009)(8936002)(42882006)(106356001)(2950100002)(7736002)(316002)(8676002)(54906003)(33646002)(9686003)(1076002)(81156014)(58126008)(81166006)(47776003)(305945005)(6246003)(66066001)(16586007)(5660300001)(101416001)(575784001)(97736004)(16526018)(6486002)(72206003)(83506002)(33716001)(53936002)(6666003)(23726003)(189998001)(76506005)(478600001)(3846002)(54356999)(50986999)(76176999)(68736007)(6116002)(50466002)(8666007)(2906002)(25786009)(4326008)(6496005)(357404004);DIR:OUT;SFP:1101;SCL:1;SRVR:CY4PR0701MB3827;H:localhost;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CY4PR0701MB3827;23:zmTzPT2YiCy9TDjOB/klIc1Xb9iqt2czio7kT4C?= =?us-ascii?Q?0vfGSoz/VJ7UF3mPykVnuno/Ri8jxgwwYi5GRvhHvhpWV989c8rrP9AQRNkT?= =?us-ascii?Q?ejX7pMrvq4J0VpH2Jwo26XvNeaJ4aO3zlsq2zIbpWoNKsZnF+K3AhPeMSa/F?= =?us-ascii?Q?FLKOuWioMOcq/8wKVgGYzt5rr4mlm8GfD1b9UjX5CS+phYjyrJVlhjmDKHIl?= =?us-ascii?Q?zQUQodAL+3QxcOqjl5QkcBNMxMQrdSzuqkLtTjPNppr+hQCF7jRcRoziG5vr?= =?us-ascii?Q?S6pO7L4faj+wIjK+8/iK9uEUnBmRs6kB+HwH3msNpbtGxA3bax4+38AZF9k5?= =?us-ascii?Q?7VvP5c9h2VqRQOCnPSzHiUchUl8B3X9CQPIJnWfmE/hEgSCtnM2qLImIbE25?= =?us-ascii?Q?SEtE3yEFR7mV2NmKVDUCdjWhyeJXzmKEJZQtayUvhhZ1vijaRASC9FMyMQQ9?= =?us-ascii?Q?HvSstTG+7bY3ZUSJcI3x+ma/GfTqcLkThomnYO/LBptPY7jd8Ne5ZEmtTESX?= =?us-ascii?Q?jqWigTTOMALgVrVuGolVZC9amTksLz2yiEne2Xuc6zdz8i5gXRPDa10IO7zs?= =?us-ascii?Q?KoRmhaNv83P66rttyiGibUX/YSk6RMta2jIwyrE8MdmjxkHvUqy3sIJA1KaE?= =?us-ascii?Q?GoqO9hzip7fHSVo0h3h1xB3sMnjiSe1TrLbYqbbdjmKWW5r017BM9A6SWBLe?= =?us-ascii?Q?V3EYrMUutgKSXLdrn4yNs0XAiEluVpIevEDwn1qjnC5VPhc6IIE4B0z2sJW3?= =?us-ascii?Q?ym/KUDv+Atj7UAFM4CXwv3BvAM8bDhr+VLDfMumGcW6DbprpGlvxCcnBOfzI?= =?us-ascii?Q?6c3vVHRhCRaHR8hFnPmdAHZWAr6E3n/JLN4S0+ly5O5xD7RkWh6Ye4gUcfWG?= =?us-ascii?Q?0dn6O0hSdkDBhmlT/oB902QjR+WeAABMULoC6og0PaYPhrVq6BSZW/XDYYhQ?= =?us-ascii?Q?388adzOFPFLgHXcGiFzdBxpBYuqHnNufQuoFhqzmRhqny+kPRc5iCSer7wey?= =?us-ascii?Q?+BnPprh2RbRiNuGGV7IIxNoUYWam7d5mTNw6jzyi47Zadm4kcbge4+sCWdnV?= =?us-ascii?Q?YVd3WB/Awv5v7HHp8EVv2qsCxakJRFMNlpC6WadfapqYCjIX3eJndiB5si+O?= =?us-ascii?Q?F5r/vDVKG06/gVtM+zWZWOq4+dtMhL6UAA1f22J5/VZkZlOAr8xKGPoOIYbx?= =?us-ascii?Q?whFMWA/9p+Y6QCDmn34kcQPJKyVO385FXf74oCP8n7gC+IwB2apmP4DoTc/4?= =?us-ascii?Q?J0UpCl5Xq6eV0oNj+HorjP8XSkURnlGS2EQvy9hPAS8ppS9DBtcpOjFNuGoS?= =?us-ascii?Q?14KZlNa0blRqyDNcdW2XOLw9NLjBuz+7fOI1iP+2yEyJcHDv7/eGWg3mMJAm?= =?us-ascii?Q?kcjZ/Gw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR0701MB3827;6:h1/ZCrBr7+078LrFyaClLnWJcqRmPpk1DdPLVludf7NhjVUBVqYyWIGD3H+sNUcNAgAZG20tVmkFuCK1CBhL513v5Rzzl7ZKWKpdwigBkqIk0hUbtKFUnGaY55fa3WLHbc5GvAwSmt18AVYMHb8gnk7RzbH362tto0lVTDAx5if/UQcRaHOjJFngHKkj4KyRIVFBhPIvSf7GDwTOi0nvx41X8LnIYvfpCNHnRwvhEoHCBi6+/hnAyfxLP8qKZ+SbM5rKp6/RYw2OauLOJ8W9+FQnGDG7w0B4hbom0j4BDU04+BpCjkwqbsFEZb/5q5M43NAzBtzm072keFFbsIHD7g==;5:utxaD/EH4+KLeqzXTxE2PA+znVbqBZQgZDd2p96m5Y2auz3W+ZY6vQHMz2yslRjT4Aej1pTMzuvgmu32r5lRvvhueYmEefbKc6bUoLX5XdxyY/z+I+M0VVy5YQdcqdR9cAAo/p43BDTlqqnU2Idcdg==;24:vVClBTyY8clrNwk2Cs+8KLIvlVHOdLPffZd02X65wB2XiAze1tpPw/a7+IOlt0zsGdIObsDvu6KVz5AGiR5YPE7L+MyycXTirer5Sa3RODM=;7:OKCuS1RqxBd7WW1HtYwA/p+j7THaf59R6LpIH1/itf5Sy+wEb1Bisb2DMaSngH0psUcsNQC+1qatn6bsn5jvxFbd9AWQFd/pozSd+wDXx9eZQcRtdJR0Zy8E49QCiu6maYPfoZXWZhHx4c8KfUs5rMZfrQXV01/WDkuspydrWIirFBRNU+JQkoKHIJUW1Me1eMj/hVeGjl2bXYmtRxTw7pTKkKtw6THUqndG9MYM0mU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2017 14:59:47.5589 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f12faa1-d36c-41ee-e997-08d51bb9093d X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR0701MB3827 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Oct 24, 2017 at 12:51:59PM +0200, Clement Courbet wrote: > We've measured that we spend ~0.6% of sys cpu time in cpumask_next_and(). > It's essentially a joined iteration in search for a non-zero bit, which > is currently implemented as a lookup join (find a nonzero bit on the > lhs, lookup the rhs to see if it's set there). > > Implement a direct join (find a nonzero bit on the incrementally built > join). Direct benchmarking shows that it's 1.17x to 14x faster with a > geometric mean of 2.1 on 32 CPUs. No impact on memory usage. > > Approximate benchmark code: > > ``` > unsigned long src1p[nr_cpumask_longs] = {pattern1}; > unsigned long src2p[nr_cpumask_longs] = {pattern2}; > for (/*a bunch of repetitions*/) { > for (int n = -1; n <= nr_cpu_ids; ++n) { > asm volatile("" : "+rm"(src1p)); // prevent any optimization > asm volatile("" : "+rm"(src2p)); > unsigned long result = cpumask_next_and(n, src1p, src2p); > asm volatile("" : "+rm"(result)); > } > } > ``` > Signed-off-by: Clement Courbet > --- > Changes in v2: > - Refactored _find_next_common_bit into _find_next_bit., as suggested > by Yury Norov. What I actually suggested is make _find_next_and_bit() similar to _find_next_bit(), not to extend _find_next_bit(). But what you did looks OK. > This has no adverse effects on the performance side, > as the compiler successfully inlines the code. I think it's not about inlining, compiler just optimizes out branches known as false at compile-time. > include/asm-generic/bitops/find.h | 16 ++++++++++++++ > include/linux/bitmap.h | 2 ++ > lib/cpumask.c | 9 ++++---- > lib/find_bit.c | 37 +++++++++++++++++++++++++-------- > tools/include/asm-generic/bitops/find.h | 16 ++++++++++++++ > 5 files changed, 67 insertions(+), 13 deletions(-) > > diff --git a/include/asm-generic/bitops/find.h b/include/asm-generic/bitops/find.h > index 998d4d544f18..130962f3a264 100644 > --- a/include/asm-generic/bitops/find.h > +++ b/include/asm-generic/bitops/find.h > @@ -15,6 +15,22 @@ extern unsigned long find_next_bit(const unsigned long *addr, unsigned long > size, unsigned long offset); > #endif > > +#ifndef find_next_and_bit > +/** > + * find_next_and_bit - find the next set bit in both memory regions > + * @addr1: The first address to base the search on > + * @addr2: The second address to base the search on > + * @offset: The bitnumber to start searching at > + * @size: The bitmap size in bits > + * > + * Returns the bit number for the next set bit > + * If no bits are set, returns @size. > + */ > +extern unsigned long find_next_and_bit(const unsigned long *addr1, > + const unsigned long *addr2, unsigned long size, > + unsigned long offset); > +#endif > + > #ifndef find_next_zero_bit > /** > * find_next_zero_bit - find the next cleared bit in a memory region > diff --git a/include/linux/bitmap.h b/include/linux/bitmap.h > index 700cf5f67118..b4606bfda52f 100644 > --- a/include/linux/bitmap.h > +++ b/include/linux/bitmap.h > @@ -77,6 +77,8 @@ > * find_first_bit(addr, nbits) Position first set bit in *addr > * find_next_zero_bit(addr, nbits, bit) Position next zero bit in *addr >= bit > * find_next_bit(addr, nbits, bit) Position next set bit in *addr >= bit > + * find_next_and_bit(addr1, addr2, nbits, bit) Same as find_first_bit, but in > + * (*addr1 & *addr2) > */ > > /* > diff --git a/lib/cpumask.c b/lib/cpumask.c > index 8b1a1bd77539..5602223837fa 100644 > --- a/lib/cpumask.c > +++ b/lib/cpumask.c > @@ -32,10 +32,11 @@ EXPORT_SYMBOL(cpumask_next); > int cpumask_next_and(int n, const struct cpumask *src1p, > const struct cpumask *src2p) > { > - while ((n = cpumask_next(n, src1p)) < nr_cpu_ids) > - if (cpumask_test_cpu(n, src2p)) > - break; > - return n; > + /* -1 is a legal arg here. */ > + if (n != -1) > + cpumask_check(n); > + return find_next_and_bit(cpumask_bits(src1p), cpumask_bits(src2p), > + nr_cpumask_bits, n + 1); > } > EXPORT_SYMBOL(cpumask_next_and); > > diff --git a/lib/find_bit.c b/lib/find_bit.c > index 6ed74f78380c..ebc08fd9fdf8 100644 > --- a/lib/find_bit.c > +++ b/lib/find_bit.c > @@ -24,19 +24,25 @@ > #if !defined(find_next_bit) || !defined(find_next_zero_bit) > > /* > - * This is a common helper function for find_next_bit and > - * find_next_zero_bit. The difference is the "invert" argument, which > - * is XORed with each fetched word before searching it for one bits. > + * This is a common helper function for find_next_bit, find_next_zero_bit, and > + * find_next_and_bit. The differences are: > + * - The "invert" argument, which is XORed with each fetched word before > + * searching it for one bits. > + * - The optional "addr2", which is anded with "addr1" if present. > */ > -static unsigned long _find_next_bit(const unsigned long *addr, > - unsigned long nbits, unsigned long start, unsigned long invert) > +static unsigned long _find_next_bit(const unsigned long *addr1, > + const unsigned long *addr2, unsigned long nbits, > + unsigned long start, unsigned long invert) > { > unsigned long tmp; > > if (unlikely(start >= nbits)) > return nbits; > > - tmp = addr[start / BITS_PER_LONG] ^ invert; > + tmp = addr1[start / BITS_PER_LONG]; > + if (addr2) > + tmp &= addr2[start / BITS_PER_LONG]; > + tmp ^= invert; > > /* Handle 1st word. */ > tmp &= BITMAP_FIRST_WORD_MASK(start); > @@ -47,7 +53,10 @@ static unsigned long _find_next_bit(const unsigned long *addr, > if (start >= nbits) > return nbits; > > - tmp = addr[start / BITS_PER_LONG] ^ invert; > + tmp = addr1[start / BITS_PER_LONG]; > + if (addr2) > + tmp &= addr2[start / BITS_PER_LONG]; > + tmp ^= invert; > } > > return min(start + __ffs(tmp), nbits); > @@ -61,7 +70,7 @@ static unsigned long _find_next_bit(const unsigned long *addr, > unsigned long find_next_bit(const unsigned long *addr, unsigned long size, > unsigned long offset) > { > - return _find_next_bit(addr, size, offset, 0UL); > + return _find_next_bit(addr, NULL, size, offset, 0UL); > } > EXPORT_SYMBOL(find_next_bit); > #endif > @@ -70,11 +79,21 @@ EXPORT_SYMBOL(find_next_bit); > unsigned long find_next_zero_bit(const unsigned long *addr, unsigned long size, > unsigned long offset) > { > - return _find_next_bit(addr, size, offset, ~0UL); > + return _find_next_bit(addr, NULL, size, offset, ~0UL); > } > EXPORT_SYMBOL(find_next_zero_bit); > #endif > > +#if !defined(find_next_and_bit) > +unsigned long find_next_and_bit(const unsigned long *addr1, > + const unsigned long *addr2, unsigned long nbits, > + unsigned long start) It should be: unsigned long find_next_and_bit(const unsigned long *addr1, const unsigned long *addr2, unsigned long size, unsigned long offset) > +{ > + return _find_next_bit(addr1, addr2, size, offset, ~0UL); > +} > +EXPORT_SYMBOL(find_next_and_bit); > +#endif > + > #ifndef find_first_bit > /* > * Find the first set bit in a memory region. If we continue this way, we'll most probably end up like this, sooner or later: diff --git a/lib/find_bit.c b/lib/find_bit.c index ebc08fd9fdf8..1b0b4aedc93a 100644 --- a/lib/find_bit.c +++ b/lib/find_bit.c @@ -31,8 +31,12 @@ * - The optional "addr2", which is anded with "addr1" if present. */ static unsigned long _find_next_bit(const unsigned long *addr1, - const unsigned long *addr2, unsigned long nbits, - unsigned long start, unsigned long invert) + const unsigned long *and, + const unsigned long *or, + const unsigned long *xor, + unsigned long nbits, + unsigned long start, + unsigned long invert) { unsigned long tmp; @@ -40,8 +44,12 @@ static unsigned long _find_next_bit(const unsigned long *addr1, return nbits; tmp = addr1[start / BITS_PER_LONG]; - if (addr2) - tmp &= addr2[start / BITS_PER_LONG]; + if (and) + tmp &= and[start / BITS_PER_LONG]; + if (or) + tmp |= or[start / BITS_PER_LONG]; + if (xor) + tmp ^= xor[start / BITS_PER_LONG]; tmp ^= invert; /* Handle 1st word. */ @@ -54,8 +62,12 @@ static unsigned long _find_next_bit(const unsigned long *addr1, return nbits; tmp = addr1[start / BITS_PER_LONG]; - if (addr2) - tmp &= addr2[start / BITS_PER_LONG]; + if (and) + tmp &= and[start / BITS_PER_LONG]; + if (or) + tmp |= or[start / BITS_PER_LONG]; + if (xor) + tmp ^= xor[start / BITS_PER_LONG]; tmp ^= invert; } Just a fantasy of course. I'm generally fine to proceed this way. It makes _find_next_bit() more complex, but lets us avoid code duplication, which is better deal for long-term maintenance. But I'd like also to keep _find_next_bit() consistent with _find_next_bit_le() Could you please send v3 with fixed find_next_and_bit() declaration, and synced LE routines? Yury