From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751556AbdF1F3u (ORCPT ); Wed, 28 Jun 2017 01:29:50 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:48822 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750934AbdF1F3p (ORCPT ); Wed, 28 Jun 2017 01:29:45 -0400 From: Nick Terrell To: "dsterba@suse.cz" CC: Adam Borowski , Kernel Team , Chris Mason , Yann Collet , "squashfs-devel@lists.sourceforge.net" , "linux-btrfs@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH] lib/zstd: use div_u64() to let it build on 32-bit Thread-Topic: [PATCH] lib/zstd: use div_u64() to let it build on 32-bit Thread-Index: AQHS7vzVjoIzn8STEEuywsnQhzrzg6I3uOiAgADy/QCAAJ/JAA== Date: Wed, 28 Jun 2017 05:29:25 +0000 Message-ID: <0237381B-1B0B-4666-BA07-ABCEBDD492AE@fb.com> References: <20170626121230.GF2866@twin.jikos.cz> <20170627041823.1300-1-kilobyte@angband.pl> <7DDA5748-8FAB-497A-9EA3-9DEA27158C61@fb.com> <20170627125730.GV2866@suse.cz> In-Reply-To: <20170627125730.GV2866@suse.cz> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: suse.cz; dkim=none (message not signed) header.d=none;suse.cz; dmarc=none action=none header.from=fb.com; x-originating-ip: [2620:10d:c090:200::4:716b] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;DM5PR15MB1660;20:6e8G9y+UqOWEpsSYfr3FA0NfvP2QPDiVKXZuPSI4n4zjKBOVSMfkyKLmMQVFslT9US4fPhsasrmzX43tZajMsx0ohLsgPb+NQ7RIcy2yUW+5h3+BAGYqw6sOzZ2/K/rfGpnRUfIqZj14WkeJm+4VWYC5Sc0w9yr9fUZ930YE7H0= x-ms-office365-filtering-correlation-id: e0b33104-a958-45a5-8059-08d4bde6a3b4 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(201703131423075)(201703031133081)(300000504095)(300135200095)(300000505095)(300135600095);SRVR:DM5PR15MB1660; x-ms-traffictypediagnostic: DM5PR15MB1660: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(236129657087228); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3002001)(100000703101)(100105400095)(6041248)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123560025)(20161123558100)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DM5PR15MB1660;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DM5PR15MB1660; x-forefront-prvs: 03524FBD26 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(39840400002)(39450400003)(39400400002)(39410400002)(6436002)(6512007)(54906002)(7736002)(54356999)(1730700003)(102836003)(82746002)(2351001)(50986999)(305945005)(2950100002)(229853002)(189998001)(6916009)(81166006)(76176999)(110136004)(53936002)(38730400002)(83716003)(99286003)(77096006)(2900100001)(6246003)(6486002)(2906002)(2501003)(86362001)(6116002)(8676002)(33656002)(3280700002)(5660300001)(478600001)(36756003)(4326008)(5640700003)(25786009)(3660700001)(8936002)(6506006)(93886004);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR15MB1660;H:DM5PR15MB1753.namprd15.prod.outlook.com;FPR:;SPF:None;MLV:sfv;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <5C667D8BB726A44FA69EACE4A3165BEE@namprd15.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2017 05:29:25.1632 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1660 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-06-28_02:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id v5S5U7Ho008331 > Please don't top post. Sorry about that. > Which function needs 1KB of stack space? That's quite a lot. FSE_buildCTable_wksp(), FSE_compress_wksp(), and HUF_readDTableX4() required over 1 KB of stack space. > I can see in [1] that there are some on-stack buffers replaced by > pointers to the workspace. That's good, but I would like to know if > there's any hidden gem that grags the precious stack space. I've been hunting down functions that use up the most stack trace and replacing buffers with pointers to the workspace. I compiled the code with -Wframe-larger-than=512 and reduced the stack usage of all offending functions. In the next version of the patch, no function uses more than 400 B of stack space. We'll be porting the changes back upstream as well. > Hm, I'd suggest to create a version optimized for kernel, eg. expecting > that 4+ GB buffer will never be used and you can use the most fittin in > type. This should affect only the function signatures, not the > algorithm implementation, so porting future zstd changes should be > straightforward. If the functions were exposed, then I would agree 100%. However, since these are internal functions, and the rest of zstd uses size_t to represent buffer sizes, I think it would be awkward to change just FSE/HUF functions. I also prefer size_t because it is friendlier to the optimizer, especially the loop optimizer, since the compiler doesn't have to worry about unsigned overflow. On a related note, zstd performs automatic optimizations to improve compression speed and reduce memory usage when given small sources, which is the common case in the kernel.