From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751734AbbCSU6i (ORCPT ); Thu, 19 Mar 2015 16:58:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36814 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751330AbbCSU6e (ORCPT ); Thu, 19 Mar 2015 16:58:34 -0400 From: Jeff Moyer To: Jens Axboe Cc: Tejun Heo , kent.overstreet@gmail.com, Benjamin LaHaise , linux-kernel@vger.kernel.org Subject: Re: serial percpu_ref draining in exit_aio() References: <20150319173413.GU25365@htj.duckdns.org> <550B30D5.9080207@kernel.dk> X-PGP-KeyID: 1F78E1B4 X-PGP-CertKey: F6FE 280D 8293 F72C 65FD 5A58 1FF8 A7CA 1F78 E1B4 X-PCLoadLetter: What the f**k does that mean? Date: Thu, 19 Mar 2015 16:58:33 -0400 In-Reply-To: <550B30D5.9080207@kernel.dk> (Jens Axboe's message of "Thu, 19 Mar 2015 14:25:57 -0600") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Jens Axboe writes: > On 03/19/2015 11:34 AM, Tejun Heo wrote: >> Hello, >> >> So, Jens noticed that fio process exiting takes seconds when there are >> multiple aio contexts and the culprit seems to be the serial >> percpu_ref draining in exit_aio(). It's generally a bad idea to >> expose RCU latencies to userland because they add up really quickly >> and are unrelated to other performance parameters. Can you guys >> please at least update the code so that it waits for all percpu_refs >> to drain at the same time rather than one after another? That should >> resolve the worst part of the problem. > > This works for me. Before: > > real 0m5.872s > user 0m0.020s > sys 0m0.040s > > after > > real 0m0.246s > user 0m0.020s > sys 0m0.040s > > It solves the exit_aio() issue, but if the app calls io_destroy(), > then we are back to square one... Do you really want to do memory allocation in the exit path? That sounds like a bad idea to me. (Of course, now you're going to point out all the places that currently happens, right? ;-) -Jeff