From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C767C32792 for ; Thu, 3 Oct 2019 06:43:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 638C0222BE for ; Thu, 3 Oct 2019 06:43:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="dLBEBmZ6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726354AbfJCGn1 (ORCPT ); Thu, 3 Oct 2019 02:43:27 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:14116 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725770AbfJCGn1 (ORCPT ); Thu, 3 Oct 2019 02:43:27 -0400 Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x936hOZB028094 for ; Wed, 2 Oct 2019 23:43:25 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=facebook; bh=6DXtwSfVTxTQsg7OTAhIZXgsLk6C+uUhHjPiAO5wwjY=; b=dLBEBmZ62kTmrmp57awYr6nJ79lFsBRMUGbKHuv6g7qkTYpUn2+OIx9UosJRybSBzX7b MTZHwBn8ltvYqJBAN5U2Jey7kn15pQmka6VP3JrJWJ2DpojP5rdE/ljWa0Tj5DJ8jEPI j0tsFdJ7j2xsQF0xp8e/rV404/9XOrTr2m8= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2vc9fw8v2u-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Wed, 02 Oct 2019 23:43:25 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Wed, 2 Oct 2019 23:43:22 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 8EA6762E311C; Wed, 2 Oct 2019 23:43:20 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: CC: , Song Liu , , Peter Zijlstra , Arnaldo Carvalho de Melo , Thomas Gleixner Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH] perf/core: fix corner case in perf_rotate_context() Date: Wed, 2 Oct 2019 23:43:17 -0700 Message-ID: <20191003064317.3961135-1-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 X-FB-Internal: Safe MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8 definitions=2019-10-03_03:2019-10-01,2019-10-03 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 clxscore=1015 adultscore=0 phishscore=0 suspectscore=1 priorityscore=1501 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1908290000 definitions=main-1910030064 X-FB-Internal: deliver Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a rare corner case, but it does happen: In perf_rotate_context(), when the first cpu flexible event fail to schedule, cpu_rotate is 1, while cpu_event is NULL. Since cpu_event is NULL, perf_rotate_context will _NOT_ call cpu_ctx_sched_out(), thus cpuctx->ctx.is_active will have EVENT_FLEXIBLE set. Then, the next perf_event_sched_in() will skip all cpu flexible events because of the EVENT_FLEXIBLE bit. In the next call of perf_rotate_context(), cpu_rotate stays 1, and cpu_event stays NULL, so this process repeats. The end result is, flexible events on this cpu will not be scheduled (until another event being added to the cpuctx). Similar issue may happen with the task_ctx. But it is usually not a problem because the task_ctx moves around different CPU. Fix this corner case by using cpu_rotate and task_rotate to gate calls for (cpu_)ctx_sched_out and rotate_ctx. Also enable rotate_ctx() to handle event == NULL case. Fixes: 8d5bce0c37fa ("perf/core: Optimize perf_rotate_context() event scheduling") Cc: stable@vger.kernel.org # v4.17+ Cc: Peter Zijlstra Cc: Arnaldo Carvalho de Melo Cc: Thomas Gleixner Signed-off-by: Song Liu --- kernel/events/core.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 4655adbbae10..50021735f367 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -3775,6 +3775,13 @@ static void rotate_ctx(struct perf_event_context *ctx, struct perf_event *event) if (ctx->rotate_disable) return; + /* if no event specified, try to rotate the first event */ + if (!event) + event = rb_entry_safe(rb_first(&ctx->flexible_groups.tree), + typeof(*event), group_node); + if (!event) + return; + perf_event_groups_delete(&ctx->flexible_groups, event); perf_event_groups_insert(&ctx->flexible_groups, event); } @@ -3816,14 +3823,14 @@ static bool perf_rotate_context(struct perf_cpu_context *cpuctx) * As per the order given at ctx_resched() first 'pop' task flexible * and then, if needed CPU flexible. */ - if (task_event || (task_ctx && cpu_event)) + if (task_rotate || (task_ctx && cpu_rotate)) ctx_sched_out(task_ctx, cpuctx, EVENT_FLEXIBLE); - if (cpu_event) + if (cpu_rotate) cpu_ctx_sched_out(cpuctx, EVENT_FLEXIBLE); - if (task_event) + if (task_rotate) rotate_ctx(task_ctx, task_event); - if (cpu_event) + if (cpu_rotate) rotate_ctx(&cpuctx->ctx, cpu_event); perf_event_sched_in(cpuctx, task_ctx, current); -- 2.17.1