From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D22CEC433E0 for ; Fri, 5 Jun 2020 07:45:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC1632074B for ; Fri, 5 Jun 2020 07:45:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SgVXZTnw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726144AbgFEHp0 (ORCPT ); Fri, 5 Jun 2020 03:45:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725280AbgFEHp0 (ORCPT ); Fri, 5 Jun 2020 03:45:26 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1498C08C5C2 for ; Fri, 5 Jun 2020 00:45:25 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id v24so3315433plo.6 for ; Fri, 05 Jun 2020 00:45:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bmrm2m7W0PSX2i69F/rKXTi11ScgSrzl4tXCB1MSrUA=; b=SgVXZTnwu/kB/WAJ0aBMI7PpNh2+yxXii1tt4cohX0MmEyfLL2qa/veUkJgpOLfnoe VPTvuKAkPV2giTt93wef9dWmTMON5iqBWQRM1QbmQ3VXYGpp4mxSCxfEEyYoXZMYi6S3 Z53SYgX/AbaJV39WHCZWrrPttqh7qUxiIWzMPm8ttl1LGcovBVrw62ObioyK0pN09HV6 i+jZeqeOatRMHduGAFr1y4A65MPY8yziA/ZW4PsfVeejGbFSbG65OpBno1lGxwulQKai t/9cQxZPzA1rHDkIsq8je4vGc5nay+QiQpoyWRKdVzMVE0XYTEIUq7zDLTNHne6+tsfF r+5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bmrm2m7W0PSX2i69F/rKXTi11ScgSrzl4tXCB1MSrUA=; b=dcf+875suXlLtXymu8eCUKDJPxZrq9bC+oFfTILR0gx+wy5SdOY43i0WHEPx9ZPvco m1L9UqQNaS8b5fREVLoQFwYZmhkEHuonaizDqaNteqcxC3rrswpEz4zdFXTCIrTAFG9N 6++0NoFpTA63J/+yYZEhhjJ0IKG33TjZss2BsNPVVgOFraQuPkJNG78X5GVqOAh6asUC 4XrWUEJ5fhjy8R1hBCfkkWLvhSMYHRENQSzC2xXVRGVXSfDTbdhj1qxX+mfzb6WYSrhm osTZFvP3N8WbXSJhEpF9lSaQmqzF0eO8I6DwzckYaoDoFxk2O0uRQsTv3vmwjCBhYwVC 7F3w== X-Gm-Message-State: AOAM531tq40Rth3VTZ9THqMhwhgFSSggwPrJhX7+b9hDDcYcnh880J7R AkeLgVA1XKBdfiy5/ug+VnTXV+wg X-Google-Smtp-Source: ABdhPJw4vZeUd7aqryBVZOFCiyyc/Phkd9UtE5FELuWOks5+AwIiDCiyLgenDYl2GB5h4biJXb3BMA== X-Received: by 2002:a17:90a:d784:: with SMTP id z4mr1597708pju.30.1591343125201; Fri, 05 Jun 2020 00:45:25 -0700 (PDT) Received: from laptop.hsd1.wa.comcast.net ([2601:600:817f:a132:df3e:521d:99d5:710d]) by smtp.gmail.com with ESMTPSA id 145sm6562544pfa.53.2020.06.05.00.45.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2020 00:45:24 -0700 (PDT) From: Andrei Vagin To: Catalin Marinas , Will Deacon , Dmitry Safonov Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Vincenzo Frascino , Mark Rutland , Thomas Gleixner , Andrei Vagin Subject: [PATCH 3/6 v4] arm64/vdso: Add time namespace page Date: Fri, 5 Jun 2020 00:45:19 -0700 Message-Id: <20200605074519.354579-1-avagin@gmail.com> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20200602180259.76361-4-avagin@gmail.com> References: <20200602180259.76361-4-avagin@gmail.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Allocate the time namespace page among VVAR pages. Provide __arch_get_timens_vdso_data() helper for VDSO code to get the code-relative position of VVARs on that special page. If a task belongs to a time namespace then the VVAR page which contains the system wide VDSO data is replaced with a namespace specific page which has the same layout as the VVAR page. That page has vdso_data->seq set to 1 to enforce the slow path and vdso_data->clock_mode set to VCLOCK_TIMENS to enforce the time namespace handling path. The extra check in the case that vdso_data->seq is odd, e.g. a concurrent update of the VDSO data is in progress, is not really affecting regular tasks which are not part of a time namespace as the task is spin waiting for the update to finish and vdso_data->seq to become even again. If a time namespace task hits that code path, it invokes the corresponding time getter function which retrieves the real VVAR page, reads host time and then adds the offset for the requested clock which is stored in the special VVAR page. Cc: Mark Rutland Reviewed-by: Vincenzo Frascino Reviewed-by: Dmitry Safonov Signed-off-by: Andrei Vagin --- v4: - fix an issue reported by the lkp robot. - vvar has the same size with/without CONFIG_TIME_NAMESPACE, but the timens page isn't allocated on !CONFIG_TIME_NAMESPACE. This simplifies criu/vdso migration between different kernel configs. arch/arm64/include/asm/vdso.h | 2 ++ .../include/asm/vdso/compat_gettimeofday.h | 12 +++++++++++ arch/arm64/include/asm/vdso/gettimeofday.h | 8 ++++++++ arch/arm64/kernel/vdso.c | 20 ++++++++++++++++--- arch/arm64/kernel/vdso/vdso.lds.S | 5 ++++- arch/arm64/kernel/vdso32/vdso.lds.S | 5 ++++- include/vdso/datapage.h | 1 + 7 files changed, 48 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h index 07468428fd29..f99dcb94b438 100644 --- a/arch/arm64/include/asm/vdso.h +++ b/arch/arm64/include/asm/vdso.h @@ -12,6 +12,8 @@ */ #define VDSO_LBASE 0x0 +#define __VVAR_PAGES 2 + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h index b6907ae78e53..b7c549d46d18 100644 --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h @@ -152,6 +152,18 @@ static __always_inline const struct vdso_data *__arch_get_vdso_data(void) return ret; } +#ifdef CONFIG_TIME_NS +static __always_inline const struct vdso_data *__arch_get_timens_vdso_data(void) +{ + const struct vdso_data *ret; + + /* See __arch_get_vdso_data(). */ + asm volatile("mov %0, %1" : "=r"(ret) : "r"(_timens_data)); + + return ret; +} +#endif + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_VDSO_GETTIMEOFDAY_H */ diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h index afba6ba332f8..cf39eae5eaaf 100644 --- a/arch/arm64/include/asm/vdso/gettimeofday.h +++ b/arch/arm64/include/asm/vdso/gettimeofday.h @@ -96,6 +96,14 @@ const struct vdso_data *__arch_get_vdso_data(void) return _vdso_data; } +#ifdef CONFIG_TIME_NS +static __always_inline +const struct vdso_data *__arch_get_timens_vdso_data(void) +{ + return _timens_data; +} +#endif + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_VDSO_GETTIMEOFDAY_H */ diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index 33df3cdf7982..fd609120386d 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -46,6 +46,12 @@ enum arch_vdso_type { #define VDSO_TYPES (ARM64_VDSO + 1) #endif /* CONFIG_COMPAT_VDSO */ +enum vvar_pages { + VVAR_DATA_PAGE_OFFSET, + VVAR_TIMENS_PAGE_OFFSET, + VVAR_NR_PAGES, +}; + struct __vdso_abi { const char *name; const char *vdso_code_start; @@ -81,6 +87,7 @@ static union { } vdso_data_store __page_aligned_data; struct vdso_data *vdso_data = vdso_data_store.data; + static int __vdso_remap(enum arch_vdso_type arch_index, const struct vm_special_mapping *sm, struct vm_area_struct *new_vma) @@ -132,6 +139,11 @@ static int __vdso_init(enum arch_vdso_type arch_index) } #ifdef CONFIG_TIME_NS +struct vdso_data *arch_get_vdso_data(void *vvar_page) +{ + return (struct vdso_data *)(vvar_page); +} + /* * The vvar page layout depends on whether a task belongs to the root or * non-root time namespace. Whenever a task changes its namespace, the VVAR @@ -180,9 +192,11 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index, unsigned long vdso_base, vdso_text_len, vdso_mapping_len; void *ret; + BUILD_BUG_ON(VVAR_NR_PAGES != __VVAR_PAGES); + vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT; /* Be sure to map the data page */ - vdso_mapping_len = vdso_text_len + PAGE_SIZE; + vdso_mapping_len = vdso_text_len + VVAR_NR_PAGES * PAGE_SIZE; vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0); if (IS_ERR_VALUE(vdso_base)) { @@ -190,13 +204,13 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index, goto up_fail; } - ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE, + ret = _install_special_mapping(mm, vdso_base, VVAR_NR_PAGES * PAGE_SIZE, VM_READ|VM_MAYREAD|VM_PFNMAP, vdso_lookup[arch_index].dm); if (IS_ERR(ret)) goto up_fail; - vdso_base += PAGE_SIZE; + vdso_base += VVAR_NR_PAGES * PAGE_SIZE; mm->context.vdso = (void *)vdso_base; ret = _install_special_mapping(mm, vdso_base, vdso_text_len, VM_READ|VM_EXEC| diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S index 7ad2d3a0cd48..d808ad31e01f 100644 --- a/arch/arm64/kernel/vdso/vdso.lds.S +++ b/arch/arm64/kernel/vdso/vdso.lds.S @@ -17,7 +17,10 @@ OUTPUT_ARCH(aarch64) SECTIONS { - PROVIDE(_vdso_data = . - PAGE_SIZE); + PROVIDE(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE); +#ifdef CONFIG_TIME_NS + PROVIDE(_timens_data = _vdso_data + PAGE_SIZE); +#endif . = VDSO_LBASE + SIZEOF_HEADERS; .hash : { *(.hash) } :text diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S index a3944927eaeb..06cc60a9630f 100644 --- a/arch/arm64/kernel/vdso32/vdso.lds.S +++ b/arch/arm64/kernel/vdso32/vdso.lds.S @@ -17,7 +17,10 @@ OUTPUT_ARCH(arm) SECTIONS { - PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE); + PROVIDE_HIDDEN(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE); +#ifdef CONFIG_TIME_NS + PROVIDE_HIDDEN(_timens_data = _vdso_data + PAGE_SIZE); +#endif . = VDSO_LBASE + SIZEOF_HEADERS; .hash : { *(.hash) } :text diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h index 7955c56d6b3c..ee810cae4e1e 100644 --- a/include/vdso/datapage.h +++ b/include/vdso/datapage.h @@ -109,6 +109,7 @@ struct vdso_data { * relocation, and this is what we need. */ extern struct vdso_data _vdso_data[CS_BASES] __attribute__((visibility("hidden"))); +extern struct vdso_data _timens_data[CS_BASES] __attribute__((visibility("hidden"))); /* * The generic vDSO implementation requires that gettimeofday.h -- 2.24.1