From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750815AbdALOPT (ORCPT ); Thu, 12 Jan 2017 09:15:19 -0500 Received: from mail-eopbgr20122.outbound.protection.outlook.com ([40.107.2.122]:12704 "EHLO EUR02-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750731AbdALOPM (ORCPT ); Thu, 12 Jan 2017 09:15:12 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=dsafonov@virtuozzo.com; Subject: Re: [PATCH 1/2] x86/mm: don't mmap() over 4GB with compat syscall To: Andy Lutomirski References: <20170111181730.12458-1-dsafonov@virtuozzo.com> <20170111181730.12458-2-dsafonov@virtuozzo.com> CC: "linux-kernel@vger.kernel.org" , Dmitry Safonov <0x7f454c46@gmail.com>, Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Andy Lutomirski , Borislav Petkov , X86 ML From: Dmitry Safonov Message-ID: <103b955a-9178-d9a8-89d6-e1501feee95b@virtuozzo.com> Date: Thu, 12 Jan 2017 17:11:28 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: AM4PR0701CA0002.eurprd07.prod.outlook.com (10.165.102.12) To AM5PR0801MB1731.eurprd08.prod.outlook.com (10.169.247.9) X-MS-Office365-Filtering-Correlation-Id: a872b6b6-f1d7-4e2f-7acd-08d43af55dc9 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:AM5PR0801MB1731; X-Microsoft-Exchange-Diagnostics: 1;AM5PR0801MB1731;3:fmN+oBoVnzfrlRHOzR5JsmVDvc6Aczn6ZYun/mezdQXXzG5pVfhjg0Cm2rbqOpFIojobDhVLTN0s0Ra+ilVzie3uXswN9M7DxS81zbk6c/HTRrWatN1AjiUxT4RcJcizaXB5NNrnVwdr3hVItJBUy3Yp2jWi5sB2e5zjaTDZk/7662k9vRpiMl6NCbV1tIXYAQ0mDez1RJX8uQbs8kynIdjTskONsiuEOC1oXNZRU2ysjcr01CyHLgkdc45fV/+E/LLo/ii6SLTryLWHHGGwJQ==;25:TvAKkhKimOzl9KcTg83FuzDg7KTzkFr7b04bWL3PGXutX3wdHQW4cxSOwLSeg/P/e6A/AbfrSwQdQndI5mKOlkhgXY2Qi0rK7rJqV86FFwu2Mapmfkmw6bOAp4IgzhQeyakC4bmT5M6B7M9eg3o3Sk2QgPcgPlvAJIkc9gdSVU/zKMDzbMftyf+nQiwAscEDcRTt3Gfe0izYzLXNlxA33u7RbY2Eit513PDOhTGaa3Y9ac2WvS8XPEpFBBTQM2kKOzIidy6RCvj05TnjbIxQKQVn0h8JLMW8rljbAfVsYgbWMaat+ZENfGmGBF+pKtPiJQg4LNE70DUf4JZodKI1be2vTNAwcnvkUu9MwbNCpiF8jiUl3iQoZO84qkay0xac1P0aLvwPLiMHL8p6y6ZoALaDyAj+xT5QoIMIUBPJhPI7150QqGo+6ZWDB8ooHAzhlODnYth/mNru8KlCo8/uCA== X-Microsoft-Exchange-Diagnostics: 1;AM5PR0801MB1731;31:keUbQXFQjSaZH529fsZAa1fnmjgDmV4G+EKB9B83pLry0tENOHLo00h/ftHrT2n25OByYllExZGUqDt6GvUYx5L3URNl8rAsR1aehmRwmS65uu0PwvjbunkLkkIijJfk1b77ILCrrMGafv0Plc+FHJltK4FWFf9Jz28XW7/3x5JJh4cppO6hCaaTXP/kOs9qT0s48fveDoifFZOpN6a3VWpV9i5c4w5yemTaKPO0ZvVwrR05mE/5Me5m3HOnT5ts;20:7W+vw9PzTJISy4lo+AqI2p+ENWWI4D1OSEgd70lVSSCQ7Y4o1KsVRse7kQdCnpxRdjMUG94jzE2XE0aaQC+U/8D732C8KhWqqntWX7GwCE65B2UYsEypIq9bGKSXCD+sZo57h6e8a+g+UxHi07arXDwfFnJZohBIxPXnpLzJmq8dMZj4RcX/g9PaHeDU6Ca6Kk8SeEqZnGxLUJ0FRQToBCTje+xtYB+EpWiVcwaT+765QuSQwq5e79sTFg7rJCSi X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(278428928389397); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123560025)(20161123555025)(20161123564025)(20161123558021)(20161123562025)(6072148);SRVR:AM5PR0801MB1731;BCL:0;PCL:0;RULEID:;SRVR:AM5PR0801MB1731; X-Microsoft-Exchange-Diagnostics: 1;AM5PR0801MB1731;4:TzrteQ+vqnWTwSPhkg98lUVdvaLUGDAlyk/EasvWjzDJN1Qp0muh2uLxMJ/Bp4W9j/HTnDeyyZd0vTZoya0GxigvR11zuNMXZxyEMVLNHHa+SVhqmfYPoaxq0S7igc44gZD9S8QI2vhk1wS5N/EsJtq7bqtjXF25c+zzIgz5zk4kUp6AkuVBFvpH7LJeFNAVfYb9WH5QjUKAXQVYp6du762U2TS7RAPgcQ89VGJZTeJDkllk4yctlrTOrxIHpvhl/MxhG4slhw48ph7pTP5nZoyW8t4pZX1WpGIJxlCA9+VKsMOj5Bf3C6ZWOYRZglD0rz9SM3jVY4AbaloOzjNzZ/o5JqB03+SKMhCosZqY6H8/Obbxg5Mv07ywB0nQVsKO9ZH6v0p5KagQv5XzneAoT2xZo6GzNFeGzixFhrw3mzqYUAKo8KwkA+YrSFZ1xQgk1mPVaK8XYVWNLRlTR14xduw/AUt4VmnyofgGXJnQRtdtgVIj8oKmKDRnD6QJvSiKf5AiNjQ49JytLHDfNlOgoenR5aIqX7/muYJsFXw1e4bFU7CDWzMkmT5BvPQkrwG6lY8GYj8MuS+xsfmRtml7D8JGN+8LeylTxU8Rw/iOgJxAbqiey8nOcu0B6ZivPGYiyaojYnWrm41l5Do8a6lV9w== X-Forefront-PRVS: 018577E36E X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(39450400003)(39830400002)(39410400002)(199003)(24454002)(189002)(377454003)(6116002)(81166006)(81156014)(6666003)(38730400001)(2950100002)(3846002)(6916009)(106356001)(105586002)(189998001)(77096006)(305945005)(54906002)(101416001)(7736002)(8676002)(4001350100001)(50466002)(76176999)(50986999)(54356999)(39060400001)(33646002)(5660300001)(23676002)(68736007)(42186005)(36756003)(2906002)(65826007)(83506001)(65956001)(229853002)(31686004)(92566002)(4326007)(6486002)(230700001)(47776003)(90366009)(86362001)(65806001)(110136003)(97736004)(64126003)(25786008)(66066001)(31696002);DIR:OUT;SFP:1102;SCL:1;SRVR:AM5PR0801MB1731;H:[172.16.25.13];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTTVQUjA4MDFNQjE3MzE7MjM6YlFLTHlFSW5yVWovVXQ1V29jaEluaVJL?= =?utf-8?B?Q3pxZm1ldWk3WERXU2MwZHkrVmV3cHdZblQ4MFQ5bk9MZi9XOFBpb3Q2N25Q?= =?utf-8?B?OE9ibDdlNGppcUtlU04zRXBGUzhGejF6OFA2elhsc0MyQkduMDNROGkweGN3?= =?utf-8?B?MVhXRnhLN0JNUnY4Mko1NmdVU2hnNXQ3Z05PejFOdXJQazFTUUlhSU95NlVZ?= =?utf-8?B?a0VneHdjWlVYTHpzaGxKYy96SFdvVkRFeDNKdEs4MXErLzI0eCtyVVorQWJt?= =?utf-8?B?bEcrNk1ZUEhkR0pKdjVSL0JyaStEU1o4cHF0SnFPVjU3TUNBZ3NueTRxSWVB?= =?utf-8?B?bWNQQ0tzODFwV3V1M2tnLzBwQkI5VGVhOW9DYkFRRE0zSmVib3QvYjh6ZE9R?= =?utf-8?B?djZ5SjJKNWxRVmFESnVJUjBsT0paSThBWVNtWWRoK21nNW90T3RaUlZSY0RC?= =?utf-8?B?RWxzU01zODl1bjBNSnlnQVBOMmJZKzJ6bkZabkhJVHlKZWIxam42dEhRZ3Rt?= =?utf-8?B?MCtONytJOEdJemt6NzJZNUcyWVRUaGFrWWNzbnA3NzJZdExhRVRWeWxJdlpP?= =?utf-8?B?UTlNR21wZFZRMHBNd2hDL1VHekdNRlFTU1UrY0JOL1YrZmVNUmw5SVFoR3JW?= =?utf-8?B?cjBOemZDemx1S3NHaEw2UGRqN2FuNkRSbWxMUnNETHkwc2dCSFQxTExrYXdO?= =?utf-8?B?M1IxblczMjVQVW5TMVU1M1oxS25kOW42MXZyS0plUGM0SlRXbU5kL2ZTeHMx?= =?utf-8?B?aitzRXpWTFEvQU9EU2RvbzRhMGt0aFFaYVNRUVhUVzFzSG5OcGFyRE1zMHVq?= =?utf-8?B?TUkrNUR0dHVOZ3AwZDVQeDF3S0grL2h4cDM2ZlNyci9jOGFVWUtFMUU4c3Bx?= =?utf-8?B?QzI1VU9xLzFBcXlLSkhVaTU4YVZDcGdHVlRDOXVGeXozVE1FRlJvaEJoNWQz?= =?utf-8?B?ODVYcUlaajk1TGRkelA0aFJlaG9mNTl2eUlpbHVIK2hUVlFGb2FkRkt0bVEv?= =?utf-8?B?bFdMVEJqdmVwWndPeUhlSlZDOFA3R3JscmNwb043QzB0ZkFHVHZheHZFbDdr?= =?utf-8?B?TUtmMFhMdjVIV2pZRW1BY0lsN283enQ4cWRUdWNWSTdvT2g4MmYyS0pYZjV2?= =?utf-8?B?b1dEU1k2QjJ3WXdRbGtpd2FadDBRRjVMdU9QTVp1MU1BdDVtaXBmVXpkdkNE?= =?utf-8?B?UTFJVWtyWGZzZmtGK3R5Smtqb1ZCaVllaFphakduam94d3hWZGJ2RnFFR1Fr?= =?utf-8?B?ckVnbnZoYURUeWJiRHAvRjhUdFpMZWZkWkJPNkczM05xZjJLZlN6MXdZbzF0?= =?utf-8?B?bVAzYy9XcXhqQ3VCUlROQUdHNDZ0c3ptN2tGcmJEN3V4OUJFWnh0YTBveGdS?= =?utf-8?B?OXpwdVFtY0VRQ2o4ZnFaejk1OGZHSHIwbWE4cnRaTWFiSS9maW9nM2tjZUZV?= =?utf-8?B?aDBOajlKRXNrUFkvZE1BUmhXK0Z5bVJLTWVhT3R5YlNhNW9ja1V3NVRsWFcy?= =?utf-8?B?VXNLL1FZWHkyVElsaFRvWU1WUDFGSFZ6MDIrN1RrK2dRSUdkS3h4SHcrMEhE?= =?utf-8?B?dVlLNm9oT0hRQzZUbWI3emFyTTVLaTZXdTNKblhLeFUzSEwzKzVKRXd5Vjhx?= =?utf-8?B?YnBEZUF0VGM3U25xQTlCS2xUT1NRSUxXQXhPTG1VaDhtckRpYWpQMjI1aUJo?= =?utf-8?B?VWIvSE9iVStnT25xc3lQYmtjaEhJWE5sbzIzOXdqMllPZk4wdis0d3FsTWlL?= =?utf-8?B?U05vSThBd0c0dkJkY1JOUFZYSlRndFI4QmlQQ0FKUjR6TjdRSzRmOG5ZaERL?= =?utf-8?B?bFZuL04wRktlTHYvMHhnUldTUUg5c0xpMy9POXNSNjh4b3IxenF2MEc5dnlB?= =?utf-8?B?enBZSHI2Q2lnRU1DcHBpY2V4S0FIK3NTQ2pLbjFzZXJhL2VGYUpnNDFiZ1hK?= =?utf-8?Q?ELMiY0vMRkBsVq/YGxD9qdhuTDCCw7Ow=3D?= X-Microsoft-Exchange-Diagnostics: 1;AM5PR0801MB1731;6:IiaU38qUbux/1VqxQpshMtGdk3MFxU/Q1Tu7qf78wGchgIZQoMOUrALwgFSPHCB/0ZXIVuxZegNIv85YhdR8rIGcQPeXF/o2yUR7Eh1jJ9sxtdQhpaP7Cycb4SJ9XcOzvaTt6o/P9/y9HxbuzeRaOxTvEDWE6Amnn2hrV9KFR9AHR3RPI27hY9BmA9owjyjJGnPy3PsfUim2ES5in9VOVmoPd4OOTXuTzGrNyLrG2FaKcHDakG26jryNuNvlkTmRYXk98/ABCeKm5CJ512Hskue+JDj8nvWgi77+9gl/IzSjOzQbo6oD2eGQ8Vl/saBgbbSQKWhQrh94yf6hNivKiXLumtf8OluHQlzsHQxhjgnX6ZD5DnxTvkPUWpDENdHUtSq+27PRLd60pQSVj0Wh86e0OHykalKOYDXhWiVnuHg=;5:Q3vc01M+De5YIgK4nvhwVHpGHJ6RB5jSQ+D37zwP/8S4JhG336EI3Lpp2sao3NFO6mtG06fRx4tj1QBAVAR+MZ39jypkZzyt6P8maOnponDWCxzTn5JPq5f5T9L8SCktrIqWEhVdlgRrY9YMcFDZ9w==;24:sDzqiI9JEg7RchhRCYzp8EhziLfqFoTkoDAmpULinBjb2r2ffrmGFa7H11HOAE6slRtvVyyghgOLOFH0shb9q6RNolPepS2k+ciGJvQb1JY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;AM5PR0801MB1731;7:kfGBIoRE7oi03uOlOSwIqN/xj4+lwas77dS/fcRUsmQ5LPPMrRaz9S6cV3hQXAWLyOO66JWU6YCrYz0N+mGFnbB6MyCc00iqDcEc/fixjH4quDQH1R5IB3pm04B7xfIj3tNGTKjGw/onBTAVZelCLx+dMjHjkRrC7gfDhKZtutCC0sm2CIqF/vFybkLJhFFaOsU3Q9/7dWST/U9f5dDxTtWuj+KksoNihUPxwWOrKVb2/UQHajnb/TMtS2c3kri0RzcVgJMHeGmDYwmIXUHtVbRkK2posZdZyK8io64nQw0d7Mu80Z6B25r2bmioA3wh1NfXr5+k1dB38a8AhqkM+QpwmqqFCytXWmdbbNxAC917WhDP7aoCFjwATioxa4mU/OvNw8Pqgoylcf/gt1xpVObBjHNz1AW+yGAnXemGLOvQs5w2yJ1YcnZaXFMSi8Rvu1j6aILJm0lnyX0WN+eO7A==;20:o/9c1djq6XEJuSfBiRIYdVyvTYzIxm8oExAx4kTp9bXiSfOb4EhYFqX+3zjA/ubk/fcIFkUGm51kgM8LFAzovIO259o/1uoMG9gfT/v3Be2tJJS/FDEAFEXgb5myK95jqWqU/m39/Lz//UdfXB6JJ0kviuxWoSG+RLuM0uoNeYk= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2017 14:14:47.5511 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1731 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/12/2017 01:26 AM, Andy Lutomirski wrote: > On Wed, Jan 11, 2017 at 10:17 AM, Dmitry Safonov wrote: >> During fixing CRIU bugs on ZDTM tests for 32-bit C/R, I found that >> compatible ia32/x32 syscalls mmap() and mmap2() can return address >> over 4Gb in x86_64 applications, which results in returning lower >> 4 bytes of address while dropping the higher bytes. >> It happens because mmap() upper limit doesn't differ native/compat >> syscalls for 64-bit task, it's: (TASK_UNMAPPED_BASE + random_factor) >> which is: (PAGE_ALIGN(TASK_SIZE / 3)) + random_factor >> (in case of legacy mmap it's just TASK_SIZE). >> This patch limits higher address that can be mmaped with compat >> syscalls in 64-bit applications with IA32_PAGE_OFFSET (+randomization). >> >> Signed-off-by: Dmitry Safonov >> --- >> arch/x86/kernel/sys_x86_64.c | 37 +++++++++++++++++++++++++++++-------- >> 1 file changed, 29 insertions(+), 8 deletions(-) >> >> diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c >> index a55ed63b9f91..0893725db6e6 100644 >> --- a/arch/x86/kernel/sys_x86_64.c >> +++ b/arch/x86/kernel/sys_x86_64.c >> @@ -100,7 +100,7 @@ SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len, >> static void find_start_end(unsigned long flags, unsigned long *begin, >> unsigned long *end) >> { >> - if (!test_thread_flag(TIF_ADDR32) && (flags & MAP_32BIT)) { >> + if (!test_thread_flag(TIF_ADDR32)) { >> /* This is usually used needed to map code in small >> model, so it needs to be in the first 31bit. Limit >> it to that. This means we need to move the >> @@ -109,14 +109,24 @@ static void find_start_end(unsigned long flags, unsigned long *begin, >> malloc knows how to fall back to mmap. Give it 1GB >> of playground for now. -AK */ >> *begin = 0x40000000; >> - *end = 0x80000000; >> - if (current->flags & PF_RANDOMIZE) { >> - *begin = randomize_page(*begin, 0x02000000); >> + >> + if (flags & MAP_32BIT) { >> + if (current->flags & PF_RANDOMIZE) >> + *begin = randomize_page(*begin, 0x02000000); >> + *end = 0x80000000; >> + return; >> + } >> + if (current->thread.status & TS_COMPAT) { >> + if (current->flags & PF_RANDOMIZE) >> + *begin = randomize_page(*begin, >> + 1UL << mmap_rnd_compat_bits); >> + *end = IA32_PAGE_OFFSET; >> + return; >> } >> - } else { >> - *begin = current->mm->mmap_legacy_base; >> - *end = TASK_SIZE; >> } >> + >> + *begin = current->mm->mmap_legacy_base; >> + *end = TASK_SIZE; >> } >> >> unsigned long >> @@ -187,10 +197,21 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, >> return addr; >> } >> >> + if (current->thread.status & TS_COMPAT) { > > in_compat_syscall(), please. > > Also, we need to verify that, if this is called execve(), it does the > right thing. > >> + if (current->flags & PF_RANDOMIZE) { >> + unsigned long rnd = 1UL << mmap_rnd_compat_bits; >> + >> + info.high_limit = >> + randomize_page(IA32_PAGE_OFFSET - rnd, rnd); >> + } else { >> + info.high_limit = IA32_PAGE_OFFSET; >> + } >> + } else { >> + info.high_limit = mm->mmap_base; >> + } > > This code was incomprehensible before and it's worse now. Could you > try to clean it up a bit? For example, a patch that simply folds > find_start_end() into its sole caller as the first patch in the series > without changing any semantics would probably help. So, I debugged it a little more. Here it is: we really should mmap here not just up to IA32_PAGE_OFFSET, but also think about stack of compat applications. Need to have a gap in the end of address space the same way it's counted in mmap_base(). And here is another bug (but less painful): for 32-bit applications, that set 64-bit CS and do native x86_64 syscalls, it's not possible to map over 4Gb (or around), 32-bit mmap_base. I think there are two ways to fix these issues: 1. Introduce mmap_compat_base (and mmap_compat_legacy_base, I guess). Initialize only one of mmap_{,compat}_base in arch_pick_mmap_layout() and initialize the second only if 64-bit app does 32-bit mmap and vice-versa. Use these two mmap bases accordingly. 2. Save random_factor in arch_pick_mmap_layout() and make mmap_base() calculations during mmap() using compat or native task_size according to in_compat_syscall() - the perf overhead in mmap_base() doesn't look that great (but I may be wrong). Need also be careful to RLIMIT_STACK and fall back to bottom-up allocations in case of infinity. So, does (1) or (2) make sense, or maybe there is some simpler solution to these bugs? -- Dmitry