From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754655AbcHARV3 (ORCPT ); Mon, 1 Aug 2016 13:21:29 -0400 Received: from mail-ve1eur01on0094.outbound.protection.outlook.com ([104.47.1.94]:53602 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754451AbcHARUy (ORCPT ); Mon, 1 Aug 2016 13:20:54 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aryabinin@virtuozzo.com; From: Andrey Ryabinin To: Andrew Morton CC: Alexander Potapenko , Dave Jones , Vegard Nossum , "Sasha Levin" , Dmitry Vyukov , , , , Andrey Ryabinin , , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim Subject: [PATCH 5/6] mm/kasan: get rid of ->state in struct kasan_alloc_meta Date: Mon, 1 Aug 2016 17:45:14 +0300 Message-ID: <1470062715-14077-5-git-send-email-aryabinin@virtuozzo.com> X-Mailer: git-send-email 2.7.3 In-Reply-To: <1470062715-14077-1-git-send-email-aryabinin@virtuozzo.com> References: <1470062715-14077-1-git-send-email-aryabinin@virtuozzo.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: AM3PR08CA0029.eurprd08.prod.outlook.com (10.160.207.167) To VI1PR0801MB1982.eurprd08.prod.outlook.com (10.173.74.15) X-MS-Office365-Filtering-Correlation-Id: 1bd80135-cda7-4270-b625-08d3ba1a4dbd X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1982;2:/oHetW2VO7x1lZHixtzlEp7lUFv8Bw74sfYWB5IiaFbYGAfbgYaarEgNQCI1c1e3+mGkCEC1g6OP/WRP78PV6DZ7kn8UdCICddSr86ervRjo3+za1j2Io+t0c9haw9fnLzqw32OPGHSkCpF/amxWO0CtZZZI70tLCz4zEHf8Crl3Vks4MZNkoRvVHzzsBrGe;3:UeB9k8q7sr30QjviyQLJEu4b9U9LjhAPxBkNCb5CqlJh/nMOwZVWqFfW5yWRUMgYD8Y5/o8ayfBBEIbyITsQqK7QeIVIQCrPQDhyYsDkCYcgLAeL5Y8i/VzWc8jQiYgn X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB1982; X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1982;25:0Eb3pQW7KVuzpwVdYkiX6tSXSMt8+WDIPm3Lmm8a8nI1eRAR9kelttZq1Hjci3445iRdqiFRHiheAwiihwWxLT8H35bElAFOeth2jVHpeU6/6DJwbDuPd0jYCvDtSEa6iyZnJAzpMmMABbRVkjS1qo6SHfw9DdXejDiUvGoadK1VMt9ln3GZ9nAQI9HO0egS3B5XwCWkEFiTphzujUtC15gBuyVSZ4M0rfWOaIMd9xAPDLBoeE8gWO9R/JiBJ1D/O3tLEtapmnUzyAJCCQSKUtos7bqFLaakVyM5L+gQDtN2L1ncGT49S5o15MnZRQtkkrz3imul2CBMj1cAA8ZSkb+ExW/Dzmay6jy1svRqq3naOIxWswforcAyCdwGhIWHHgPWlOJFKQ53XSI3Gig3WGQqMcr/8O6YmewkP6Fi9Q7/lNPJxnY5ttU3ihh+1UOMF8ieij5KOaBiNG2iJfmYe5q61dNp0/9t0kiNKGdLN8EEvh5lYOW1rABnFjyzOxNtU6eedjvt5ncugqKu6r/cmiojMduUe5bi+gTDhBVPdCHv5E7jEThzq9FCN81hlZoMTn+IMW0ZKAZ68xPCMfg55CRHHoUTWaNih9N5HfwBY3HSR8sMncF2H31Q5R29dFEYki4bNX3P4hpbILcCvAfw5+yHLsS4xAhl8XA/4aTMYM8dJ+QPekXYDRsXty1reTZ2IN5qu0oUPabStNWsk8jUli3RU5b1PinjMkr57OHmfkAFEVRuzok+LnkjCmT/wZzv X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1982;31:Fa7kbVWMwp//PvrulyG+VWXXB2kmM9EslIUory5AWtd9v0X2rbS9LNmRlSi7v52V71iWS14Vs/rXv3VMrH553+PmM7Q/IAxgUhS18u2aF9IleK/qgP2GYxzY+TSOInF8XCFIMaFfCXqHjdP/lNq4ArQRIApdwxQWMb8a/3yH21mByPQzANW5SCz7DCS+6K+CSGpzTS/Cg0KxpHmIPKHJHf7c2uAMbxqVAjgg1267Ekg=;4:snmnKl7AQ3P/cBd0z968G1BdHV7SdGuiOOevV10eaSlZbO/qJPuwUS6vRpfo812hksNrC3fIh3n6HmJvnJMcr4iDkGuisSE7OsdLv7A4P/bSeZlDsCuMuO7ayqTu/cvRw2Da2/F4j/sQMcjr4IiV/JU+GufkhD6tAeWyYNGFQPNpI/8VbKLqsMsaVi8FBr8P1dTcofRYQ/nr4c31BkKB5oFAmJnTuhpXZJZwzM6GbNWeESuahuuIAlPm4dIeD7LD/an4HZ+1y+yS+NCzsfFVDHVLLsMpyZt3jIgab990nFbHmKhG4eWaea0b1I1QmzXDqkMHQDSiQZ1v/yJ4wujvH++NMfPum4aTM7v5+RlWehmnziV5BHWuZR65KbXnQl/2b8EMjU/UBk30UtU0P6zZYqYpW/nf0J4V04xeKLmoPvFV0tRoF63jjYUplw7TPgYaLikumY2Hci8og6315xx6fpDMGVyVUmXnbeJZS6gHqe+EKsmVFLvGpCeRwVjtgY+I X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(211936372134217)(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040130)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6041072)(6043046);SRVR:VI1PR0801MB1982;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB1982; X-Forefront-PRVS: 0021920B5A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6069001)(7916002)(199003)(189002)(97736004)(92566002)(76176999)(36756003)(47776003)(8676002)(101416001)(81166006)(42186005)(81156014)(189998001)(3846002)(6116002)(5003940100001)(7416002)(4326007)(50466002)(110136002)(305945005)(48376002)(66066001)(586003)(69596002)(2906002)(77096005)(50226002)(86362001)(33646002)(106356001)(2950100001)(229853001)(68736007)(50986999)(19580405001)(19580395003)(53416004)(7846002)(7736002)(105586002)(76506005)(26583001);DIR:OUT;SFP:1102;SCL:1;SRVR:VI1PR0801MB1982;H:localhost.sw.ru;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;VI1PR0801MB1982;23:IMEeAjD+r3v52ahYvOXWoGacy3JlZKpUL9wWdje?= =?us-ascii?Q?emuEcbNdawPICd6bowC9MVqdBplH/CD1QE3kntou7a3HmsbuRdabA12wFdJt?= =?us-ascii?Q?Z4sUNkQcJcF6YeZk40maSDOnaWOvonVbLYdhlqdra/kGIYOKS+96B/LftKwq?= =?us-ascii?Q?TkbnEktq2j6dP2os5ro7RFHe1fHfMQMmi15XMxPxzeQqomnl+AU35HRNAI6+?= =?us-ascii?Q?xdM5ShmUOLmrW+pCUM3uT189800j5cVz6QpLMJAJkShU5aNJI4xI6Mvmg1mm?= =?us-ascii?Q?hoLVdOuS6M1fIt7/IbwsJyWTlYr2bYp4RPs8FxDapITYrVLxhgyNinbIODWT?= =?us-ascii?Q?MZzX/1AEj9iQZiHj1vd34016hGy/91qQl6RjTjj7QvWkR0F0BjwBMD8MaKol?= =?us-ascii?Q?6g5fu9IWCqxVlOv0ypHe8tTT0i0NpLQZaacT45EzG4pjf2kEQyjK5tT15TK1?= =?us-ascii?Q?tgdUbnvxcvdqSM9zoz5OThbZRm47xOExcRnKiv82lnqnNKIgR8CoZkhdoN8X?= =?us-ascii?Q?UixqQksYbgKIEGCuM5eZQb1K/ZeeRb/S7jzP+ztmc769G0DaMZzhVeOg28cC?= =?us-ascii?Q?M2tVzmqobYHQ44v8I8eRfELIFdeV/nTW1cy4gmXYmOYsFj0spknEJ3UFX3G5?= =?us-ascii?Q?sH/o6PfIMLq/0Mi9PHloY5YHq3ix883m1tWVoHaDHIL3CYti4iS79HKxHKfp?= =?us-ascii?Q?ZedX9kMDuJYpdYHTmGuwl1yKv6lfwsDU4RPrgkFa/0vIx5rEArsXImNkukSs?= =?us-ascii?Q?afCNNwmzv3yu7V0REtBlltXNbhoZlXSA+xeV7LhuWUbajeBYxJgdnue4vEx3?= =?us-ascii?Q?ZZXEI5I4/5ZSB9EESeLdPnQ8MnfF10/Y8IxwFCT9B5qS8D3kqmj6nHEu9h5O?= =?us-ascii?Q?1L20sW06nJayTlIorDkgn91Jg70BrGUgBfhi/4klm9lPdwnnYkBjaoTCxE6/?= =?us-ascii?Q?vhcGuIzK9gThW3204o9itQyHkCF6QovlM+62SvZqF5lu7JeQErh/eCJqy4Sv?= =?us-ascii?Q?89dYRxu1PNlPN1bekQbTklDq+CShRll7TEJR08RFOERg1TucT9XsD6gcgDLc?= =?us-ascii?Q?KdW+KNrMr0VBfNB37YbQlyB0toCG5FP9EznrqXxpoAl4HTE7P1fjBqvymxgY?= =?us-ascii?Q?AhJnPkh/r+11Yp2C0UaHuSB7gYHwYJG8YIj5BOx31R/0sjW6gEoGvPg=3D?= =?us-ascii?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1982;6:0+Jcni3EknrfiOuALNV0KD9dayAR5Vq18b05HTisdObyHRwZH9nn5wEktr27/x5SGDkzTvi0XMgLXnpN+Z7//0H8O4uAk/FGp5JgI6H0vKWb54WApPgi3FZb9JWylW/PFGY3KINKxXI5m4EW/LKTDOMXNu3QUJgBJNPMjwHYrmEDS2JaQ2IxARUtIDol432glEPQaOkUlQjQKtlEiEWjbsRf7usDIQEFrnIPyooTPh3qvKjwYD7zdO1+NvR3Bat5SlY0QsdMQSodH0nHyLSA9VzkT2tQJMOmHeJ7kdBxQNiIq+ckcH6OtvPeFDAczMkf;5:vGNxT+hof88n23AG+iYyz/C6ahz+JwGeUPSjaie+bbutntll+ToGi4vVHsp2GAUWOI6kz/7HCdAenk5fispDT+/lAKux593GR6Z8u9ShHpXor7O5djH8hZlnAYSACsappwYb9f/1mkwFDhBgArBp3g==;24:KbqxAVa7UbJH2ChMFvL/cF4FAcnwmkpcv5ZCm70DYwD11M7WTtZ0ug35tiM+BEB/yaN779fwhb/UWPGYzXUPRvrejpaAbW/ydokEJ0z2yHA=;7:tBK3NsgmhiYLrLyWF+/q8tCz8e/QaE8bRC+gjlnwcCoq2jMZsQQcXM+ozUDWHb8/40PBdasOaYqJYy4gmr3lXP9tCzx43KvCTTO5rIElMUPI1G8WoHkEGYNNYpgHDe+6XrUbOPNAh1BzvQtSDkhZq9RA2KUl8f6ZHXRPhGZxW1K8AZOs6LFhgcAYEVBY/eTSE+Y0oUbz59C98FdYTS5VaJt9p32R2AR2UbxT1fgEUWtwDVdCclogVwjEd4G0E8aE SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1982;20:9oJl+HLZVukwoqnR6DOknlWPDaV6luJb8ZMLKJJHzg4Vp6n8FkKxkHg+io1R9Cr1OekGAbtD+kECkyiBvBJLP8hiwQ8m+egbnQG57OrIcvanM0vHIXP5SXeJCvF/hRz5TB6h/KCG8/KJDrKiUFwFIrSIu539tv20QW7mJ88N56U= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2016 14:44:11.3326 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1982 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The state of object currently tracked in two places - shadow memory, and the ->state field in struct kasan_alloc_meta. We can get rid of the latter. The will save us a little bit of memory. Also, this allow us to move free stack into struct kasan_alloc_meta, without increasing memory consumption. So now we should always know when the last time the object was freed. This may be useful for long delayed use-after-free bugs. As a side effect this fixes following UBSAN warning: UBSAN: Undefined behaviour in mm/kasan/quarantine.c:102:13 member access within misaligned address ffff88000d1efebc for type 'struct qlist_node' which requires 8 byte alignment Reported-by: kernel test robot Signed-off-by: Andrey Ryabinin Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim --- include/linux/kasan.h | 3 +++ mm/kasan/kasan.c | 61 +++++++++++++++++++++++---------------------------- mm/kasan/kasan.h | 12 ++-------- mm/kasan/quarantine.c | 2 -- mm/kasan/report.c | 23 +++++-------------- mm/slab.c | 4 +++- mm/slub.c | 1 + 7 files changed, 42 insertions(+), 64 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index c9cf374..d600303 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -56,6 +56,7 @@ void kasan_cache_destroy(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); +void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(const void *ptr); @@ -102,6 +103,8 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) {} +static inline void kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) {} static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} static inline void kasan_kfree_large(const void *ptr) {} diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 388e812..92750e3 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -442,11 +442,6 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object) kasan_poison_shadow(object, round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) { - struct kasan_alloc_meta *alloc_info = - get_alloc_info(cache, object); - alloc_info->state = KASAN_STATE_INIT; - } } static inline int in_irqentry_text(unsigned long ptr) @@ -510,6 +505,17 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, return (void *)object + cache->kasan_info.free_meta_offset; } +void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); +} + void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { kasan_kmalloc(cache, object, cache->object_size, flags); @@ -529,34 +535,27 @@ static void kasan_poison_slab_free(struct kmem_cache *cache, void *object) bool kasan_slab_free(struct kmem_cache *cache, void *object) { + s8 shadow_byte; + /* RCU slabs could be legally used after free within the RCU period */ if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU)) return false; - if (likely(cache->flags & SLAB_KASAN)) { - struct kasan_alloc_meta *alloc_info; - struct kasan_free_meta *free_info; + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + pr_err("Double free"); + dump_stack(); + return true; + } - alloc_info = get_alloc_info(cache, object); - free_info = get_free_info(cache, object); + kasan_poison_slab_free(cache, object); - switch (alloc_info->state) { - case KASAN_STATE_ALLOC: - alloc_info->state = KASAN_STATE_QUARANTINE; - set_track(&free_info->track, GFP_NOWAIT); - kasan_poison_slab_free(cache, object); - quarantine_put(free_info, cache); - return true; - case KASAN_STATE_QUARANTINE: - case KASAN_STATE_FREE: - pr_err("Double free"); - dump_stack(); - break; - default: - break; - } - } - return false; + if (unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; } void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, @@ -579,13 +578,9 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, kasan_unpoison_shadow(object, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) { - struct kasan_alloc_meta *alloc_info = - get_alloc_info(cache, object); - alloc_info->state = KASAN_STATE_ALLOC; - set_track(&alloc_info->track, flags); - } + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); } EXPORT_SYMBOL(kasan_kmalloc); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index aa17546..9b7b31e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -59,13 +59,6 @@ struct kasan_global { * Structures to keep alloc and free tracks * */ -enum kasan_state { - KASAN_STATE_INIT, - KASAN_STATE_ALLOC, - KASAN_STATE_QUARANTINE, - KASAN_STATE_FREE -}; - #define KASAN_STACK_DEPTH 64 struct kasan_track { @@ -74,8 +67,8 @@ struct kasan_track { }; struct kasan_alloc_meta { - struct kasan_track track; - u32 state; + struct kasan_track alloc_track; + struct kasan_track free_track; }; struct qlist_node { @@ -86,7 +79,6 @@ struct kasan_free_meta { * Otherwise it might be used for the allocator freelist. */ struct qlist_node quarantine_link; - struct kasan_track track; }; struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 4852625..7fd121d 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -144,13 +144,11 @@ static void *qlink_to_object(struct qlist_node *qlink, struct kmem_cache *cache) static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache) { void *object = qlink_to_object(qlink, cache); - struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object); unsigned long flags; if (IS_ENABLED(CONFIG_SLAB)) local_irq_save(flags); - alloc_info->state = KASAN_STATE_FREE; ___cache_free(cache, object, _THIS_IP_); if (IS_ENABLED(CONFIG_SLAB)) diff --git a/mm/kasan/report.c b/mm/kasan/report.c index d67a7e0..f437398 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -133,7 +133,6 @@ static void kasan_object_err(struct kmem_cache *cache, struct page *page, void *object, char *unused_reason) { struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object); - struct kasan_free_meta *free_info; dump_stack(); pr_err("Object at %p, in cache %s size: %d\n", object, cache->name, @@ -141,23 +140,11 @@ static void kasan_object_err(struct kmem_cache *cache, struct page *page, if (!(cache->flags & SLAB_KASAN)) return; - switch (alloc_info->state) { - case KASAN_STATE_INIT: - pr_err("Object not allocated yet\n"); - break; - case KASAN_STATE_ALLOC: - pr_err("Allocation:\n"); - print_track(&alloc_info->track); - break; - case KASAN_STATE_FREE: - case KASAN_STATE_QUARANTINE: - free_info = get_free_info(cache, object); - pr_err("Allocation:\n"); - print_track(&alloc_info->track); - pr_err("Deallocation:\n"); - print_track(&free_info->track); - break; - } + + pr_err("Allocated:\n"); + print_track(&alloc_info->alloc_track); + pr_err("Freed:\n"); + print_track(&alloc_info->free_track); } static void print_address_description(struct kasan_access_info *info) diff --git a/mm/slab.c b/mm/slab.c index 09771ed..ca135bd 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2604,9 +2604,11 @@ static void cache_init_objs(struct kmem_cache *cachep, } for (i = 0; i < cachep->num; i++) { + objp = index_to_obj(cachep, page, i); + kasan_init_slab_obj(cachep, objp); + /* constructor could break poison info */ if (DEBUG == 0 && cachep->ctor) { - objp = index_to_obj(cachep, page, i); kasan_unpoison_object_data(cachep, objp); cachep->ctor(objp); kasan_poison_object_data(cachep, objp); diff --git a/mm/slub.c b/mm/slub.c index 74e7c8c..26eb6a99 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1384,6 +1384,7 @@ static void setup_object(struct kmem_cache *s, struct page *page, void *object) { setup_object_debug(s, page, object); + kasan_init_slab_obj(s, object); if (unlikely(s->ctor)) { kasan_unpoison_object_data(s, object); s->ctor(object); -- 2.7.3