* [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) @ 2023-08-29 9:39 syzbot 2023-08-29 18:29 ` Yonghong Song 0 siblings, 1 reply; 6+ messages in thread From: syzbot @ 2023-08-29 9:39 UTC (permalink / raw) To: andrii, ast, bpf, daniel, haoluo, john.fastabend, jolsa, kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs Hello, syzbot found the following issue on: HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000 kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25 dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 Unfortunately, I don't have any reproducer for this issue yet. Downloadable assets: disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz IMPORTANT: if you fix the issue, please add the following tag to the commit: Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com ================================================================== BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: bpf_long_memcpy include/linux/bpf.h:428 [inline] bpf_obj_memcpy include/linux/bpf.h:441 [inline] copy_map_value_long include/linux/bpf.h:464 [inline] bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 __sys_bpf+0x28a/0x780 __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: bpf_long_memcpy include/linux/bpf.h:428 [inline] bpf_obj_memcpy include/linux/bpf.h:441 [inline] copy_map_value_long include/linux/bpf.h:464 [inline] bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 __sys_bpf+0x28a/0x780 __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd value changed: 0x0000000000000000 -> 0xfffffff000002788 Reported by Kernel Concurrency Sanitizer on: CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023 ================================================================== --- This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller@googlegroups.com. syzbot will keep track of this issue. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. If the bug is already fixed, let syzbot know by replying with: #syz fix: exact-commit-title If you want to overwrite bug's subsystems, reply with: #syz set subsystems: new-subsystem (See the list of subsystem names on the web dashboard) If the bug is a duplicate of another bug, reply with: #syz dup: exact-subject-of-another-report If you want to undo deduplication, reply with: #syz undup ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) 2023-08-29 9:39 [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) syzbot @ 2023-08-29 18:29 ` Yonghong Song 2023-08-29 18:53 ` Marco Elver 0 siblings, 1 reply; 6+ messages in thread From: Yonghong Song @ 2023-08-29 18:29 UTC (permalink / raw) To: syzbot, andrii, ast, bpf, daniel, haoluo, john.fastabend, jolsa, kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs On 8/29/23 5:39 AM, syzbot wrote: > Hello, > > syzbot found the following issue on: > > HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k.. > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000 > kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25 > dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 > > Unfortunately, I don't have any reproducer for this issue yet. > > Downloadable assets: > disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz > vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz > kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com > > ================================================================== > BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update > > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: > bpf_long_memcpy include/linux/bpf.h:428 [inline] > bpf_obj_memcpy include/linux/bpf.h:441 [inline] > copy_map_value_long include/linux/bpf.h:464 [inline] > bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 > bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 > generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 > bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 > __sys_bpf+0x28a/0x780 > __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] > __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] > __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: > bpf_long_memcpy include/linux/bpf.h:428 [inline] > bpf_obj_memcpy include/linux/bpf.h:441 [inline] > copy_map_value_long include/linux/bpf.h:464 [inline] > bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 > bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 > generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 > bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 > __sys_bpf+0x28a/0x780 > __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] > __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] > __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > value changed: 0x0000000000000000 -> 0xfffffff000002788 > > Reported by Kernel Concurrency Sanitizer on: > CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023 > ================================================================== This case is with two tasks doing bpf_map batch update together for the same map and key. > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: So concurrency is introduced by user applications. In my opinion, this probably not an issue from kernel perspective. > > > --- > This report is generated by a bot. It may contain errors. > See https://goo.gl/tpsmEJ for more information about syzbot. > syzbot engineers can be reached at syzkaller@googlegroups.com. > > syzbot will keep track of this issue. See: > https://goo.gl/tpsmEJ#status for how to communicate with syzbot. > > If the bug is already fixed, let syzbot know by replying with: > #syz fix: exact-commit-title > > If you want to overwrite bug's subsystems, reply with: > #syz set subsystems: new-subsystem > (See the list of subsystem names on the web dashboard) > > If the bug is a duplicate of another bug, reply with: > #syz dup: exact-subject-of-another-report > > If you want to undo deduplication, reply with: > #syz undup > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) 2023-08-29 18:29 ` Yonghong Song @ 2023-08-29 18:53 ` Marco Elver 2023-08-29 20:04 ` Daniel Borkmann 0 siblings, 1 reply; 6+ messages in thread From: Marco Elver @ 2023-08-29 18:53 UTC (permalink / raw) To: yonghong.song Cc: syzbot, andrii, ast, bpf, daniel, haoluo, john.fastabend, jolsa, kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs, Paul E. McKenney On Tue, 29 Aug 2023 at 20:30, Yonghong Song <yonghong.song@linux.dev> wrote: > > > > On 8/29/23 5:39 AM, syzbot wrote: > > Hello, > > > > syzbot found the following issue on: > > > > HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k.. > > git tree: upstream > > console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000 > > kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25 > > dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f > > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 > > > > Unfortunately, I don't have any reproducer for this issue yet. > > > > Downloadable assets: > > disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz > > vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz > > kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz > > > > IMPORTANT: if you fix the issue, please add the following tag to the commit: > > Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com > > > > ================================================================== > > BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update > > > > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: > > bpf_long_memcpy include/linux/bpf.h:428 [inline] > > bpf_obj_memcpy include/linux/bpf.h:441 [inline] > > copy_map_value_long include/linux/bpf.h:464 [inline] > > bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 > > bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 > > generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 > > bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 > > __sys_bpf+0x28a/0x780 > > __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] > > __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] > > __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 > > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > > > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: > > bpf_long_memcpy include/linux/bpf.h:428 [inline] > > bpf_obj_memcpy include/linux/bpf.h:441 [inline] > > copy_map_value_long include/linux/bpf.h:464 [inline] > > bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 > > bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 > > generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 > > bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 > > __sys_bpf+0x28a/0x780 > > __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] > > __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] > > __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 > > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > > > value changed: 0x0000000000000000 -> 0xfffffff000002788 > > > > Reported by Kernel Concurrency Sanitizer on: > > CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0 > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023 > > ================================================================== > > This case is with two tasks doing bpf_map batch update together for the > same map and key. > > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: > > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: > > So concurrency is introduced by user applications. > In my opinion, this probably not an issue from kernel perspective. Perhaps not, but I recall there being a discussion about making KCSAN aware of memory accesses done by BPF programs (memcpy being a tiny subset of those). Not sure if the above data race qualifies as something we might want to still detect, i.e. a kernel dev testing their kernel might be interested in such a report. Regardless, in this case we should teach syzkaller to ignore KCSAN data races that originate from bpf user operations whatever the origin. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) 2023-08-29 18:53 ` Marco Elver @ 2023-08-29 20:04 ` Daniel Borkmann 2023-08-29 20:36 ` Marco Elver 0 siblings, 1 reply; 6+ messages in thread From: Daniel Borkmann @ 2023-08-29 20:04 UTC (permalink / raw) To: Marco Elver, yonghong.song Cc: syzbot, andrii, ast, bpf, haoluo, john.fastabend, jolsa, kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs, Paul E. McKenney On 8/29/23 8:53 PM, Marco Elver wrote: > On Tue, 29 Aug 2023 at 20:30, Yonghong Song <yonghong.song@linux.dev> wrote: >> On 8/29/23 5:39 AM, syzbot wrote: >>> Hello, >>> >>> syzbot found the following issue on: >>> >>> HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k.. >>> git tree: upstream >>> console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000 >>> kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25 >>> dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f >>> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 >>> >>> Unfortunately, I don't have any reproducer for this issue yet. >>> >>> Downloadable assets: >>> disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz >>> vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz >>> kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz >>> >>> IMPORTANT: if you fix the issue, please add the following tag to the commit: >>> Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com >>> >>> ================================================================== >>> BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update >>> >>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: >>> bpf_long_memcpy include/linux/bpf.h:428 [inline] >>> bpf_obj_memcpy include/linux/bpf.h:441 [inline] >>> copy_map_value_long include/linux/bpf.h:464 [inline] >>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 >>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 >>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 >>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 >>> __sys_bpf+0x28a/0x780 >>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] >>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] >>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 >>> do_syscall_x64 arch/x86/entry/common.c:50 [inline] >>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 >>> entry_SYSCALL_64_after_hwframe+0x63/0xcd >>> >>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: >>> bpf_long_memcpy include/linux/bpf.h:428 [inline] >>> bpf_obj_memcpy include/linux/bpf.h:441 [inline] >>> copy_map_value_long include/linux/bpf.h:464 [inline] >>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 >>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 >>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 >>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 >>> __sys_bpf+0x28a/0x780 >>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] >>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] >>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 >>> do_syscall_x64 arch/x86/entry/common.c:50 [inline] >>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 >>> entry_SYSCALL_64_after_hwframe+0x63/0xcd >>> >>> value changed: 0x0000000000000000 -> 0xfffffff000002788 >>> >>> Reported by Kernel Concurrency Sanitizer on: >>> CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0 >>> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023 >>> ================================================================== >> >> This case is with two tasks doing bpf_map batch update together for the >> same map and key. >> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: >> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: >> >> So concurrency is introduced by user applications. >> In my opinion, this probably not an issue from kernel perspective. > > Perhaps not, but I recall there being a discussion about making KCSAN > aware of memory accesses done by BPF programs (memcpy being a tiny > subset of those). Not sure if the above data race qualifies as > something we might want to still detect, i.e. a kernel dev testing > their kernel might be interested in such a report. > > Regardless, in this case we should teach syzkaller to ignore KCSAN > data races that originate from bpf user operations whatever the > origin. I presume KCSAN could be silenced here via READ_ONCE/WRITE_ONCE conversion? diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f58895830ada..32c4a37045f2 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -424,8 +424,11 @@ static inline void bpf_long_memcpy(void *dst, const void *src, u32 size) long *ldst = dst; size /= sizeof(long); - while (size--) - *ldst++ = *lsrc++; + while (size--) { + WRITE_ONCE(*ldst, READ_ONCE(*lsrc)); + ldst++; + lsrc++; + } } /* copy everything but bpf_spin_lock, bpf_timer, and kptrs. There could be one of each. */ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) 2023-08-29 20:04 ` Daniel Borkmann @ 2023-08-29 20:36 ` Marco Elver 2023-08-29 20:54 ` Daniel Borkmann 0 siblings, 1 reply; 6+ messages in thread From: Marco Elver @ 2023-08-29 20:36 UTC (permalink / raw) To: Daniel Borkmann Cc: yonghong.song, syzbot, andrii, ast, bpf, haoluo, john.fastabend, jolsa, kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs, Paul E. McKenney On Tue, 29 Aug 2023 at 22:04, Daniel Borkmann <daniel@iogearbox.net> wrote: > > On 8/29/23 8:53 PM, Marco Elver wrote: > > On Tue, 29 Aug 2023 at 20:30, Yonghong Song <yonghong.song@linux.dev> wrote: > >> On 8/29/23 5:39 AM, syzbot wrote: > >>> Hello, > >>> > >>> syzbot found the following issue on: > >>> > >>> HEAD commit: 727dbda16b83 Merge tag 'hardening-v6.6-rc1' of git://git.k.. > >>> git tree: upstream > >>> console output: https://syzkaller.appspot.com/x/log.txt?x=136f39dfa80000 > >>> kernel config: https://syzkaller.appspot.com/x/.config?x=dea9c2ce3f646a25 > >>> dashboard link: https://syzkaller.appspot.com/bug?extid=97522333291430dd277f > >>> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 > >>> > >>> Unfortunately, I don't have any reproducer for this issue yet. > >>> > >>> Downloadable assets: > >>> disk image: https://storage.googleapis.com/syzbot-assets/9923a023ab11/disk-727dbda1.raw.xz > >>> vmlinux: https://storage.googleapis.com/syzbot-assets/650dbc695d77/vmlinux-727dbda1.xz > >>> kernel image: https://storage.googleapis.com/syzbot-assets/361da71276bf/bzImage-727dbda1.xz > >>> > >>> IMPORTANT: if you fix the issue, please add the following tag to the commit: > >>> Reported-by: syzbot+97522333291430dd277f@syzkaller.appspotmail.com > >>> > >>> ================================================================== > >>> BUG: KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update > >>> > >>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: > >>> bpf_long_memcpy include/linux/bpf.h:428 [inline] > >>> bpf_obj_memcpy include/linux/bpf.h:441 [inline] > >>> copy_map_value_long include/linux/bpf.h:464 [inline] > >>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 > >>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 > >>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 > >>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 > >>> __sys_bpf+0x28a/0x780 > >>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] > >>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] > >>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 > >>> do_syscall_x64 arch/x86/entry/common.c:50 [inline] > >>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > >>> entry_SYSCALL_64_after_hwframe+0x63/0xcd > >>> > >>> write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: > >>> bpf_long_memcpy include/linux/bpf.h:428 [inline] > >>> bpf_obj_memcpy include/linux/bpf.h:441 [inline] > >>> copy_map_value_long include/linux/bpf.h:464 [inline] > >>> bpf_percpu_array_update+0x3bb/0x500 kernel/bpf/arraymap.c:380 > >>> bpf_map_update_value+0x190/0x370 kernel/bpf/syscall.c:175 > >>> generic_map_update_batch+0x3ae/0x4f0 kernel/bpf/syscall.c:1749 > >>> bpf_map_do_batch+0x2df/0x3d0 kernel/bpf/syscall.c:4648 > >>> __sys_bpf+0x28a/0x780 > >>> __do_sys_bpf kernel/bpf/syscall.c:5241 [inline] > >>> __se_sys_bpf kernel/bpf/syscall.c:5239 [inline] > >>> __x64_sys_bpf+0x43/0x50 kernel/bpf/syscall.c:5239 > >>> do_syscall_x64 arch/x86/entry/common.c:50 [inline] > >>> do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > >>> entry_SYSCALL_64_after_hwframe+0x63/0xcd > >>> > >>> value changed: 0x0000000000000000 -> 0xfffffff000002788 > >>> > >>> Reported by Kernel Concurrency Sanitizer on: > >>> CPU: 0 PID: 8268 Comm: syz-executor.4 Not tainted 6.5.0-syzkaller-00453-g727dbda16b83 #0 > >>> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023 > >>> ================================================================== > >> > >> This case is with two tasks doing bpf_map batch update together for the > >> same map and key. > >> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8257 on cpu 1: > >> > write to 0xffffe8fffe7425d8 of 8 bytes by task 8268 on cpu 0: > >> > >> So concurrency is introduced by user applications. > >> In my opinion, this probably not an issue from kernel perspective. > > > > Perhaps not, but I recall there being a discussion about making KCSAN > > aware of memory accesses done by BPF programs (memcpy being a tiny > > subset of those). Not sure if the above data race qualifies as > > something we might want to still detect, i.e. a kernel dev testing > > their kernel might be interested in such a report. > > > > Regardless, in this case we should teach syzkaller to ignore KCSAN > > data races that originate from bpf user operations whatever the > > origin. > > I presume KCSAN could be silenced here via READ_ONCE/WRITE_ONCE conversion? > > diff --git a/include/linux/bpf.h b/include/linux/bpf.h > index f58895830ada..32c4a37045f2 100644 > --- a/include/linux/bpf.h > +++ b/include/linux/bpf.h > @@ -424,8 +424,11 @@ static inline void bpf_long_memcpy(void *dst, const void *src, u32 size) > long *ldst = dst; > > size /= sizeof(long); > - while (size--) > - *ldst++ = *lsrc++; > + while (size--) { > + WRITE_ONCE(*ldst, READ_ONCE(*lsrc)); > + ldst++; > + lsrc++; > + } In this case, maybe data_race(*ldst++ = *lsrc++) would be more appropriate and efficient. Unlikely that READ_ONCE()/WRITE_ONCE() here helps make this any safer, i.e. the memcpy is still not atomic and if it's a bug on the user's side, it'll corrupt data either way. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) 2023-08-29 20:36 ` Marco Elver @ 2023-08-29 20:54 ` Daniel Borkmann 0 siblings, 0 replies; 6+ messages in thread From: Daniel Borkmann @ 2023-08-29 20:54 UTC (permalink / raw) To: Marco Elver Cc: yonghong.song, syzbot, andrii, ast, bpf, haoluo, john.fastabend, jolsa, kpsingh, linux-kernel, martin.lau, sdf, song, syzkaller-bugs, yhs, Paul E. McKenney On 8/29/23 10:36 PM, Marco Elver wrote: [...] > In this case, maybe data_race(*ldst++ = *lsrc++) would be more > appropriate and efficient. Unlikely that READ_ONCE()/WRITE_ONCE() here > helps make this any safer, i.e. the memcpy is still not atomic and if > it's a bug on the user's side, it'll corrupt data either way. I wasn't aware of data_race(), thanks for the suggestion! Just flushed out a small patch. Thanks, Daniel ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-08-29 20:55 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-08-29 9:39 [syzbot] [bpf?] KCSAN: data-race in bpf_percpu_array_update / bpf_percpu_array_update (2) syzbot 2023-08-29 18:29 ` Yonghong Song 2023-08-29 18:53 ` Marco Elver 2023-08-29 20:04 ` Daniel Borkmann 2023-08-29 20:36 ` Marco Elver 2023-08-29 20:54 ` Daniel Borkmann
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®