Re: Potential data race in flush_to_ldisc

[Peter Hurley <peter@hurleysoftware.com>]

On 08/28/2015 12:57 PM, Dmitry Vyukov wrote:
> Hello,
> 
> We are working on a dynamic data race detector for the Linux kernel,
> KernelThreadSanitizer (ktsan):
> https://github.com/google/ktsan/wiki
> 
> While booting kernel (upstream revision 21bdb584af8c) we got a report:
> 
> ThreadSanitizer: data-race in release_tty
> 
> Write of size 8 by thread T325 (K2579):
>  [<ffffffff81655c43>] release_tty+0xf3/0x1c0 drivers/tty/tty_io.c:1688
>  [<ffffffff816563a8>] tty_release+0x698/0x7c0 drivers/tty/tty_io.c:1920
>  [<ffffffff8126154f>] __fput+0x15f/0x310 fs/file_table.c:207
>  [<ffffffff8126176d>] ____fput+0x1d/0x30 fs/file_table.c:243
>  [<ffffffff810b9485>] task_work_run+0x115/0x130 kernel/task_work.c:123
> (discriminator 1)
>  [<     inlined    >] do_notify_resume+0x73/0x80
> tracehook_notify_resume include/linux/tracehook.h:190
>  [<ffffffff81006da3>] do_notify_resume+0x73/0x80 arch/x86/kernel/signal.c:757
>  [<ffffffff81ee25fc>] int_signal+0x12/0x17 arch/x86/entry/entry_64.S:326
> 
> Previous read of size 8 by thread T19 (K16):
>  [<ffffffff816624d9>] flush_to_ldisc+0x29/0x300 drivers/tty/tty_buffer.c:472
>  [<ffffffff810b1fce>] process_one_work+0x47e/0x930 kernel/workqueue.c:2036
>  [<ffffffff810b2530>] worker_thread+0xb0/0x900 kernel/workqueue.c:2170
>  [<ffffffff810bbbd0>] kthread+0x150/0x170 kernel/kthread.c:207
>  [<ffffffff81ee281f>] ret_from_fork+0x3f/0x70 arch/x86/entry/entry_64.S:526
> 
> 
> flush_to_ldisc accesses port->itty:
> 
> static void flush_to_ldisc(struct work_struct *work)
> {
>    ...
>     tty = port->itty;
>     if (tty == NULL)
>         return;
>     disc = tty_ldisc_ref(tty);
> 
> while release_tty concurrently sets itty to NULL:
> 
> static void release_tty(struct tty_struct *tty, int idx)
> {
>     ...
>     tty->port->itty = NULL;
>     if (tty->link)
>         tty->link->port->itty = NULL;
>     cancel_work_sync(&tty->port->buf.work);
>     tty_kref_put(tty->link);
>     tty_kref_put(tty);
> }
> 
> It seems that read of port->itty requires to be at least READ_ONCE,

Agree; it should be READ_ONCE.

> because otherwise flush_to_ldisc can check that itty is not NULL, then
> re-read it again and crash with NULL deref.
> I don't know what is ownership and locking story here. There can be
> larger issue here: either a lock is missing, or itty can be deleted
> under flush_to_ldisc feet.
> 
> Please confirm that this is real but. If so please fix it.

Not a race.

The cancel_work_sync() waits for flush_to_ldisc() to complete, if already
running. For example,

CPU 0                                   | CPU 1
                                        |
release_tty()                           | flush_to_ldisc()
                                        |   tty = port->itty;
                                        |   tty == NULL? no
                                        |   ...
  port->itty = NULL                     |
  cancel_work_sync()                    |
    sleep here since flush_to_ldisc()   |
         running on CPU1                |
                                        | worker ends
    woken   <===========================| wake waiters

If flush_to_ldisc() was scheduled but not yet running, it will be cancelled
and not run.

Also, if flush_to_ldisc() is scheduled from some other cpu after cancel_work_sync(),
flush_to_ldisc() is guaranteed to 'see' the NULL port->itty.

Regards,
Peter Hurley

PS - And what Greg said; analyzing what is and is not a race will rapidly
improve your kernel familiarity.