From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Google-Smtp-Source: ACJfBosoPUt6O1ONyntjvMjDe0eD7IBV1fH4C1EtWGdhKHhbwiuY3u8dZtgt2pnVtGwnDosb17Kj ARC-Seal: i=1; a=rsa-sha256; t=1515044863; cv=none; d=google.com; s=arc-20160816; b=YuKMg9GxicQ2VV37D4Id9Xm0HqfMw3ilqG1E6uEZ+XXttg2yXc7y+ZEwoo3rgqc2Zf qaDimbMA6Blic198uMPqiPWqsu6l8K4K/gtYesE0tLi5obK+8rovD1p+cx50ZT38/GO1 blgpPb6EuXmgA9jnkZjSpokHjMmdBFBqLhV+dVaViBup+xek8Aks0Sl7dbGyTvyI7LRd sUqTS7vhqLDBip9LQYEIOeTxGxzzZufxgCYgK5upZixSkWlgplPu20c0LVZBXNYpvqke Zor+tinF3xJJjzj4x9YgF5WQGxqgZxd235c2jrSqxZ4Aj+RTqy2g/RF0hYuNnNhSH4sN vfZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dmarc-filter :dkim-signature:dkim-signature:arc-authentication-results; bh=lmxFOAH+SD3EsZCFApnpIJg81vyF7re74dxGxufwiEY=; b=UHcmRQ0pJSGMtQKM321L8XswhXEMKVz72Z2lUx7SrTvgJj8+NaJ+o9ctLBq23jqva8 O/f8uBD4PBlETOeW0Zj+SLlTu7Hqhe1iFKHgroIC5siDupsuegvJjjYbC6qunuzP9r31 BuJ8Inyr5q+POvkHsKNQEz83KAH408Fj1ZrHn+reu8QTYbdrf66+KpokugRfua9R94wA Da7zkVnnluOF+QA+foUxy6Ex+9CM+C2m/JwjnwU1qG5OUy095J3NfADd8gEDYO/cv9HI Dh/swrFB0QJBrzYd3P5PWPHccD7GBfWeCZRs0LixpMIbhY11f9mdxfQBtoQW992J7bDG v39g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@codeaurora.org header.s=default header.b=K0aL59wu; dkim=pass header.i=@codeaurora.org header.s=default header.b=RuSlDXHI; spf=pass (google.com: domain of gkohli@codeaurora.org designates 198.145.29.96 as permitted sender) smtp.mailfrom=gkohli@codeaurora.org Authentication-Results: mx.google.com; dkim=pass header.i=@codeaurora.org header.s=default header.b=K0aL59wu; dkim=pass header.i=@codeaurora.org header.s=default header.b=RuSlDXHI; spf=pass (google.com: domain of gkohli@codeaurora.org designates 198.145.29.96 as permitted sender) smtp.mailfrom=gkohli@codeaurora.org DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 2B8C4602B9 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=gkohli@codeaurora.org Subject: Re: [PATCH] tty: fix data race in n_tty_receive_buf_common To: Alan Cox Cc: jslaby@suse.com, gregkh@linuxfoundation.org, mikey@neuling.org, linux-kernel@vger.kernel.org References: <1514987332-14122-1-git-send-email-gkohli@codeaurora.org> <20180103193807.465e054e@alans-desktop> From: "Kohli, Gaurav" Message-ID: <0a456419-c836-08cf-070b-a254fb702b75@codeaurora.org> Date: Thu, 4 Jan 2018 11:17:38 +0530 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <20180103193807.465e054e@alans-desktop> Content-Type: multipart/alternative; boundary="------------5DFA091F55CBB447C7801E6C" Content-Language: en-US X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1588579369033162085?= X-GMAIL-MSGID: =?utf-8?q?1588639683193055654?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: This is a multi-part message in MIME format. --------------5DFA091F55CBB447C7801E6C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi Alan , In our case we are seeing this, when tty_init_dev call is running on one core and flush_to_ldisc is running on other core: So in below case tty_port is intact , only tty  is reinitialized again.    kworker  -000|n_tty_receive_buf_common() -001|n_tty_receive_buf2() -002|tty_ldisc_receive_buf(inline) -002|receive_buf(inline) -002|flush_to_ldisc() -003|__read_once_size(inline) -003|static_key_count(inline) -003|static_key_false(inline) -003|trace_workqueue_execute_end(inline) -003|process_one_work() init -005|tty_unthrottle() -006|n_tty_open() -007|tty_ldisc_open.isra.3() -008|tty_ldisc_setup() -009|tty_init_dev() -010|tty_open_by_driver(inline) -010|tty_open() -011|chrdev_open() -012|do_dentry_open() -013|vfs_open() -014|do_last() > > > 2. tty_init_dev when we set up a new tty > > At that point the tty is not supposed to be receiving data and sure > enough we don't call tty->ops->open until it has finished the ldisc set > up, nor do we tty_init_dev a port that is already running. > > So given we don't activate the port until tty->ops->open() calls > tty_port_open calls the port activation routine I don't see a bug. So it seems this case only , open call has not completed but we have request for receive . > > What does a full (all CPU) trace of the bug look like and what tty driver > are you using when you capture the trace ? We are using tty for console logging,     |    tty = 0xFFFFFFFF477AC880 -> (     |      magic = 21505,     |      kref = (refcount = (counter = 2)),     |      dev = 0xFFFFFFFFEDE3DA80,     |      driver = 0xFFFFFFFFEDE2A480,     |      ops = 0xFFFFFF9F26F7D0D0,     |      index = 0,     |      ldisc_sem = (count = 1, wait_lock = (raw_lock = (owner = 0, next = 0), magic = 3735899821, own     |      termiox = 0x0,     |      name = "ttyMSM0",     |      pgrp = 0x0, We have seen this issue on 4.9 and also one thing i have observed, before tty is getting reinit in tty_init_dev(), there is console service exited before it in all the dumps.  35206.969644:   <2> init: Service 'console' (pid 7440) exited with status 130  35206.969690:   <2> init: Sending signal 9 to service 'console' (pid 7440) process group...  35206.970857:   <2> init: kill(7440, 9) failed: No such process. So how can we stop request of receive buff, if we already have tty_port and tty is getting reinitialized in midway like above case? Regards Gaurav -- Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project. --------------5DFA091F55CBB447C7801E6C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


Hi Alan ,

In our case we are seeing this, when tty_init_dev call is running on one core and flush_to_ldisc is
running on other core:

So in below case tty_port is intact , only tty  is reinitialized again.
   kworker

 -000|n_tty_receive_buf_common()
-001|n_tty_receive_buf2()
-002|tty_ldisc_receive_buf(inline)
-002|receive_buf(inline)
-002|flush_to_ldisc()
-003|__read_once_size(inline)
-003|static_key_count(inline)
-003|static_key_false(inline)
-003|trace_workqueue_execute_end(inline)
-003|process_one_work()
 init

-005|tty_unthrottle()
-006|n_tty_open()
-007|tty_ldisc_open.isra.3()
-008|tty_ldisc_setup()
-009|tty_init_dev()
-010|tty_open_by_driver(inline)
-010|tty_open()
-011|chrdev_open()
-012|do_dentry_open()
-013|vfs_open()
-014|do_last()
 



2. tty_init_dev when we set up a new tty

At that point the tty is not supposed to be receiving data and sure
enough we don't call tty->ops->open until it has finished the ldisc set
up, nor do we tty_init_dev a port that is already running.

So given we don't activate the port until tty->ops->open() calls
tty_port_open calls the port activation routine I don't see a bug.
So it seems this case only , open call has not completed but we have request for receive . 

What does a full (all CPU) trace of the bug look like and what tty driver
are you using when you capture the trace ?
We are using tty for console logging,
    |    tty = 0xFFFFFFFF477AC880 -> (
    |      magic = 21505,
    |      kref = (refcount = (counter = 2)),
    |      dev = 0xFFFFFFFFEDE3DA80,
    |      driver = 0xFFFFFFFFEDE2A480,
    |      ops = 0xFFFFFF9F26F7D0D0,
    |      index = 0,
    |      ldisc_sem = (count = 1, wait_lock = (raw_lock = (owner = 0, next = 0), magic = 3735899821, own
    |      termiox = 0x0,
    |      name = "ttyMSM0",
    |      pgrp = 0x0,

We have seen this issue on 4.9 and also one thing i have observed, before tty is getting reinit in tty_init_dev(),
there is console service exited before it in all the dumps.
 35206.969644:   <2> init: Service 'console' (pid 7440) exited with status 130
 35206.969690:   <2> init: Sending signal 9 to service 'console' (pid 7440) process group...
 35206.970857:   <2> init: kill(7440, 9) failed: No such process.

So how can we stop request of receive buff, if we already have tty_port and tty is getting reinitialized in midway like above
case?

Regards
Gaurav



-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
--------------5DFA091F55CBB447C7801E6C--