From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 472241E9B2E for ; Tue, 1 Apr 2025 07:45:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743493519; cv=none; b=tWWb8slaXHEZmXM6xo6CioklbtSfvH+M7/u5GhVNveJdct1O74sscBv6Cj8dC0lH0jxxdSkn6Lqo8bM71Nd0VykPBVGkeX8Xs+7BwO3weOxKzIomIoWaXbxI6HlstndJDwCAMzfP2RFd/o+G8iZSbqxWqTNHbSnPr8iv5wrUrBI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743493519; c=relaxed/simple; bh=NJg9loH5DPwUT9RIGGEOKSUIUmJpxXqcl7s9uyAryf0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=lp9IjISLAUWkilO4PLl4I/v/1sZz6xnwSyz2iJQnBDMRDVCHMGorAbN4bP+Ca5JbgFVrQhbBI+bcmZwfZsePeDGNkvUJt4tj62nntnaFt4+iRU21QwVOC6VC66uOOCV9rHCS5rfFdaXQMc3ZOpe+3u0sqhNaTyDBctuTbqibKJY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MwMKzKWa; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MwMKzKWa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743493514; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uPVEmBMiCehllT7+kdVA6tb5Y19MZOZXc3+AWueTqfA=; b=MwMKzKWaTwMWMDOSrDwMytvj4aEHcCL4aHPIGyOp2iTmsN/x1tBaUymo4MVyqhU53gRT5N 4xjIj9GLwcho2XWv6Atp+lbrEoG17Ya9zEws8w22yHTirilWrVVK+p7BVnFPjjanXqeAJf TI1Y+CURshlWrP/WIrMJNQwWix4sr7c= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-544-P4LOQHBTPVuIgve1ra3ItA-1; Tue, 01 Apr 2025 03:45:13 -0400 X-MC-Unique: P4LOQHBTPVuIgve1ra3ItA-1 X-Mimecast-MFC-AGG-ID: P4LOQHBTPVuIgve1ra3ItA_1743493512 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3912d9848a7so3129698f8f.0 for ; Tue, 01 Apr 2025 00:45:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743493512; x=1744098312; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uPVEmBMiCehllT7+kdVA6tb5Y19MZOZXc3+AWueTqfA=; b=ZCxZ3OYaCPQichpTnmojj3oyKqRyhGIjE6LGUZ1kpteKl2mGk+jogICRL+VAQUbcRX IdAp6vs+5dhAWcS7XvvkA8gEVsstodaTf6rx0R360MqyH0cG+SueOUq3cuPhFFFpUmTD hUGtAwvJEwmj/mPeCKM4EidRafznHxA84I4EGWXbuzrwO2l7Bd1hlVD0JExED6ov8Dv5 R7wBBkkFGUhios8eml3oRokAqqlQlGhi8T4RNl2O0NKG3uDv8f8uVbdme05xnCBXuD01 Za7PftkiXHFf6yNAK4Dmct5vLF5V5zNsdDUy3jKDO+fhQvZG0zyv26XbUXOtA1uCO/7H 4CuA== X-Forwarded-Encrypted: i=1; AJvYcCVJG7t8GcvNYxvQEsRAnKticNddxXq1bHV+bZHIrzuuSmjVcQvtONWLjpAMJs1S1pImtAiXzdTFK7p1gnw=@vger.kernel.org X-Gm-Message-State: AOJu0YzON3RVedL5LhvSxEbbVp7C4jOH1qjDlMSukIFZGprkfFRP/kgd rFU7TPGuZWGk68OR1TN3Sgxzl3wo6xNFJm5ey3uI90VP8VcABoZNaLKrXAUAitUykhw6+h9zWwj EuJ/7bidwxnfqCZPzHnUJEJkT+GmTctpaeLzGe4W685/6a2E7wR1TXalHnpHAcg== X-Gm-Gg: ASbGncsfJAVLC/h4qLok4x2YlGn28VmPg66J/Gj9lyRnpNpzVo/IpqrflTWEku2wUtw 5K7EWSP5LNcSDTpQm2YMO4SFzaBCyXkyaF/b8QYkl74MSvZ1P1iXyVPoxQLASkO6zvro/WZ8FRj upybi+C2nU11VOMnRljhuwBMHEZxFOWTZq2W4zo8da3AN1nYO3WS+0Z4h7/a9M9bYciZsqIb85s iOjP+o3uAf4sOoCRX2N5fwH+qL3fm4VZzmDub1a00Y+M9IYN78q+t82Jw93VIeV3gMGqx0OIXWe YmpcU7ob5B8QO35GCDDXHN+XsMC6ti9amvTxmGHZ+ene9w== X-Received: by 2002:a05:6000:1448:b0:399:6d6a:90d9 with SMTP id ffacd0b85a97d-39c0c136bd4mr13298975f8f.18.1743493511681; Tue, 01 Apr 2025 00:45:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFpGjDvgRuv3rPh+sZzuIo7a1UqtEX2dbLL0x3yl3Qc1zwqWs0cDqxIOuutwoJPuKYJIFAmdw== X-Received: by 2002:a05:6000:1448:b0:399:6d6a:90d9 with SMTP id ffacd0b85a97d-39c0c136bd4mr13298938f8f.18.1743493511262; Tue, 01 Apr 2025 00:45:11 -0700 (PDT) Received: from [192.168.88.253] (146-241-68-231.dyn.eolo.it. [146.241.68.231]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-39c0b6630a3sm13423505f8f.30.2025.04.01.00.45.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 01 Apr 2025 00:45:10 -0700 (PDT) Message-ID: Date: Tue, 1 Apr 2025 09:45:08 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Concurrent slab-use-after-free in netdev_next_lower_dev To: Dylan Wolff , Wenjia Zhang , Jan Karcher , "D. Wythe" , Tony Lu , Wen Gu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Simon Horman , linux-rdma@vger.kernel.org, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Jiacheng Xu <3170103308@zju.edu.cn> References: Content-Language: en-US From: Paolo Abeni In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 3/31/25 3:00 PM, Dylan Wolff wrote: > From the report, it looks like the net_device is freed at the end of an > rtnl critical section in netdev_run_todo. At the time of the crash, the > *use* thread has acquired rtnl_lock() in smc_vlan_by_tcpsk. The crash > occurred at the line preceded by `>>>` below in 6.13 rc4 while iterating > over devices with netdev_walk_all_lower_dev: > > ``` > static struct net_device *netdev_next_lower_dev(struct net_device *dev, > struct list_head **iter) > { > struct netdev_adjacent *lower; > >>>> lower = list_entry((*iter)->next, struct netdev_adjacent, list); > > if (&lower->list == &dev->adj_list.lower) > return NULL; > > *iter = &lower->list; > > return lower->dev; > } > ``` Please share the decoded syzkaller/kasan splat in plaintext instead of describing it in natural language, and please avoid attachments unless explicitly asked for. Also, can you reproduce the issue on top of the current net tree? > This looks to me like it is an issue with reference counting; I see that > netdev_refcnt_read is checked in netdev_run_todo before the device is > freed, but I don't see anything in netdev_walk_all_lower_dev / > netdev_next_lower_dev that is incrementing netdev_refcnt_read when it is > iterating over the devices. I'm guessing the fix is to either add reference > counting to netdev_walk_all_lower_dev or to use a different, > concurrency-safe iterator over the devices in the caller (smc_vlan_by_tcpsk > ). > > Could someone confirm if I am on the right track here? If so I am happy to > try to come up with the patch. netdev_walk_all_lower_dev() should not need additional refcounting, as it is traversing the list under rtnl lock, and device should be removed from the adjacency list before the actual device free under such lock, too. Thanks, Paolo