From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4337127BF93 for ; Thu, 12 Feb 2026 21:06:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770930362; cv=none; b=q3o/Nn/Ix9Zz0xsLi5yCg20A6NrbGSfLRDJ+z6HBXwTpgg71en4ZbjxFoX6RoDjOlQwJs58vLk/8h795tJalQXIfWEk50WTB7U2nm6UGNPhnkbbERB3bBFdSSyWg5J4CBuhMSJIO9rr4hg1qU5JWTnJSj0TQRc1lokp7EsOdLwU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770930362; c=relaxed/simple; bh=Ya+CGTJLrEsA5sJn5yXU05k9CEnmFuK5qm9N962RkiQ=; h=Message-ID:Date:MIME-Version:Subject:To:References:From: In-Reply-To:Content-Type; b=KiIzk7RR1XtqMS2xEoRM5SehUK/S1I07gJvMZ9OKBUScmTp+EhK7U0XxLRfyTgRc0V9VSefF1Fs7HFBv/4Hst9zsPis/NG7tnt+3VPesNwRu6f1jeePXUcEknoUZ0UeDFe0xhORMw1u6PcoldVPK9kNX+7JKpLrCnLI5q462pfQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b=Ws30356k; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b="Ws30356k" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4806e0f6b69so1954985e9.3 for ; Thu, 12 Feb 2026 13:06:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1770930360; x=1771535160; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=ADy427avKCLRuNPr+99fOoTmdbxhgZXkgFfd8ipl3Es=; b=Ws30356kNHYma1xCGDeouDMp1Z6vRs11jfZnkPl5vhxcNJUZXKyWrys4h3EwrhBr41 fmQ2bndQgzD7Tk0ZsMxVCX6e5lwTGPQK0rufTtelTwRhlNdAbGKsw+x8CAn3jwz34Ayc U+M++N3cb12uKLcpox2lzOqd6FzW0R3If1DlzmR4s8F+KW9kvpH+4D7S4Lc0EpOWoIDQ ZB9C6OpRg/1qnxdgL7uOQp5CzZUJwW7mn/uVoZ3mAxOHoa1tDjK1/msrIS0MFq32u+Yr ajMUnBVPc0CnmFZKrmuv7sWTMNCdMxexp6ODQDduYjagnilv0qfoHG5e/TZEobgbVYne OBkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770930360; x=1771535160; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ADy427avKCLRuNPr+99fOoTmdbxhgZXkgFfd8ipl3Es=; b=OQ4BCajrjIgHAFC1DXyI23ZDT6yVeSoM/+wGQh7need5FmN0jj9HbNaAvlVNrjgklr JTFm/OZsfS1TGg3mFOLrtu8IypJMiKOJolfSanCX4XfS8RL4ZH3xujzWLavJGSHrjRcr Qzxc++9ycr1wIt9szlTLsa9RM7wi+YvA5RYBTIVnxdxhVuwoxppNJBwFj6eLallbvkvR FwjK0TLCRM6S2C1kq003KHSxwXQLlYXoJGCrciaw18wvsXIpriJlQNduGuUPeDL0nPXb /8oNR+Va8HDQTssP3sBaUQ/np7Eq6LbLzgBBluE8xNGDKYOuYIBxGmwL6OrSpLrgbtT9 /vMQ== X-Forwarded-Encrypted: i=1; AJvYcCUqHTNTu4w2ZtPbxvpVFMetCZZCZ9PquUrnW19AlXEJlzObP4ninjXMGMGfCXiLxsbTrCm99zXi1l5NgmQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwwtkrY/WFlCvfvsUIsk85JSU9lsuv0o4aeZMHX5Nbgt0ESYzww svJYJsahVrJ9ABwR1AdWc8HsPeqQOHEKeQEQ14ESXvntKb3yLClZrUOK6neXosUxxWU= X-Gm-Gg: AZuq6aKNY6XZI6kl9Y+87WaAEC1wo5ftI7mkjieQ7c6mRoDe/O9fqMyloLznqClcRoM i2XS/Ysf+uCVZU2DKcXtyMcZTvhxkJeEJx7eH/pAoBmHrPcIrR0LMjkxmHgc4c2DVcxeW5C+kr+ Wzhx04YOd4Z3AnOMnawOc0uiEdYTsADAT9ZV/9Umpj/JLB4IyGjPBF5u7wOVWPZ2aG5KGrGQQUm WUPnORZXgM3pLXYdaIooYbvcwevLG2Aw+33JK17zEQNgi9xxm+PQ9aemmAVvojyvm1acsJmRbyc yOJlufd2CwcDzJWEy+U5KMT3MYN4+0hbOGveZScGocLEmU5NXxWPH4azejf41K7QY7E/08o3Mgo cF4DJR76jjfynYMJQ1ZXJeaVyhcXUq9k07C2NwQlMRhzpAbzowTWDFeM+JuGBpqsnGFw92YJKr8 YztdR0rOmKVZNdfxyEJ4upRceXeMaoQ70hyHjljE4jA+WPsJjKpie2u2mCNg1fMg== X-Received: by 2002:a05:600c:1547:b0:477:6d96:b3c8 with SMTP id 5b1f17b1804b1-48371085837mr6027235e9.23.1770930359475; Thu, 12 Feb 2026 13:05:59 -0800 (PST) Received: from ?IPV6:2403:580d:fda1::299? (2403-580d-fda1--299.ip6.aussiebb.net. [2403:580d:fda1::299]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c6e197d5e1csm5735387a12.22.2026.02.12.13.05.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Feb 2026 13:05:58 -0800 (PST) Message-ID: Date: Fri, 13 Feb 2026 07:35:45 +1030 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Orange PI 5 MAX: very unstable using kernel 6.19.0 and 6.18.10, 6.18.9 perfectly stable To: David Arendt , LKML , linux-rockchip@vger.kernel.org, Btrfs BTRFS References: <8ebe4d76-eb07-499b-b140-1f300c1b8d7e@prnet.org> Content-Language: en-US From: Qu Wenruo Autocrypt: addr=wqu@suse.com; keydata= xsBNBFnVga8BCACyhFP3ExcTIuB73jDIBA/vSoYcTyysFQzPvez64TUSCv1SgXEByR7fju3o 8RfaWuHCnkkea5luuTZMqfgTXrun2dqNVYDNOV6RIVrc4YuG20yhC1epnV55fJCThqij0MRL 1NxPKXIlEdHvN0Kov3CtWA+R1iNN0RCeVun7rmOrrjBK573aWC5sgP7YsBOLK79H3tmUtz6b 9Imuj0ZyEsa76Xg9PX9Hn2myKj1hfWGS+5og9Va4hrwQC8ipjXik6NKR5GDV+hOZkktU81G5 gkQtGB9jOAYRs86QG/b7PtIlbd3+pppT0gaS+wvwMs8cuNG+Pu6KO1oC4jgdseFLu7NpABEB AAHNGFF1IFdlbnJ1byA8d3F1QHN1c2UuY29tPsLAlAQTAQgAPgIbAwULCQgHAgYVCAkKCwIE FgIDAQIeAQIXgBYhBC3fcuWlpVuonapC4cI9kfOhJf6oBQJnEXVgBQkQ/lqxAAoJEMI9kfOh Jf6o+jIH/2KhFmyOw4XWAYbnnijuYqb/obGae8HhcJO2KIGcxbsinK+KQFTSZnkFxnbsQ+VY fvtWBHGt8WfHcNmfjdejmy9si2jyy8smQV2jiB60a8iqQXGmsrkuR+AM2V360oEbMF3gVvim 2VSX2IiW9KERuhifjseNV1HLk0SHw5NnXiWh1THTqtvFFY+CwnLN2GqiMaSLF6gATW05/sEd V17MdI1z4+WSk7D57FlLjp50F3ow2WJtXwG8yG8d6S40dytZpH9iFuk12Sbg7lrtQxPPOIEU rpmZLfCNJJoZj603613w/M8EiZw6MohzikTWcFc55RLYJPBWQ+9puZtx1DopW2jOwE0EWdWB rwEIAKpT62HgSzL9zwGe+WIUCMB+nOEjXAfvoUPUwk+YCEDcOdfkkM5FyBoJs8TCEuPXGXBO Cl5P5B8OYYnkHkGWutAVlUTV8KESOIm/KJIA7jJA+Ss9VhMjtePfgWexw+P8itFRSRrrwyUf E+0WcAevblUi45LjWWZgpg3A80tHP0iToOZ5MbdYk7YFBE29cDSleskfV80ZKxFv6koQocq0 vXzTfHvXNDELAuH7Ms/WJcdUzmPyBf3Oq6mKBBH8J6XZc9LjjNZwNbyvsHSrV5bgmu/THX2n g/3be+iqf6OggCiy3I1NSMJ5KtR0q2H2Nx2Vqb1fYPOID8McMV9Ll6rh8S8AEQEAAcLAfAQY AQgAJgIbDBYhBC3fcuWlpVuonapC4cI9kfOhJf6oBQJnEXWBBQkQ/lrSAAoJEMI9kfOhJf6o cakH+QHwDszsoYvmrNq36MFGgvAHRjdlrHRBa4A1V1kzd4kOUokongcrOOgHY9yfglcvZqlJ qfa4l+1oxs1BvCi29psteQTtw+memmcGruKi+YHD7793zNCMtAtYidDmQ2pWaLfqSaryjlzR /3tBWMyvIeWZKURnZbBzWRREB7iWxEbZ014B3gICqZPDRwwitHpH8Om3eZr7ygZck6bBa4MU o1XgbZcspyCGqu1xF/bMAY2iCDcq6ULKQceuKkbeQ8qxvt9hVxJC2W3lHq8dlK1pkHPDg9wO JoAXek8MF37R8gpLoGWl41FIUb3hFiu3zhDDvslYM4BmzI18QgQTQnotJH8= In-Reply-To: <8ebe4d76-eb07-499b-b140-1f300c1b8d7e@prnet.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit 在 2026/2/13 06:41, David Arendt 写道: > Hello, > > I am using a Kubernetes Cluster with 3 Orange PI5 MAX nodes. The data is > stored using a btrfs filesystem as backend. If using kernel 6.19.0 or > kernel 6.18.10 I have experienced many crashes during high IO load on > all 3 nodes. Reverting back to 6.18.9 solves the problems completely. > Unfortunately the crashes are spontaneous reboots without leaving a > trace in any logfile, so I have no stacktrace of them. After the crashes > I have sometimes incorrect btrfs csums for a file but these may also be > a result of a partial write due to the crash. On one node I had a btrfs > error logged without crashing, but I am not sure if this is the root > cause or a result of a prior crash. A scrub after reboot returned no > error with 6.19.0. The offending tree dump items are: Feb 10 13:31:07 opi02 kernel: item 92 key (13218356101120 Feb 10 13:31:07 opi02 kernel: item 93 key (13216208642048 Feb 10 13:31:07 opi02 kernel: item 94 key (13218356162560 Obviously item 93 is smaller than all its previous and next item keys. hex(13218356101120) = 0xc05a36b8000 hex(13216208642048) = 0xc05236be000 hex(13218356162560) = 0xc05a36c7000 It looks like something fliped, "0xc05a3" -> "0xc0523" 0xa -> 0x2 is exactly one bit flipped. So either the memory hardware has something wrong and resulting a sticking bit (always 0), or there is something inside the kernel touching memory it shouldn't. And this exactly matches the symptom, changing random bit of your kernel, crash always expected. Can you run a memtest to make sure it is not hardware problems first? Thanks, Qu > > Unfortunately I don't have more information at the moment. > > Thanks in advance, > > David Arendt > >