From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA9D039060B for ; Tue, 28 Jul 2026 04:14:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785212073; cv=none; b=fcQrIJqgSxaUjzqQXZPWR+SM0qco+f4ZwG8iv5qmLRMMRHj3siY1y3sIfL8Xml2VjNap301pcUd7YcxvTkAAuDxB1d+Wxk0bkndCLEpy2rD3WLei+assrPv2oTVmNYjhBgNzrP5YYaZ+5YcEx+/s229DypYAWWCE78N0vKG50ak= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785212073; c=relaxed/simple; bh=RH42sLA4MuGPpZli7KVq7uoOU5s+9wzFRhN0P04gNHs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=CenyWX8w1s16tSzSPASJGv+tAFd1cq2QaPFokDVDOFwkznclr3VNqPbzDuj1YWmcsHdAcAqeW53RyYoeiwimw4nai4ziwDN2HZe+7Xidf8563+qoK2wrA41N1IcduW2LeBBWhz8fY43Qb1tPq6gGp1MAQ6PYoLBu0heRGLZtqRU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=l7WRVR7M; arc=none smtp.client-ip=209.85.160.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="l7WRVR7M" Received: by mail-qt1-f174.google.com with SMTP id d75a77b69052e-51c16ac21acso21979631cf.0 for ; Mon, 27 Jul 2026 21:14:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1785212065; x=1785816865; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=aN5W11Hsh6MzAnuT/d7MxNwGLL+Nh74FFhNYp722hFA=; b=l7WRVR7M0bkfomr7NEcDJmiOpwSHNzsM+ofVk9YD4ioMJAeZ1laeytn7S8AEjoKprb Pms233v+Xme92oHA9yXs/C5tG/eMsooav9dzuXEKSgvAvy6dypAU4cQJTIXETdXnD6+c BMnuZmJ3e1nXYnFSnZomIGLiMGw2nWTo0EEmMRFG2uDtZ85nG/udlvm7jy7F1YaIs1kD IRxGNW1lNPYtRuHJgGjTt+wa6WIUUEdZw6jX0hgMKJGk3e4+N9OK7kZujb5ySMFeGIRN ioQmgJ30H+YE09c4hyNdZeXr24yoHstncEcMarlJjkoSnmCmHR9m7c/L1ZmHqzOLGV9Q GvWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785212065; x=1785816865; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=aN5W11Hsh6MzAnuT/d7MxNwGLL+Nh74FFhNYp722hFA=; b=cWqmqMv14PMubFfcKPz9AH8Y9+IdJMeTx4AueGM9sIdiMyOxLzBi2nzHDjgl0KevpF /bs0vFhislniq5qrYqPds2o4vLcIIrD1yVI+B/83SCf28xO+mVbYLFl9jYeRmdBbFCCj YJvdcGXchGrwfMq+LfAfbb7wj+EyzLz5NdVuDSmpC+WBOeM1ZGvdYK06s0HGBnk1tL2b X94CDCQfcSQSJPDQGhGB8xHaSBsKhMIGcWeHXATzAwNTLXsgiydh01WLg1e2GrnrV+ot XSitPfj10ngPx5pehw7d95tdCNyyoYsvrhoXrjpVS0SIu9tsPIPF/YHFiBc6gWZYwcg4 nwpQ== X-Forwarded-Encrypted: i=1; AHgh+RrXBnAFQhyhhlJaVDwyil1kzp0ZlXAl24T45exHqFpQrzWoVImX2pWsMBc5HW/3PDzS/glT9g8sMJhi1XY=@vger.kernel.org X-Gm-Message-State: AOJu0YyU9Atz0oMV3VmJdv25QMKuC4pLGOodCfMOgUBZRyQj1VTmYsfX ImpNOTcmgFL1gX0oVIEQVwryLVdgN3IXNO1iUad1fgIDlQmfO4TMiXynKAWVxnfftTA= X-Gm-Gg: AR+sD10KUrTgvmKcrnh+LWgwZxUaLyXKJw/cft1aLDiVq9N08tkYQNJBXat8jY4SYh5 smv5iJNuFliRP+w/hvgK0mkY90fiMlnggn8PpSSv2eqcWGC/F+WoF9YoNsk8Ucn2XCD9rX1rysr PnbvbxFee++ldS4D0Nd82qVuTVpcHgw8Q1cK0K58wjhJe5qKJpmvUeEYdtKV2GOsCmXogLOUWxK vL5n1CwYTODfsxSagS5KSwo3uOghwFx+ZbKhLm2m3nHuNoUzko7rhM2FEcPqGHBMWNY6siwwgiW fPqpfAo2XYNWLggd4olRlGTjY3zEKGrP5dQ10/+7HXaP6LFeJwpEVc9pJUcwx54ay7pUUVo3rp0 7EQOY4/YM6wOX7+v46ezmQ54U348tTBNCm/2ChrpSxKvTERlYUqE8hNpjy85uM0pFrgosm3EBee nRnawwvSIz2hqSXKeoTWcdjEXteApWJLxNb0zIrSihE+JK+XYlUm3O/zyueqV5OQrmRJte X-Received: by 2002:a05:620a:199f:b0:930:b440:a63 with SMTP id af79cd13be357-93302748796mr59604885a.80.1785212064648; Mon, 27 Jul 2026 21:14:24 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-932de657111sm760520085a.38.2026.07.27.21.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jul 2026 21:14:24 -0700 (PDT) Date: Tue, 28 Jul 2026 00:14:18 -0400 From: Gregory Price To: Richard Cheng Cc: Dave Jiang , dave@stgolabs.net, jic23@kernel.org, alison.schofield@intel.com, vishal.l.verma@intel.com, djbw@kernel.org, danwilliams@nvidia.com, iweiny@kernel.org, ming.li@zohomail.com, rrichter@amd.com, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, sreddym@nvidia.com, smadhavan@nvidia.com, kaihengf@nvidia.com, kobak@nvidia.com, newtonl@nvidia.com, kristinc@nvidia.com, mochs@nvidia.com Subject: Re: [PATCH] cxl/region: Restore passthrough decoder enable on region re-assembly Message-ID: References: <20260727103743.63343-1-icheng@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Tue, Jul 28, 2026 at 11:03:37AM +0800, Richard Cheng wrote: > On Mon, Jul 27, 2026 at 01:25:48PM +0800, Dave Jiang wrote: > > I think this part is talking about physical remove/insert ? sorry maybe I commit message was too vague about the scenario. > The case here is a SW-only teardown via sysfs, this is what I did. > > """ > $ echo 1 > /sys/bus/pci/devices/$BDF/remove > $ echo 1 > /sys/bus/pci/devices/$BDF/rescan > """ > > No physical removal, no power cycle, no link-down, no reset. Linux drops and re-enumerates the same device, which never stopped running. > The endpoint HDM decoder is still committed. > > The device is byte-identical before and after, the kernel doesn't clear it either, the decoder is locked and cxl_decoder_reset() returns > early for CXL_DECODER_F_LOCK before touching any register. The memory keeps decoding. > > I don't think anything needs re-programming here? the only thing lost is kernel-side bookkeeping on the passthrough decoder, which is > freed with port and reallocated with F_ENABLE clear on rescan. The endpoint recovers its state from HW. > The passthrough decoder has no HW to recover from, that asymmetry is the bug. > Being a passthrough decoder is just a special state of a switch decoder, it doesn't necessarily imply programmability (Enable, Commit, Lock all technically still apply, it's just ambiguous what they should be). Wonder if we're just not setting these intermediate decoders up sanely. If the endpoint is locked / not reprogrammable (commit / lock) on the first go around, maybe the intermediate decoders should be force-locked and not have F_ENABLE cleared on teardown? It's not like this actually changes anything on the hardware, it's just bookkeeping. (although i will say it's been a bit since i looked at the flag state machine here, so i could be misremembering what the rules around ENABLE are i this context). Curious - what happens if you fully unload cxl_pci between remove/rescan? ~Gregory