Re: [PATCH 3/3] cxl: Avoid to create dax regions for type2 accelerators

["Huang, Ying" <ying.huang@intel.com>]

Alejandro Lucero Palau <alucerop@amd.com> writes:

> On 8/6/24 06:52, Huang, Ying wrote:
>> Jonathan Cameron <Jonathan.Cameron@Huawei.com> writes:
>>
>>> On Mon, 29 Jul 2024 16:46:11 +0800
>>> Huang Ying <ying.huang@intel.com> wrote:
>>>
>>>> The memory range of a type2 accelerator should be managed by the type2
>>>> accelerator specific driver instead of the common dax region drivers,
>>>> as discussed in [1].
>>>>
>>>> [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/
>>>>
>>>> So, in this patch, we skip dax regions creation for type2 accelerator
>>>> device memory regions.
>>>>
>>>> Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/
>>>>
>>>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>>>> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
>>>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>>>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>>>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>>>> Cc: Dave Jiang <dave.jiang@intel.com>
>>>> Cc: Alison Schofield <alison.schofield@intel.com>
>>>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>>>> Cc: Ira Weiny <ira.weiny@intel.com>
>>>> Cc: Alejandro Lucero <alucerop@amd.com>
>>>> ---
>>>>   drivers/cxl/core/region.c | 8 ++++++++
>>>>   1 file changed, 8 insertions(+)
>>>>
>>>> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
>>>> index 9a483c8a32fd..b37e12bb4a35 100644
>>>> --- a/drivers/cxl/core/region.c
>>>> +++ b/drivers/cxl/core/region.c
>>>> @@ -3435,6 +3435,14 @@ static int cxl_region_probe(struct device *dev)
>>>>   					p->res->start, p->res->end, cxlr,
>>>>   					is_system_ram) > 0)
>>>>   			return 0;
>>>> +		/*
>>>> +		 * HDM-D[B] (device-memory) regions have accelerator
>>>> +		 * specific usage, skip device-dax registration.
>>>> +		 */
>>>> +		if (cxlr->type == CXL_DECODER_DEVMEM)
>>>> +			return 0;
>>> As in previous need to be careful as that may not mean it's
>>> an accelerator.
>> Yes.  We need some other way to identify type2 devices.
>
>
> Maybe the easier option is the accel driver specifying if DAX should
> be used.
>
> We are adding mailbox and hdm as optional for accel/type2, with the
> driver specifying what is supported. Another optional param could be
> this DAX usage.

Another way is let accel/cxl_pci driver specify the device type (type2
vs. type3).  cxl_pci_probe()->cxl_memdev_state_create() will set
cxl_dev_state.type = CXL_DEVTYPE_CLASSMEM. And
accel_X_probe()->cxl_accel_state_create() can set cxl_dev_state.type =
CXL_DEVTYPE_DEVMEM.  This can be passed to cxl_region.  Then we can
create cxl_dax_region for type3 devices only by default.

>
>>> However, we do need to deal with BI setup for HDM-DB type 3 devices
>>> etc and to check the HDM Decoder capability registers to make sure
>>> Supported Coherence model is appropriate. (e.g. 11 for host only or
>>> device coherency - HDM-H/HDM-DB)
>> Yes.  We need to check BI configuration too.
>>
>>>> +
>>>> +		/* HDM-H routes to device-dax */
>>>>   		return devm_cxl_add_dax_region(cxlr);
>>>>   	default:
>>>>   		dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",

--
Best Regards,
Huang, Ying