mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Li Zhijian <lizhijian@cn.fujitsu.com>
To: <linux-rdma@vger.kernel.org>, <zyjzyj2000@gmail.com>,
	<jgg@ziepe.ca>, <aharonl@nvidia.com>, <leon@kernel.org>,
	<tom@talpey.com>, <tomasz.gromadzki@intel.com>
Cc: <linux-kernel@vger.kernel.org>, <mbloch@nvidia.com>,
	<liangwenpeng@huawei.com>, <yangx.jy@fujitsu.com>,
	<y-goto@fujitsu.com>, <rpearsonhpe@gmail.com>,
	<dan.j.williams@intel.com>, Li Zhijian <lizhijian@cn.fujitsu.com>
Subject: [RFC PATCH v2 3/9] RDMA/rxe: Allow registering persistent flag for pmem MR only
Date: Tue, 25 Jan 2022 16:50:35 +0800	[thread overview]
Message-ID: <20220125085041.49175-4-lizhijian@cn.fujitsu.com> (raw)
In-Reply-To: <20220125085041.49175-1-lizhijian@cn.fujitsu.com>

Memory region should support 2 placement types: IB_ACCESS_FLUSH_PERSISTENT
and IB_ACCESS_FLUSH_GLOBAL_VISIBILITY, and only pmem/nvdimm has ability to
persist data(IB_ACCESS_FLUSH_PERSISTENT).

It prevents local user from registering a persistent access flag to
a non-pmem MR.

+------------------------+------------------+--------------+
| HCA attributes         |    register access flags        |
|        and             +-----------------+---------------+
| MR attribute(is_pmem)  |global visibility |  persistence |
|------------------------+------------------+--------------+
| global visibility(DRAM)|        O         |      X       |
|------------------------+------------------+--------------+
| global visibility(PMEM)|        O         |      X       |
|------------------------+------------------+--------------+
| persistence(DRAM)      |        X         |      X       |
|------------------------+------------------+--------------+
| persistence(PMEM)      |        X         |      O       |
+------------------------+------------------+--------------+
PMEM: is_pmem is true
DRAM: is_pmem is false
O: allow to register such access flag
X: otherwise

Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
---
V2: update commit message, get rid of confusing ib_check_flush_access_flags() # Tom
---
 drivers/infiniband/sw/rxe/rxe_mr.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index 0427baea8c06..89a3bb4e8b71 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -258,7 +258,15 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova,
 	set->offset = ib_umem_offset(umem);
 
 	// iova_in_pmem() must be called after set is updated
-	mr->ibmr.is_pmem = iova_in_pmem(mr, iova, length);
+	if (iova_in_pmem(mr, iova, length))
+		mr->ibmr.is_pmem = true;
+	else if (access & IB_ACCESS_FLUSH_PERSISTENT) {
+		pr_warn("Cannot register IB_ACCESS_FLUSH_PERSISTENT for non-pmem memory\n");
+		mr->state = RXE_MR_STATE_INVALID;
+		mr->umem = NULL;
+		err = -EINVAL;
+		goto err_release_umem;
+	}
 
 	return 0;
 
-- 
2.31.1




  parent reply	other threads:[~2022-01-25  8:49 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-25  8:50 [RFC PATCH v2 0/9] RDMA/rxe: Add RDMA FLUSH operation Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 1/9] RDMA: mr: Introduce is_pmem Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 2/9] RDMA: Allow registering MR with flush access flags Li Zhijian
2022-01-25  8:50 ` Li Zhijian [this message]
2022-01-25  8:50 ` [RFC PATCH v2 4/9] RDMA/rxe: Implement RC RDMA FLUSH service in requester side Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 5/9] RDMA/rxe: Set BTH's SE to zero for FLUSH packet Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 6/9] RDMA/rxe: Implement flush execution in responder side Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 7/9] RDMA/rxe: Implement flush completion Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 8/9] RDMA/rxe: Enable RDMA FLUSH capability for rxe device Li Zhijian
2022-01-25  8:50 ` [RFC PATCH v2 9/9] RDMA/rxe: Add RD FLUSH service support Li Zhijian
2022-01-25  8:57 ` [RFC PATCH v2 0/9] RDMA/rxe: Add RDMA FLUSH operation Zhu Yanjun
2022-01-25 10:32   ` lizhijian

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220125085041.49175-4-lizhijian@cn.fujitsu.com \
    --to=lizhijian@cn.fujitsu.com \
    --cc=aharonl@nvidia.com \
    --cc=dan.j.williams@intel.com \
    --cc=jgg@ziepe.ca \
    --cc=leon@kernel.org \
    --cc=liangwenpeng@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mbloch@nvidia.com \
    --cc=rpearsonhpe@gmail.com \
    --cc=tom@talpey.com \
    --cc=tomasz.gromadzki@intel.com \
    --cc=y-goto@fujitsu.com \
    --cc=yangx.jy@fujitsu.com \
    --cc=zyjzyj2000@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®