From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Cyrus-Session-Id: sloti22d1t05-927638-1516769980-2-2557200721701716076 X-Sieve: CMU Sieve 3.0 X-Spam-known-sender: no X-Spam-score: 0.0 X-Spam-hits: BAYES_00 -1.9, HEADER_FROM_DIFFERENT_DOMAINS 0.25, RCVD_IN_DNSWL_HI -5, T_RP_MATCHES_RCVD -0.01, URIBL_BLACK 1.7, LANGUAGES en, BAYES_USED global, SA_VERSION 3.4.0 X-Spam-source: IP='209.132.180.67', Host='vger.kernel.org', Country='US', FromHeader='com', MailFrom='org', XOriginatingCountry='US' X-Spam-charsets: plain='iso-8859-1' X-Resolved-to: greg@kroah.com X-Delivered-to: greg@kroah.com X-Mail-from: stable-owner@vger.kernel.org ARC-Seal: i=1; a=rsa-sha256; cv=none; d=messagingengine.com; s=arctest; t=1516769980; b=UNS+NUkr0VA9HUd9kCdaWlPamtWHXj0pA2yBQQEDiCNHbjo oOllAFisBBFNrWmFkfmBlf7o+MdNdJs8Q58EHvSzF9I1GCnNTTNl3hbcDOe4HqRq GmTnZeG8b9vmfumomRYTGCfr8D3KNQsRp3tPn4glkiKWD4gnbYECYkK/4GeGJApv dCrkuGQ5tXCMcLblFhN2Kec/O5jo9lRSxExxaTFC3ZGZPAUyo4qDdxWp7E7Dg7UQ +5pe0IRtKzPe/t0AFBIjygsDT4z799UMUuSzm0ucnfCdnfnGCGVsDap8O5+XDwXr r+w5lttLTjMHIECMwFQNtCTrcRMjD4pvph0O8iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=from:to:cc:subject:date:message-id :references:in-reply-to:content-type:content-transfer-encoding :mime-version:sender:list-id; s=arctest; t=1516769980; bh=3DDB1O t1fig3jBifY7PhiHu8CdK+kXw1FhhCVwyeWOg=; b=mMpkFYZCvxYnmTvlxwc6cz mv4TYmJHZW/SHj3qWYOjT4cEdXMhQ81zqvwrN9jUCKXqvWG0Ny2/RUWFJt+rwhSD SJVPlN1sZqacoGVAtVOauyl/b+CERx2wA69pSur55MiGny/ypQan56FSdna4fM/N ngHsXLD7i03aQxQFxwo8wpBe707t9z9a+ABfmp/pAWQgzAjBoYQ5IlcPvBOWcR4W A+PgDqiZZIfDuzmBQ2gx01JmKl0jaqwth/wB7oJXoyO+CMATGvXNSCQ9Qrw+TrI1 VeZBUSPIuGDK9faVSxi9/dZnwF8bzg9kSv2MxDQJMw6iDJcGa98iynV1KLQTyEgA == ARC-Authentication-Results: i=1; mx4.messagingengine.com; arc=none (no signatures found); dkim=pass (1024-bit rsa key sha256) header.d=microsoft.com header.i=@microsoft.com header.b=JB2m5Ju8 x-bits=1024 x-keytype=rsa x-algorithm=sha256 x-selector=selector1; dmarc=pass (p=reject,has-list-id=yes,d=none) header.from=microsoft.com; iprev=pass policy.iprev=209.132.180.67 (vger.kernel.org); spf=none smtp.mailfrom=stable-owner@vger.kernel.org smtp.helo=vger.kernel.org; x-aligned-from=fail; x-ptr=pass x-ptr-helo=vger.kernel.org x-ptr-lookup=vger.kernel.org; x-return-mx=pass smtp.domain=vger.kernel.org smtp.result=pass smtp_org.domain=kernel.org smtp_org.result=pass smtp_is_org_domain=no header.domain=microsoft.com header.result=pass header_is_org_domain=yes Authentication-Results: mx4.messagingengine.com; arc=none (no signatures found); dkim=pass (1024-bit rsa key sha256) header.d=microsoft.com header.i=@microsoft.com header.b=JB2m5Ju8 x-bits=1024 x-keytype=rsa x-algorithm=sha256 x-selector=selector1; dmarc=pass (p=reject,has-list-id=yes,d=none) header.from=microsoft.com; iprev=pass policy.iprev=209.132.180.67 (vger.kernel.org); spf=none smtp.mailfrom=stable-owner@vger.kernel.org smtp.helo=vger.kernel.org; x-aligned-from=fail; x-ptr=pass x-ptr-helo=vger.kernel.org x-ptr-lookup=vger.kernel.org; x-return-mx=pass smtp.domain=vger.kernel.org smtp.result=pass smtp_org.domain=kernel.org smtp_org.result=pass smtp_is_org_domain=no header.domain=microsoft.com header.result=pass header_is_org_domain=yes Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932237AbeAXE71 (ORCPT ); Tue, 23 Jan 2018 23:59:27 -0500 Received: from mail-sn1nam01on0111.outbound.protection.outlook.com ([104.47.32.111]:45037 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752671AbeAXEPQ (ORCPT ); Tue, 23 Jan 2018 23:15:16 -0500 From: Sasha Levin To: "linux-kernel@vger.kernel.org" , "stable@vger.kernel.org" CC: Sagi Grimberg , Christoph Hellwig , "Sasha Levin" Subject: [PATCH AUTOSEL for 4.14 037/100] nvme-rdma: don't complete requests before a send work request has completed Thread-Topic: [PATCH AUTOSEL for 4.14 037/100] nvme-rdma: don't complete requests before a send work request has completed Thread-Index: AQHTlMnbvtRRfEgrakeckeoGFvHdZw== Date: Wed, 24 Jan 2018 04:14:42 +0000 Message-ID: <20180124041414.32065-37-alexander.levin@microsoft.com> References: <20180124041414.32065-1-alexander.levin@microsoft.com> In-Reply-To: <20180124041414.32065-1-alexander.levin@microsoft.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [52.168.54.252] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;DM5PR2101MB0998;7:1gz3/yPp0YYZk8DfL24sgJTGo6deMoc7kLDhd42mt4Ba1OmNjMyHAWMsk042qN0F6p91sZUNN0ODSMT7V0aYwoM8yikiA0tyt1CUMbsiMS3o/Y/T3JYwtKP1j1ppn8dmzJbGgzMRf/ktrzH8qW6T8WM4MmQ+qCHkp8Mfvukz3NnxrRYdl7VZJyS3KCDvRIbF2TqlnN0QBXOEIycrlTFfUIdMw5KRmu1TOpOkUruSp9p+JT/qLlQR6zkMYHOglvnx x-ms-office365-filtering-correlation-id: 2f780e52-5ab8-477b-a37a-08d562e10ff7 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(2017052603307)(7193020);SRVR:DM5PR2101MB0998; x-ms-traffictypediagnostic: DM5PR2101MB0998: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(89211679590171); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(61425038)(6040501)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231046)(2400081)(944501161)(10201501046)(3002001)(6055026)(61426038)(61427038)(6041288)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011);SRVR:DM5PR2101MB0998;BCL:0;PCL:0;RULEID:;SRVR:DM5PR2101MB0998; x-forefront-prvs: 056297E276 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(39380400002)(346002)(39860400002)(396003)(376002)(199004)(189003)(14454004)(3660700001)(6346003)(10290500003)(105586002)(66066001)(6506007)(8676002)(22452003)(86612001)(54906003)(10090500001)(81166006)(81156014)(76176011)(110136005)(86362001)(68736007)(106356001)(59450400001)(8936002)(3280700002)(36756003)(25786009)(107886003)(2501003)(6666003)(99286004)(2906002)(26005)(2950100002)(72206003)(102836004)(305945005)(6116002)(3846002)(5660300001)(4326008)(478600001)(316002)(53936002)(97736004)(2900100001)(5250100002)(6512007)(6436002)(6486002)(1076002)(7736002)(22906009)(217873001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR2101MB0998;H:DM5PR2101MB1032.namprd21.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; authentication-results: spf=none (sender IP is ) smtp.mailfrom=Alexander.Levin@microsoft.com; x-microsoft-antispam-message-info: RIxeH+6nSH1auac2rGkTANZdXs35ZXBkfF3RDP7lnrEIJE+l9+GXoBrqC49od9Pi9LLNHVjqK9/OzuVvNtTUbQ== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f780e52-5ab8-477b-a37a-08d562e10ff7 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jan 2018 04:14:42.3565 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR2101MB0998 Sender: stable-owner@vger.kernel.org X-Mailing-List: stable@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-Mailing-List: linux-kernel@vger.kernel.org List-ID: From: Sagi Grimberg [ Upstream commit 4af7f7ff92a42b6c713293c99e7982bcfcf51a70 ] In order to guarantee that the HCA will never get an access violation (either from invalidated rkey or from iommu) when retrying a send operation we must complete a request only when both send completion and the nvme cqe has arrived. We need to set the send/recv completions flags atomically because we might have more than a single context accessing the request concurrently (one is cq irq-poll context and the other is user-polling used in IOCB_HIPRI). Only then we are safe to invalidate the rkey (if needed), unmap the host buffers, and complete the IO. Signed-off-by: Sagi Grimberg Reviewed-by: Max Gurtovoy Signed-off-by: Christoph Hellwig Signed-off-by: Sasha Levin --- drivers/nvme/host/rdma.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 6de163e6c9eb..33d4431c2b4b 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -67,6 +67,9 @@ struct nvme_rdma_request { struct nvme_request req; struct ib_mr *mr; struct nvme_rdma_qe sqe; + union nvme_result result; + __le16 status; + refcount_t ref; struct ib_sge sge[1 + NVME_RDMA_MAX_INLINE_SEGMENTS]; u32 num_sge; int nents; @@ -1177,6 +1180,7 @@ static int nvme_rdma_map_data(struct nvme_rdma_queue = *queue, req->num_sge =3D 1; req->inline_data =3D false; req->mr->need_inval =3D false; + refcount_set(&req->ref, 2); /* send and recv completions */ =20 c->common.flags |=3D NVME_CMD_SGL_METABUF; =20 @@ -1213,8 +1217,19 @@ static int nvme_rdma_map_data(struct nvme_rdma_queue= *queue, =20 static void nvme_rdma_send_done(struct ib_cq *cq, struct ib_wc *wc) { - if (unlikely(wc->status !=3D IB_WC_SUCCESS)) + struct nvme_rdma_qe *qe =3D + container_of(wc->wr_cqe, struct nvme_rdma_qe, cqe); + struct nvme_rdma_request *req =3D + container_of(qe, struct nvme_rdma_request, sqe); + struct request *rq =3D blk_mq_rq_from_pdu(req); + + if (unlikely(wc->status !=3D IB_WC_SUCCESS)) { nvme_rdma_wr_error(cq, wc, "SEND"); + return; + } + + if (refcount_dec_and_test(&req->ref)) + nvme_end_request(rq, req->status, req->result); } =20 /* @@ -1359,14 +1374,19 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_r= dma_queue *queue, } req =3D blk_mq_rq_to_pdu(rq); =20 - if (rq->tag =3D=3D tag) - ret =3D 1; + req->status =3D cqe->status; + req->result =3D cqe->result; =20 if ((wc->wc_flags & IB_WC_WITH_INVALIDATE) && wc->ex.invalidate_rkey =3D=3D req->mr->rkey) req->mr->need_inval =3D false; =20 - nvme_end_request(rq, cqe->status, cqe->result); + if (refcount_dec_and_test(&req->ref)) { + if (rq->tag =3D=3D tag) + ret =3D 1; + nvme_end_request(rq, req->status, req->result); + } + return ret; } =20 --=20 2.11.0