From: Roland Dreier <rolandd@cisco.com>
To: linux-kernel@vger.kernel.org, openib-general@openib.org
Subject: [git patch review 7/8] IB/mthca: fix posting of send lists of length >= 255 on mem-free HCAs
Date: Wed, 30 Nov 2005 00:57:25 +0000 [thread overview]
Message-ID: <1133312245799-fb80cd19aa5b232b@cisco.com> (raw)
In-Reply-To: <1133312245799-51b50fe9f024aec5@cisco.com>
On mem-free HCAs, when posting a long list of send requests, a
doorbell must be rung every 255 requests. Add code to handle this.
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
---
drivers/infiniband/hw/mthca/mthca_qp.c | 31 +++++++++++++++++++++++++++++--
drivers/infiniband/hw/mthca/mthca_wqe.h | 3 ++-
2 files changed, 31 insertions(+), 3 deletions(-)
applies-to: 1f53cd0db55372192cc088788dadbed102845a17
e0ae9ecf469fdd3c1ad999efbf4fe6b782f49900
diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c
index f9c8eb9..7450550 100644
--- a/drivers/infiniband/hw/mthca/mthca_qp.c
+++ b/drivers/infiniband/hw/mthca/mthca_qp.c
@@ -1822,6 +1822,7 @@ int mthca_arbel_post_send(struct ib_qp *
{
struct mthca_dev *dev = to_mdev(ibqp->device);
struct mthca_qp *qp = to_mqp(ibqp);
+ __be32 doorbell[2];
void *wqe;
void *prev_wqe;
unsigned long flags;
@@ -1841,6 +1842,34 @@ int mthca_arbel_post_send(struct ib_qp *
ind = qp->sq.head & (qp->sq.max - 1);
for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (unlikely(nreq == MTHCA_ARBEL_MAX_WQES_PER_SEND_DB)) {
+ nreq = 0;
+
+ doorbell[0] = cpu_to_be32((MTHCA_ARBEL_MAX_WQES_PER_SEND_DB << 24) |
+ ((qp->sq.head & 0xffff) << 8) |
+ f0 | op0);
+ doorbell[1] = cpu_to_be32((qp->qpn << 8) | size0);
+
+ qp->sq.head += MTHCA_ARBEL_MAX_WQES_PER_SEND_DB;
+ size0 = 0;
+
+ /*
+ * Make sure that descriptors are written before
+ * doorbell record.
+ */
+ wmb();
+ *qp->sq.db = cpu_to_be32(qp->sq.head & 0xffff);
+
+ /*
+ * Make sure doorbell record is written before we
+ * write MMIO send doorbell.
+ */
+ wmb();
+ mthca_write64(doorbell,
+ dev->kar + MTHCA_SEND_DOORBELL,
+ MTHCA_GET_DOORBELL_LOCK(&dev->doorbell_lock));
+ }
+
if (mthca_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) {
mthca_err(dev, "SQ %06x full (%u head, %u tail,"
" %d max, %d nreq)\n", qp->qpn,
@@ -2017,8 +2046,6 @@ int mthca_arbel_post_send(struct ib_qp *
out:
if (likely(nreq)) {
- __be32 doorbell[2];
-
doorbell[0] = cpu_to_be32((nreq << 24) |
((qp->sq.head & 0xffff) << 8) |
f0 | op0);
diff --git a/drivers/infiniband/hw/mthca/mthca_wqe.h b/drivers/infiniband/hw/mthca/mthca_wqe.h
index 73f1c0b..e7d2c1e 100644
--- a/drivers/infiniband/hw/mthca/mthca_wqe.h
+++ b/drivers/infiniband/hw/mthca/mthca_wqe.h
@@ -50,7 +50,8 @@ enum {
enum {
MTHCA_INVAL_LKEY = 0x100,
- MTHCA_TAVOR_MAX_WQES_PER_RECV_DB = 256
+ MTHCA_TAVOR_MAX_WQES_PER_RECV_DB = 256,
+ MTHCA_ARBEL_MAX_WQES_PER_SEND_DB = 255
};
struct mthca_next_seg {
---
0.99.9k
next prev parent reply other threads:[~2005-11-30 0:57 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-11-30 0:57 [git patch review 1/8] IPoIB: reinitialize path struct's completion for every query Roland Dreier
2005-11-30 0:57 ` [git patch review 2/8] IPoIB: always set path->query to NULL when query finishes Roland Dreier
2005-11-30 0:57 ` [git patch review 3/8] IPoIB: reinitialize mcast structs' completions for every query Roland Dreier
2005-11-30 0:57 ` [git patch review 4/8] IPoIB: don't zero members after we allocate with kzalloc Roland Dreier
2005-11-30 0:57 ` [git patch review 5/8] IPoIB: protect child list in ipoib_ib_dev_flush Roland Dreier
2005-11-30 0:57 ` [git patch review 6/8] IPoIB: fix error handling in ipoib_open Roland Dreier
2005-11-30 0:57 ` Roland Dreier [this message]
2005-11-30 0:57 ` [git patch review 8/8] IB/uverbs: track multicast group membership for userspace QPs Roland Dreier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1133312245799-fb80cd19aa5b232b@cisco.com \
--to=rolandd@cisco.com \
--cc=linux-kernel@vger.kernel.org \
--cc=openib-general@openib.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®