From: James Simmons <jsimmons@infradead.org>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
devel@driverdev.osuosl.org,
Andreas Dilger <andreas.dilger@intel.com>,
Oleg Drokin <oleg.drokin@intel.com>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Lustre Development List <lustre-devel@lists.lustre.org>,
Chris Horn <hornc@cray.com>,
James Simmons <jsimmons@infradead.org>
Subject: [PATCH 58/58] staging: lustre: ptlrpc: Early replies need to honor at_max
Date: Thu, 21 Jul 2016 22:44:51 -0400 [thread overview]
Message-ID: <1469155491-15265-59-git-send-email-jsimmons@infradead.org> (raw)
In-Reply-To: <1469155491-15265-1-git-send-email-jsimmons@infradead.org>
From: Chris Horn <hornc@cray.com>
When determining whether an early reply can be sent the server will
calculate the new deadline based on an offset from the request
arrival time. However, when actually setting the new deadline
the server offsets the current time. This can result in deadlines
being extended more than at_max seconds past the request arrival
time. Instead, the server should offset the arrival time when updating
its request timeout.
When a client receives an early reply it doesn't know the server side
arrival time so we use the original sent time as an approximation.
Signed-off-by: Chris Horn <hornc@cray.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4578
Reviewed-on: http://review.whamcloud.com/9100
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
drivers/staging/lustre/lustre/ptlrpc/client.c | 8 +++++---
drivers/staging/lustre/lustre/ptlrpc/import.c | 11 +++++++----
drivers/staging/lustre/lustre/ptlrpc/service.c | 18 +++++++++++++-----
3 files changed, 25 insertions(+), 12 deletions(-)
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index d4463d7..ddb08ea 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -385,10 +385,12 @@ static int ptlrpc_at_recv_early_reply(struct ptlrpc_request *req)
spin_lock(&req->rq_lock);
olddl = req->rq_deadline;
/*
- * server assumes it now has rq_timeout from when it sent the
- * early reply, so client should give it at least that long.
+ * server assumes it now has rq_timeout from when the request
+ * arrived, so the client should give it at least that long.
+ * since we don't know the arrival time we'll use the original
+ * sent time
*/
- req->rq_deadline = ktime_get_real_seconds() + req->rq_timeout +
+ req->rq_deadline = req->rq_sent + req->rq_timeout +
ptlrpc_at_get_net_latency(req);
DEBUG_REQ(D_ADAPTTO, req,
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
index 3292e6e..af8ffbc 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/import.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c
@@ -1497,10 +1497,13 @@ EXPORT_SYMBOL(ptlrpc_disconnect_import);
/* Adaptive Timeout utils */
extern unsigned int at_min, at_max, at_history;
-/* Bin into timeslices using AT_BINS bins.
- * This gives us a max of the last binlimit*AT_BINS secs without the storage,
- * but still smoothing out a return to normalcy from a slow response.
- * (E.g. remember the maximum latency in each minute of the last 4 minutes.)
+/*
+ *Update at_current with the specified value (bounded by at_min and at_max),
+ * as well as the AT history "bins".
+ * - Bin into timeslices using AT_BINS bins.
+ * - This gives us a max of the last at_history seconds without the storage,
+ * but still smoothing out a return to normalcy from a slow response.
+ * - (E.g. remember the maximum latency in each minute of the last 4 minutes.)
*/
int at_measured(struct adaptive_timeout *at, unsigned int val)
{
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 4788c49..30d8b72 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -1005,13 +1005,16 @@ ptlrpc_at_remove_timed(struct ptlrpc_request *req)
array->paa_count--;
}
+/*
+ * Attempt to extend the request deadline by sending an early reply to the
+ * client.
+ */
static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
{
struct ptlrpc_service_part *svcpt = req->rq_rqbd->rqbd_svcpt;
struct ptlrpc_request *reqcopy;
struct lustre_msg *reqmsg;
long olddl = req->rq_deadline - ktime_get_real_seconds();
- time64_t newdl;
int rc;
/* deadline is when the client expects us to reply, margin is the
@@ -1039,8 +1042,13 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
return -ENOSYS;
}
- /* Fake our processing time into the future to ask the clients
- * for some extra amount of time
+ /*
+ * We want to extend the request deadline by at_extra seconds,
+ * so we set our service estimate to reflect how much time has
+ * passed since this request arrived plus an additional
+ * at_extra seconds. The client will calculate the new deadline
+ * based on this service estimate (plus some additional time to
+ * account for network latency). See ptlrpc_at_recv_early_reply
*/
at_measured(&svcpt->scp_at_estimate, at_extra +
ktime_get_real_seconds() - req->rq_arrival_time.tv_sec);
@@ -1056,7 +1064,6 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
ktime_get_real_seconds());
return -ETIMEDOUT;
}
- newdl = ktime_get_real_seconds() + at_get(&svcpt->scp_at_estimate);
reqcopy = ptlrpc_request_cache_alloc(GFP_NOFS);
if (!reqcopy)
@@ -1110,7 +1117,8 @@ static int ptlrpc_at_send_early_reply(struct ptlrpc_request *req)
if (!rc) {
/* Adjust our own deadline to what we told the client */
- req->rq_deadline = newdl;
+ req->rq_deadline = req->rq_arrival_time.tv_sec +
+ at_get(&svcpt->scp_at_estimate);
req->rq_early_count++; /* number sent, server side */
} else {
DEBUG_REQ(D_ERROR, req, "Early reply send failed %d", rc);
--
1.7.1
prev parent reply other threads:[~2016-07-22 4:10 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-22 2:43 [PATCH 00/58] staging: lustre: bug fixes from lustre 2.5.[56-58] James Simmons
2016-07-22 2:43 ` [PATCH 01/58] staging: lustre: obd: expand op_cli_flags James Simmons
2016-07-22 2:43 ` [PATCH 02/58] staging: lustre: obd: rename struct lmv_stripe_md field mea to lmv James Simmons
2016-07-22 2:43 ` [PATCH 03/58] staging: lustre: ptlrpc: remove wirecheck for struct lmv_stripe_md James Simmons
2016-07-22 2:43 ` [PATCH 04/58] staging: lustre: llite: cache directory striping information James Simmons
2016-07-22 2:43 ` [PATCH 05/58] staging: lustre: lmv: add new lmv structures James Simmons
2016-07-22 2:43 ` [PATCH 06/58] staging: lustre: mdc: handle IT_READDIR operations James Simmons
2016-07-22 2:44 ` [PATCH 07/58] staging: lustre: llite: label the debug info James Simmons
2016-07-22 2:44 ` [PATCH 08/58] staging: lustre: llite: pass struct md_op_data to ll_dir_read James Simmons
2016-07-22 2:44 ` [PATCH 09/58] staging: lustre: llite: remove debug message in ll_dir_read James Simmons
2016-07-22 2:44 ` [PATCH 10/58] staging: lustre: llite: reduce indent " James Simmons
2016-07-22 2:44 ` [PATCH 11/58] staging: lustre: llite: set next only when needed " James Simmons
2016-07-22 2:44 ` [PATCH 12/58] staging: lustre: llite: handle done flags differently " James Simmons
2016-07-22 2:44 ` [PATCH 13/58] staging: lustre: llite: change done flag in ll_dir_read to bool James Simmons
2016-07-22 2:44 ` [PATCH 14/58] staging: lustre: llite: rename some variables for ll_dir_read James Simmons
2016-07-22 2:44 ` [PATCH 15/58] staging: lustre: llite: clarify some debug messages for statahead James Simmons
2016-07-22 2:44 ` [PATCH 16/58] staging: lustre: llite: remove code never called James Simmons
2016-07-22 2:44 ` [PATCH 17/58] staging: lustre: llite: pass in __u64 pos for ll_dir_read James Simmons
2016-07-22 2:44 ` [PATCH 18/58] staging: lustre: llite: do post work for statahead in readdir case James Simmons
2016-07-22 2:44 ` [PATCH 19/58] staging: lustre: llite: add md_op_data parameter to ll_get_dir_page James Simmons
2016-08-15 16:38 ` Greg Kroah-Hartman
2016-08-15 21:40 ` James Simmons
2016-08-16 10:51 ` Greg Kroah-Hartman
2016-07-22 2:44 ` [PATCH 20/58] staging: lustre: llite: remove comment from ll_dir_read James Simmons
2016-07-22 2:44 ` [PATCH 21/58] staging: lustre: llite: style cleanup for llite_internal.h James Simmons
2016-07-22 2:44 ` [PATCH 22/58] staging: lustre: llite: pass inode to ll_release_page James Simmons
2016-07-22 2:44 ` [PATCH 23/58] staging: lustre: llite: change remove parameter to bool James Simmons
2016-07-22 2:44 ` [PATCH 24/58] staging: lustre: mdc: don't take rpc lock for readdir case James Simmons
2016-07-22 2:44 ` [PATCH 25/58] staging: lustre: lmv: remove unused lmv_get_mea function James Simmons
2016-07-22 2:44 ` [PATCH 26/58] staging: lustre: lmv: remove duplicate MAX_HASH_* James Simmons
2016-07-22 2:44 ` [PATCH 27/58] staging: lustre: lmv: change handling of lmv striping information James Simmons
2016-07-22 2:44 ` [PATCH 28/58] staging: lustre: lmv: remove lmv_get_easize James Simmons
2016-07-22 2:44 ` [PATCH 29/58] staging: lustre: lmv: replace obd_free_memmd with lmv_free_memmd James Simmons
2016-07-22 2:44 ` [PATCH 30/58] staging: lustre: create striped directory James Simmons
2016-07-22 14:38 ` kbuild test robot
2016-07-26 16:36 ` [PATCH v2 " James Simmons
2016-07-26 16:59 ` Joe Perches
2016-07-26 18:09 ` James Simmons
2016-07-22 2:44 ` [PATCH 31/58] staging: lustre: llite: fix "getdirstripe" to show stripe info James Simmons
2016-07-22 2:44 ` [PATCH 32/58] staging: lustre: delete striped directory James Simmons
2016-07-22 2:44 ` [PATCH 33/58] staging: lustre: obdclass: fix lmd_parse() to handle comma-separated NIDs James Simmons
2016-07-22 2:44 ` [PATCH 34/58] staging: lustre: obdclass: bug fixes for lu_device_type handling James Simmons
2016-07-22 2:44 ` [PATCH 35/58] staging: lustre: add ability to migrate inodes James Simmons
2016-07-22 2:44 ` [PATCH 36/58] staging: lustre: lmv: fix issue found by Klocwork Insight tool James Simmons
2016-07-22 2:44 ` [PATCH 37/58] staging: lustre: libcfs: Only dump log once per sec. to avoid EEXIST James Simmons
2016-07-22 2:44 ` [PATCH 38/58] staging: lustre: llite: enable clients to inject error for lfsck James Simmons
2016-07-22 2:44 ` [PATCH 39/58] staging: lustre: osc: allow to call brw_commit() multiple times James Simmons
2016-07-22 2:44 ` [PATCH 40/58] staging: lustre: llite: a few fixes for migration James Simmons
2016-07-22 2:44 ` [PATCH 41/58] staging: lustre: mdc: fixup MDS_SWAP_LAYOUTS ELC handling James Simmons
2016-07-22 2:44 ` [PATCH 42/58] staging: lustre: don't need to const __u64 parameters for lustre_idl.h James Simmons
2016-07-22 2:44 ` [PATCH 43/58] staging: lustre: const correct FID/OSTID/... helpers James Simmons
2016-07-22 2:44 ` [PATCH 44/58] staging: lustre: use bool for several function in lustre_idl.h/lustre_fid.h James Simmons
2016-07-22 2:44 ` [PATCH 45/58] staging: lustre: simplify inline functions in lustre_fid.h James Simmons
2016-07-22 2:44 ` [PATCH 46/58] staging: lustre: lmv: access lum_stripe_offset as little endian James Simmons
2016-07-22 2:44 ` [PATCH 47/58] staging: lustre: lmv: lookup remote migrating object in LMV James Simmons
2016-07-22 2:44 ` [PATCH 48/58] staging: lustre: lmv: Ensure lmv_intent_lookup cleans up reqp James Simmons
2016-07-22 2:44 ` [PATCH 49/58] staging: lustre: llite: avoid a deadlock in page write James Simmons
2016-07-22 2:44 ` [PATCH 50/58] staging: lustre: lov: handle the case of stripe size is not power 2 James Simmons
2016-07-22 2:44 ` [PATCH 51/58] staging: lustre: lmv: cleanup req in lmv_getattr_name() James Simmons
2016-07-22 2:44 ` [PATCH 52/58] staging: lustre: lmv: rename request to preq " James Simmons
2016-07-22 2:44 ` [PATCH 53/58] staging: lustre: obdclass: unified flow control interfaces James Simmons
2016-07-22 2:44 ` [PATCH 54/58] staging: lustre: reorder LOV_MAGIC_* definition James Simmons
2016-07-22 2:44 ` [PATCH 55/58] staging: lustre: ldlm: flock completion fixes James Simmons
2016-07-22 2:44 ` [PATCH 56/58] staging: lustre: move ioctls to lustre_ioctl.h James Simmons
2016-07-22 2:44 ` [PATCH 57/58] staging: lustre: llite: add error handler in inode prepare phase James Simmons
2016-07-22 2:44 ` James Simmons [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1469155491-15265-59-git-send-email-jsimmons@infradead.org \
--to=jsimmons@infradead.org \
--cc=andreas.dilger@intel.com \
--cc=devel@driverdev.osuosl.org \
--cc=gregkh@linuxfoundation.org \
--cc=hornc@cray.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lustre-devel@lists.lustre.org \
--cc=oleg.drokin@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®