* [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd
@ 2011-09-30 12:48 Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
` (10 more replies)
0 siblings, 11 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
This the first request for review of drbd-8.4. The complete set has
471 patches. This is the twelfth installment containing 11 patches.
The whole set is available here:
git://git.drbd.org/linux-2.6-drbd.git for-jens
and is jens_for-3.2_drivers...for-jens
and this part is f23bb45^...ef9bce0
This patch series contains:
* Start to use RCU for the objects containing the configuration.
(Changed seldom, read accessed frequently)
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place Philipp Reisner
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 2 +-
drivers/block/drbd/drbd_main.c | 12 ++++--------
drivers/block/drbd/drbd_nl.c | 2 +-
3 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 9aca1f4..ce56aa0 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1367,7 +1367,7 @@ extern rwlock_t global_state_lock;
extern int conn_lowest_minor(struct drbd_tconn *tconn);
enum drbd_ret_code conn_new_minor(struct drbd_tconn *tconn, unsigned int minor, int vnr);
extern void drbd_free_mdev(struct drbd_conf *mdev);
-extern void drbd_delete_device(unsigned int minor);
+extern void drbd_delete_device(struct drbd_conf *mdev);
struct drbd_tconn *drbd_new_tconn(const char *name);
extern void drbd_free_tconn(struct drbd_tconn *tconn);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 8097d7a..8466ee7 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2195,15 +2195,10 @@ static void drbd_release_all_peer_reqs(struct drbd_conf *mdev)
}
/* caution. no locking. */
-void drbd_delete_device(unsigned int minor)
+void drbd_delete_device(struct drbd_conf *mdev)
{
- struct drbd_conf *mdev = minor_to_mdev(minor);
-
- if (!mdev)
- return;
-
idr_remove(&mdev->tconn->volumes, mdev->vnr);
- idr_remove(&minors, minor);
+ idr_remove(&minors, mdev_to_minor(mdev));
synchronize_rcu();
/* paranoia asserts */
@@ -2256,7 +2251,8 @@ static void drbd_cleanup(void)
drbd_genl_unregister();
idr_for_each_entry(&minors, mdev, i)
- drbd_delete_device(i);
+ drbd_delete_device(mdev);
+
drbd_destroy_mempools();
unregister_blkdev(DRBD_MAJOR, "drbd");
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 4b60673..be2ca81 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -2916,7 +2916,7 @@ static enum drbd_ret_code adm_delete_minor(struct drbd_conf *mdev)
* we may want to delete a minor from a live replication group.
*/
mdev->state.role == R_SECONDARY) {
- drbd_delete_device(mdev_to_minor(mdev));
+ drbd_delete_device(mdev);
return NO_ERROR;
} else
return ERR_MINOR_CONFIGURED;
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep Philipp Reisner
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 1 -
drivers/block/drbd/drbd_main.c | 25 +++++++------------------
2 files changed, 7 insertions(+), 19 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index ce56aa0..c49dc08 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1366,7 +1366,6 @@ extern rwlock_t global_state_lock;
extern int conn_lowest_minor(struct drbd_tconn *tconn);
enum drbd_ret_code conn_new_minor(struct drbd_tconn *tconn, unsigned int minor, int vnr);
-extern void drbd_free_mdev(struct drbd_conf *mdev);
extern void drbd_delete_device(struct drbd_conf *mdev);
struct drbd_tconn *drbd_new_tconn(const char *name);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 8466ee7..03b3ffe 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2224,10 +2224,13 @@ void drbd_delete_device(struct drbd_conf *mdev)
kfree(mdev->p_uuid);
/* mdev->p_uuid = NULL; */
- /* cleanup the rest that has been
- * allocated from drbd_new_device
- * and actually free the mdev itself */
- drbd_free_mdev(mdev);
+ kfree(mdev->current_epoch);
+ if (mdev->bitmap) /* should no longer be there. */
+ drbd_bm_cleanup(mdev);
+ __free_page(mdev->md_io_page);
+ put_disk(mdev->vdisk);
+ blk_cleanup_queue(mdev->rq_queue);
+ kfree(mdev);
}
static void drbd_cleanup(void)
@@ -2542,20 +2545,6 @@ out_no_q:
return err;
}
-/* counterpart of drbd_new_device.
- * last part of drbd_delete_device. */
-void drbd_free_mdev(struct drbd_conf *mdev)
-{
- kfree(mdev->current_epoch);
- if (mdev->bitmap) /* should no longer be there. */
- drbd_bm_cleanup(mdev);
- __free_page(mdev->md_io_page);
- put_disk(mdev->vdisk);
- blk_cleanup_queue(mdev->rq_queue);
- kfree(mdev);
-}
-
-
int __init drbd_init(void)
{
int err;
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
2011-09-30 12:48 ` [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem Philipp Reisner
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_main.c | 12 +++++++---
drivers/block/drbd/drbd_nl.c | 41 +++++++++++++++++++++++++----------
drivers/block/drbd/drbd_receiver.c | 2 +
drivers/block/drbd/drbd_state.c | 34 +++++++++++++++++++++++++++--
drivers/block/drbd/drbd_worker.c | 4 +++
5 files changed, 74 insertions(+), 19 deletions(-)
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 03b3ffe..c4c0bc5 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -453,8 +453,10 @@ void tl_clear(struct drbd_tconn *tconn)
}
/* ensure bit indicating barrier is required is clear */
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
clear_bit(CREATE_BARRIER, &mdev->flags);
+ rcu_read_unlock();
spin_unlock_irq(&tconn->req_lock);
}
@@ -634,13 +636,15 @@ char *drbd_task_to_thread_name(struct drbd_tconn *tconn, struct task_struct *tas
int conn_lowest_minor(struct drbd_tconn *tconn)
{
- int vnr = 0;
struct drbd_conf *mdev;
+ int vnr = 0, m;
+ rcu_read_lock();
mdev = idr_get_next(&tconn->volumes, &vnr);
- if (!mdev)
- return -1;
- return mdev_to_minor(mdev);
+ m = mdev ? mdev_to_minor(mdev) : -1;
+ rcu_read_unlock();
+
+ return m;
}
#ifdef CONFIG_SMP
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index be2ca81..fc918d0 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -407,12 +407,14 @@ static enum drbd_fencing_p highest_fencing_policy(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (get_ldev_if_state(mdev, D_CONSISTENT)) {
fp = max_t(enum drbd_fencing_p, fp, mdev->ldev->dc.fencing);
put_ldev(mdev);
}
}
+ rcu_read_unlock();
return fp;
}
@@ -1664,29 +1666,41 @@ out:
static bool conn_resync_running(struct drbd_tconn *tconn)
{
struct drbd_conf *mdev;
+ bool rv = false;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (mdev->state.conn == C_SYNC_SOURCE ||
mdev->state.conn == C_SYNC_TARGET ||
mdev->state.conn == C_PAUSED_SYNC_S ||
- mdev->state.conn == C_PAUSED_SYNC_T)
- return true;
+ mdev->state.conn == C_PAUSED_SYNC_T) {
+ rv = true;
+ break;
+ }
}
- return false;
+ rcu_read_unlock();
+
+ return rv;
}
static bool conn_ov_running(struct drbd_tconn *tconn)
{
struct drbd_conf *mdev;
+ bool rv = false;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (mdev->state.conn == C_VERIFY_S ||
- mdev->state.conn == C_VERIFY_T)
- return true;
+ mdev->state.conn == C_VERIFY_T) {
+ rv = true;
+ break;
+ }
}
- return false;
+ rcu_read_unlock();
+
+ return rv;
}
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
@@ -1912,26 +1926,28 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
if (get_ldev(mdev)) {
enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
put_ldev(mdev);
if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
retcode = ERR_STONITH_AND_PROT_A;
- goto fail;
+ goto fail_rcu_unlock;
}
}
if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
retcode = ERR_DISCARD;
- goto fail;
+ goto fail_rcu_unlock;
}
if (!mdev->bitmap) {
if(drbd_bm_init(mdev)) {
retcode = ERR_NOMEM;
- goto fail;
+ goto fail_rcu_unlock;
}
}
}
+ rcu_read_unlock();
if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A) {
retcode = ERR_CONG_NOT_PROTO_A;
@@ -2045,15 +2061,19 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
retcode = _conn_request_state(tconn, NS(conn, C_UNCONNECTED), CS_VERBOSE);
spin_unlock_irq(&tconn->req_lock);
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
mdev->send_cnt = 0;
mdev->recv_cnt = 0;
kobject_uevent(&disk_to_dev(mdev->vdisk)->kobj, KOBJ_CHANGE);
}
+ rcu_read_unlock();
conn_reconfig_done(tconn);
drbd_adm_finish(info, retcode);
return 0;
+fail_rcu_unlock:
+ rcu_read_unlock();
fail:
kfree(int_dig_in);
kfree(int_dig_vv);
@@ -2616,8 +2636,6 @@ int drbd_adm_get_status_all(struct sk_buff *skb, struct netlink_callback *cb)
/* synchronize with drbd_new_tconn/drbd_free_tconn */
mutex_lock(&drbd_cfg_mutex);
- /* synchronize with drbd_delete_device */
- rcu_read_lock();
next_tconn:
/* revalidate iterator position */
list_for_each_entry(tmp, &drbd_tconns, all_tconn) {
@@ -2680,7 +2698,6 @@ next_tconn:
}
out:
- rcu_read_unlock();
mutex_unlock(&drbd_cfg_mutex);
/* where to start the next iteration */
cb->args[0] = (long)pos;
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 0c45eba..92ecb22 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -4828,11 +4828,13 @@ static int tconn_finish_peer_reqs(struct drbd_tconn *tconn)
set_bit(SIGNAL_ASENDER, &tconn->flags);
spin_lock_irq(&tconn->req_lock);
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
not_empty = !list_empty(&mdev->done_ee);
if (not_empty)
break;
}
+ rcu_read_unlock();
spin_unlock_irq(&tconn->req_lock);
} while (not_empty);
diff --git a/drivers/block/drbd/drbd_state.c b/drivers/block/drbd/drbd_state.c
index 694bd3f..0aed33b 100644
--- a/drivers/block/drbd/drbd_state.c
+++ b/drivers/block/drbd/drbd_state.c
@@ -56,15 +56,21 @@ static inline bool is_susp(union drbd_state s)
bool conn_all_vols_unconf(struct drbd_tconn *tconn)
{
struct drbd_conf *mdev;
+ bool rv = true;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (mdev->state.disk != D_DISKLESS ||
mdev->state.conn != C_STANDALONE ||
- mdev->state.role != R_SECONDARY)
- return false;
+ mdev->state.role != R_SECONDARY) {
+ rv = false;
+ break;
+ }
}
- return true;
+ rcu_read_unlock();
+
+ return rv;
}
enum drbd_role conn_highest_role(struct drbd_tconn *tconn)
@@ -73,8 +79,10 @@ enum drbd_role conn_highest_role(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
role = max_t(enum drbd_role, role, mdev->state.role);
+ rcu_read_unlock();
return role;
}
@@ -85,8 +93,10 @@ enum drbd_role conn_highest_peer(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
peer = max_t(enum drbd_role, peer, mdev->state.peer);
+ rcu_read_unlock();
return peer;
}
@@ -97,8 +107,10 @@ enum drbd_disk_state conn_highest_disk(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
ds = max_t(enum drbd_disk_state, ds, mdev->state.disk);
+ rcu_read_unlock();
return ds;
}
@@ -109,8 +121,10 @@ enum drbd_disk_state conn_lowest_disk(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
ds = min_t(enum drbd_disk_state, ds, mdev->state.disk);
+ rcu_read_unlock();
return ds;
}
@@ -121,8 +135,10 @@ enum drbd_disk_state conn_highest_pdsk(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
ds = max_t(enum drbd_disk_state, ds, mdev->state.pdsk);
+ rcu_read_unlock();
return ds;
}
@@ -133,8 +149,10 @@ enum drbd_disk_state conn_lowest_conn(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
conn = min_t(enum drbd_conns, conn, mdev->state.conn);
+ rcu_read_unlock();
return conn;
}
@@ -1388,12 +1406,14 @@ static int w_after_conn_state_ch(struct drbd_work *w, int unused)
/* case1: The outdate peer handler is successful: */
if (ns_max.pdsk <= D_OUTDATED) {
tl_clear(tconn);
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (test_bit(NEW_CUR_UUID, &mdev->flags)) {
drbd_uuid_new_current(mdev);
clear_bit(NEW_CUR_UUID, &mdev->flags);
}
}
+ rcu_read_unlock();
conn_request_state(tconn,
(union drbd_state) { { .susp_fen = 1 } },
(union drbd_state) { { .susp_fen = 0 } },
@@ -1401,8 +1421,10 @@ static int w_after_conn_state_ch(struct drbd_work *w, int unused)
}
/* case2: The connection was established again: */
if (ns_min.conn >= C_CONNECTED) {
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
clear_bit(NEW_CUR_UUID, &mdev->flags);
+ rcu_read_unlock();
spin_lock_irq(&tconn->req_lock);
_tl_restart(tconn, RESEND);
_conn_request_state(tconn,
@@ -1427,6 +1449,7 @@ void conn_old_common_state(struct drbd_tconn *tconn, union drbd_state *pcs, enum
struct drbd_conf *mdev;
int vnr, first_vol = 1;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
os = mdev->state;
@@ -1451,6 +1474,7 @@ void conn_old_common_state(struct drbd_tconn *tconn, union drbd_state *pcs, enum
if (cs.pdsk != os.pdsk)
flags &= ~CS_DC_PDSK;
}
+ rcu_read_unlock();
*pf |= CS_DC_MASK;
*pf &= flags;
@@ -1466,6 +1490,7 @@ conn_is_valid_transition(struct drbd_tconn *tconn, union drbd_state mask, union
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
os = drbd_read_state(mdev);
ns = sanitize_state(mdev, apply_mask_val(os, mask, val), NULL);
@@ -1491,6 +1516,7 @@ conn_is_valid_transition(struct drbd_tconn *tconn, union drbd_state mask, union
if (rv < SS_SUCCESS)
break;
}
+ rcu_read_unlock();
if (rv < SS_SUCCESS && flags & CS_VERBOSE)
print_st_err(mdev, os, ns, rv);
@@ -1516,6 +1542,7 @@ conn_set_state(struct drbd_tconn *tconn, union drbd_state mask, union drbd_state
if (mask.conn == C_MASK)
tconn->cstate = val.conn;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
os = drbd_read_state(mdev);
ns = apply_mask_val(os, mask, val);
@@ -1541,6 +1568,7 @@ conn_set_state(struct drbd_tconn *tconn, union drbd_state mask, union drbd_state
ns_min.disk = min_t(enum drbd_disk_state, ns.disk, ns_min.disk);
ns_min.pdsk = min_t(enum drbd_disk_state, ns.pdsk, ns_min.pdsk);
}
+ rcu_read_unlock();
ns_min.susp = ns_max.susp = tconn->susp;
ns_min.susp_nod = ns_max.susp_nod = tconn->susp_nod;
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index 62536bc..536eb05 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -1350,6 +1350,7 @@ static int _drbd_pause_after(struct drbd_conf *mdev)
struct drbd_conf *odev;
int i, rv = 0;
+ rcu_read_lock();
idr_for_each_entry(&minors, odev, i) {
if (odev->state.conn == C_STANDALONE && odev->state.disk == D_DISKLESS)
continue;
@@ -1357,6 +1358,7 @@ static int _drbd_pause_after(struct drbd_conf *mdev)
rv |= (__drbd_set_state(_NS(odev, aftr_isp, 1), CS_HARD, NULL)
!= SS_NOTHING_TO_DO);
}
+ rcu_read_unlock();
return rv;
}
@@ -1372,6 +1374,7 @@ static int _drbd_resume_next(struct drbd_conf *mdev)
struct drbd_conf *odev;
int i, rv = 0;
+ rcu_read_lock();
idr_for_each_entry(&minors, odev, i) {
if (odev->state.conn == C_STANDALONE && odev->state.disk == D_DISKLESS)
continue;
@@ -1382,6 +1385,7 @@ static int _drbd_resume_next(struct drbd_conf *mdev)
!= SS_NOTHING_TO_DO) ;
}
}
+ rcu_read_unlock();
return rv;
}
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (2 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem Philipp Reisner
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 4 +++-
drivers/block/drbd/drbd_main.c | 10 +++++-----
drivers/block/drbd/drbd_nl.c | 27 ++++++++++++++++-----------
3 files changed, 24 insertions(+), 17 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index c49dc08..7896a64 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -171,7 +171,9 @@ drbd_insert_fault(struct drbd_conf *mdev, unsigned int type) {
extern struct ratelimit_state drbd_ratelimit_state;
extern struct idr minors;
extern struct list_head drbd_tconns;
-extern struct mutex drbd_cfg_mutex;
+extern struct rw_semaphore drbd_cfg_rwsem;
+/* drbd_cfg_rwsem protects: drbd_tconns list,
+ note: non sleeping iterations over the idrs are protoected by RCU */
/* on the wire */
enum drbd_packet {
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index c4c0bc5..d48efb4 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -120,7 +120,7 @@ module_param_string(usermode_helper, usermode_helper, sizeof(usermode_helper), 0
*/
struct idr minors;
struct list_head drbd_tconns; /* list of struct drbd_tconn */
-DEFINE_MUTEX(drbd_cfg_mutex);
+DECLARE_RWSEM(drbd_cfg_rwsem);
struct kmem_cache *drbd_request_cache;
struct kmem_cache *drbd_ee_cache; /* peer requests */
@@ -2321,14 +2321,14 @@ struct drbd_tconn *conn_by_name(const char *name)
if (!name || !name[0])
return NULL;
- mutex_lock(&drbd_cfg_mutex);
+ down_read(&drbd_cfg_rwsem);
list_for_each_entry(tconn, &drbd_tconns, all_tconn) {
if (!strcmp(tconn->name, name))
goto found;
}
tconn = NULL;
found:
- mutex_unlock(&drbd_cfg_mutex);
+ up_read(&drbd_cfg_rwsem);
return tconn;
}
@@ -2395,9 +2395,9 @@ struct drbd_tconn *drbd_new_tconn(const char *name)
DRBD_ON_NO_DATA_DEF, /* on_no_data */
};
- mutex_lock(&drbd_cfg_mutex);
+ down_write(&drbd_cfg_rwsem);
list_add_tail(&tconn->all_tconn, &drbd_tconns);
- mutex_unlock(&drbd_cfg_mutex);
+ up_write(&drbd_cfg_rwsem);
return tconn;
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index fc918d0..5667536 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -1959,7 +1959,7 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
new_my_addr = (struct sockaddr *)&new_conf->my_addr;
new_peer_addr = (struct sockaddr *)&new_conf->peer_addr;
- /* No need to take drbd_cfg_mutex here. All reconfiguration is
+ /* No need to take drbd_cfg_rwsem here. All reconfiguration is
* strictly serialized on genl_lock(). We are protected against
* concurrent reconfiguration/addition/deletion */
list_for_each_entry(oconn, &drbd_tconns, all_tconn) {
@@ -2635,7 +2635,7 @@ int drbd_adm_get_status_all(struct sk_buff *skb, struct netlink_callback *cb)
*/
/* synchronize with drbd_new_tconn/drbd_free_tconn */
- mutex_lock(&drbd_cfg_mutex);
+ down_read(&drbd_cfg_rwsem);
next_tconn:
/* revalidate iterator position */
list_for_each_entry(tmp, &drbd_tconns, all_tconn) {
@@ -2698,7 +2698,7 @@ next_tconn:
}
out:
- mutex_unlock(&drbd_cfg_mutex);
+ up_read(&drbd_cfg_rwsem);
/* where to start the next iteration */
cb->args[0] = (long)pos;
cb->args[1] = (pos == tconn) ? volume + 1 : 0;
@@ -2949,9 +2949,9 @@ int drbd_adm_delete_minor(struct sk_buff *skb, struct genl_info *info)
if (retcode != NO_ERROR)
goto out;
- mutex_lock(&drbd_cfg_mutex);
+ down_write(&drbd_cfg_rwsem);
retcode = adm_delete_minor(adm_ctx.mdev);
- mutex_unlock(&drbd_cfg_mutex);
+ up_write(&drbd_cfg_rwsem);
/* if this was the last volume of this connection,
* this will terminate all threads */
if (retcode == NO_ERROR)
@@ -2979,7 +2979,7 @@ int drbd_adm_down(struct sk_buff *skb, struct genl_info *info)
goto out;
}
- mutex_lock(&drbd_cfg_mutex);
+ down_read(&drbd_cfg_rwsem);
/* demote */
idr_for_each_entry(&adm_ctx.tconn->volumes, mdev, i) {
retcode = drbd_set_role(mdev, R_SECONDARY, 0);
@@ -3006,14 +3006,17 @@ int drbd_adm_down(struct sk_buff *skb, struct genl_info *info)
goto out_unlock;
}
}
+ up_read(&drbd_cfg_rwsem);
/* delete volumes */
+ down_write(&drbd_cfg_rwsem);
idr_for_each_entry(&adm_ctx.tconn->volumes, mdev, i) {
retcode = adm_delete_minor(mdev);
if (retcode != NO_ERROR) {
/* "can not happen" */
drbd_msg_put_info("failed to delete volume");
- goto out_unlock;
+ up_write(&drbd_cfg_rwsem);
+ goto out;
}
}
@@ -3028,10 +3031,12 @@ int drbd_adm_down(struct sk_buff *skb, struct genl_info *info)
/* "can not happen" */
retcode = ERR_CONN_IN_USE;
drbd_msg_put_info("failed to delete connection");
- goto out_unlock;
}
+
+ up_write(&drbd_cfg_rwsem);
+ goto out;
out_unlock:
- mutex_unlock(&drbd_cfg_mutex);
+ up_read(&drbd_cfg_rwsem);
out:
drbd_adm_finish(info, retcode);
return 0;
@@ -3047,14 +3052,14 @@ int drbd_adm_delete_connection(struct sk_buff *skb, struct genl_info *info)
if (retcode != NO_ERROR)
goto out;
- mutex_lock(&drbd_cfg_mutex);
+ down_write(&drbd_cfg_rwsem);
if (conn_lowest_minor(adm_ctx.tconn) < 0) {
drbd_free_tconn(adm_ctx.tconn);
retcode = NO_ERROR;
} else {
retcode = ERR_CONN_IN_USE;
}
- mutex_unlock(&drbd_cfg_mutex);
+ up_write(&drbd_cfg_rwsem);
out:
drbd_adm_finish(info, retcode);
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (3 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 06/11] drbd: Runtime changeable wire protocol Philipp Reisner
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
With this commit the locking for all accesses to IDRs is complete:
* Non sleeping read accesses are protected by RCU
* sleeping read accesses are protocted by a read lock on drbd_cfg_rwsem
* accesses that add anything are protected by a write lock
* accesses that remove an object are protoected by a write lock
and a call to synchronize_rcu() after it is removed from the IDR
and before the object is actually free()ed.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 2 +-
drivers/block/drbd/drbd_main.c | 2 ++
drivers/block/drbd/drbd_nl.c | 4 ++++
drivers/block/drbd/drbd_proc.c | 2 ++
drivers/block/drbd/drbd_receiver.c | 13 +++++++++++--
drivers/block/drbd/drbd_worker.c | 2 ++
6 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 7896a64..2119d9b 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -172,7 +172,7 @@ extern struct ratelimit_state drbd_ratelimit_state;
extern struct idr minors;
extern struct list_head drbd_tconns;
extern struct rw_semaphore drbd_cfg_rwsem;
-/* drbd_cfg_rwsem protects: drbd_tconns list,
+/* drbd_cfg_rwsem protects: drbd_tconns list, minors idr, tconn->volumes idr
note: non sleeping iterations over the idrs are protoected by RCU */
/* on the wire */
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index d48efb4..a2cac15 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2257,8 +2257,10 @@ static void drbd_cleanup(void)
drbd_genl_unregister();
+ down_write(&drbd_cfg_rwsem);
idr_for_each_entry(&minors, mdev, i)
drbd_delete_device(mdev);
+ up_write(&drbd_cfg_rwsem);
drbd_destroy_mempools();
unregister_blkdev(DRBD_MAJOR, "drbd");
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 5667536..331da4f 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -363,8 +363,10 @@ static void conn_md_sync(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&tconn->volumes, mdev, vnr)
drbd_md_sync(mdev);
+ up_read(&drbd_cfg_rwsem);
}
int conn_khelper(struct drbd_tconn *tconn, char *cmd)
@@ -2920,7 +2922,9 @@ int drbd_adm_add_minor(struct sk_buff *skb, struct genl_info *info)
goto out;
}
+ down_write(&drbd_cfg_rwsem);
retcode = conn_new_minor(adm_ctx.tconn, dh->minor, adm_ctx.volume);
+ up_write(&drbd_cfg_rwsem);
out:
drbd_adm_finish(info, retcode);
return 0;
diff --git a/drivers/block/drbd/drbd_proc.c b/drivers/block/drbd/drbd_proc.c
index a4dbdbc..4025d08 100644
--- a/drivers/block/drbd/drbd_proc.c
+++ b/drivers/block/drbd/drbd_proc.c
@@ -227,6 +227,7 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
oos .. known out-of-sync kB
*/
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&minors, mdev, i) {
if (prev_i != i - 1)
seq_printf(seq, "\n");
@@ -293,6 +294,7 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
}
}
}
+ up_read(&drbd_cfg_rwsem);
return 0;
}
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 92ecb22..4d9d71a 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -964,7 +964,10 @@ retry:
if (drbd_send_protocol(tconn) == -EOPNOTSUPP)
return -1;
- return !idr_for_each(&tconn->volumes, drbd_connected, tconn);
+ down_read(&drbd_cfg_rwsem);
+ h = !idr_for_each(&tconn->volumes, drbd_connected, tconn);
+ up_read(&drbd_cfg_rwsem);
+ return h;
out_release_sockets:
if (tconn->data.socket) {
@@ -4084,7 +4087,9 @@ static void drbd_disconnect(struct drbd_tconn *tconn)
drbd_thread_stop(&tconn->asender);
drbd_free_sock(tconn);
+ down_read(&drbd_cfg_rwsem);
idr_for_each(&tconn->volumes, drbd_disconnected, tconn);
+ up_read(&drbd_cfg_rwsem);
conn_info(tconn, "Connection closed\n");
if (conn_highest_role(tconn) == R_PRIMARY && conn_highest_pdsk(tconn) >= D_UNKNOWN)
@@ -4821,10 +4826,14 @@ static int tconn_finish_peer_reqs(struct drbd_tconn *tconn)
do {
clear_bit(SIGNAL_ASENDER, &tconn->flags);
flush_signals(current);
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&tconn->volumes, mdev, i) {
- if (drbd_finish_peer_reqs(mdev))
+ if (drbd_finish_peer_reqs(mdev)) {
+ up_read(&drbd_cfg_rwsem);
return 1; /* error */
+ }
}
+ up_read(&drbd_cfg_rwsem);
set_bit(SIGNAL_ASENDER, &tconn->flags);
spin_lock_irq(&tconn->req_lock);
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index 536eb05..b63c08e 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -1733,12 +1733,14 @@ int drbd_worker(struct drbd_thread *thi)
spin_unlock_irq(&tconn->data.work.q_lock);
drbd_thread_stop(&tconn->receiver);
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
D_ASSERT(mdev->state.disk == D_DISKLESS && mdev->state.conn == C_STANDALONE);
/* _drbd_set_state only uses stop_nowait.
* wait here for the exiting receiver. */
drbd_mdev_cleanup(mdev);
}
+ up_read(&drbd_cfg_rwsem);
clear_bit(OBJECT_DYING, &tconn->flags);
clear_bit(CONFIG_PENDING, &tconn->flags);
wake_up(&tconn->ping_wait);
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 06/11] drbd: Runtime changeable wire protocol
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (4 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 07/11] drbd: Check consistency of net options when the get changed online Philipp Reisner
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
The wire protocol is no longer a property that is negotiated
between the two peers. It is now expressed with two bits
(DP_SEND_WRITE_ACK and DP_SEND_RECEIVE_ACK) in each data
packet. Therefore the primary node is free to change the
wire protocol at any time without disconnect/reconnect.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 6 +++++
drivers/block/drbd/drbd_main.c | 8 ++++++-
drivers/block/drbd/drbd_receiver.c | 39 +++++++++++++++++------------------
drivers/block/drbd/drbd_req.c | 18 ++++++++++++----
drivers/block/drbd/drbd_req.h | 8 +++++++
5 files changed, 53 insertions(+), 26 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 2119d9b..c57cedb 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -327,6 +327,8 @@ extern unsigned int drbd_header_size(struct drbd_tconn *tconn);
#define DP_FUA 16 /* equals REQ_FUA */
#define DP_FLUSH 32 /* equals REQ_FLUSH */
#define DP_DISCARD 64 /* equals REQ_DISCARD */
+#define DP_SEND_RECEIVE_ACK 128 /* This is a proto B write request */
+#define DP_SEND_WRITE_ACK 256 /* This is a proto C write request */
struct p_data {
u64 sector; /* 64 bits sector number */
@@ -656,6 +658,9 @@ enum {
/* Conflicting local requests need to be restarted after this request */
__EE_RESTART_REQUESTS,
+
+ /* The peer wants a write ACK for this (wire proto C) */
+ __EE_SEND_WRITE_ACK,
};
#define EE_CALL_AL_COMPLETE_IO (1<<__EE_CALL_AL_COMPLETE_IO)
#define EE_MAY_SET_IN_SYNC (1<<__EE_MAY_SET_IN_SYNC)
@@ -663,6 +668,7 @@ enum {
#define EE_WAS_ERROR (1<<__EE_WAS_ERROR)
#define EE_HAS_DIGEST (1<<__EE_HAS_DIGEST)
#define EE_RESTART_REQUESTS (1<<__EE_RESTART_REQUESTS)
+#define EE_SEND_WRITE_ACK (1<<__EE_SEND_WRITE_ACK)
/* flag bits per mdev */
enum {
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index a2cac15..6a11e9d 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -1672,6 +1672,12 @@ int drbd_send_dblock(struct drbd_conf *mdev, struct drbd_request *req)
if (mdev->state.conn >= C_SYNC_SOURCE &&
mdev->state.conn <= C_PAUSED_SYNC_T)
dp_flags |= DP_MAY_SET_IN_SYNC;
+ if (mdev->tconn->agreed_pro_version >= 100) {
+ if (req->rq_state & RQ_EXP_RECEIVE_ACK)
+ dp_flags |= DP_SEND_RECEIVE_ACK;
+ if (req->rq_state & RQ_EXP_WRITE_ACK)
+ dp_flags |= DP_SEND_WRITE_ACK;
+ }
p->dp_flags = cpu_to_be32(dp_flags);
if (dgs)
drbd_csum_bio(mdev, mdev->tconn->integrity_w_tfm, req->master_bio, p + 1);
@@ -1688,7 +1694,7 @@ int drbd_send_dblock(struct drbd_conf *mdev, struct drbd_request *req)
* out ok after sending on this side, but does not fit on the
* receiving side, we sure have detected corruption elsewhere.
*/
- if (mdev->tconn->net_conf->wire_protocol == DRBD_PROT_A || dgs)
+ if (!(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK)) || dgs)
err = _drbd_send_bio(mdev, req->master_bio);
else
err = _drbd_send_zc_bio(mdev, req->master_bio);
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 4d9d71a..0b648dc 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1697,7 +1697,7 @@ static int e_end_block(struct drbd_work *w, int cancel)
sector_t sector = peer_req->i.sector;
int err = 0, pcmd;
- if (mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C) {
+ if (peer_req->flags & EE_SEND_WRITE_ACK) {
if (likely((peer_req->flags & EE_WAS_ERROR) == 0)) {
pcmd = (mdev->state.conn >= C_SYNC_SOURCE &&
mdev->state.conn <= C_PAUSED_SYNC_T &&
@@ -2074,20 +2074,28 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
list_add(&peer_req->w.list, &mdev->active_ee);
spin_unlock_irq(&mdev->tconn->req_lock);
- switch (mdev->tconn->net_conf->wire_protocol) {
- case DRBD_PROT_C:
+ if (mdev->tconn->agreed_pro_version < 100) {
+ switch (mdev->tconn->net_conf->wire_protocol) {
+ case DRBD_PROT_C:
+ dp_flags |= DP_SEND_WRITE_ACK;
+ break;
+ case DRBD_PROT_B:
+ dp_flags |= DP_SEND_RECEIVE_ACK;
+ break;
+ }
+ }
+
+ if (dp_flags & DP_SEND_WRITE_ACK) {
+ peer_req->flags |= EE_SEND_WRITE_ACK;
inc_unacked(mdev);
/* corresponding dec_unacked() in e_end_block()
* respective _drbd_clear_done_ee */
- break;
- case DRBD_PROT_B:
+ }
+
+ if (dp_flags & DP_SEND_RECEIVE_ACK) {
/* I really don't like it that the receiver thread
* sends on the msock, but anyways */
drbd_send_ack(mdev, P_RECV_ACK, peer_req);
- break;
- case DRBD_PROT_A:
- /* nothing to do */
- break;
}
if (mdev->state.pdsk < D_INCONSISTENT) {
@@ -2932,7 +2940,7 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
if (cf & CF_DRY_RUN)
set_bit(CONN_DRY_RUN, &tconn->flags);
- if (p_proto != tconn->net_conf->wire_protocol) {
+ if (p_proto != tconn->net_conf->wire_protocol && tconn->agreed_pro_version < 100) {
conn_err(tconn, "incompatible communication protocols\n");
goto disconnect;
}
@@ -4622,23 +4630,18 @@ static int got_BlockAck(struct drbd_tconn *tconn, struct packet_info *pi)
}
switch (pi->cmd) {
case P_RS_WRITE_ACK:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = WRITE_ACKED_BY_PEER_AND_SIS;
break;
case P_WRITE_ACK:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = WRITE_ACKED_BY_PEER;
break;
case P_RECV_ACK:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_B);
what = RECV_ACKED_BY_PEER;
break;
case P_DISCARD_WRITE:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = DISCARD_WRITE;
break;
case P_RETRY_WRITE:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = POSTPONE_WRITE;
break;
default:
@@ -4656,8 +4659,6 @@ static int got_NegAck(struct drbd_tconn *tconn, struct packet_info *pi)
struct p_block_ack *p = pi->data;
sector_t sector = be64_to_cpu(p->sector);
int size = be32_to_cpu(p->blksize);
- bool missing_ok = tconn->net_conf->wire_protocol == DRBD_PROT_A ||
- tconn->net_conf->wire_protocol == DRBD_PROT_B;
int err;
mdev = vnr_to_mdev(tconn, pi->vnr);
@@ -4674,15 +4675,13 @@ static int got_NegAck(struct drbd_tconn *tconn, struct packet_info *pi)
err = validate_req_change_req_state(mdev, p->block_id, sector,
&mdev->write_requests, __func__,
- NEG_ACKED, missing_ok);
+ NEG_ACKED, true);
if (err) {
/* Protocol A has no P_WRITE_ACKs, but has P_NEG_ACKs.
The master bio might already be completed, therefore the
request is no longer in the collision hash. */
/* In Protocol B we might already have got a P_RECV_ACK
but then get a P_NEG_ACK afterwards. */
- if (!missing_ok)
- return err;
drbd_set_out_of_sync(mdev, sector, size);
}
return 0;
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index fd0b452..0f1a29f 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -323,7 +323,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
struct bio_and_error *m)
{
struct drbd_conf *mdev = req->w.mdev;
- int rv = 0;
+ int p, rv = 0;
if (m)
m->bio = NULL;
@@ -344,6 +344,10 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
* and from w_read_retry_remote */
D_ASSERT(!(req->rq_state & RQ_NET_MASK));
req->rq_state |= RQ_NET_PENDING;
+ p = mdev->tconn->net_conf->wire_protocol;
+ req->rq_state |=
+ p == DRBD_PROT_C ? RQ_EXP_WRITE_ACK :
+ p == DRBD_PROT_B ? RQ_EXP_RECEIVE_ACK : 0;
inc_ap_pending(mdev);
break;
@@ -500,7 +504,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
atomic_add(req->i.size >> 9, &mdev->ap_in_flight);
if (bio_data_dir(req->master_bio) == WRITE &&
- mdev->tconn->net_conf->wire_protocol == DRBD_PROT_A) {
+ !(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK))) {
/* this is what is dangerous about protocol A:
* pretend it was successfully written on the peer. */
if (req->rq_state & RQ_NET_PENDING) {
@@ -550,6 +554,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
req->rq_state |= RQ_NET_DONE;
/* fall through */
case WRITE_ACKED_BY_PEER:
+ D_ASSERT(req->rq_state & RQ_EXP_WRITE_ACK);
/* protocol C; successfully written on peer.
* Nothing to do here.
* We want to keep the tl in place for all protocols, to cater
@@ -560,11 +565,14 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
* request could set NET_DONE right here, and not wait for the
* P_BARRIER_ACK, but that is an unnecessary optimization. */
+ goto ack_common;
/* this makes it effectively the same as for: */
case RECV_ACKED_BY_PEER:
+ D_ASSERT(req->rq_state & RQ_EXP_RECEIVE_ACK);
/* protocol B; pretends to be successfully written on peer.
* see also notes above in HANDED_OVER_TO_NETWORK about
* protocol != C */
+ ack_common:
req->rq_state |= RQ_NET_OK;
D_ASSERT(req->rq_state & RQ_NET_PENDING);
dec_ap_pending(mdev);
@@ -574,8 +582,8 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
break;
case POSTPONE_WRITE:
- /*
- * If this node has already detected the write conflict, the
+ D_ASSERT(req->rq_state & RQ_EXP_WRITE_ACK);
+ /* If this node has already detected the write conflict, the
* worker will be waiting on misc_wait. Wake it up once this
* request has completed locally.
*/
@@ -646,7 +654,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
}
if ((req->rq_state & RQ_NET_MASK) != 0) {
req->rq_state |= RQ_NET_DONE;
- if (mdev->tconn->net_conf->wire_protocol == DRBD_PROT_A)
+ if (!(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK)))
atomic_sub(req->i.size>>9, &mdev->ap_in_flight);
}
_req_may_be_done(req, m); /* Allowed while state.susp */
diff --git a/drivers/block/drbd/drbd_req.h b/drivers/block/drbd/drbd_req.h
index b9f474a..dfd2cef 100644
--- a/drivers/block/drbd/drbd_req.h
+++ b/drivers/block/drbd/drbd_req.h
@@ -198,6 +198,12 @@ enum drbd_req_state_bits {
/* The peer has sent a retry ACK */
__RQ_POSTPONED,
+
+ /* We expect a receive ACK (wire proto B) */
+ __RQ_EXP_RECEIVE_ACK,
+
+ /* We expect a write ACK (wite proto C) */
+ __RQ_EXP_WRITE_ACK,
};
#define RQ_LOCAL_PENDING (1UL << __RQ_LOCAL_PENDING)
@@ -219,6 +225,8 @@ enum drbd_req_state_bits {
#define RQ_WRITE (1UL << __RQ_WRITE)
#define RQ_IN_ACT_LOG (1UL << __RQ_IN_ACT_LOG)
#define RQ_POSTPONED (1UL << __RQ_POSTPONED)
+#define RQ_EXP_RECEIVE_ACK (1UL << __RQ_EXP_RECEIVE_ACK)
+#define RQ_EXP_WRITE_ACK (1UL << __RQ_EXP_WRITE_ACK)
/* For waking up the frozen transfer log mod_req() has to return if the request
should be counted in the epoch object*/
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 07/11] drbd: Check consistency of net options when the get changed online
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (5 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 06/11] drbd: Runtime changeable wire protocol Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100 Philipp Reisner
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_nl.c | 79 ++++++++++++++++++++++++------------------
1 files changed, 45 insertions(+), 34 deletions(-)
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 331da4f..0bf1c2e 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -1705,6 +1705,45 @@ static bool conn_ov_running(struct drbd_tconn *tconn)
return rv;
}
+static enum drbd_ret_code
+check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
+{
+ struct drbd_conf *mdev;
+ int i;
+
+ if (new_conf->two_primaries &&
+ (new_conf->wire_protocol != DRBD_PROT_C))
+ return ERR_NOT_PROTO_C;
+
+ rcu_read_lock();
+ idr_for_each_entry(&tconn->volumes, mdev, i) {
+ if (get_ldev(mdev)) {
+ enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
+ put_ldev(mdev);
+ if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
+ rcu_read_unlock();
+ return ERR_STONITH_AND_PROT_A;
+ }
+ }
+ if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
+ rcu_read_unlock();
+ return ERR_DISCARD;
+ }
+ if (!mdev->bitmap) {
+ if(drbd_bm_init(mdev)) {
+ rcu_read_unlock();
+ return ERR_NOMEM;
+ }
+ }
+ }
+ rcu_read_unlock();
+
+ if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A)
+ return ERR_CONG_NOT_PROTO_A;
+
+ return NO_ERROR;
+}
+
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
{
enum drbd_ret_code retcode;
@@ -1763,6 +1802,10 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
+ retcode = check_net_options(tconn, new_conf);
+ if (retcode != NO_ERROR)
+ goto fail;
+
/* re-sync running */
rsr = conn_resync_running(tconn);
if (rsr && strcmp(new_conf->csums_alg, tconn->net_conf->csums_alg)) {
@@ -1922,39 +1965,9 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
- if (new_conf->two_primaries
- && (new_conf->wire_protocol != DRBD_PROT_C)) {
- retcode = ERR_NOT_PROTO_C;
- goto fail;
- }
-
- rcu_read_lock();
- idr_for_each_entry(&tconn->volumes, mdev, i) {
- if (get_ldev(mdev)) {
- enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
- put_ldev(mdev);
- if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
- retcode = ERR_STONITH_AND_PROT_A;
- goto fail_rcu_unlock;
- }
- }
- if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
- retcode = ERR_DISCARD;
- goto fail_rcu_unlock;
- }
- if (!mdev->bitmap) {
- if(drbd_bm_init(mdev)) {
- retcode = ERR_NOMEM;
- goto fail_rcu_unlock;
- }
- }
- }
- rcu_read_unlock();
-
- if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A) {
- retcode = ERR_CONG_NOT_PROTO_A;
+ retcode = check_net_options(tconn, new_conf);
+ if (retcode != NO_ERROR)
goto fail;
- }
retcode = NO_ERROR;
@@ -2074,8 +2087,6 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
drbd_adm_finish(info, retcode);
return 0;
-fail_rcu_unlock:
- rcu_read_unlock();
fail:
kfree(int_dig_in);
kfree(int_dig_vv);
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (6 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 07/11] drbd: Check consistency of net options when the get changed online Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf Philipp Reisner
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_nl.c | 5 +++++
include/linux/drbd.h | 1 +
2 files changed, 6 insertions(+), 0 deletions(-)
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 0bf1c2e..bb41af1 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -1711,6 +1711,11 @@ check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
struct drbd_conf *mdev;
int i;
+ if (tconn->net_conf && tconn->agreed_pro_version < 100 &&
+ tconn->cstate == C_WF_REPORT_PARAMS &&
+ new_conf->wire_protocol != tconn->net_conf->wire_protocol)
+ return ERR_NEED_APV_100;
+
if (new_conf->two_primaries &&
(new_conf->wire_protocol != DRBD_PROT_C))
return ERR_NOT_PROTO_C;
diff --git a/include/linux/drbd.h b/include/linux/drbd.h
index fe8d6ba..6c7c85d 100644
--- a/include/linux/drbd.h
+++ b/include/linux/drbd.h
@@ -160,6 +160,7 @@ enum drbd_ret_code {
ERR_MINOR_CONFIGURED = 160,
ERR_MINOR_EXISTS = 161,
ERR_INVALID_REQUEST = 162,
+ ERR_NEED_APV_100 = 163,
/* insert new ones above this line */
AFTER_LAST_ERR_CODE
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (7 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100 Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU Philipp Reisner
2011-09-30 12:48 ` [PATCH 11/11] drbd: Considering that the two_primaries config flag can change Philipp Reisner
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Removing the get_net_conf()/put_net_conf() calls
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 15 ++-
drivers/block/drbd/drbd_main.c | 78 ++++++++----
drivers/block/drbd/drbd_nl.c | 171 +++++++++++++++-----------
drivers/block/drbd/drbd_proc.c | 9 +-
drivers/block/drbd/drbd_receiver.c | 245 ++++++++++++++++++++++++------------
drivers/block/drbd/drbd_req.c | 40 ++++--
drivers/block/drbd/drbd_state.c | 14 ++-
drivers/block/drbd/drbd_worker.c | 26 +++-
8 files changed, 389 insertions(+), 209 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index c57cedb..99da54c 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -832,7 +832,7 @@ struct drbd_tconn { /* is a resource from the config file */
struct mutex cstate_mutex; /* Protects graceful disconnects */
unsigned long flags;
- struct net_conf *net_conf; /* protected by get_net_conf() and put_net_conf() */
+ struct net_conf *net_conf; /* content protected by rcu */
atomic_t net_cnt; /* Users of net_conf */
wait_queue_head_t net_cnt_wait;
wait_queue_head_t ping_wait; /* Woken upon reception of a ping, and a state change */
@@ -2059,11 +2059,14 @@ static inline void drbd_get_syncer_progress(struct drbd_conf *mdev,
* maybe re-implement using semaphores? */
static inline int drbd_get_max_buffers(struct drbd_conf *mdev)
{
- int mxb = 1000000; /* arbitrary limit on open requests */
- if (get_net_conf(mdev->tconn)) {
- mxb = mdev->tconn->net_conf->max_buffers;
- put_net_conf(mdev->tconn);
- }
+ struct net_conf *nc;
+ int mxb;
+
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ mxb = nc ? nc->max_buffers : 1000000; /* arbitrary limit on open requests */
+ rcu_read_unlock();
+
return mxb;
}
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 6a11e9d..d335ecc 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -843,15 +843,19 @@ int drbd_send_sync_param(struct drbd_conf *mdev)
int size;
const int apv = mdev->tconn->agreed_pro_version;
enum drbd_packet cmd;
+ struct net_conf *nc;
sock = &mdev->tconn->data;
p = drbd_prepare_command(mdev, sock);
if (!p)
return -EIO;
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+
size = apv <= 87 ? sizeof(struct p_rs_param)
: apv == 88 ? sizeof(struct p_rs_param)
- + strlen(mdev->tconn->net_conf->verify_alg) + 1
+ + strlen(nc->verify_alg) + 1
: apv <= 94 ? sizeof(struct p_rs_param_89)
: /* apv >= 95 */ sizeof(struct p_rs_param_95);
@@ -876,9 +880,10 @@ int drbd_send_sync_param(struct drbd_conf *mdev)
}
if (apv >= 88)
- strcpy(p->verify_alg, mdev->tconn->net_conf->verify_alg);
+ strcpy(p->verify_alg, nc->verify_alg);
if (apv >= 89)
- strcpy(p->csums_alg, mdev->tconn->net_conf->csums_alg);
+ strcpy(p->csums_alg, nc->csums_alg);
+ rcu_read_unlock();
return drbd_send_command(mdev, sock, cmd, size, NULL, 0);
}
@@ -887,36 +892,44 @@ int drbd_send_protocol(struct drbd_tconn *tconn)
{
struct drbd_socket *sock;
struct p_protocol *p;
+ struct net_conf *nc;
int size, cf;
- if (tconn->net_conf->dry_run && tconn->agreed_pro_version < 92) {
- conn_err(tconn, "--dry-run is not supported by peer");
- return -EOPNOTSUPP;
- }
-
sock = &tconn->data;
p = conn_prepare_command(tconn, sock);
if (!p)
return -EIO;
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+
+ if (nc->dry_run && tconn->agreed_pro_version < 92) {
+ rcu_read_unlock();
+ mutex_unlock(&sock->mutex);
+ conn_err(tconn, "--dry-run is not supported by peer");
+ return -EOPNOTSUPP;
+ }
+
size = sizeof(*p);
if (tconn->agreed_pro_version >= 87)
- size += strlen(tconn->net_conf->integrity_alg) + 1;
+ size += strlen(nc->integrity_alg) + 1;
- p->protocol = cpu_to_be32(tconn->net_conf->wire_protocol);
- p->after_sb_0p = cpu_to_be32(tconn->net_conf->after_sb_0p);
- p->after_sb_1p = cpu_to_be32(tconn->net_conf->after_sb_1p);
- p->after_sb_2p = cpu_to_be32(tconn->net_conf->after_sb_2p);
- p->two_primaries = cpu_to_be32(tconn->net_conf->two_primaries);
+ p->protocol = cpu_to_be32(nc->wire_protocol);
+ p->after_sb_0p = cpu_to_be32(nc->after_sb_0p);
+ p->after_sb_1p = cpu_to_be32(nc->after_sb_1p);
+ p->after_sb_2p = cpu_to_be32(nc->after_sb_2p);
+ p->two_primaries = cpu_to_be32(nc->two_primaries);
cf = 0;
- if (tconn->net_conf->want_lose)
+ if (nc->want_lose)
cf |= CF_WANT_LOSE;
- if (tconn->net_conf->dry_run)
+ if (nc->dry_run)
cf |= CF_DRY_RUN;
p->conn_flags = cpu_to_be32(cf);
if (tconn->agreed_pro_version >= 87)
- strcpy(p->integrity_alg, tconn->net_conf->integrity_alg);
+ strcpy(p->integrity_alg, nc->integrity_alg);
+ rcu_read_unlock();
+
return conn_send_command(tconn, sock, P_PROTOCOL, size, NULL, 0);
}
@@ -938,7 +951,9 @@ int _drbd_send_uuids(struct drbd_conf *mdev, u64 uuid_flags)
mdev->comm_bm_set = drbd_bm_total_weight(mdev);
p->uuid[UI_SIZE] = cpu_to_be64(mdev->comm_bm_set);
- uuid_flags |= mdev->tconn->net_conf->want_lose ? 1 : 0;
+ rcu_read_lock();
+ uuid_flags |= rcu_dereference(mdev->tconn->net_conf)->want_lose ? 1 : 0;
+ rcu_read_unlock();
uuid_flags |= test_bit(CRASHED_PRIMARY, &mdev->flags) ? 2 : 0;
uuid_flags |= mdev->new_state_tmp.disk == D_INCONSISTENT ? 4 : 0;
p->uuid[UI_FLAGS] = cpu_to_be64(uuid_flags);
@@ -1134,12 +1149,14 @@ int fill_bitmap_rle_bits(struct drbd_conf *mdev,
unsigned long rl;
unsigned len;
unsigned toggle;
- int bits;
+ int bits, use_rle;
/* may we use this feature? */
- if ((mdev->tconn->net_conf->use_rle == 0) ||
- (mdev->tconn->agreed_pro_version < 90))
- return 0;
+ rcu_read_lock();
+ use_rle = rcu_dereference(mdev->tconn->net_conf)->use_rle;
+ rcu_read_unlock();
+ if (!use_rle || mdev->tconn->agreed_pro_version < 90)
+ return 0;
if (c->bit_offset >= c->bm_bits)
return 0; /* nothing to do. */
@@ -1803,7 +1820,9 @@ int drbd_send(struct drbd_tconn *tconn, struct socket *sock,
msg.msg_flags = msg_flags | MSG_NOSIGNAL;
if (sock == tconn->data.socket) {
- tconn->ko_count = tconn->net_conf->ko_count;
+ rcu_read_lock();
+ tconn->ko_count = rcu_dereference(tconn->net_conf)->ko_count;
+ rcu_read_unlock();
drbd_update_congested(tconn);
}
do {
@@ -3226,15 +3245,18 @@ const char *cmdname(enum drbd_packet cmd)
*/
int drbd_wait_misc(struct drbd_conf *mdev, struct drbd_interval *i)
{
- struct net_conf *net_conf = mdev->tconn->net_conf;
+ struct net_conf *nc;
DEFINE_WAIT(wait);
long timeout;
- if (!net_conf)
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (!nc) {
+ rcu_read_unlock();
return -ETIMEDOUT;
- timeout = MAX_SCHEDULE_TIMEOUT;
- if (net_conf->ko_count)
- timeout = net_conf->timeout * HZ / 10 * net_conf->ko_count;
+ }
+ timeout = nc->ko_count ? nc->timeout * HZ / 10 * nc->ko_count : MAX_SCHEDULE_TIMEOUT;
+ rcu_read_unlock();
/* Indicate to wake up mdev->misc_wait on progress. */
i->waiting = true;
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index bb41af1..6d28e7c 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -290,29 +290,32 @@ static int drbd_adm_finish(struct genl_info *info, int retcode)
static void setup_khelper_env(struct drbd_tconn *tconn, char **envp)
{
char af[20], ad[60], *afs;
+ struct net_conf *nc;
- if (get_net_conf(tconn)) {
- switch (((struct sockaddr *)tconn->net_conf->peer_addr)->sa_family) {
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ if (nc) {
+ switch (((struct sockaddr *)nc->peer_addr)->sa_family) {
case AF_INET6:
afs = "ipv6";
snprintf(ad, 60, "DRBD_PEER_ADDRESS=%pI6",
- &((struct sockaddr_in6 *)tconn->net_conf->peer_addr)->sin6_addr);
+ &((struct sockaddr_in6 *)nc->peer_addr)->sin6_addr);
break;
case AF_INET:
afs = "ipv4";
snprintf(ad, 60, "DRBD_PEER_ADDRESS=%pI4",
- &((struct sockaddr_in *)tconn->net_conf->peer_addr)->sin_addr);
+ &((struct sockaddr_in *)nc->peer_addr)->sin_addr);
break;
default:
afs = "ssocks";
snprintf(ad, 60, "DRBD_PEER_ADDRESS=%pI4",
- &((struct sockaddr_in *)tconn->net_conf->peer_addr)->sin_addr);
+ &((struct sockaddr_in *)nc->peer_addr)->sin_addr);
}
snprintf(af, 20, "DRBD_PEER_AF=%s", afs);
envp[3]=af;
envp[4]=ad;
- put_net_conf(tconn);
}
+ rcu_read_unlock();
}
int drbd_khelper(struct drbd_conf *mdev, char *cmd)
@@ -519,6 +522,7 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
{
const int max_tries = 4;
enum drbd_state_rv rv = SS_UNKNOWN_ERROR;
+ struct net_conf *nc;
int try = 0;
int forced = 0;
union drbd_state mask, val;
@@ -576,7 +580,12 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
if (rv == SS_TWO_PRIMARIES) {
/* Maybe the peer is detected as dead very soon...
retry at most once more in this case. */
- schedule_timeout_interruptible((mdev->tconn->net_conf->ping_timeo+1)*HZ/10);
+ int timeo;
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ timeo = nc ? (nc->ping_timeo + 1) * HZ / 10 : 1;
+ rcu_read_unlock();
+ schedule_timeout_interruptible(timeo);
if (try < max_tries)
try = max_tries - 1;
continue;
@@ -606,10 +615,11 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
put_ldev(mdev);
}
} else {
- if (get_net_conf(mdev->tconn)) {
- mdev->tconn->net_conf->want_lose = 0;
- put_net_conf(mdev->tconn);
- }
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc)
+ nc->want_lose = 0;
+ rcu_read_unlock();
set_disk_ro(mdev->vdisk, false);
if (get_ldev(mdev)) {
if (((mdev->state.conn < C_CONNECTED ||
@@ -1233,6 +1243,7 @@ int drbd_adm_attach(struct sk_buff *skb, struct genl_info *info)
struct lru_cache *resync_lru = NULL;
union drbd_state ns, os;
enum drbd_state_rv rv;
+ struct net_conf *nc;
int cp_discovered = 0;
retcode = drbd_adm_prepare(skb, info, DRBD_ADM_NEED_MINOR);
@@ -1296,14 +1307,16 @@ int drbd_adm_attach(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
- if (get_net_conf(mdev->tconn)) {
- int prot = mdev->tconn->net_conf->wire_protocol;
- put_net_conf(mdev->tconn);
- if (nbc->dc.fencing == FP_STONITH && prot == DRBD_PROT_A) {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc) {
+ if (nbc->dc.fencing == FP_STONITH && nc->wire_protocol == DRBD_PROT_A) {
+ rcu_read_unlock();
retcode = ERR_STONITH_AND_PROT_A;
goto fail;
}
}
+ rcu_read_unlock();
bdev = blkdev_get_by_path(nbc->dc.backing_dev,
FMODE_READ | FMODE_WRITE | FMODE_EXCL, mdev);
@@ -1706,42 +1719,35 @@ static bool conn_ov_running(struct drbd_tconn *tconn)
}
static enum drbd_ret_code
-check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
+_check_net_options(struct drbd_tconn *tconn, struct net_conf *old_conf, struct net_conf *new_conf)
{
struct drbd_conf *mdev;
int i;
- if (tconn->net_conf && tconn->agreed_pro_version < 100 &&
+ if (old_conf && tconn->agreed_pro_version < 100 &&
tconn->cstate == C_WF_REPORT_PARAMS &&
- new_conf->wire_protocol != tconn->net_conf->wire_protocol)
+ new_conf->wire_protocol != old_conf->wire_protocol)
return ERR_NEED_APV_100;
if (new_conf->two_primaries &&
(new_conf->wire_protocol != DRBD_PROT_C))
return ERR_NOT_PROTO_C;
- rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
if (get_ldev(mdev)) {
enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
put_ldev(mdev);
- if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
- rcu_read_unlock();
+ if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH)
return ERR_STONITH_AND_PROT_A;
- }
}
- if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
- rcu_read_unlock();
+ if (mdev->state.role == R_PRIMARY && new_conf->want_lose)
return ERR_DISCARD;
- }
+
if (!mdev->bitmap) {
- if(drbd_bm_init(mdev)) {
- rcu_read_unlock();
+ if(drbd_bm_init(mdev))
return ERR_NOMEM;
- }
}
}
- rcu_read_unlock();
if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A)
return ERR_CONG_NOT_PROTO_A;
@@ -1749,11 +1755,23 @@ check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
return NO_ERROR;
}
+static enum drbd_ret_code
+check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
+{
+ static enum drbd_ret_code rv;
+
+ rcu_read_lock();
+ rv = _check_net_options(tconn, rcu_dereference(tconn->net_conf), new_conf);
+ rcu_read_unlock();
+
+ return rv;
+}
+
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
{
enum drbd_ret_code retcode;
struct drbd_tconn *tconn;
- struct net_conf *new_conf = NULL;
+ struct net_conf *old_conf, *new_conf = NULL;
int err;
int ovr; /* online verify running */
int rsr; /* re-sync running */
@@ -1789,17 +1807,20 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
goto out;
}
- /* we also need a net config
- * to change the options on */
- if (!get_net_conf(tconn)) {
+ conn_reconfig_start(tconn);
+
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+
+ if (!old_conf) {
drbd_msg_put_info("net conf missing, try connect");
retcode = ERR_INVALID_REQUEST;
- goto out;
+ goto fail_rcu_unlock;
}
- conn_reconfig_start(tconn);
+ *new_conf = *old_conf;
+ rcu_read_unlock();
- memcpy(new_conf, tconn->net_conf, sizeof(*new_conf));
err = net_conf_from_attrs(new_conf, info);
if (err) {
retcode = ERR_MANDATORY_TAG;
@@ -1813,10 +1834,13 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* re-sync running */
rsr = conn_resync_running(tconn);
- if (rsr && strcmp(new_conf->csums_alg, tconn->net_conf->csums_alg)) {
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+ if (rsr && old_conf && strcmp(new_conf->csums_alg, old_conf->csums_alg)) {
retcode = ERR_CSUMS_RESYNC_RUNNING;
- goto fail;
+ goto fail_rcu_unlock;
}
+ rcu_read_unlock();
if (!rsr && new_conf->csums_alg[0]) {
csums_tfm = crypto_alloc_hash(new_conf->csums_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1834,12 +1858,15 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* online verify running */
ovr = conn_ov_running(tconn);
- if (ovr) {
- if (strcmp(new_conf->verify_alg, tconn->net_conf->verify_alg)) {
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+ if (ovr && old_conf) {
+ if (strcmp(new_conf->verify_alg, old_conf->verify_alg)) {
retcode = ERR_VERIFY_RUNNING;
- goto fail;
+ goto fail_rcu_unlock;
}
}
+ rcu_read_unlock();
if (!ovr && new_conf->verify_alg[0]) {
verify_tfm = crypto_alloc_hash(new_conf->verify_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1855,16 +1882,9 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
}
}
-
- /* For now, use struct assignment, not pointer assignment.
- * We don't have any means to determine who might still
- * keep a local alias into the struct,
- * so we cannot just free it and hope for the best :(
- * FIXME
- * To avoid someone looking at a half-updated struct, we probably
- * should have a rw-semaphor on net_conf and disk_conf.
- */
- *tconn->net_conf = *new_conf;
+ rcu_assign_pointer(tconn->net_conf, new_conf);
+ synchronize_rcu();
+ kfree(old_conf);
if (!rsr) {
crypto_free_hash(tconn->csums_tfm);
@@ -1880,11 +1900,12 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
if (tconn->cstate >= C_WF_REPORT_PARAMS)
drbd_send_sync_param(minor_to_mdev(conn_lowest_minor(tconn)));
+ fail_rcu_unlock:
+ rcu_read_unlock();
fail:
crypto_free_hash(csums_tfm);
crypto_free_hash(verify_tfm);
kfree(new_conf);
- put_net_conf(tconn);
conn_reconfig_done(tconn);
out:
drbd_adm_finish(info, retcode);
@@ -1895,7 +1916,7 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
{
char hmac_name[CRYPTO_MAX_ALG_NAME];
struct drbd_conf *mdev;
- struct net_conf *new_conf = NULL;
+ struct net_conf *old_conf, *new_conf = NULL;
struct crypto_hash *tfm = NULL;
struct crypto_hash *integrity_w_tfm = NULL;
struct crypto_hash *integrity_r_tfm = NULL;
@@ -1983,23 +2004,26 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
* strictly serialized on genl_lock(). We are protected against
* concurrent reconfiguration/addition/deletion */
list_for_each_entry(oconn, &drbd_tconns, all_tconn) {
+ struct net_conf *nc;
if (oconn == tconn)
continue;
- if (get_net_conf(oconn)) {
- taken_addr = (struct sockaddr *)&oconn->net_conf->my_addr;
- if (new_conf->my_addr_len == oconn->net_conf->my_addr_len &&
+
+ rcu_read_lock();
+ nc = rcu_dereference(oconn->net_conf);
+ if (nc) {
+ taken_addr = (struct sockaddr *)&nc->my_addr;
+ if (new_conf->my_addr_len == nc->my_addr_len &&
!memcmp(new_my_addr, taken_addr, new_conf->my_addr_len))
retcode = ERR_LOCAL_ADDR;
- taken_addr = (struct sockaddr *)&oconn->net_conf->peer_addr;
- if (new_conf->peer_addr_len == oconn->net_conf->peer_addr_len &&
+ taken_addr = (struct sockaddr *)&nc->peer_addr;
+ if (new_conf->peer_addr_len == nc->peer_addr_len &&
!memcmp(new_peer_addr, taken_addr, new_conf->peer_addr_len))
retcode = ERR_PEER_ADDR;
-
- put_net_conf(oconn);
- if (retcode != NO_ERROR)
- goto fail;
}
+ rcu_read_unlock();
+ if (retcode != NO_ERROR)
+ goto fail;
}
if (new_conf->cram_hmac_alg[0] != 0) {
@@ -2058,12 +2082,15 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
conn_flush_workqueue(tconn);
spin_lock_irq(&tconn->req_lock);
- if (tconn->net_conf != NULL) {
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+ if (old_conf != NULL) {
retcode = ERR_NET_CONFIGURED;
+ rcu_read_unlock();
spin_unlock_irq(&tconn->req_lock);
goto fail;
}
- tconn->net_conf = new_conf;
+ rcu_assign_pointer(tconn->net_conf, new_conf);
crypto_free_hash(tconn->cram_hmac_tfm);
tconn->cram_hmac_tfm = tfm;
@@ -2518,9 +2545,9 @@ int nla_put_status_info(struct sk_buff *skb, struct drbd_conf *mdev,
const struct sib_info *sib)
{
struct state_info *si = NULL; /* for sizeof(si->member); */
+ struct net_conf *nc;
struct nlattr *nla;
int got_ldev;
- int got_net;
int err = 0;
int exclude_sensitive;
@@ -2538,7 +2565,6 @@ int nla_put_status_info(struct sk_buff *skb, struct drbd_conf *mdev,
exclude_sensitive = sib || !capable(CAP_SYS_ADMIN);
got_ldev = get_ldev(mdev);
- got_net = get_net_conf(mdev->tconn);
/* We need to add connection name and volume number information still.
* Minor number is in drbd_genlmsghdr. */
@@ -2551,9 +2577,14 @@ int nla_put_status_info(struct sk_buff *skb, struct drbd_conf *mdev,
if (got_ldev)
if (disk_conf_to_skb(skb, &mdev->ldev->dc, exclude_sensitive))
goto nla_put_failure;
- if (got_net)
- if (net_conf_to_skb(skb, mdev->tconn->net_conf, exclude_sensitive))
- goto nla_put_failure;
+
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc)
+ err = net_conf_to_skb(skb, nc, exclude_sensitive);
+ rcu_read_unlock();
+ if (err)
+ goto nla_put_failure;
nla = nla_nest_start(skb, DRBD_NLA_STATE_INFO);
if (!nla)
@@ -2600,8 +2631,6 @@ nla_put_failure:
err = -EMSGSIZE;
if (got_ldev)
put_ldev(mdev);
- if (got_net)
- put_net_conf(mdev->tconn);
return err;
}
diff --git a/drivers/block/drbd/drbd_proc.c b/drivers/block/drbd/drbd_proc.c
index 4025d08..792a71e 100644
--- a/drivers/block/drbd/drbd_proc.c
+++ b/drivers/block/drbd/drbd_proc.c
@@ -197,6 +197,8 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
int i, prev_i = -1;
const char *sn;
struct drbd_conf *mdev;
+ struct net_conf *nc;
+ char wp;
static char write_ordering_chars[] = {
[WO_none] = 'n',
@@ -240,6 +242,10 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
mdev->state.role == R_SECONDARY) {
seq_printf(seq, "%2d: cs:Unconfigured\n", i);
} else {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ wp = nc ? nc->wire_protocol - DRBD_PROT_A + 'A' : ' ';
+ rcu_read_unlock();
seq_printf(seq,
"%2d: cs:%s ro:%s/%s ds:%s/%s %c %c%c%c%c%c%c\n"
" ns:%u nr:%u dw:%u dr:%u al:%u bm:%u "
@@ -249,8 +255,7 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
drbd_role_str(mdev->state.peer),
drbd_disk_str(mdev->state.disk),
drbd_disk_str(mdev->state.pdsk),
- (mdev->tconn->net_conf == NULL ? ' ' :
- (mdev->tconn->net_conf->wire_protocol - DRBD_PROT_A+'A')),
+ wp,
drbd_suspended(mdev) ? 's' : 'r',
mdev->state.aftr_isp ? 'a' : '-',
mdev->state.peer_isp ? 'p' : '-',
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 0b648dc..a37f2ee 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -244,11 +244,18 @@ struct page *drbd_alloc_pages(struct drbd_conf *mdev, unsigned int number,
bool retry)
{
struct page *page = NULL;
+ struct net_conf *nc;
DEFINE_WAIT(wait);
+ int mxb;
/* Yes, we may run up to @number over max_buffers. If we
* follow it strictly, the admin will get it wrong anyways. */
- if (atomic_read(&mdev->pp_in_use) < mdev->tconn->net_conf->max_buffers)
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ mxb = nc ? nc->max_buffers : 1000000;
+ rcu_read_unlock();
+
+ if (atomic_read(&mdev->pp_in_use) < mxb)
page = __drbd_alloc_pages(mdev, number);
while (page == NULL) {
@@ -256,7 +263,7 @@ struct page *drbd_alloc_pages(struct drbd_conf *mdev, unsigned int number,
drbd_kick_lo_and_reclaim_net(mdev);
- if (atomic_read(&mdev->pp_in_use) < mdev->tconn->net_conf->max_buffers) {
+ if (atomic_read(&mdev->pp_in_use) < mxb) {
page = __drbd_alloc_pages(mdev, number);
if (page)
break;
@@ -607,24 +614,47 @@ static struct socket *drbd_try_connect(struct drbd_tconn *tconn)
const char *what;
struct socket *sock;
struct sockaddr_in6 src_in6;
- int err;
+ struct sockaddr_in6 peer_in6;
+ struct net_conf *nc;
+ int err, peer_addr_len, my_addr_len;
+ int sndbuf_size, rcvbuf_size, try_connect_int;
int disconnect_on_error = 1;
- if (!get_net_conf(tconn))
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ if (!nc) {
+ rcu_read_unlock();
return NULL;
+ }
+
+ sndbuf_size = nc->sndbuf_size;
+ rcvbuf_size = nc->rcvbuf_size;
+ try_connect_int = nc->try_connect_int;
+
+ my_addr_len = min_t(int, nc->my_addr_len, sizeof(src_in6));
+ memcpy(&src_in6, nc->my_addr, my_addr_len);
+
+ if (((struct sockaddr *)nc->my_addr)->sa_family == AF_INET6)
+ src_in6.sin6_port = 0;
+ else
+ ((struct sockaddr_in *)&src_in6)->sin_port = 0; /* AF_INET & AF_SCI */
+
+ peer_addr_len = min_t(int, nc->peer_addr_len, sizeof(src_in6));
+ memcpy(&peer_in6, nc->peer_addr, peer_addr_len);
+
+ rcu_read_unlock();
what = "sock_create_kern";
- err = sock_create_kern(((struct sockaddr *)tconn->net_conf->my_addr)->sa_family,
- SOCK_STREAM, IPPROTO_TCP, &sock);
+ err = sock_create_kern(((struct sockaddr *)&src_in6)->sa_family,
+ SOCK_STREAM, IPPROTO_TCP, &sock);
if (err < 0) {
sock = NULL;
goto out;
}
sock->sk->sk_rcvtimeo =
- sock->sk->sk_sndtimeo = tconn->net_conf->try_connect_int*HZ;
- drbd_setbufsize(sock, tconn->net_conf->sndbuf_size,
- tconn->net_conf->rcvbuf_size);
+ sock->sk->sk_sndtimeo = try_connect_int * HZ;
+ drbd_setbufsize(sock, sndbuf_size, rcvbuf_size);
/* explicitly bind to the configured IP as source IP
* for the outgoing connections.
@@ -633,17 +663,8 @@ static struct socket *drbd_try_connect(struct drbd_tconn *tconn)
* Make sure to use 0 as port number, so linux selects
* a free one dynamically.
*/
- memcpy(&src_in6, tconn->net_conf->my_addr,
- min_t(int, tconn->net_conf->my_addr_len, sizeof(src_in6)));
- if (((struct sockaddr *)tconn->net_conf->my_addr)->sa_family == AF_INET6)
- src_in6.sin6_port = 0;
- else
- ((struct sockaddr_in *)&src_in6)->sin_port = 0; /* AF_INET & AF_SCI */
-
what = "bind before connect";
- err = sock->ops->bind(sock,
- (struct sockaddr *) &src_in6,
- tconn->net_conf->my_addr_len);
+ err = sock->ops->bind(sock, (struct sockaddr *) &src_in6, my_addr_len);
if (err < 0)
goto out;
@@ -651,9 +672,7 @@ static struct socket *drbd_try_connect(struct drbd_tconn *tconn)
* stay C_WF_CONNECTION, don't go Disconnecting! */
disconnect_on_error = 0;
what = "connect";
- err = sock->ops->connect(sock,
- (struct sockaddr *)tconn->net_conf->peer_addr,
- tconn->net_conf->peer_addr_len, 0);
+ err = sock->ops->connect(sock, (struct sockaddr *) &peer_in6, peer_addr_len, 0);
out:
if (err < 0) {
@@ -676,40 +695,52 @@ out:
if (disconnect_on_error)
conn_request_state(tconn, NS(conn, C_DISCONNECTING), CS_HARD);
}
- put_net_conf(tconn);
+
return sock;
}
static struct socket *drbd_wait_for_connect(struct drbd_tconn *tconn)
{
- int timeo, err;
+ int timeo, err, my_addr_len;
+ int sndbuf_size, rcvbuf_size, try_connect_int;
struct socket *s_estab = NULL, *s_listen;
+ struct sockaddr_in6 my_addr;
+ struct net_conf *nc;
const char *what;
- if (!get_net_conf(tconn))
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ if (!nc) {
+ rcu_read_unlock();
return NULL;
+ }
+
+ sndbuf_size = nc->sndbuf_size;
+ rcvbuf_size = nc->rcvbuf_size;
+ try_connect_int = nc->try_connect_int;
+
+ my_addr_len = min_t(int, nc->my_addr_len, sizeof(struct sockaddr_in6));
+ memcpy(&my_addr, nc->my_addr, my_addr_len);
+ rcu_read_unlock();
what = "sock_create_kern";
- err = sock_create_kern(((struct sockaddr *)tconn->net_conf->my_addr)->sa_family,
+ err = sock_create_kern(((struct sockaddr *)&my_addr)->sa_family,
SOCK_STREAM, IPPROTO_TCP, &s_listen);
if (err) {
s_listen = NULL;
goto out;
}
- timeo = tconn->net_conf->try_connect_int * HZ;
+ timeo = try_connect_int * HZ;
timeo += (random32() & 1) ? timeo / 7 : -timeo / 7; /* 28.5% random jitter */
s_listen->sk->sk_reuse = 1; /* SO_REUSEADDR */
s_listen->sk->sk_rcvtimeo = timeo;
s_listen->sk->sk_sndtimeo = timeo;
- drbd_setbufsize(s_listen, tconn->net_conf->sndbuf_size,
- tconn->net_conf->rcvbuf_size);
+ drbd_setbufsize(s_listen, sndbuf_size, rcvbuf_size);
what = "bind before listen";
- err = s_listen->ops->bind(s_listen,
- (struct sockaddr *) tconn->net_conf->my_addr,
- tconn->net_conf->my_addr_len);
+ err = s_listen->ops->bind(s_listen, (struct sockaddr *)&my_addr, my_addr_len);
if (err < 0)
goto out;
@@ -724,7 +755,6 @@ out:
conn_request_state(tconn, NS(conn, C_DISCONNECTING), CS_HARD);
}
}
- put_net_conf(tconn);
return s_estab;
}
@@ -817,7 +847,8 @@ int drbd_connected(int vnr, void *p, void *data)
static int drbd_connect(struct drbd_tconn *tconn)
{
struct socket *sock, *msock;
- int try, h, ok;
+ struct net_conf *nc;
+ int timeout, try, h, ok;
if (conn_request_state(tconn, NS(conn, C_WF_CONNECTION), CS_VERBOSE) < SS_SUCCESS)
return -2;
@@ -924,11 +955,17 @@ retry:
* sock->sk->sk_rcvtimeo = MAX_SCHEDULE_TIMEOUT;
* first set it to the P_CONNECTION_FEATURES timeout,
* which we set to 4x the configured ping_timeout. */
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+
sock->sk->sk_sndtimeo =
- sock->sk->sk_rcvtimeo = tconn->net_conf->ping_timeo*4*HZ/10;
+ sock->sk->sk_rcvtimeo = nc->ping_timeo*4*HZ/10;
+
+ msock->sk->sk_rcvtimeo = nc->ping_int*HZ;
+ timeout = nc->timeout * HZ / 10;
+ rcu_read_unlock();
- msock->sk->sk_sndtimeo = tconn->net_conf->timeout*HZ/10;
- msock->sk->sk_rcvtimeo = tconn->net_conf->ping_int*HZ;
+ msock->sk->sk_sndtimeo = timeout;
/* we don't want delays.
* we use TCP_CORK where appropriate, though */
@@ -956,7 +993,7 @@ retry:
if (conn_request_state(tconn, NS(conn, C_WF_REPORT_PARAMS), CS_VERBOSE) < SS_SUCCESS)
return 0;
- sock->sk->sk_sndtimeo = tconn->net_conf->timeout*HZ/10;
+ sock->sk->sk_sndtimeo = timeout;
sock->sk->sk_rcvtimeo = MAX_SCHEDULE_TIMEOUT;
drbd_thread_start(&tconn->asender);
@@ -1842,7 +1879,9 @@ static int wait_for_and_update_peer_seq(struct drbd_conf *mdev, const u32 peer_s
}
prepare_to_wait(&mdev->seq_wait, &wait, TASK_INTERRUPTIBLE);
spin_unlock(&mdev->peer_seq_lock);
- timeout = mdev->tconn->net_conf->ping_timeo*HZ/10;
+ rcu_read_lock();
+ timeout = rcu_dereference(mdev->tconn->net_conf)->ping_timeo*HZ/10;
+ rcu_read_unlock();
timeout = schedule_timeout(timeout);
spin_lock(&mdev->peer_seq_lock);
if (!timeout) {
@@ -2075,7 +2114,8 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
spin_unlock_irq(&mdev->tconn->req_lock);
if (mdev->tconn->agreed_pro_version < 100) {
- switch (mdev->tconn->net_conf->wire_protocol) {
+ rcu_read_lock();
+ switch (rcu_dereference(mdev->tconn->net_conf)->wire_protocol) {
case DRBD_PROT_C:
dp_flags |= DP_SEND_WRITE_ACK;
break;
@@ -2083,6 +2123,7 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
dp_flags |= DP_SEND_RECEIVE_ACK;
break;
}
+ rcu_read_unlock();
}
if (dp_flags & DP_SEND_WRITE_ACK) {
@@ -2385,6 +2426,7 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
{
int self, peer, rv = -100;
unsigned long ch_self, ch_peer;
+ enum drbd_after_sb_p after_sb_0p;
self = mdev->ldev->md.uuid[UI_BITMAP] & 1;
peer = mdev->p_uuid[UI_BITMAP] & 1;
@@ -2392,10 +2434,14 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
ch_peer = mdev->p_uuid[UI_SIZE];
ch_self = mdev->comm_bm_set;
- switch (mdev->tconn->net_conf->after_sb_0p) {
+ rcu_read_lock();
+ after_sb_0p = rcu_dereference(mdev->tconn->net_conf)->after_sb_0p;
+ rcu_read_unlock();
+ switch (after_sb_0p) {
case ASB_CONSENSUS:
case ASB_DISCARD_SECONDARY:
case ASB_CALL_HELPER:
+ case ASB_VIOLENTLY:
dev_err(DEV, "Configuration error.\n");
break;
case ASB_DISCONNECT:
@@ -2431,7 +2477,7 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
if (ch_peer == 0) { rv = 1; break; }
if (ch_self == 0) { rv = -1; break; }
}
- if (mdev->tconn->net_conf->after_sb_0p == ASB_DISCARD_ZERO_CHG)
+ if (after_sb_0p == ASB_DISCARD_ZERO_CHG)
break;
case ASB_DISCARD_LEAST_CHG:
if (ch_self < ch_peer)
@@ -2456,13 +2502,18 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
static int drbd_asb_recover_1p(struct drbd_conf *mdev) __must_hold(local)
{
int hg, rv = -100;
+ enum drbd_after_sb_p after_sb_1p;
- switch (mdev->tconn->net_conf->after_sb_1p) {
+ rcu_read_lock();
+ after_sb_1p = rcu_dereference(mdev->tconn->net_conf)->after_sb_1p;
+ rcu_read_unlock();
+ switch (after_sb_1p) {
case ASB_DISCARD_YOUNGER_PRI:
case ASB_DISCARD_OLDER_PRI:
case ASB_DISCARD_LEAST_CHG:
case ASB_DISCARD_LOCAL:
case ASB_DISCARD_REMOTE:
+ case ASB_DISCARD_ZERO_CHG:
dev_err(DEV, "Configuration error.\n");
break;
case ASB_DISCONNECT:
@@ -2505,8 +2556,12 @@ static int drbd_asb_recover_1p(struct drbd_conf *mdev) __must_hold(local)
static int drbd_asb_recover_2p(struct drbd_conf *mdev) __must_hold(local)
{
int hg, rv = -100;
+ enum drbd_after_sb_p after_sb_2p;
- switch (mdev->tconn->net_conf->after_sb_2p) {
+ rcu_read_lock();
+ after_sb_2p = rcu_dereference(mdev->tconn->net_conf)->after_sb_2p;
+ rcu_read_unlock();
+ switch (after_sb_2p) {
case ASB_DISCARD_YOUNGER_PRI:
case ASB_DISCARD_OLDER_PRI:
case ASB_DISCARD_LEAST_CHG:
@@ -2514,6 +2569,7 @@ static int drbd_asb_recover_2p(struct drbd_conf *mdev) __must_hold(local)
case ASB_DISCARD_REMOTE:
case ASB_CONSENSUS:
case ASB_DISCARD_SECONDARY:
+ case ASB_DISCARD_ZERO_CHG:
dev_err(DEV, "Configuration error.\n");
break;
case ASB_VIOLENTLY:
@@ -2758,9 +2814,10 @@ static int drbd_uuid_compare(struct drbd_conf *mdev, int *rule_nr) __must_hold(l
static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_role peer_role,
enum drbd_disk_state peer_disk) __must_hold(local)
{
- int hg, rule_nr;
enum drbd_conns rv = C_MASK;
enum drbd_disk_state mydisk;
+ struct net_conf *nc;
+ int hg, rule_nr, rr_conflict, dry_run;
mydisk = mdev->state.disk;
if (mydisk == D_NEGOTIATING)
@@ -2797,7 +2854,10 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
if (abs(hg) == 100)
drbd_khelper(mdev, "initial-split-brain");
- if (hg == 100 || (hg == -100 && mdev->tconn->net_conf->always_asbp)) {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+
+ if (hg == 100 || (hg == -100 && nc->always_asbp)) {
int pcount = (mdev->state.role == R_PRIMARY)
+ (peer_role == R_PRIMARY);
int forced = (hg == -100);
@@ -2826,9 +2886,9 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
}
if (hg == -100) {
- if (mdev->tconn->net_conf->want_lose && !(mdev->p_uuid[UI_FLAGS]&1))
+ if (nc->want_lose && !(mdev->p_uuid[UI_FLAGS]&1))
hg = -1;
- if (!mdev->tconn->net_conf->want_lose && (mdev->p_uuid[UI_FLAGS]&1))
+ if (!nc->want_lose && (mdev->p_uuid[UI_FLAGS]&1))
hg = 1;
if (abs(hg) < 100)
@@ -2836,6 +2896,9 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
"Sync from %s node\n",
(hg < 0) ? "peer" : "this");
}
+ rr_conflict = nc->rr_conflict;
+ dry_run = nc->dry_run;
+ rcu_read_unlock();
if (hg == -100) {
/* FIXME this log message is not correct if we end up here
@@ -2854,7 +2917,7 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
if (hg < 0 && /* by intention we do not use mydisk here. */
mdev->state.role == R_PRIMARY && mdev->state.disk >= D_CONSISTENT) {
- switch (mdev->tconn->net_conf->rr_conflict) {
+ switch (rr_conflict) {
case ASB_CALL_HELPER:
drbd_khelper(mdev, "pri-lost");
/* fall through */
@@ -2867,7 +2930,7 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
}
}
- if (mdev->tconn->net_conf->dry_run || test_bit(CONN_DRY_RUN, &mdev->tconn->flags)) {
+ if (dry_run || test_bit(CONN_DRY_RUN, &mdev->tconn->flags)) {
if (hg == 0)
dev_info(DEV, "dry-run connect: No resync, would become Connected immediately.\n");
else
@@ -2926,6 +2989,8 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
int p_proto, p_after_sb_0p, p_after_sb_1p, p_after_sb_2p;
int p_want_lose, p_two_primaries, cf;
char p_integrity_alg[SHARED_SECRET_MAX] = "";
+ unsigned char *my_alg;
+ struct net_conf *nc;
p_proto = be32_to_cpu(p->protocol);
p_after_sb_0p = be32_to_cpu(p->after_sb_0p);
@@ -2940,38 +3005,43 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
if (cf & CF_DRY_RUN)
set_bit(CONN_DRY_RUN, &tconn->flags);
- if (p_proto != tconn->net_conf->wire_protocol && tconn->agreed_pro_version < 100) {
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+
+ if (p_proto != nc->wire_protocol && tconn->agreed_pro_version < 100) {
conn_err(tconn, "incompatible communication protocols\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (cmp_after_sb(p_after_sb_0p, tconn->net_conf->after_sb_0p)) {
+ if (cmp_after_sb(p_after_sb_0p, nc->after_sb_0p)) {
conn_err(tconn, "incompatible after-sb-0pri settings\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (cmp_after_sb(p_after_sb_1p, tconn->net_conf->after_sb_1p)) {
+ if (cmp_after_sb(p_after_sb_1p, nc->after_sb_1p)) {
conn_err(tconn, "incompatible after-sb-1pri settings\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (cmp_after_sb(p_after_sb_2p, tconn->net_conf->after_sb_2p)) {
+ if (cmp_after_sb(p_after_sb_2p, nc->after_sb_2p)) {
conn_err(tconn, "incompatible after-sb-2pri settings\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (p_want_lose && tconn->net_conf->want_lose) {
+ if (p_want_lose && nc->want_lose) {
conn_err(tconn, "both sides have the 'want_lose' flag set\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (p_two_primaries != tconn->net_conf->two_primaries) {
+ if (p_two_primaries != nc->two_primaries) {
conn_err(tconn, "incompatible setting of the two-primaries options\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
+ my_alg = nc->integrity_alg;
+ rcu_read_unlock();
+
if (tconn->agreed_pro_version >= 87) {
- unsigned char *my_alg = tconn->net_conf->integrity_alg;
int err;
err = drbd_recv_all(tconn, p_integrity_alg, pi->size);
@@ -2989,6 +3059,8 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
return 0;
+disconnect_rcu_unlock:
+ rcu_read_unlock();
disconnect:
conn_request_state(tconn, NS(conn, C_DISCONNECTING), CS_HARD);
return -EIO;
@@ -4322,19 +4394,26 @@ static int drbd_do_auth(struct drbd_tconn *tconn)
char *response = NULL;
char *right_response = NULL;
char *peers_ch = NULL;
- unsigned int key_len = strlen(tconn->net_conf->shared_secret);
+ unsigned int key_len;
+ char secret[SHARED_SECRET_MAX]; /* 64 byte */
unsigned int resp_size;
struct hash_desc desc;
struct packet_info pi;
+ struct net_conf *nc;
int err, rv;
/* FIXME: Put the challenge/response into the preallocated socket buffer. */
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ key_len = strlen(nc->shared_secret);
+ memcpy(secret, nc->shared_secret, key_len);
+ rcu_read_unlock();
+
desc.tfm = tconn->cram_hmac_tfm;
desc.flags = 0;
- rv = crypto_hash_setkey(tconn->cram_hmac_tfm,
- (u8 *)tconn->net_conf->shared_secret, key_len);
+ rv = crypto_hash_setkey(tconn->cram_hmac_tfm, (u8 *)secret, key_len);
if (rv) {
conn_err(tconn, "crypto_hash_setkey() failed with %d\n", rv);
rv = -1;
@@ -4456,8 +4535,8 @@ static int drbd_do_auth(struct drbd_tconn *tconn)
rv = !memcmp(response, right_response, resp_size);
if (rv)
- conn_info(tconn, "Peer authenticated using %d bytes of '%s' HMAC\n",
- resp_size, tconn->net_conf->cram_hmac_alg);
+ conn_info(tconn, "Peer authenticated using %d bytes HMAC\n",
+ resp_size);
else
rv = -1;
@@ -4884,33 +4963,42 @@ int drbd_asender(struct drbd_thread *thi)
int received = 0;
unsigned int header_size = drbd_header_size(tconn);
int expect = header_size;
- int ping_timeout_active = 0;
+ bool ping_timeout_active = false;
+ struct net_conf *nc;
+ int ping_timeo, no_cork, ping_int;
current->policy = SCHED_RR; /* Make this a realtime task! */
current->rt_priority = 2; /* more important than all other tasks */
while (get_t_state(thi) == RUNNING) {
drbd_thread_current_set_cpu(thi);
+
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ ping_timeo = nc->ping_timeo;
+ no_cork = nc->no_cork;
+ ping_int = nc->ping_int;
+ rcu_read_unlock();
+
if (test_and_clear_bit(SEND_PING, &tconn->flags)) {
if (drbd_send_ping(tconn)) {
conn_err(tconn, "drbd_send_ping has failed\n");
goto reconnect;
}
- tconn->meta.socket->sk->sk_rcvtimeo =
- tconn->net_conf->ping_timeo*HZ/10;
- ping_timeout_active = 1;
+ tconn->meta.socket->sk->sk_rcvtimeo = ping_timeo * HZ / 10;
+ ping_timeout_active = true;
}
/* TODO: conditionally cork; it may hurt latency if we cork without
much to send */
- if (!tconn->net_conf->no_cork)
+ if (!no_cork)
drbd_tcp_cork(tconn->meta.socket);
if (tconn_finish_peer_reqs(tconn)) {
conn_err(tconn, "tconn_finish_peer_reqs() failed\n");
goto reconnect;
}
/* but unconditionally uncork unless disabled */
- if (!tconn->net_conf->no_cork)
+ if (!no_cork)
drbd_tcp_uncork(tconn->meta.socket);
/* short circuit, recv_msg would return EINTR anyways. */
@@ -4984,10 +5072,11 @@ int drbd_asender(struct drbd_thread *thi)
tconn->last_received = jiffies;
- /* the idle_timeout (ping-int)
- * has been restored in got_PingAck() */
- if (cmd == &asender_tbl[P_PING_ACK])
- ping_timeout_active = 0;
+ if (cmd == &asender_tbl[P_PING_ACK]) {
+ /* restore idle timeout */
+ tconn->meta.socket->sk->sk_rcvtimeo = ping_int * HZ;
+ ping_timeout_active = false;
+ }
buf = tconn->meta.rbuf;
received = 0;
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index 0f1a29f..c4e4553 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -323,6 +323,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
struct bio_and_error *m)
{
struct drbd_conf *mdev = req->w.mdev;
+ struct net_conf *nc;
int p, rv = 0;
if (m)
@@ -344,7 +345,10 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
* and from w_read_retry_remote */
D_ASSERT(!(req->rq_state & RQ_NET_MASK));
req->rq_state |= RQ_NET_PENDING;
- p = mdev->tconn->net_conf->wire_protocol;
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ p = nc->wire_protocol;
+ rcu_read_unlock();
req->rq_state |=
p == DRBD_PROT_C ? RQ_EXP_WRITE_ACK :
p == DRBD_PROT_B ? RQ_EXP_RECEIVE_ACK : 0;
@@ -474,7 +478,11 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
drbd_queue_work(&mdev->tconn->data.work, &req->w);
/* close the epoch, in case it outgrew the limit */
- if (mdev->tconn->newest_tle->n_writes >= mdev->tconn->net_conf->max_epoch_size)
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ p = nc->max_epoch_size;
+ rcu_read_unlock();
+ if (mdev->tconn->newest_tle->n_writes >= p)
queue_barrier(mdev);
break;
@@ -729,6 +737,7 @@ int __drbd_make_request(struct drbd_conf *mdev, struct bio *bio, unsigned long s
const sector_t sector = bio->bi_sector;
struct drbd_tl_epoch *b = NULL;
struct drbd_request *req;
+ struct net_conf *nc;
int local, remote, send_oos = 0;
int err;
int ret = 0;
@@ -935,17 +944,19 @@ allocate_barrier:
if (send_oos && drbd_set_out_of_sync(mdev, sector, size))
_req_mod(req, QUEUE_FOR_SEND_OOS);
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
if (remote &&
- mdev->tconn->net_conf->on_congestion != OC_BLOCK && mdev->tconn->agreed_pro_version >= 96) {
+ nc->on_congestion != OC_BLOCK && mdev->tconn->agreed_pro_version >= 96) {
int congested = 0;
- if (mdev->tconn->net_conf->cong_fill &&
- atomic_read(&mdev->ap_in_flight) >= mdev->tconn->net_conf->cong_fill) {
+ if (nc->cong_fill &&
+ atomic_read(&mdev->ap_in_flight) >= nc->cong_fill) {
dev_info(DEV, "Congestion-fill threshold reached\n");
congested = 1;
}
- if (mdev->act_log->used >= mdev->tconn->net_conf->cong_extents) {
+ if (mdev->act_log->used >= nc->cong_extents) {
dev_info(DEV, "Congestion-extents threshold reached\n");
congested = 1;
}
@@ -953,12 +964,13 @@ allocate_barrier:
if (congested) {
queue_barrier(mdev); /* last barrier, after mirrored writes */
- if (mdev->tconn->net_conf->on_congestion == OC_PULL_AHEAD)
+ if (nc->on_congestion == OC_PULL_AHEAD)
_drbd_set_state(_NS(mdev, conn, C_AHEAD), 0, NULL);
- else /*mdev->tconn->net_conf->on_congestion == OC_DISCONNECT */
+ else /*nc->on_congestion == OC_DISCONNECT */
_drbd_set_state(_NS(mdev, conn, C_DISCONNECTING), 0, NULL);
}
}
+ rcu_read_unlock();
spin_unlock_irq(&mdev->tconn->req_lock);
kfree(b); /* if someone else has beaten us to it... */
@@ -1058,12 +1070,14 @@ void request_timer_fn(unsigned long data)
struct drbd_tconn *tconn = mdev->tconn;
struct drbd_request *req; /* oldest request */
struct list_head *le;
- unsigned long et = 0; /* effective timeout = ko_count * timeout */
+ struct net_conf *nc;
+ unsigned long et; /* effective timeout = ko_count * timeout */
+
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ et = nc ? nc->timeout * HZ/10 * nc->ko_count : 0;
+ rcu_read_unlock();
- if (get_net_conf(tconn)) {
- et = tconn->net_conf->timeout*HZ/10 * tconn->net_conf->ko_count;
- put_net_conf(tconn);
- }
if (!et || mdev->state.conn < C_WF_REPORT_PARAMS)
return; /* Recurring timer stopped */
diff --git a/drivers/block/drbd/drbd_state.c b/drivers/block/drbd/drbd_state.c
index 0aed33b..2423283 100644
--- a/drivers/block/drbd/drbd_state.c
+++ b/drivers/block/drbd/drbd_state.c
@@ -464,6 +464,7 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
enum drbd_fencing_p fp;
enum drbd_state_rv rv = SS_SUCCESS;
+ struct net_conf *nc;
fp = FP_DONT_CARE;
if (get_ldev(mdev)) {
@@ -471,14 +472,15 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
put_ldev(mdev);
}
- if (get_net_conf(mdev->tconn)) {
- if (!mdev->tconn->net_conf->two_primaries && ns.role == R_PRIMARY) {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc) {
+ if (!nc->two_primaries && ns.role == R_PRIMARY) {
if (ns.peer == R_PRIMARY)
rv = SS_TWO_PRIMARIES;
else if (conn_highest_peer(mdev->tconn) == R_PRIMARY)
rv = SS_O_VOL_PEER_PRI;
- }
- put_net_conf(mdev->tconn);
+ }
}
if (rv <= 0)
@@ -513,7 +515,7 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
rv = SS_CONNECTED_OUTDATES;
else if ((ns.conn == C_VERIFY_S || ns.conn == C_VERIFY_T) &&
- (mdev->tconn->net_conf->verify_alg[0] == 0))
+ (nc->verify_alg[0] == 0))
rv = SS_NO_VERIFY_ALG;
else if ((ns.conn == C_VERIFY_S || ns.conn == C_VERIFY_T) &&
@@ -523,6 +525,8 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
else if (ns.conn >= C_CONNECTED && ns.pdsk == D_UNKNOWN)
rv = SS_CONNECTED_OUTDATES;
+ rcu_read_unlock();
+
return rv;
}
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index b63c08e..c54c191 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -1621,10 +1621,16 @@ void drbd_start_resync(struct drbd_conf *mdev, enum drbd_conns side)
* detect connection loss, then waiting for a ping
* response (implicit in drbd_resync_finished) reduces
* the race considerably, but does not solve it. */
- if (side == C_SYNC_SOURCE)
- schedule_timeout_interruptible(
- mdev->tconn->net_conf->ping_int * HZ +
- mdev->tconn->net_conf->ping_timeo*HZ/9);
+ if (side == C_SYNC_SOURCE) {
+ struct net_conf *nc;
+ int timeo;
+
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ timeo = nc->ping_int * HZ + nc->ping_timeo * HZ / 9;
+ rcu_read_unlock();
+ schedule_timeout_interruptible(timeo);
+ }
drbd_resync_finished(mdev);
}
@@ -1647,22 +1653,30 @@ int drbd_worker(struct drbd_thread *thi)
struct drbd_tconn *tconn = thi->tconn;
struct drbd_work *w = NULL;
struct drbd_conf *mdev;
+ struct net_conf *nc;
LIST_HEAD(work_list);
int vnr, intr = 0;
+ int cork;
while (get_t_state(thi) == RUNNING) {
drbd_thread_current_set_cpu(thi);
if (down_trylock(&tconn->data.work.s)) {
mutex_lock(&tconn->data.mutex);
- if (tconn->data.socket && !tconn->net_conf->no_cork)
+
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ cork = nc ? !nc->no_cork : 0;
+ rcu_read_unlock();
+
+ if (tconn->data.socket && cork)
drbd_tcp_uncork(tconn->data.socket);
mutex_unlock(&tconn->data.mutex);
intr = down_interruptible(&tconn->data.work.s);
mutex_lock(&tconn->data.mutex);
- if (tconn->data.socket && !tconn->net_conf->no_cork)
+ if (tconn->data.socket && cork)
drbd_tcp_cork(tconn->data.socket);
mutex_unlock(&tconn->data.mutex);
}
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (8 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 11/11] drbd: Considering that the two_primaries config flag can change Philipp Reisner
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Removing the get_net_conf()/put_net_conf() functions
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 29 +-----------
drivers/block/drbd/drbd_main.c | 17 ++++++-
drivers/block/drbd/drbd_nl.c | 67 ++++++++++++----------------
drivers/block/drbd/drbd_receiver.c | 83 +++++++++++++++++++++++-------------
4 files changed, 100 insertions(+), 96 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 99da54c..83e6cad 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -833,9 +833,8 @@ struct drbd_tconn { /* is a resource from the config file */
unsigned long flags;
struct net_conf *net_conf; /* content protected by rcu */
- atomic_t net_cnt; /* Users of net_conf */
- wait_queue_head_t net_cnt_wait;
- wait_queue_head_t ping_wait; /* Woken upon reception of a ping, and a state change */
+ struct mutex net_conf_update; /* mutex for ready-copy-update of net_conf */
+ wait_queue_head_t ping_wait; /* Woken upon reception of a ping, and a state change */
struct res_opts res_opts;
struct drbd_socket data; /* data/barrier/cstate/parameter packets */
@@ -1379,6 +1378,7 @@ extern void drbd_delete_device(struct drbd_conf *mdev);
struct drbd_tconn *drbd_new_tconn(const char *name);
extern void drbd_free_tconn(struct drbd_tconn *tconn);
struct drbd_tconn *conn_by_name(const char *name);
+extern void conn_free_crypto(struct drbd_tconn *tconn);
extern int proc_details;
@@ -1935,29 +1935,6 @@ static inline void _sub_unacked(struct drbd_conf *mdev, int n, const char *func,
ERR_IF_CNT_IS_NEGATIVE(unacked_cnt, func, line);
}
-static inline void put_net_conf(struct drbd_tconn *tconn)
-{
- if (atomic_dec_and_test(&tconn->net_cnt))
- wake_up(&tconn->net_cnt_wait);
-}
-
-/**
- * get_net_conf() - Increase ref count on mdev->tconn->net_conf; Returns 0 if nothing there
- * @mdev: DRBD device.
- *
- * You have to call put_net_conf() when finished working with mdev->tconn->net_conf.
- */
-static inline int get_net_conf(struct drbd_tconn *tconn)
-{
- int have_net_conf;
-
- atomic_inc(&tconn->net_cnt);
- have_net_conf = tconn->cstate >= C_UNCONNECTED;
- if (!have_net_conf)
- put_net_conf(tconn);
- return have_net_conf;
-}
-
/**
* get_ldev() - Increase the ref count on mdev->ldev. Returns 0 if there is no ldev
* @M: DRBD device.
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index d335ecc..5eb60e8 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2376,6 +2376,20 @@ static void drbd_free_socket(struct drbd_socket *socket)
free_page((unsigned long) socket->rbuf);
}
+void conn_free_crypto(struct drbd_tconn *tconn)
+{
+ crypto_free_hash(tconn->cram_hmac_tfm);
+ crypto_free_hash(tconn->integrity_w_tfm);
+ crypto_free_hash(tconn->integrity_r_tfm);
+ kfree(tconn->int_dig_in);
+ kfree(tconn->int_dig_vv);
+ tconn->cram_hmac_tfm = NULL;
+ tconn->integrity_w_tfm = NULL;
+ tconn->integrity_r_tfm = NULL;
+ tconn->int_dig_in = NULL;
+ tconn->int_dig_vv = NULL;
+}
+
struct drbd_tconn *drbd_new_tconn(const char *name)
{
struct drbd_tconn *tconn;
@@ -2402,8 +2416,7 @@ struct drbd_tconn *drbd_new_tconn(const char *name)
tconn->cstate = C_STANDALONE;
mutex_init(&tconn->cstate_mutex);
spin_lock_init(&tconn->req_lock);
- atomic_set(&tconn->net_cnt, 0);
- init_waitqueue_head(&tconn->net_cnt_wait);
+ mutex_init(&tconn->net_conf_update);
init_waitqueue_head(&tconn->ping_wait);
idr_init(&tconn->volumes);
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 6d28e7c..bafa457 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -615,11 +615,12 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
put_ldev(mdev);
}
} else {
- rcu_read_lock();
- nc = rcu_dereference(mdev->tconn->net_conf);
+ mutex_lock(&mdev->tconn->net_conf_update);
+ nc = mdev->tconn->net_conf;
if (nc)
- nc->want_lose = 0;
- rcu_read_unlock();
+ nc->want_lose = 0; /* without copy; single bit op is atomic */
+ mutex_unlock(&mdev->tconn->net_conf_update);
+
set_disk_ro(mdev->vdisk, false);
if (get_ldev(mdev)) {
if (((mdev->state.conn < C_CONNECTED ||
@@ -1809,17 +1810,16 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
conn_reconfig_start(tconn);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
+ mutex_lock(&tconn->net_conf_update);
+ old_conf = tconn->net_conf;
if (!old_conf) {
drbd_msg_put_info("net conf missing, try connect");
retcode = ERR_INVALID_REQUEST;
- goto fail_rcu_unlock;
+ goto fail;
}
*new_conf = *old_conf;
- rcu_read_unlock();
err = net_conf_from_attrs(new_conf, info);
if (err) {
@@ -1834,13 +1834,10 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* re-sync running */
rsr = conn_resync_running(tconn);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
if (rsr && old_conf && strcmp(new_conf->csums_alg, old_conf->csums_alg)) {
retcode = ERR_CSUMS_RESYNC_RUNNING;
- goto fail_rcu_unlock;
+ goto fail;
}
- rcu_read_unlock();
if (!rsr && new_conf->csums_alg[0]) {
csums_tfm = crypto_alloc_hash(new_conf->csums_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1858,15 +1855,12 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* online verify running */
ovr = conn_ov_running(tconn);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
- if (ovr && old_conf) {
+ if (ovr) {
if (strcmp(new_conf->verify_alg, old_conf->verify_alg)) {
retcode = ERR_VERIFY_RUNNING;
- goto fail_rcu_unlock;
+ goto fail;
}
}
- rcu_read_unlock();
if (!ovr && new_conf->verify_alg[0]) {
verify_tfm = crypto_alloc_hash(new_conf->verify_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1883,8 +1877,6 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
}
rcu_assign_pointer(tconn->net_conf, new_conf);
- synchronize_rcu();
- kfree(old_conf);
if (!rsr) {
crypto_free_hash(tconn->csums_tfm);
@@ -1897,12 +1889,17 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
verify_tfm = NULL;
}
+ mutex_unlock(&tconn->net_conf_update);
+ synchronize_rcu();
+ kfree(old_conf);
+
if (tconn->cstate >= C_WF_REPORT_PARAMS)
drbd_send_sync_param(minor_to_mdev(conn_lowest_minor(tconn)));
- fail_rcu_unlock:
- rcu_read_unlock();
+ goto out;
+
fail:
+ mutex_unlock(&tconn->net_conf_update);
crypto_free_hash(csums_tfm);
crypto_free_hash(verify_tfm);
kfree(new_conf);
@@ -2081,32 +2078,26 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
}
conn_flush_workqueue(tconn);
- spin_lock_irq(&tconn->req_lock);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
- if (old_conf != NULL) {
+
+ mutex_lock(&tconn->net_conf_update);
+ old_conf = tconn->net_conf;
+ if (old_conf) {
retcode = ERR_NET_CONFIGURED;
- rcu_read_unlock();
- spin_unlock_irq(&tconn->req_lock);
+ mutex_unlock(&tconn->net_conf_update);
goto fail;
}
rcu_assign_pointer(tconn->net_conf, new_conf);
- crypto_free_hash(tconn->cram_hmac_tfm);
+ conn_free_crypto(tconn);
tconn->cram_hmac_tfm = tfm;
-
- crypto_free_hash(tconn->integrity_w_tfm);
tconn->integrity_w_tfm = integrity_w_tfm;
-
- crypto_free_hash(tconn->integrity_r_tfm);
tconn->integrity_r_tfm = integrity_r_tfm;
+ tconn->int_dig_in = int_dig_in;
+ tconn->int_dig_vv = int_dig_vv;
- kfree(tconn->int_dig_in);
- kfree(tconn->int_dig_vv);
- tconn->int_dig_in=int_dig_in;
- tconn->int_dig_vv=int_dig_vv;
- retcode = _conn_request_state(tconn, NS(conn, C_UNCONNECTED), CS_VERBOSE);
- spin_unlock_irq(&tconn->req_lock);
+ mutex_unlock(&tconn->net_conf_update);
+
+ retcode = conn_request_state(tconn, NS(conn, C_UNCONNECTED), CS_VERBOSE);
rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index a37f2ee..05b214c 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -3138,6 +3138,7 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
unsigned int header_size, data_size, exp_max_sz;
struct crypto_hash *verify_tfm = NULL;
struct crypto_hash *csums_tfm = NULL;
+ struct net_conf *old_conf, *new_conf = NULL;
const int apv = tconn->agreed_pro_version;
int *rs_plan_s = NULL;
int fifo_size = 0;
@@ -3212,10 +3213,13 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
p->csums_alg[SHARED_SECRET_MAX-1] = 0;
}
- if (strcmp(mdev->tconn->net_conf->verify_alg, p->verify_alg)) {
+ mutex_lock(&mdev->tconn->net_conf_update);
+ old_conf = mdev->tconn->net_conf;
+
+ if (strcmp(old_conf->verify_alg, p->verify_alg)) {
if (mdev->state.conn == C_WF_REPORT_PARAMS) {
dev_err(DEV, "Different verify-alg settings. me=\"%s\" peer=\"%s\"\n",
- mdev->tconn->net_conf->verify_alg, p->verify_alg);
+ old_conf->verify_alg, p->verify_alg);
goto disconnect;
}
verify_tfm = drbd_crypto_alloc_digest_safe(mdev,
@@ -3226,10 +3230,10 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
}
}
- if (apv >= 89 && strcmp(mdev->tconn->net_conf->csums_alg, p->csums_alg)) {
+ if (apv >= 89 && strcmp(old_conf->csums_alg, p->csums_alg)) {
if (mdev->state.conn == C_WF_REPORT_PARAMS) {
dev_err(DEV, "Different csums-alg settings. me=\"%s\" peer=\"%s\"\n",
- mdev->tconn->net_conf->csums_alg, p->csums_alg);
+ old_conf->csums_alg, p->csums_alg);
goto disconnect;
}
csums_tfm = drbd_crypto_alloc_digest_safe(mdev,
@@ -3259,22 +3263,38 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
put_ldev(mdev);
}
- spin_lock(&mdev->peer_seq_lock);
- /* lock against drbd_nl_syncer_conf() */
- if (verify_tfm) {
- strcpy(mdev->tconn->net_conf->verify_alg, p->verify_alg);
- mdev->tconn->net_conf->verify_alg_len = strlen(p->verify_alg) + 1;
- crypto_free_hash(mdev->tconn->verify_tfm);
- mdev->tconn->verify_tfm = verify_tfm;
- dev_info(DEV, "using verify-alg: \"%s\"\n", p->verify_alg);
+ if (verify_tfm || csums_tfm) {
+ new_conf = kzalloc(sizeof(struct net_conf), GFP_KERNEL);
+ if (!new_conf) {
+ dev_err(DEV, "Allocation of new net_conf failed\n");
+ goto disconnect;
+ }
+
+ *new_conf = *old_conf;
+
+ if (verify_tfm) {
+ strcpy(new_conf->verify_alg, p->verify_alg);
+ new_conf->verify_alg_len = strlen(p->verify_alg) + 1;
+ crypto_free_hash(mdev->tconn->verify_tfm);
+ mdev->tconn->verify_tfm = verify_tfm;
+ dev_info(DEV, "using verify-alg: \"%s\"\n", p->verify_alg);
+ }
+ if (csums_tfm) {
+ strcpy(new_conf->csums_alg, p->csums_alg);
+ new_conf->csums_alg_len = strlen(p->csums_alg) + 1;
+ crypto_free_hash(mdev->tconn->csums_tfm);
+ mdev->tconn->csums_tfm = csums_tfm;
+ dev_info(DEV, "using csums-alg: \"%s\"\n", p->csums_alg);
+ }
+ rcu_assign_pointer(tconn->net_conf, new_conf);
}
- if (csums_tfm) {
- strcpy(mdev->tconn->net_conf->csums_alg, p->csums_alg);
- mdev->tconn->net_conf->csums_alg_len = strlen(p->csums_alg) + 1;
- crypto_free_hash(mdev->tconn->csums_tfm);
- mdev->tconn->csums_tfm = csums_tfm;
- dev_info(DEV, "using csums-alg: \"%s\"\n", p->csums_alg);
+ mutex_unlock(&mdev->tconn->net_conf_update);
+ if (new_conf) {
+ synchronize_rcu();
+ kfree(old_conf);
}
+
+ spin_lock(&mdev->peer_seq_lock);
if (fifo_size != mdev->rs_plan_s.size) {
kfree(mdev->rs_plan_s.values);
mdev->rs_plan_s.values = rs_plan_s;
@@ -3286,6 +3306,7 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
return 0;
disconnect:
+ mutex_lock(&mdev->tconn->net_conf_update);
/* just for completeness: actually not needed,
* as this is not reached if csums_tfm was ok. */
crypto_free_hash(csums_tfm);
@@ -3715,7 +3736,9 @@ static int receive_state(struct drbd_tconn *tconn, struct packet_info *pi)
}
}
- mdev->tconn->net_conf->want_lose = 0;
+ mutex_lock(&mdev->tconn->net_conf_update);
+ mdev->tconn->net_conf->want_lose = 0; /* without copy; single bit op is atomic */
+ mutex_unlock(&mdev->tconn->net_conf_update);
drbd_md_sync(mdev); /* update connected indicator, la_size, ... */
@@ -4183,13 +4206,17 @@ static void drbd_disconnect(struct drbd_tconn *tconn)
spin_unlock_irq(&tconn->req_lock);
if (oc == C_DISCONNECTING) {
- wait_event(tconn->net_cnt_wait, atomic_read(&tconn->net_cnt) == 0);
+ struct net_conf *old_conf;
- crypto_free_hash(tconn->cram_hmac_tfm);
- tconn->cram_hmac_tfm = NULL;
+ mutex_lock(&tconn->net_conf_update);
+ old_conf = tconn->net_conf;
+ rcu_assign_pointer(tconn->net_conf, NULL);
+ conn_free_crypto(tconn);
+ mutex_unlock(&tconn->net_conf_update);
+
+ synchronize_rcu();
+ kfree(old_conf);
- kfree(tconn->net_conf);
- tconn->net_conf = NULL;
conn_request_state(tconn, NS(conn, C_STANDALONE), CS_VERBOSE);
}
}
@@ -4568,12 +4595,8 @@ int drbdd_init(struct drbd_thread *thi)
}
} while (h == 0);
- if (h > 0) {
- if (get_net_conf(tconn)) {
- drbdd(tconn);
- put_net_conf(tconn);
- }
- }
+ if (h > 0)
+ drbdd(tconn);
drbd_disconnect(tconn);
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 11/11] drbd: Considering that the two_primaries config flag can change
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (9 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Now since it is possible to change the two_primaries config
flag while the connection is up, make sure we treat a peer_req
in a consistent way if the config flag changes while the peer_req
is under IO.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 4 ++++
drivers/block/drbd/drbd_receiver.c | 19 ++++++++++++++-----
2 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 83e6cad..3833d56 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -661,6 +661,9 @@ enum {
/* The peer wants a write ACK for this (wire proto C) */
__EE_SEND_WRITE_ACK,
+
+ /* Is set when net_conf had two_primaries set while creating this peer_req */
+ __EE_IN_INTERVAL_TREE,
};
#define EE_CALL_AL_COMPLETE_IO (1<<__EE_CALL_AL_COMPLETE_IO)
#define EE_MAY_SET_IN_SYNC (1<<__EE_MAY_SET_IN_SYNC)
@@ -669,6 +672,7 @@ enum {
#define EE_HAS_DIGEST (1<<__EE_HAS_DIGEST)
#define EE_RESTART_REQUESTS (1<<__EE_RESTART_REQUESTS)
#define EE_SEND_WRITE_ACK (1<<__EE_SEND_WRITE_ACK)
+#define EE_IN_INTERVAL_TREE (1<<__EE_IN_INTERVAL_TREE)
/* flag bits per mdev */
enum {
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 05b214c..ec5b923 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1752,7 +1752,7 @@ static int e_end_block(struct drbd_work *w, int cancel)
}
/* we delete from the conflict detection hash _after_ we sent out the
* P_WRITE_ACK / P_NEG_ACK, to get the sequence number right. */
- if (mdev->tconn->net_conf->two_primaries) {
+ if (peer_req->flags & EE_IN_INTERVAL_TREE) {
spin_lock_irq(&mdev->tconn->req_lock);
D_ASSERT(!drbd_interval_empty(&peer_req->i));
drbd_remove_epoch_entry_interval(mdev, peer_req);
@@ -1811,14 +1811,19 @@ static u32 seq_max(u32 a, u32 b)
static bool need_peer_seq(struct drbd_conf *mdev)
{
struct drbd_tconn *tconn = mdev->tconn;
+ int tp;
/*
* We only need to keep track of the last packet_seq number of our peer
* if we are in dual-primary mode and we have the discard flag set; see
* handle_write_conflicts().
*/
- return tconn->net_conf->two_primaries &&
- test_bit(DISCARD_CONCURRENT, &tconn->flags);
+
+ rcu_read_lock();
+ tp = rcu_dereference(mdev->tconn->net_conf)->two_primaries;
+ rcu_read_unlock();
+
+ return tp && test_bit(DISCARD_CONCURRENT, &tconn->flags);
}
static void update_peer_seq(struct drbd_conf *mdev, unsigned int peer_seq)
@@ -2049,7 +2054,7 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
u32 peer_seq = be32_to_cpu(p->seq_num);
int rw = WRITE;
u32 dp_flags;
- int err;
+ int err, tp;
mdev = vnr_to_mdev(tconn, pi->vnr);
if (!mdev)
@@ -2094,7 +2099,11 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
atomic_inc(&peer_req->epoch->active);
spin_unlock(&mdev->epoch_lock);
- if (mdev->tconn->net_conf->two_primaries) {
+ rcu_read_lock();
+ tp = rcu_dereference(mdev->tconn->net_conf)->two_primaries;
+ rcu_read_unlock();
+ if (tp) {
+ peer_req->flags |= EE_IN_INTERVAL_TREE;
err = wait_for_and_update_peer_seq(mdev, peer_seq);
if (err)
goto out_interrupted;
--
1.7.4.1
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2011-09-30 12:50 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
2011-09-30 12:48 ` [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place Philipp Reisner
2011-09-30 12:48 ` [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep Philipp Reisner
2011-09-30 12:48 ` [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem Philipp Reisner
2011-09-30 12:48 ` [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem Philipp Reisner
2011-09-30 12:48 ` [PATCH 06/11] drbd: Runtime changeable wire protocol Philipp Reisner
2011-09-30 12:48 ` [PATCH 07/11] drbd: Check consistency of net options when the get changed online Philipp Reisner
2011-09-30 12:48 ` [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100 Philipp Reisner
2011-09-30 12:48 ` [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf Philipp Reisner
2011-09-30 12:48 ` [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU Philipp Reisner
2011-09-30 12:48 ` [PATCH 11/11] drbd: Considering that the two_primaries config flag can change Philipp Reisner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®