[PATCH net v12 10/15] rxrpc: Fix the cleanup of service calls when socket shut down

[David Howells <dhowells@redhat.com>]

When a kernel AF_RXRPC socket is shut down, rxrpc_release_call() detaches
each outstanding service call from the socket, but doesn't send the app a
notification for each call that the socket to end the linkage from the app
side, assuming that the app will do this - but neither afs nor rxperf do.
The notification is prevented by rxrpc_notify_socket() rejecting the
notification if the socket in the CLOSE state.  This could lead to calls
not being cleaned up and rmmod of rxrpc stalling indefinitely.

Fix this by:

 (1) Making rxrpc_release_calls_on_socket() wait for the call to be
     transitioned to the completed state when the I/O thread processes the
     abort proposal.  This prevents the call from having the RELEASED flag
     set before rxrpc_notify_socket() runs (which would otherwise cause the
     notification to be skipped).

 (2) Making rxrpc_notify_socket() call ->notify_rx() even if the socket is
     in the RXRPC_CLOSE state.  The wait added in (1) makes sure that the
     notification is done before the call is released from the socket.

Note that this isn't relevant to userspace as the userspace app doesn't
have its own structures in the kernel that need to be cleaned up.

Fixes: 248f219cb8bc ("rxrpc: Rewrite the data and ack handling code")
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Marc Dionne <marc.dionne@auristor.com>
cc: Jeffrey Altman <jaltman@auristor.com>
cc: Eric Dumazet <edumazet@google.com>
cc: "David S. Miller" <davem@davemloft.net>
cc: Jakub Kicinski <kuba@kernel.org>
cc: Paolo Abeni <pabeni@redhat.com>
cc: Simon Horman <horms@kernel.org>
cc: linux-afs@lists.infradead.org
cc: stable@vger.kernel.org
---
 net/rxrpc/call_object.c |  1 +
 net/rxrpc/recvmsg.c     | 12 ++++++------
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c
index 817ed9acb91e..68d4096994bd 100644
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -628,6 +628,7 @@ void rxrpc_release_calls_on_socket(struct rxrpc_sock *rx)
 		rxrpc_get_call(call, rxrpc_call_get_release_sock);
 		rxrpc_propose_abort(call, RX_CALL_DEAD, -ECONNRESET,
 				    rxrpc_abort_call_sock_release);
+		wait_event(call->waitq, rxrpc_call_is_complete(call));
 		rxrpc_release_call(rx, call);
 		rxrpc_put_call(call, rxrpc_call_put_release_sock);
 	}
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index 0c960f13b5fc..214eea04b1c2 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -37,12 +37,12 @@ void rxrpc_notify_socket(struct rxrpc_call *call)
 
 	rx = rcu_dereference(call->socket);
 	sk = &rx->sk;
-	if (rx && sk->sk_state < RXRPC_CLOSE) {
-		if (call->notify_rx) {
-			spin_lock_irqsave(&call->notify_lock, flags);
-			call->notify_rx(sk, call, call->user_call_ID);
-			spin_unlock_irqrestore(&call->notify_lock, flags);
-		} else {
+	if (call->notify_rx) {
+		spin_lock_irqsave(&call->notify_lock, flags);
+		call->notify_rx(sk, call, call->user_call_ID);
+		spin_unlock_irqrestore(&call->notify_lock, flags);
+	} else {
+		if (rx && sk->sk_state < RXRPC_CLOSE) {
 			spin_lock_irqsave(&rx->recvmsg_lock, flags);
 			if (list_empty(&call->recvmsg_link)) {
 				rxrpc_get_call(call, rxrpc_call_get_notify_socket);