[PATCH 1/1] file: Avoid acquire on shared fd table lookups

[PATCH 1/1] file: Avoid acquire on shared fd table lookups

[Pranith Kumar]

__fget_light() reads files->count to choose whether it can borrow an
fd without taking a file reference. The current acquire load applies
to both branches. Only the single-owner branch uses the count load
to order its subsequent fd table lookup against a concurrent
close_fd() and put_files_struct(). The shared-table branch obtains a
file reference under RCU instead.

Read the count without an acquire and use
smp_acquire__after_ctrl_dep() on the single-owner branch. The branch
provides the control dependency, and the barrier preserves acquire
ordering before the borrowed fd is read. This removes an acquire
barrier from each shared-table fdget() lookup on arm64.

With two threads sharing an fd table and each performing 10,000 pipe
write/read pairs on separate arm64 cores, performance improves 1.75%.

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 fs/file.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/fs/file.c b/fs/file.c
index 4cd40dfa6129..20ab46434abe 100644
--- a/fs/file.c
+++ b/fs/file.c
@@ -1337,10 +1337,12 @@ static inline struct fd __fget_light(unsigned int fd, fmode_t mask)
 	 * entry combined with the new refcount - otherwise we could
 	 * return a file that is concurrently being freed.
 	 *
-	 * atomic_read_acquire() pairs with atomic_dec_and_test() in
-	 * put_files_struct().
+	 * The acquire on the single-owner path pairs with
+	 * atomic_dec_and_test() in put_files_struct(). The shared-table
+	 * path obtains its own file reference under RCU.
 	 */
-	if (likely(atomic_read_acquire(&files->count) == 1)) {
+	if (likely(atomic_read(&files->count) == 1)) {
+		smp_acquire__after_ctrl_dep();
 		file = files_lookup_fd_raw(files, fd);
 		if (!file || unlikely(file->f_mode & mask))
 			return EMPTY_FD;
-- 
2.47.3