From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr2-f2.google.com (mail-wr2-f2.google.com [74.125.225.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58B3E377023 for ; Sat, 25 Jul 2026 18:26:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.66 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785004001; cv=none; b=m98yqT0UGYRiUwGEgzDwh2Kb1kIdN7PU7G560lCYu9vfzilU8+kp+2VFxxZz8+338pgJL/4AI2h5Waoa41jXZUApfXejWJ5/8HtEhOFh42FhbVjLkC7F6eqAx5rqQB9zbAtKCmfsklEiMTs0LPQLazNRG4PwC/vXo3MM9UmkxTc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785004001; c=relaxed/simple; bh=qDqp3t4OgY9x0Mmm22X0ekdlnOfnjeEH5dfeVfKQDms=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i0nsiRo+53J2/aCaEZhDkPzwQQGk8zchet0YBixha4yatCfarbOymZek+Jx2eCPVkkB6CG+uZrMZBfYH5V4O4eTILg6iiF3OyCET8GDyR1TFw1aXx2vP9LOKG6Ape0V3M8TJmpZNkHju/gzqCQKzOEg2RR2/gHRMH+WOrJIvYUg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=g8MxGxrz; arc=none smtp.client-ip=74.125.225.66 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="g8MxGxrz" Received: by mail-wr2-f2.google.com with SMTP id ffacd0b85a97d-4730b8edae6so496176f8f.0 for ; Sat, 25 Jul 2026 11:26:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785003997; x=1785608797; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=K0+z0feMX7JoKO2phMVBw4gbQDIWDzSyQ5EIA1ldW5E=; b=g8MxGxrzfvyNC20g/WABErO/tefQwXNJj8T4x255JnFtM4bCWhBQmMSxYdBL+lt6/z qw9I+WyBpDiGK6x2/hb7WMkeI8DBpK5Z4Y12MovKTCXoWKwHdiA4A17E4EUMlPVFFRbC GMLlJYzlkbhoKC/kjVJPXC6bWBLTgypEwN3S+5XmgeEHu6xAILZXnQLmsuhVWufJoOSn Er3GkwVNi/R/Un6v9aGPzxlXVWYB7eP3DGnpF51+TIG9rPpqRJF0prc5Z33TuJpVoPmm vRoanuTHZ7u/erJfJWAg0lcHibdPu6pHZLrOotDADLx1fEHV2yYlZRu1FPdq80LmqOgW KY6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785003997; x=1785608797; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=K0+z0feMX7JoKO2phMVBw4gbQDIWDzSyQ5EIA1ldW5E=; b=iR+L6GDHiqBl0L68orf2RPdD39xDCzRGL473Ke4MUqeGZLSIDe3ALQF+Beg3+pv/lc tjyMUSUg3vUYPFo7No9bdj4lxG/+za6vFem9lKXRebUf7+iGHCy53NgNqcxNiUdgzNdE QdTbbrW0VUWiOa+Gy2BCQSyjwbIdRWykcWJu7nLXsUZd5JugKc1MWdfeAjcGJcF/WL5f pxH1aCmVspAulp0lo4VFRLoPVaNwfFNOWoIFJh2S2XzKlh9T8wYuWoRkhGzQ45lX/o+I pI76HEbt2FuJaytTmaCdvlA9OaawtNG+zgrRKKI/BLEWF2nNOI7VvqM4QWoff0IlQKGT OzZg== X-Gm-Message-State: AOJu0YzXSg35Cn01WnlgXWi4IulvthXOB/eizTt4SxOOGIaX+RCuvvQh rCQHl0ip0CTp9Gqf6h4asPIy2bZCVKpo+mL4otCO9qyoGaj8Qyp8cGC8YW7e+5Wvxtw= X-Gm-Gg: AR+sD11rtECvGHueJxPojrNTxmzRhGyd8psNV3EkoOwkpekJMVTnbAllhltkUofHqW9 tbbzTK961yS6xNwgDxjfb0+Vk2IculGsXAn7Shvl1VMQre/yYmf/s4nLZeKQ7wSL/nooK8iSXt5 1/mPIOC00dOcMgbW+RU6kRavyoprY74WijEfMpCLGalZqiBp8uVvsB2woPqBAsJYHASWxpWWEpC z4t+98Rdrdr5IQEekgaRoKwrbLF4IxPZFvXkMPhaFprwVMPVf8SweL5QDmJDf7p+TNroFdfsEMw OPHbJclUFKT2RrV1xSW4Jkx51wNApH6Q49zuvakrRO9NO4gwMVGjsAChpiWPDfyFmm8jtU+nWtZ B/WF4V05sG4El4jbZGU3q+EJbutaHrHBZ5KBzXKxNh6pzG78oeyN/sGKrcuXFv5eO6YQby9epGm NOCK36YUg= X-Received: by 2002:a05:600c:35d5:b0:490:e5c1:b8bf with SMTP id 5b1f17b1804b1-496b56f385dmr37477295e9.13.1785003997454; Sat, 25 Jul 2026 11:26:37 -0700 (PDT) Received: from localhost ([178.241.98.184]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-496b499c473sm84741065e9.11.2026.07.25.11.26.36 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 25 Jul 2026 11:26:37 -0700 (PDT) From: Ayhan Aydin To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: nsd.project.dev@gmail.com Subject: [RFC PATCH 3/3] Documentation: Add NSD filesystem documentation Date: Sat, 25 Jul 2026 21:26:28 +0300 Message-ID: <20260725182628.221603-4-nsd.project.dev@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260725182628.221603-1-nsd.project.dev@gmail.com> References: <20260725182628.221603-1-nsd.project.dev@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add initial documentation for the Neural Storage Driver covering architecture, sysfs interface, and performance. Signed-off-by: Ayhan Aydin --- Documentation/filesystems/nsd.rst | 77 +++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 Documentation/filesystems/nsd.rst diff --git a/Documentation/filesystems/nsd.rst b/Documentation/filesystems/nsd.rst new file mode 100644 index 0000000..a78018a --- /dev/null +++ b/Documentation/filesystems/nsd.rst @@ -0,0 +1,77 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================================== +Neural Storage Driver (NSD) - v1.0.0 +======================================== + +Overview +======== +NSD is a learning prefetcher for the Linux kernel page cache. It monitors +I/O patterns via a hook in :c:func:`filemap_read` and prefetches pages +ahead of the application using :c:func:`page_cache_sync_readahead`. + +Unlike the kernel existing readahead (fixed window), NSD builds +a synaptic Markov chain model of access patterns at 4 KB region +granularity. It detects sequential strides, repeating patterns, and +learned transitions. + +Architecture +============:: + + Application + | + v + filemap_read() <-- NSD hook + | + +---> nsd_notify_read() + | + +---> Synaptic Table (Markov chain) + | | + | +---> Region hash + stride predictor + | + +---> Prefetch Worker + | + +---> page_cache_sync_readahead(WILLNEED) + +Sysfs Interface +=============== +/sys/kernel/nsd/:: + + stats Current statistics (hit rate, prefetch count, etc.) + observe_only 0=active, 1=observe-only (no prefetch) + penalty Enable penalty weakening on waste + waste_track Enable waste tracking + +Statistics fields:: + + prefetched Total pages prefetched + used Prefetched pages that were actually accessed + wasted Prefetched pages never accessed + hit_rate_real Used / Prefetched ratio + stride_preds Stride predictions made + chain_depth Average chain prediction depth + synapse_ents Current synapse table entries + ring_events Pending ring buffer events + +Module Parameters +================= + observe_only Start in observe-only mode (default: false) + penalty Enable penalty on waste (default: true) + waste_track Enable waste tracking (default: false) + +Performance +=========== +Tested on: x86_64, SATA SSD, kernel 7.0.0:: + + Workload Improvement + --------------------------- ----------- + SQLite FTS (4 GB table) -18.8% query time + Sequential 64K buffered +22.6% throughput + Random 4K buffered +1.1% (noise) + +The prefetcher achieves 98% real hit rate on SSD workloads. + +See Also +======== +Documentation/filesystems/fscache.rst +Documentation/filesystems/squashfs.rst \ No newline at end of file -- 2.43.0