From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A445E37F8DD for ; Fri, 3 Apr 2026 07:47:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775202454; cv=none; b=ksmgC8WkrXyc6f/mhP1PQCMYH18zumdRvvljOCTKv3V9Kit4JS+IBG5gB8I0Js0GEE5VDLtfR9v0nkKuQsy7+5c6WsaUcg/JuKWpaHkgDRW0Ddz3yv7x5eeLWW/vvUtVg7elSYsH0jLUGdTN+V8+h6xBpyvwNyNT0/bcR3Wke0A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775202454; c=relaxed/simple; bh=lYjog9cOhXXGU6rje935IS8T+D5KK6v5oOcIIHPU8jA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=gtHd9jEuiirjzaTEIx1MOki2oDs+y3gQowQ7cxAxe9iTjQHAEevlAkhtkfW3wZL8x9GPTKDI0TIKUN18De4U+k0vwbK2IxFhX+yYKa+Tt1MNgQpdhVBJxGQux/1qqPF68od7flUelg+lPHDgyO9GtluShuKzswuii6j6DjqzPVU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=M8qMiXeL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="M8qMiXeL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20072C4CEF7; Fri, 3 Apr 2026 07:47:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1775202454; bh=lYjog9cOhXXGU6rje935IS8T+D5KK6v5oOcIIHPU8jA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=M8qMiXeLlaFDbV6Oq/lEU649CTuXN/cgjPZBdUCXhMXmFtdurC42laQDzgvRlrjkY 2Vbgp4KSNDvlJxB7Xn7TVAt2h1GJYcx8iz5UeMxLjY5LcAtgqWPAb+OrxkHXl7u3zO +yIhtKfX+/kgvrDSopbQJqYWtyHj1/o8vrP1EfYdiGboXkgEYaQZxFmi3YKtuwFehY Rwa4eVq3/wenL0YhmtnU/F/flw6Jmxw0F9sAjlcS2E/fmEd75HUL3i506RaATlc9Yk TmnY1R3Bg+EP1+OJsaecOt24zfpww/8sWc6zZTBxPWnAkUC7nFKGvcYI96LUZPKfoG /cQLtUS6E40wQ== Date: Fri, 3 Apr 2026 08:47:27 +0100 From: "Lorenzo Stoakes (Oracle)" To: Roman Gushchin Cc: Sean Christopherson , linux-kernel , Andrew Morton , Theodore Ts'o , Guenter Roeck , Konstantin Ryabitsev , Chris Mason , SeongJae Park , elkin@google.com, Christian Brauner , Dmitry Vyukov , Sasha Levin , Shakeel Butt , Lorenzo Stoakes , Ian Rogers , Venkatesh Srinivas , Paolo Bonzini Subject: Re: Introduce Sashiko (agentic review of Linux kernel changes) Message-ID: <50fa537f-65e2-4e8a-b9ab-932a22943a3a@lucifer.local> References: <7ia4o6kmpj5s.fsf@castle.c.googlers.com> <39e6b4d2-8a30-4eaa-908d-5d11b746f8d5@lucifer.local> <87v7etugwd.fsf@linux.dev> <34630bb5-840b-4a99-8e19-51fd4fc8ba96@lucifer.local> <87jyv7a1q5.fsf@linux.dev> <875x68n7a5.fsf@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <875x68n7a5.fsf@linux.dev> On Thu, Apr 02, 2026 at 06:48:18PM -0700, Roman Gushchin wrote: > Sean Christopherson writes: > > > +Venkatesh and Paolo > > > > On Thu, Mar 19, 2026, Roman Gushchin wrote: > >> "Lorenzo Stoakes (Oracle)" writes: > >> > On Wed, Mar 18, 2026 at 11:33:22AM -0700, Roman Gushchin wrote: > >> >> >> Finally, some subsystems have a good prompts coverage and some don't. It > >> >> >> doesn't have to be lengthy documentation (and it might actually be > >> >> >> counter-productive), but having a small list of things to look at - some > >> >> >> high-level concepts which are hard to grasp from the code, etc. - can > >> >> >> help a lot with both bug discovery and false positives. > >> >> > > >> >> > I guess best contributed to Chris's review-prompts repo right? > >> >> > >> >> Both works for me now, we'll figure out with Chris how to sync our > >> >> prompts. The small problem is that we're using various models, tools and > >> >> review protocols and barely can test each other's setup. And it's all > >> >> very fragile, so it's not exactly trivial. > >> >> But we'll figure out something soon. > >> > Yeah, part of the fun I guess :) > >> > > >> >> In general we need to carefully separate instructions (like which tools > >> >> to use, which prompts to load etc) from factual data. Then we can easily > >> >> use the factual data with various tooling around. > > > > In an offline conversation, Venkatesh had a very (IMO) insightful observation > > regarding the factual data of the prompts: the information is also very useful > > documentation for *humans*. And in response to me lamenting about having to > > potentially review an external repo, Venkatesh also suggested putting the gory > > details about subsystem behavior in the kernel's Documentation/. > > > > To me, that suggestion seems like a no brainer. The existing subject matter > > experts are already in place to review and help maintain the documentation, the > > documentation can be updated in lockstep with the code, those of us that like > > email-based review don't need to change our ways, etc. :-) > > > > And irrespective of AI domination, I'd love to have detailed documenation of some > > of KVM's gnarlier internals. If AI review is what gets us the staffing/motivation > > to write and maintain that documentation, then so be it. It would be a shame if > > some of the most comprehensive documentation for the kernel is buried in AI > > specific prompts. > > > > Naively, synchronizing from Documentation to model-specific bots doesn't seem > > like it'd be a hard problem to solve. > > I think so too, thanks Sean! > > First, I agree improving the documentation is a no-brainer with or > without AI. And AI will benefit from it too. > > The only part which is slightly less obvious is what should go into > prompts vs what the model gets through training. I'm pretty sure that > the Linux kernel source code is used during the training, so at least > big frontier models "know" a lot about the kernel code already, so > prompts should really only close the gap between the cut date and new > developments plus all kinds of tribal knowledge which is not easy to > derive from the code or documentation (e.g. the outcome of some > hallway conversation during the last conference). Well :) we constantly struggle with documentation as the catch-22 is that the people who know enough to document are all too busy to write it. We have tried for years to encourage but it is what it is. And we do kinda require the documentation to be up to a certain standard and applicable to $CURRENT_KERNEL. [Side note: My book side-stepped this by a. being tied to a frozen kernel version and b. I could skip any area I deemed out of scope (though it differs in approach too - isn't documentation so much as a code commentary with lots of explanation).] I mean I've made my own efforts on this to some degree, e.g. https://docs.kernel.org/mm/process_addrs.html so have tossed my hat into the ring, but how consumable that'd be I'm not sure :) and presumably the LLMs gobble all that up anyway. > > But I'm hopeful we can figure out a way to auto-generate prompts from > the documentation by stripping part which are "obvious" to the model. Yeah I also wonder to what degree finessing things for LLM consumption matters too, I have no expereince with that but I hear that there's some of that going on in Chris's prompts? I did imagine mm-specific stuff for the LLM could be in the form of _generalised_ concepts and imperative instructions instructions. > > Thanks! Anyway semi-rambling here so TL;DR: It'd probably be quicker to move on LLM-specific prompts - they'll have less scrutiny than generalised documentation, less effort required to format, and more able to finesse for LLM purposes. OTOH documentation would be more broadly useful for humans too, but I just wonder if anybody will have the time to do that :>) Cheers, Lorenzo