FYI, git output messages are being used by other tooling

FYI, git output messages are being used by other tooling

[Steven Rostedt]

Hi,

I just want to let you folks know that ktest.pl (tools/testing/ktest/ in
the Linux kernel repository) parses the output of git to display the first
bad commit found in an automated git bisect.

The output was recently updated via: 0c0f93e7fa64 ("bisect: print bisect
terms in single quotes") which made ktest.pl fail to find the bad commit.

The change is being addressed:

  https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/

But I figured it would be good for you to know about this dependency.

-- Steve

Re: FYI, git output messages are being used by other tooling

[Junio C Hamano]

Steven Rostedt <rostedt@goodmis.org> writes:

> Hi,
>
> I just want to let you folks know that ktest.pl (tools/testing/ktest/ in
> the Linux kernel repository) parses the output of git to display the first
> bad commit found in an automated git bisect.
>
> The output was recently updated via: 0c0f93e7fa64 ("bisect: print bisect
> terms in single quotes") which made ktest.pl fail to find the bad commit.
>
> The change is being addressed:
>
>   https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/
>
> But I figured it would be good for you to know about this dependency.

Shouldn't the script instead seeing "git show bisect/bad" instead?

Re: FYI, git output messages are being used by other tooling

[Kristoffer Haugsbakk]

On Tue, Oct 20, 2026, at 20:10, Steven Rostedt wrote:
> Hi,
>
> I just want to let you folks know that ktest.pl (tools/testing/ktest/ in
> the Linux kernel repository) parses the output of git to display the first
> bad commit found in an automated git bisect.
>
> The output was recently updated via: 0c0f93e7fa64 ("bisect: print bisect
> terms in single quotes") which made ktest.pl fail to find the bad commit.
>
> The change is being addressed:
>
>   
> https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/
>
> But I figured it would be good for you to know about this dependency.

There were recent threads about how to
query for the first bad commit.

https://lore.kernel.org/git/asAbOSQ4BkuCTPY5@debian/

I had a recent script that I put on hold and didn't
finish. But it involved using exit codes and querying
for things like ref state. And maybe using
git rev-list --bisect.

Re: FYI, git output messages are being used by other tooling

[Steven Rostedt]

On Tue, 20 Oct 2026 11:37:01 -0700
Junio C Hamano <gitster@pobox.com> wrote:

> Steven Rostedt <rostedt@goodmis.org> writes:
> 
> > Hi,
> >
> > I just want to let you folks know that ktest.pl (tools/testing/ktest/ in
> > the Linux kernel repository) parses the output of git to display the first
> > bad commit found in an automated git bisect.
> >
> > The output was recently updated via: 0c0f93e7fa64 ("bisect: print bisect
> > terms in single quotes") which made ktest.pl fail to find the bad commit.
> >
> > The change is being addressed:
> >
> >   https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/
> >
> > But I figured it would be good for you to know about this dependency.  
> 
> Shouldn't the script instead seeing "git show bisect/bad" instead?

I could look into changing it. The code was written in 2010 so it could
perhaps be updated ;-)

Thanks,

-- Steve

Re: FYI, git output messages are being used by other tooling

[Theodore Tso]

On Tue, Oct 20, 2026 at 11:37:01AM -0500, Junio C Hamano wrote:
> > The change is being addressed:
> >
> >   https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/
> >
> > But I figured it would be good for you to know about this dependency.
> 
> Shouldn't the script instead seeing "git show bisect/bad" instead?
>

I wonder if we should have something like a "git bisect --format=json"
option to facilitate programs that are running git bisect in some kind
of automation.  Steven's email caused me to check my test appliance[1]
but fortunately the only thing I depend on is whether the otuput of
"git bisect good/bad" has a string which contains "is the first bad
commit" or not.  If someone has a commit with the one-line summary
which contains that string, or if git ever changed the git bisect text
output, or someone tried to set LANG to Chinese or some such, hilarity
would ensue.   :-)

[1] https://github.com/tytso/xfstests-bld/blob/master/test-appliance/files/usr/local/lib/gce-server/util/git/git.go

A structured output that contained the result of the "git bisect
good|bad", and the first bad commit if it has been found, would be
certainly be a more reliable way for automated git bisection harneses
to rely upon.  It'd be great if that was something that was in git
3.0.

						- Ted

Re: FYI, git output messages are being used by other tooling

[Steven Rostedt]

On Tue, 20 Oct 2026 11:37:01 -0700
Junio C Hamano <gitster@pobox.com> wrote:

> Steven Rostedt <rostedt@goodmis.org> writes:
> 
> > Hi,
> >
> > I just want to let you folks know that ktest.pl (tools/testing/ktest/ in
> > the Linux kernel repository) parses the output of git to display the first
> > bad commit found in an automated git bisect.
> >
> > The output was recently updated via: 0c0f93e7fa64 ("bisect: print bisect
> > terms in single quotes") which made ktest.pl fail to find the bad commit.
> >
> > The change is being addressed:
> >
> >   https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/
> >
> > But I figured it would be good for you to know about this dependency.  
> 
> Shouldn't the script instead seeing "git show bisect/bad" instead?

Hmm, it also is used to see if the bisect finished or not. It looks at the
output of the last git bisect action and has:

    if ($output =~ m/^(Bisecting: .*\(roughly \d+ steps?\))\s+\[([[:xdigit:]]+)\]/) {
	doprint "$1 [$2]\n";
    } elsif ($output =~ m/^([[:xdigit:]]+) is the first bad commit/) {
	$bisect_bad_commit = $1;
	doprint "Found bad commit... $1\n";
	return 0;
    } else {
	# we already logged it, just print it now.
	print $output;
    }
    return 1;

Where returning 0 means the bad commit has been found.
Otherwise it keeps looping.

Is there a git command to know if the bad bisect has been found or not?

-- Steve

Re: FYI, git output messages are being used by other tooling

[Kristoffer Haugsbakk]

On Tue, Oct 20, 2026, at 21:23, Steven Rostedt wrote:
> On Tue, 20 Oct 2026 11:37:01 -0700
> Junio C Hamano <gitster@pobox.com> wrote:
>
>> Steven Rostedt <rostedt@goodmis.org> writes:
>> 
>> > Hi,
>> >
>> > I just want to let you folks know that ktest.pl (tools/testing/ktest/ in
>> > the Linux kernel repository) parses the output of git to display the first
>> > bad commit found in an automated git bisect.
>> >
>> > The output was recently updated via: 0c0f93e7fa64 ("bisect: print bisect
>> > terms in single quotes") which made ktest.pl fail to find the bad commit.
>> >
>> > The change is being addressed:
>> >
>> >   https://lore.kernel.org/all/20261009-ktest-git-bisect-v1-1-1d1428f6cbf4@marliere.net/
>> >
>> > But I figured it would be good for you to know about this dependency.  
>> 
>> Shouldn't the script instead seeing "git show bisect/bad" instead?
>
> Hmm, it also is used to see if the bisect finished or not. It looks at the
> output of the last git bisect action and has:
>
>     if ($output =~ m/^(Bisecting: .*\(roughly \d+ 
> steps?\))\s+\[([[:xdigit:]]+)\]/) {
> 	doprint "$1 [$2]\n";
>     } elsif ($output =~ m/^([[:xdigit:]]+) is the first bad commit/) {
> 	$bisect_bad_commit = $1;
> 	doprint "Found bad commit... $1\n";
> 	return 0;
>     } else {
> 	# we already logged it, just print it now.
> 	print $output;
>     }
>     return 1;
>
> Where returning 0 means the bad commit has been found.
> Otherwise it keeps looping.

Is this different from git bisect run?

>
> Is there a git command to know if the bad bisect has been found or not?
>
> -- Steve

Sent from mobile

Re: FYI, git output messages are being used by other tooling

[Steven Rostedt]

On Tue, 20 Oct 2026 21:28:10 +0200
"Kristoffer Haugsbakk" <kristofferhaugsbakk@fastmail.com> wrote:

> > Where returning 0 means the bad commit has been found.
> > Otherwise it keeps looping.  
> 
> Is this different from git bisect run?

Yes. It's ktest.pl that was created back in 2010 for automated testing of
Linux kernels. It will build, install, boot and test each kernel and based
on the result it will do a git bisect good/bad.

Thus it is much more advanced that "git bisect run".

-- Steve

Re: FYI, git output messages are being used by other tooling

[Theodore Tso]

On Tue, Oct 20, 2026 at 09:28:10PM -0500, Kristoffer Haugsbakk wrote:
> > Where returning 0 means the bad commit has been found.
> > Otherwise it keeps looping.
> 
> Is this different from git bisect run?

"git bisect run" isn't always the best way to manage a bisection.  For
example, I might run a command like:

   gce-xfstests ltm -c ext4/4k,ext4/1k  generic/245 --repo stable-gc.git \
       --bisect-good v6.18.57 --bisect-bad queue/6.18

This results in an RPC call to the Lightweight Test Manager (ltm),
which is a micro VM running in Google Compute Engine.  If there isn't
a Kernel Compile Service (kcs) VM already launched, the ltm VM will
launch a much larger build VM.  Once the kcs VM is ready, the bisect
request is then forwarded to the kcs VM.  The kcs VM will check out
relevant repo, run "git bisect start", and will do the build.  Once
the kernel is built, it then sends a RPC to the ltm VM, and the ltm VM
will launch one or more test VM's (in above example, one for the
ext4/4k config, and one of the ext4/1k config).  The ltm VM will
monitor the serial console of the systems under test, so if the kernel
hangs, the ltm VM can force reboot a failing VM and either continue
with the next text (if there is more than one test that is being run).
If we are doing a bisect, then the ltm VM will send a bisect update to
the kcs VM via an RPC.

This architecture allows multiple biseccts or test watches to be
managed by a single ltm VM and (if necessary the kcs VM --- the kcs VM
will shut itself down if it's not needed to save on cloud costs).

So yeah.  Sometimes "git bisect run" isn't really enough, and it would
be useful to have a way for programs to be able to call other "git
bisect" subcommands, and have a standardized way to get the results
from the command without having to just parse the output and be
dependant on the precise format of the human-readable output.

Cheers,

					- Ted

P.S.  For more information about how my test appliance works, and the
other features/functionality of the ltm VM, see [1] and [2].

[1] https://thunk.org/gce-xfstests
[2] https://github.com/tytso/xfstests-bld/blob/master/Documentation/gce-xfstests.md