regression tracking

regression tracking

[Bjorn Helgaas]

The regression tracking that you folks used to do before the demise of
bugzilla.kernel.org was incredibly valuable.

I'm very concerned about the fact that we don't have a systematic way
to do this any more.  Some things do get reported on LKML, of course,
but without the legwork you guys did, I'm afraid we're just dropping
things on the floor because there's no summary of what the issues are,
how many people are seeing them, who's responsible, etc.

Consider this as (1) kudos for performing an essential community
service, and (2) a plea for the resurrection or replacement of
bugzilla.kernel.org or some other mechanism for reviving regression
tracking.

Bjorn

Re: regression tracking

[Maciej Rutecki]

Hi

On środa, 26 października 2011 o 19:23:11 Bjorn Helgaas wrote:
> The regression tracking that you folks used to do before the demise of
> bugzilla.kernel.org was incredibly valuable.
> 
> I'm very concerned about the fact that we don't have a systematic way
> to do this any more.  Some things do get reported on LKML, of course,
> but without the legwork you guys did, I'm afraid we're just dropping
> things on the floor because there's no summary of what the issues are,
> how many people are seeing them, who's responsible, etc.
> 
> Consider this as (1) kudos for performing an essential community
> service, and (2) a plea for the resurrection or replacement of
> bugzilla.kernel.org or some other mechanism for reviving regression
> tracking.

Because bugzilla is still down, I yesterday started send regressions (found in 
LKML) report directly to Rafael, to create summary report (etc.) of them. 
Until bugzilla comes back, we plan to collect regression in this way, but any 
proposal for a better solution is welcome.

Regards
-- 
Maciej Rutecki
http://www.mrutecki.pl

Re: regression tracking

[Rafael J. Wysocki]

On Wednesday, October 26, 2011, Maciej Rutecki wrote:
> Hi
> 
> On środa, 26 października 2011 o 19:23:11 Bjorn Helgaas wrote:
> > The regression tracking that you folks used to do before the demise of
> > bugzilla.kernel.org was incredibly valuable.
> > 
> > I'm very concerned about the fact that we don't have a systematic way
> > to do this any more.  Some things do get reported on LKML, of course,
> > but without the legwork you guys did, I'm afraid we're just dropping
> > things on the floor because there's no summary of what the issues are,
> > how many people are seeing them, who's responsible, etc.
> > 
> > Consider this as (1) kudos for performing an essential community
> > service, and (2) a plea for the resurrection or replacement of
> > bugzilla.kernel.org or some other mechanism for reviving regression
> > tracking.
> 
> Because bugzilla is still down, I yesterday started send regressions (found in 
> LKML) report directly to Rafael, to create summary report (etc.) of them. 
> Until bugzilla comes back, we plan to collect regression in this way, but any 
> proposal for a better solution is welcome.

Seconded.

Especially if there's a password-protected Wiki service somewhere that we
could use to store data temporarily until the BZ is back, it would be much
appreciated.

Thanks,
Rafael

Re: regression tracking

[Bjorn Helgaas]

On Thu, Oct 27, 2011 at 2:35 PM, Rafael J. Wysocki <rjw@sisk.pl> wrote:
> On Wednesday, October 26, 2011, Maciej Rutecki wrote:
>> Hi
>>
>> On środa, 26 października 2011 o 19:23:11 Bjorn Helgaas wrote:
>> > The regression tracking that you folks used to do before the demise of
>> > bugzilla.kernel.org was incredibly valuable.
>> >
>> > I'm very concerned about the fact that we don't have a systematic way
>> > to do this any more.  Some things do get reported on LKML, of course,
>> > but without the legwork you guys did, I'm afraid we're just dropping
>> > things on the floor because there's no summary of what the issues are,
>> > how many people are seeing them, who's responsible, etc.
>> >
>> > Consider this as (1) kudos for performing an essential community
>> > service, and (2) a plea for the resurrection or replacement of
>> > bugzilla.kernel.org or some other mechanism for reviving regression
>> > tracking.
>>
>> Because bugzilla is still down, I yesterday started send regressions (found in
>> LKML) report directly to Rafael, to create summary report (etc.) of them.
>> Until bugzilla comes back, we plan to collect regression in this way, but any
>> proposal for a better solution is welcome.
>
> Seconded.
>
> Especially if there's a password-protected Wiki service somewhere that we
> could use to store data temporarily until the BZ is back, it would be much
> appreciated.

I don't know exactly what you would like to do with a wiki, but Google
Docs might be a possibility.

Disclaimer: I work for Google but this is not any sort of official
offer of help or support beyond the standard functionality that's
available to everybody.

Bjorn

Re: Regression tracking

[Florian Mickler]

On Thu, 23 Feb 2012 18:05:32 -0500
Dave Jones <davej@redhat.com> wrote:

> On Thu, Feb 23, 2012 at 11:52:13PM +0100, Florian Mickler wrote:
> 
>  > If you don't do that, I fear that I will annoy you from time to time
>  > with: "is this issue fixed already?" or "hey, what about that 4 month
>  > old regression? Is it already fixed?".
> 
> If you intend to do this, please add me to your "do not mail" list.
> 
> For people working at distributions acting as middle-man, this isn't
> helpful at all. And it's just not feasible for me to go dig out every
> thread I posted to just to reply with "fixed".
> 
> 	Dave
> 

You are not on my go-to-guy list since you do not maintain a kernel
subsystem and I don't expect you to keep track of anything. If you post
a link to a rh-bugzilla, I will of course check if that one is still
open or how it is closed. I'm not stupid, I just need some
silver linen threads to find how a regression is solved.

I.e. I don't want a "fixed". I want a "commit xyz fixed this"
or a "I can't reproduce the bug anymore". Either is fine. 

Re: Regression tracking

[Dave Jones]

On Thu, Feb 23, 2012 at 11:52:13PM +0100, Florian Mickler wrote:

 > If you don't do that, I fear that I will annoy you from time to time
 > with: "is this issue fixed already?" or "hey, what about that 4 month
 > old regression? Is it already fixed?".

If you intend to do this, please add me to your "do not mail" list.

For people working at distributions acting as middle-man, this isn't
helpful at all. And it's just not feasible for me to go dig out every
thread I posted to just to reply with "fixed".

	Dave

Re: Regression tracking

[Florian Mickler]

On Mon, 23 Jan 2012 01:52:25 +0100
"Rafael J. Wysocki" <rjw@sisk.pl> wrote:

> On Sunday, January 22, 2012, David Miller wrote:
> > From: Maciej Rutecki <maciej.rutecki@gmail.com>
> > Date: Sun, 22 Jan 2012 20:52:02 +0100
> > 
> > > OK. But tracking regressions in two (or more) places is nonsense. And this 
> > > puts into question all of my current work, such as how to analyze 
> > > (automatically) the progress of the whole kernel and its quality per each -rc. 
> > > Problem to discussion, but if everyone will do as you wish, it will all work 
> > > went to waste.
> > 
> > If someone else wants to maintain the state of bugs on some web
> > site and click buttons all day long, that is their perogative.
> > 
> > But it's not something I'm going to do.
> > 
> > You can't force people to use tools, and frankly that's the end of
> > this conversation as far as I'm concerned.

What I would like to ask from you and other developers in order to make
life easier for all users reporting bugs and people tracking
regressions is to post a solution for an issue to the mail thread it
was first reported in. That way we can just read through that mail
thread and see that it is resolved.

Also if you are aware, that someone (Maciej!) did open a bug for an
issue, please(!) stick that number in the commit changelog. Full Url
would be wonderful for automated processing, but "bug [nr]" is
sufficient for manual scanning of the commit log. 


If you don't do that, I fear that I will annoy you from time to time
with: "is this issue fixed already?" or "hey, what about that 4 month
old regression? Is it already fixed?". (And knowing netdev, it is of
course already fixed in 99.9% of the cases. So I just need the pointer
to the solution...)

[If I have much time at hand, I will even scan the mail chatter of the
people involved during the time frame the bug was reported, but that is
a tedious task... still I do that sometimes in order to avoid annoying
people with this silly stuff]

Regards,
Flo

Re: Regression tracking

[Rafael J. Wysocki]

On Sunday, January 22, 2012, David Miller wrote:
> From: Maciej Rutecki <maciej.rutecki@gmail.com>
> Date: Sun, 22 Jan 2012 20:52:02 +0100
> 
> > OK. But tracking regressions in two (or more) places is nonsense. And this 
> > puts into question all of my current work, such as how to analyze 
> > (automatically) the progress of the whole kernel and its quality per each -rc. 
> > Problem to discussion, but if everyone will do as you wish, it will all work 
> > went to waste.
> 
> If someone else wants to maintain the state of bugs on some web
> site and click buttons all day long, that is their perogative.
> 
> But it's not something I'm going to do.
> 
> You can't force people to use tools, and frankly that's the end of
> this conversation as far as I'm concerned.

Well, people who are tracking regressions need to keep the record of
what's been reported etc. somewhere and we use the Bugzilla for this
purpose (basically, as a database).  I, personally, have never been
expecting developers to follow the entries put in there by us, unless
they want to, but then we'd like the _reporters_ to update the status
(resolved/closed) which saves us quite some time (e.g. if someone
closes a bug entry created for a regression reported by him, we don't
need to dig through the git history and mailing lists archives or ask
developers whether or not the given bug has been fixed).

I hope that clarifies things.

Thanks,
Rafael

Re: Regression tracking

[David Miller]

From: Maciej Rutecki <maciej.rutecki@gmail.com>
Date: Sun, 22 Jan 2012 20:52:02 +0100

> OK. But tracking regressions in two (or more) places is nonsense. And this 
> puts into question all of my current work, such as how to analyze 
> (automatically) the progress of the whole kernel and its quality per each -rc. 
> Problem to discussion, but if everyone will do as you wish, it will all work 
> went to waste.

If someone else wants to maintain the state of bugs on some web
site and click buttons all day long, that is their perogative.

But it's not something I'm going to do.

You can't force people to use tools, and frankly that's the end of
this conversation as far as I'm concerned.