KernelCI switch to new testing architecture - plan

KernelCI switch to new testing architecture - plan

[Gustavo Padovan]

Hello everyone,

As most of you know, the KernelCI community developed a new testing 
architecture.
It is now time to move over to it. In this message, we want to share 
some details about
the upcoming movements, especially about the shutdown of the existing 
legacy system.

In our latest blog post[1], we shared an update about what's coming. 
 From a strategic
point of view, not much has changed since that blog post. We made 
progress on the
implementations, making the systems more stable, adding more tests and 
labs and
refining the data connection with KCIDB.

As shared at that time, the primary focus in the new testing 
architecture is on quality
rather than quantity. We want high quality test results that will reduce 
the community's
workload, and not waste their time with low-quality test results. **As a 
direct
consequence of our focus on quality, we will not port all tests from 
legacy KernelCI
to the new system**. Reach out to us at kernelci@lists.linux.dev, if you 
want to see
specific tests ported and we will give you instructions and support on 
how to do it.

Current status:
* ready to receive new trees and tests
* moving existing labs over
* developing the new Web Dashboard

The new Web Dashboard should be ready in the coming months for basic 
visualization.
In the meantime, we have the grafana instance to build boards to 
visualize test results
and also some interim tooling that pulls the data from the API and 
generates reports.
If you want to bring your tree and tests over and use our tooling to 
analyze the results,
we can onboard you while also benefiting from your feedback to evolve 
our regression
tracking intelligence.

We will share more information about the transition as the work evolves 
in the
coming months. Don't worry!

The target is to shutdown the legacy system around the end of August[2]. 
Some labs
will probably disconnect themselves from legacy KernelCI before that. We 
can not delay
the shutdown date much as the amount of cloud resources and engineering 
is a high
toll to the project.

**If you are depending on legacy KernelCI, please speak up, so we can 
help you move
your stuff over to the new system.**

To be clear, the KernelCI project will not maintain 100% compatibility 
with what was
provided by its legacy system. As we create a better future for 
KernelCI, some hard decisions
had to be taken; not porting all tests and usecases was one of them.

As we leave our legacy behind, a vibrant community has been gathering 
around KernelCI in
the past few months, corroborating the new strategic directions we are 
putting in place for
the project. Bear with us during this transition and the future will be 
brighter!

We are happy to hear thoughts and questions from everyone about the 
transition. We are
probably forgetting some important details in this message.

Thank you for your continued support and your patience as the KernelCI 
project goes
through its biggest change since its inception.

Best Regards,

- Gus

[1] https://kernelci.org/blog/posts/2024/strategic-updates/
[2] https://github.com/kernelci/kernelci-project/issues/337

-- 
Gustavo Padovan
Kernel Lead

Collabora Ltd.
Platinum Building, St John's Innovation Park
Cambridge CB4 0DS, UK
Registered in England & Wales, no. 5513718

Re: KernelCI switch to new testing architecture - plan

[Gustavo Padovan]

Hello everyone,

It is for another update on the switch to the new testing architecture. 
As already highlighted in the previous
  email,we keep enabling new trees and tests, and enabling labs in the 
system. The new Web Dashboard is
  also under fast development, through a specialized supplier company 
hired by the KernelCI Foundation
  and a first beta version is expected by LPC.

We are also going through a major overhaul of the KernelCI project 
documentation to highlight the
  different ways different users can engage with KernelCI. For example: 
upstream kernel maintainers
  are interested in adding tests, Lab owners are interested in adding 
the labs, CI systems across the
  Linux community are interested in sending their data to KCIDB. It will 
take some time for all
  documentation changes to land, so hold tight and send an email to us 
at kernelci@lists.linux.dev
  if you have any questions.

As for shutting down our legacy system, in the coming days both the 
BayLibre and Collabora
  will be disconnected from the legacy system, removing a significant 
amount of devices. The
  message is pretty clear: no one should be using legacy KernelCI anymore.

**If you are depending on legacy KernelCI, please speak up, so we can 
help you move your
  stuff over to the new system.**

## What is legacy KernelCI?

It is the old Jenkins based system with results showing up at 
linux.kernelci.org - if you are still
  using it you are using legacy KernelCI


## Where is the dashboard for the new KernelCI?

As highlighted in the beginning of this message, the new dashboard is 
under fast development
  after a good phase of UX research. In the meantime, we created a 
grafana dashboard that we
  will deploy in the coming days and share the link here.

Let me know any other questions you have. We'd be happy to answer them.

Best regards,

- Gus

On 6/6/24 3:24 PM, Gustavo Padovan wrote:
> Hello everyone,
>
> As most of you know, the KernelCI community developed a new testing 
> architecture.
> It is now time to move over to it. In this message, we want to share 
> some details about
> the upcoming movements, especially about the shutdown of the existing 
> legacy system.
>
> In our latest blog post[1], we shared an update about what's coming. 
> From a strategic
> point of view, not much has changed since that blog post. We made 
> progress on the
> implementations, making the systems more stable, adding more tests and 
> labs and
> refining the data connection with KCIDB.
>
> As shared at that time, the primary focus in the new testing 
> architecture is on quality
> rather than quantity. We want high quality test results that will 
> reduce the community's
> workload, and not waste their time with low-quality test results. **As 
> a direct
> consequence of our focus on quality, we will not port all tests from 
> legacy KernelCI
> to the new system**. Reach out to us at kernelci@lists.linux.dev, if 
> you want to see
> specific tests ported and we will give you instructions and support on 
> how to do it.
>
> Current status:
> * ready to receive new trees and tests
> * moving existing labs over
> * developing the new Web Dashboard
>
> The new Web Dashboard should be ready in the coming months for basic 
> visualization.
> In the meantime, we have the grafana instance to build boards to 
> visualize test results
> and also some interim tooling that pulls the data from the API and 
> generates reports.
> If you want to bring your tree and tests over and use our tooling to 
> analyze the results,
> we can onboard you while also benefiting from your feedback to evolve 
> our regression
> tracking intelligence.
>
> We will share more information about the transition as the work 
> evolves in the
> coming months. Don't worry!
>
> The target is to shutdown the legacy system around the end of 
> August[2]. Some labs
> will probably disconnect themselves from legacy KernelCI before that. 
> We can not delay
> the shutdown date much as the amount of cloud resources and 
> engineering is a high
> toll to the project.
>
> **If you are depending on legacy KernelCI, please speak up, so we can 
> help you move
> your stuff over to the new system.**
>
> To be clear, the KernelCI project will not maintain 100% compatibility 
> with what was
> provided by its legacy system. As we create a better future for 
> KernelCI, some hard decisions
> had to be taken; not porting all tests and usecases was one of them.
>
> As we leave our legacy behind, a vibrant community has been gathering 
> around KernelCI in
> the past few months, corroborating the new strategic directions we are 
> putting in place for
> the project. Bear with us during this transition and the future will 
> be brighter!
>
> We are happy to hear thoughts and questions from everyone about the 
> transition. We are
> probably forgetting some important details in this message.
>
> Thank you for your continued support and your patience as the KernelCI 
> project goes
> through its biggest change since its inception.
>
> Best Regards,
>
> - Gus
>
> [1] https://kernelci.org/blog/posts/2024/strategic-updates/
> [2] https://github.com/kernelci/kernelci-project/issues/337
>

-- 
Gustavo Padovan
Kernel Lead

Collabora Ltd.
Platinum Building, St John's Innovation Park
Cambridge CB4 0DS, UK
Registered in England & Wales, no. 5513718