Re: stable/LTS test report from KernelCI (2023-12-08)

[Greg KH]

On Fri, Dec 08, 2023 at 12:29:35PM -0300, Gustavo Padovan wrote:
> Hello,
> 
> As discussed with Greg at LPC, we are starting an iterative process to
> deliver meaningful stable test reports from KernelCI. As Greg pointed out,
> he doesn't look at the current reports sent automatically from KernelCI.
> Those are not clean enough to help the stable release process, so we
> discussed starting over again.
> 
> This reporting process is a learning exercise, growing over time. We are
> starting small with data we can verify manually (at first) to make sure we
> are not introducing noise or reporting flakes and false-positives. The
> feedback loop will teach us how to filter the results and report with
> incremental automation of the steps.
> 
> Today we are starting with build and boot tests (for the hardware platforms
> in KernelCI with sustained availability over time). Then, at every iteration
> we try to improve it, increasing the coverage and data visualization.
> Feedback is really important. Eventually, we will also have this report
> implemented in the upcoming KernelCI Web Dashboard.
> 
> This work is a contribution from Collabora(on behalf of its clients) to
> improve the Kernel Integration as whole. Moving forward, Shreeya Patel, from
> the Collabora team will be taking on the responsibilities of delivering
> these reports.
> 
> Without further ado, here's our first report:
> 
> 
> ## stable-rc HEADs:
> 
> Date: 2023-12-08
> 6.1: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/log/?h=45deeed0dade29f16e1949365688ea591c20cf2c
> 5:15: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/log/?h=e5a5d1af708eced93db167ad55998166e9d893e1
> 5.10: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/log/?h=ce575ec88a51a60900cd0995928711df8258820a
> 5:4: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/log/?h=f47279cbca2ca9f2bbe1178634053024fd9faff3
> 
> * 6.6 stable-rc was not added in KernelCI yet, but we plan to add it next
> week.
> 
> 
> ## Build failures:
> 
> No build failures seen for the stable-rc/queue commit heads for
> 6.1/5.15/5.10/5.4  \o/
> 
> 
> ## Boot failures:
> 
> No **new** boot failures seen for the stable-rc/queue commit heads for
> 6.1/5.15/5.10/5.4  \o/
> 
> (for the time being we are leaving existing failures behind)
> 
> 
> ## Considerations
> 
> All this data is available in the legacy KernelCI Web Dashboard -
> https://linux.kernelci.org/ - but not easily filtered there. The data in
> this report was checked manually. As we evolve this report, we want to add
> traceability of the information, making it really easy for anyone to dig
> deeper for more info, logs, etc.
> 
> The report covers  the hardware platforms in KernelCI with sustained
> availability over time - we will detail this further in future reports.
> 
> We opted to make the report really simple as you can see above. It is just
> an initial spark. From here your feedback will drive the process. So really
> really tell us what you want to see next. We want FEEDBACK!

Looks great!

A few notes, it can be a bit more verbose if you want :)

One email per -rc release (i.e. one per branch) is fine, and that way if
you add a:
	Tested-by: kernelci-bot <email goes here>
or something like that, to the email, my systems will pick it up and it
will get added to the final commit message.

But other than that, hey, I'll take the above, it's better than what was
there before!

How about if something breaks, what will it look like?  That's where it
gets more "interesting" :)

thanks,

greg k-h