Re: LTS testing with latest kselftests - some failures

[Shuah Khan]

Hi Sumit,

On 06/15/2017 11:56 AM, Sumit Semwal wrote:
> Hello Greg, Shuah,
> 
> While testing 4.4.y and 4.9.y LTS kernels with latest kselftest, we
> found a couple more test failures due to test-kernel mismatch:
> 
> 1. firmware tests: - linux 4.5 [1] and 4.10 [2] added a few updates to
> tests, and related updates to lib/test_firmware.c to improve the
> tests. Stable-4.4 misses these patches to lib/test_firmware.c. Stable
> 4.9 misses the second update.

I will take a look the commit list your provided and see if it makes sense
to back-port. As we discussed earlier in this thread, fixes are something
that can be back-ported to include in the stable releases. Updates will not
be.

> 2. Bitmap test - test got added in 4.5, fails if test_bitmap.ko isn't present.

If you can send a patch to have it exit gracefully and change the message
to say unsupported feature, I can pull that into mainline can mark it for
stable inclusion if it qualifies.

> 
> 3. 'seccomp ptrace hole closure' patches got added in 4.7 [3] -
> feature and test together.
> - This one also seems like a security hole being closed, and the
> 'feature' could be a candidate for stable backports, but Arnd tried
> that, and it was quite non-trivial. So perhaps  we'll need some help
> from the subsystem developers here.

This is something I need to look at and consult with Security maintainers.

> 
> For all the 3 listed above, we will try and update the tests to gracefully exit.

That is great.

> 
> 
> 4. bpf tests: These seem to have build failures in mainline as well -
> I also tried to build kselftest-next, but a simple 'make -C
> tools/testing/selftests/bpf' seems to error out. Are there any special
> instructions to build these? [I tried x86_64, arm64 cross-compile on x86_64]

Hmm. I didn't notice this one. I will check and see what's going on there.

> 
> 
> I will also individually request subsystem authors / mailing lists for
> each of these towards help in improving these tests if required, but
> wanted to use this thread as a converging point.

Please cc me and linux-kselftest mailing list.

Thanks for reporting the problems.

> 
> 
> [1]: https://lkml.org/lkml/2015/12/8/816
> Patches added via [1]:
> eb910947c82f (test: firmware_class: add asynchronous request trigger)
> be4a1326d12c (test: firmware_class: use kstrndup() where appropriate)
> 47e0bbb7fa98 (test: firmware_class: report errors properly on failure)
> 
> [2]: https://lkml.org/lkml/2017/1/23/440
> Patch added via [2]:
> 061132d2b9c9 (test_firmware: add test custom fallback trigger)
> 
> [3]: https://lkml.org/lkml/2016/6/9/627
> 
> 

thanks,
-- Shuah