[PATCH v3 00/12] selftests/nolibc: add minimal kernel config support - part1

[Zhangjin Wu]

Hi, Willy

Most of the suggestions of v2 [1] have been applied in this v3 revision,
except the local menuconfig and mrproper targets, as explained in [2].

A fresh run with tinyconfig for ppc, ppc64 and ppc64le:

    $ for arch in ppc ppc64 ppc64le; do \
        mkdir -p $PWD/kernel-$arch;
        time make defconfig run DEFCONFIG=tinyconfig ARCH=$arch O=$PWD/kernel-$arch RUN_OUT=$PWD/run.$arch.out;
      done

rerun for ppc, ppc64 and ppc64le:

    $ for arch in ppc ppc64 ppc64le; do \
        make rerun ARCH=$arch O=$PWD/kernel-$arch RUN_OUT=$PWD/run.$arch.out;
      done
    Running /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/kernel-ppc/vmlinux on qemu-system-ppc
    >> [ppc] Kernel command line: console=ttyS0 panic=-1 
    printk: console [ttyS0] enabled
    Run /init as init process
    Running test 'startup'
    Running test 'syscall'
    Running test 'stdlib'
    Running test 'vfprintf'
    Running test 'protection'
    Leaving init with final status: 0
    reboot: Power down
    powered off, test finish
    qemu-system-ppc: terminating on signal 15 from pid 190248 ()

    165 test(s): 156 passed,   9 skipped,   0 failed => status: warning

    See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.ppc.out
    Running /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/kernel-ppc64/vmlinux on qemu-system-ppc64
    Linux version 6.4.0+ (ubuntu@linux-lab) (powerpc64le-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2 SMP Fri Jul 28 01:40:55 CST 2023
    Kernel command line: console=hvc0 panic=-1 
    printk: console [hvc0] enabled
    printk: console [hvc0] enabled
    Run /init as init process
    Running test 'startup'
    Running test 'syscall'
    Running test 'stdlib'
    Running test 'vfprintf'
    Running test 'protection'
    Leaving init with final status: 0
    reboot: Power down
    powered off, test finish

    165 test(s): 156 passed,   9 skipped,   0 failed => status: warning

    See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.ppc64.out
    Running /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/kernel-ppc64le/arch/powerpc/boot/zImage on qemu-system-ppc64le
    Linux version 6.4.0+ (ubuntu@linux-lab) (powerpc64le-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2 SMP Fri Jul 28 01:41:12 CST 2023
    Kernel command line: console=hvc0 panic=-1 
    Run /init as init process
    Running test 'startup'
    Running test 'syscall'
    Running test 'stdlib'
    Running test 'vfprintf'
    Running test 'protection'
    Leaving init with final status: 0
    reboot: Power down
    powered off, test finish

    165 test(s): 156 passed,   9 skipped,   0 failed => status: warning

    See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.ppc64le.out

A fast report on existing test logs:

    $ for arch in ppc ppc64 ppc64le; do \
        make report ARCH=$arch RUN_OUT=$PWD/run.$arch.out | grep status; \
      done
    165 test(s): 156 passed,   9 skipped,   0 failed => status: warning
    165 test(s): 156 passed,   9 skipped,   0 failed => status: warning
    165 test(s): 156 passed,   9 skipped,   0 failed => status: warning

Changes from v2 --> v3:

* selftests/nolibc: allow report with existing test log
  selftests/nolibc: fix up O= option support
  selftests/nolibc: allow customize CROSS_COMPILE by architecture
  selftests/nolibc: customize CROSS_COMPILE for 32/64-bit powerpc
  selftests/nolibc: tinyconfig: add extra common options

    No Change.

* selftests/nolibc: add macros to reduce duplicated changes

    Remove REPORT_RUN_OUT and LOG_OUT.

* selftests/nolibc: string the core targets

    Removed extconfig target from our v3 powerpc patchset [3], the
    operations have been merged into the defconfig target.

    Let kernel depends on $(KERNEL_CONFIG) instead of the removed
    extconfig.

* selftests/nolibc: add menuconfig and mrproper for development

    like the other local nolibc targets, still require local menuconfig
    and mrproper targets for consistent usage with the same ARCH and no -C
    /path/to/srctree

    Merge them together to reduce duplicated entries.

* selftests/nolibc: allow quit qemu-system when poweroff fails

    Enhance timeout logic with more expected strings print and detection
    about the booting of bios, kernel, init and test.

    Add a default 10 seconds of QEMU_TIMEOUT for every architecture to
    detect all of the potential boog hang or failed poweroff.

* selftests/nolibc: customize QEMU_TIMEOUT for ppc64/ppc64le

    Reduce QEMU_TIMEOUT from 60 seconds to a more normal 15 and 20
    seconds for ppc64 and ppc64le respectively. the main time cost is
    the slow bios used.

* selftests/nolibc: tinyconfig: add support for 32/64-bit powerpc

    Rename the file names to shorter ones as suggestions from the powerpc
    patchset.

* selftests/nolibc: speed up some targets with multiple jobs

    New to speed up with -j<N> by default.

Best regards,
Zhangjin
---
[1]: https://lore.kernel.org/lkml/cover.1689759351.git.falcon@xxxxxxxxxxx/
[2]: https://lore.kernel.org/lkml/20230727132418.117924-1-falcon@xxxxxxxxxxx/
[3]: https://lore.kernel.org/lkml/8e9e5ac6283c6ec2ecf10a70ce55b219028497c1.1690468707.git.falcon@xxxxxxxxxxx/


Zhangjin Wu (12):
  selftests/nolibc: allow report with existing test log
  selftests/nolibc: add macros to reduce duplicated changes
  selftests/nolibc: fix up O= option support
  selftests/nolibc: string the core targets
  selftests/nolibc: allow customize CROSS_COMPILE by architecture
  selftests/nolibc: customize CROSS_COMPILE for 32/64-bit powerpc
  selftests/nolibc: add menuconfig and mrproper for development
  selftests/nolibc: allow quit qemu-system when poweroff fails
  selftests/nolibc: customize QEMU_TIMEOUT for ppc64/ppc64le
  selftests/nolibc: tinyconfig: add extra common options
  selftests/nolibc: tinyconfig: add support for 32/64-bit powerpc
  selftests/nolibc: speed up some targets with multiple jobs

 tools/testing/selftests/nolibc/Makefile       | 102 ++++++++++++++----
 .../selftests/nolibc/configs/common.config    |   4 +
 .../selftests/nolibc/configs/ppc.config       |   3 +
 .../selftests/nolibc/configs/ppc64.config     |   3 +
 .../selftests/nolibc/configs/ppc64le.config   |   4 +
 5 files changed, 98 insertions(+), 18 deletions(-)
 create mode 100644 tools/testing/selftests/nolibc/configs/common.config
 create mode 100644 tools/testing/selftests/nolibc/configs/ppc64.config
 create mode 100644 tools/testing/selftests/nolibc/configs/ppc64le.config

-- 
2.25.1