Re: [PATCH 1/2] selftests: ftrace: Do not failure if there is unsupported tests

[Steven Rostedt]

On Mon,  3 Jul 2017 14:25:36 +0900
Masami Hiramatsu <mhiramat@xxxxxxxxxx> wrote:

> Do not return failure exit code (1) for unsupported testcases,
> since it is expected for stable kernels.
> 
> Previously, ftracetest is expected to run only on current
> release for avoiding regressions. However, nowadays we run
> it on stable kernels. This means some test cases must return
> unsupported result. In such case, we should NOT exit
> ftracetest with error status for unsupported results so that
> kselftest (upper tests wrapper) shows it passed correctly.

I wonder if we should change kselftest instead. There are case where we
want to report "unsupported" as a failure. For instance, I have tests
where I enable everything, and if a test returns "unsupported" then it
is a failure for me.

Can we add an option in kselftest, or to ftracetest that decides if
unsupported is a failure or not? Otherwise I can not ack this patch.

-- Steve


> 
> Note that we continue to treat unresolved results as failure,
> if test writers would like to notice user that the test result
> should be reviewed, they can use exit_unresolved.
> 
> Signed-off-by: Masami Hiramatsu <mhiramat@xxxxxxxxxx>
> ---
>  tools/testing/selftests/ftrace/ftracetest |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index 14a03ea..290cd42 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -187,7 +187,7 @@ eval_result() { # sigval
>      $UNSUPPORTED)
>        prlog "	[UNSUPPORTED]"
>        UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO"
> -      return 1 # this is not a bug, but the result should be reported.
> +      return 0 # this is not a bug.
>      ;;
>      $XFAIL)
>        prlog "	[XFAIL]"