[BUG] sefltests: forwarding: gre_custom_multipath_hash.sh: [FAIL] and syntax errors

[Mirsad Todorovac]

Hi,

It's me again.

Don't throw things at me, but it is the same old vanilla torvalds tree kernel 6.5-rc2
kselftest run, and after applying the patches I ran into the new problems:

I have been free to add the timeout SIGTERM trap to the test, and this is the only
change, save for the Ido's lib.sh patch (for filling the veth NETIFS array in default
execution case).

Here is the command output:

# selftests: net/forwarding: gre_custom_multipath_hash.sh
# TEST: ping                                                          [ OK ]
# TEST: ping6                                                         [ OK ]
# INFO: Running IPv4 overlay custom multipath hash tests
# TEST: Multipath hash field: Inner source IP (balanced)              [ OK ]
# INFO: Packets sent on path1 / path2: 6760 / 5859
# TEST: Multipath hash field: Inner source IP (unbalanced)            [ OK ]
# INFO: Packets sent on path1 / path2: 2 / 12603
# TEST: Multipath hash field: Inner destination IP (balanced)         [ OK ]
# INFO: Packets sent on path1 / path2: 5850 / 6750
# TEST: Multipath hash field: Inner destination IP (unbalanced)       [ OK ]
# INFO: Packets sent on path1 / path2: 12602 / 1
# TEST: Multipath hash field: Inner source port (balanced)            [ OK ]
# INFO: Packets sent on path1 / path2: 16411 / 16364
# TEST: Multipath hash field: Inner source port (unbalanced)          [ OK ]
# INFO: Packets sent on path1 / path2: 32773 / 3
# TEST: Multipath hash field: Inner destination port (balanced)       [ OK ]
# INFO: Packets sent on path1 / path2: 16410 / 16362
# TEST: Multipath hash field: Inner destination port (unbalanced)     [ OK ]
# INFO: Packets sent on path1 / path2: 32770 / 0
# INFO: Running IPv6 overlay custom multipath hash tests
# TEST: Multipath hash field: Inner source IP (balanced)              [ OK ]
# INFO: Packets sent on path1 / path2: 6150 / 6452
# TEST: Multipath hash field: Inner source IP (unbalanced)            [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 12600
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# TEST: Multipath hash field: Inner destination IP (balanced)         [FAIL]
# 	Expected traffic to be balanced, but it is not
# INFO: Packets sent on path1 / path2: -129763 / -77050
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Invalid VRF name
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner destination IP (unbalanced)       [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 0
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner flowlabel (balanced)              [FAIL]
# 	Expected traffic to be balanced, but it is not
# INFO: Packets sent on path1 / path2: 0 / 0
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Invalid VRF name
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner flowlabel (unbalanced)            [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 0
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Invalid VRF name
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner source port (balanced)            [FAIL]
# 	Expected traffic to be balanced, but it is not
# INFO: Packets sent on path1 / path2: 0 / 0
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Invalid VRF name
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner source port (unbalanced)          [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 0
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Invalid VRF name
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner destination port (balanced)       [FAIL]
# 	Expected traffic to be balanced, but it is not
# INFO: Packets sent on path1 / path2: 0 / 0
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Invalid VRF name
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# jq: error (at <stdin>:1): Cannot iterate over null (null)
# Runtime error (func=(main), adr=3): Divide by zero
# (standard_in) 1: syntax error
# TEST: Multipath hash field: Inner destination port (unbalanced)     [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 0
# Error: argument "vveth9" is wrong: Invalid VRF
#
# Error: argument "vveth9" is wrong: Invalid VRF
#
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: No such process
# RTNETLINK answers: No such process
# Cannot find device "vveth9"
# Error: argument "vveth8" is wrong: Invalid VRF
#
# Error: argument "vveth8" is wrong: Invalid VRF
#
# Error: argument "vveth8" is wrong: Invalid VRF
#
# Cannot find device "g2"
# Cannot find device "g2"
# Cannot find device "g2"
# Cannot find device "g2"
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: No such process
# RTNETLINK answers: No such process
# Cannot find device "vveth8"
# Error: Invalid handle.
# Error: argument "vveth6" is wrong: Invalid VRF
#
# Error: argument "vveth6" is wrong: Invalid VRF
#
# Cannot find device "veth5.222"
# Cannot find device "veth5.111"
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: No such process
# RTNETLINK answers: No such process
# Cannot find device "vveth6"
# Error: argument "vveth3" is wrong: Invalid VRF
#
# Error: argument "vveth3" is wrong: Invalid VRF
#
# Cannot find device "veth4.222"
# Cannot find device "veth4.111"
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: No such process
# RTNETLINK answers: No such process
# Cannot find device "vveth3"
# Error: argument "vveth1" is wrong: Invalid VRF
#
# Error: argument "vveth1" is wrong: Invalid VRF
#
# Error: argument "vveth1" is wrong: Invalid VRF
#
# Cannot find device "g1"
# Cannot find device "g1"
# Cannot find device "g1"
# Cannot find device "g1"
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: No such process
# RTNETLINK answers: No such process
# Cannot find device "vveth1"
# Error: argument "vveth0" is wrong: Invalid VRF
#
# Error: argument "vveth0" is wrong: Invalid VRF
#
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: Cannot assign requested address
# RTNETLINK answers: No such process
# RTNETLINK answers: No such process
# Cannot find device "vveth0"
# RTNETLINK answers: File exists
# RTNETLINK answers: No such file or directory
# RTNETLINK answers: File exists
# RTNETLINK answers: No such file or directory
#
not ok 18 selftests: net/forwarding: gre_custom_multipath_hash.sh # TIMEOUT 240 seconds

Those things like RTNETLINK errors probably mess up things beyond what cleanup
does, so the reproducer might not be that easy to devise.

I have tried to reduce the complexity by adding:

trap cleanup INT TERM EXIT

to the tests that timed out.

Kind regards,
Mirsad