Re: [PATCH bpf-next] bpf, riscv: Fix stack layout of JITed code on RV32

From: Daniel Borkmann
Date: Thu Apr 30 2020 - 10:23:50 EST


On 4/30/20 4:14 AM, Xi Wang wrote:
On Wed, Apr 29, 2020 at 5:51 PM Luke Nelson <lukenels@xxxxxxxxxxxxxxxxx> wrote:

This patch fixes issues with stackframe unwinding and alignment in the
current stack layout for BPF programs on RV32.

In the current layout, RV32 fp points to the JIT scratch registers, rather
than to the callee-saved registers. This breaks stackframe unwinding,
which expects fp to point just above the saved ra and fp registers.

This patch fixes the issue by moving the callee-saved registers to be
stored on the top of the stack, pointed to by fp. This satisfies the
assumptions of stackframe unwinding.

This patch also fixes an issue with the old layout that the stack was
not aligned to 16 bytes.

Stacktrace from JITed code using the old stack layout:

[ 12.196249 ] [<c0402200>] walk_stackframe+0x0/0x96

Stacktrace using the new stack layout:

[ 13.062888 ] [<c0402200>] walk_stackframe+0x0/0x96
[ 13.063028 ] [<c04023c6>] show_stack+0x28/0x32
[ 13.063253 ] [<a403e778>] bpf_prog_82b916b2dfa00464+0x80/0x908
[ 13.063417 ] [<c09270b2>] bpf_test_run+0x124/0x39a
[ 13.063553 ] [<c09276c0>] bpf_prog_test_run_skb+0x234/0x448
[ 13.063704 ] [<c048510e>] __do_sys_bpf+0x766/0x13b4
[ 13.063840 ] [<c0485d82>] sys_bpf+0xc/0x14
[ 13.063961 ] [<c04010f0>] ret_from_syscall+0x0/0x2

The new code is also simpler to understand and includes an ASCII diagram
of the stack layout.

Tested on riscv32 QEMU virt machine.

Signed-off-by: Luke Nelson <luke.r.nels@xxxxxxxxx>

Thanks for the fix!

Acked-by: Xi Wang <xi.wang@xxxxxxxxx>

Applied, thanks everyone!