Re: current objtool warnings from randconfig builds

From: Arnd Bergmann
Date: Tue Feb 07 2023 - 15:39:13 EST

On Tue, Feb 7, 2023, at 17:16, Peter Zijlstra wrote:
>
>> 80 vmlinux.o: warning: objtool: ibt_selftest+0x11: sibling call from callable instruction with modified stack frame
>
> Can you provide details on how to reproduce this?

I see this one with clang-14 and clang-16, presumably other versions
but not gcc, the function disassembles the clang version into

0000000000000140 <ibt_selftest>:
140: f3 0f 1e fa endbr64
144: 55 push %rbp
145: 48 89 e5 mov %rsp,%rbp
148: 48 8d 05 02 00 00 00 lea 0x2(%rip),%rax # 151 <ibt_selftest_ip>
14f: ff e0 jmp *%rax

while the gcc version looks correct with

00000000000005c0 <ibt_selftest>:
5c0: f3 0f 1e fa endbr64
5c4: 48 8d 05 02 00 00 00 lea 0x2(%rip),%rax # 5cd <ibt_selftest_ip>
5cb: ff e0 jmp *%rax

Attached as 0x1A63498B-traps.o.gz and 0x1A63498B-config.gz, the
hexadecimal number is the randconfig seed I used.

>> 49 vmlinux.o: warning: objtool: lkdtm_UNSET_SMEP+0x100: relocation to !ENDBR: native_write_cr4+0x4
>
> That one was on purpose I think.

Do you mean objtool intentionally warns, or the code intentionally does
it but objtool should ignore it? I would assume that only one of them
is right.

>> 29 mm/kasan/generic.o: warning: objtool: kasan_check_range+0x1e: call to addr_has_metadata() with UACCESS enabled
>> 29 mm/kasan/generic.o: warning: objtool: __asan_load2+0x11: call to addr_has_metadata() with UACCESS enabled
>
> I'm thinking addr_has_metadata() needs the __always_inline treatment, it
> seems to be present a lot.

Makes sense, I'll send a patch.

>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_out+0x15e: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_out+0x11d: stack state mismatch: cfa1=5+16 cfa2=4+8
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_in+0x150: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_in+0x13d: stack state mismatch: cfa1=5+16 cfa2=4+8
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_out+0xe2: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_out+0xd1: stack state mismatch: cfa1=5+16 cfa2=4+8
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_in+0xea: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_in+0xcc: stack state mismatch: cfa1=5+16 cfa2=4+8
>
> Repro details?

last seen with gcc-13.0.1, but I think this happened with most older
versions already, attached 0x2D9ADB20-vmwgfx_msg.o

>> 15 vmlinux.o: warning: objtool: ftrace_likely_update+0xd2: call to __stack_chk_fail() with UACCESS enabled
>> 15 kernel/trace/trace_branch.o: warning: objtool: ftrace_likely_update+0x201: call to __stack_chk_fail() with UACCESS enabled
>
> -EWONTFIX, TRACE_BRANCH_PROFILING is fundamentally incompatible with
> lots of things.

I see CONFIG_X86_SMAP is now always enabled, does that mean that
TRACE_BRANCH_PROFILING no longer works at all on x86, or would it
still work most of the time?

>> 14 arch/x86/kvm/kvm.o: warning: objtool: .text+0x0: unreachable instruction
>
> details?

I see this with clang-14 through clang-16 or gcc. From an initial
analysis I see that virt/kvm/kvm_main.o contains empty __weak versions
of kvm_arch_mmu_notifier_invalidate_range and kvm_arch_guest_memory_reclaimed.

For some reason, after linking (with ld.bfd or ld.lld) the file into
arch/x86/kvm/kvm.o, the code for those two functions remains, but without
a symbol, which leads to a funny disassembly:

Disassembly of section .text:

0000000000000000 <kvm_is_zone_device_page-0x80>:
0: f3 0f 1e fa endbr64
4: c3 ret
5: 66 66 66 66 66 66 2e data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
c: 0f 1f 84 00 00 00 00
13: 00
14: 66 66 66 66 66 66 2e data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
1b: 0f 1f 84 00 00 00 00
22: 00
23: 66 66 66 66 66 66 2e data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
2a: 0f 1f 84 00 00 00 00
31: 00
32: 66 66 66 66 66 2e 0f data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
39: 1f 84 00 00 00 00 00
40: f3 0f 1e fa endbr64
44: c3 ret
45: 66 66 66 66 66 66 2e data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
4c: 0f 1f 84 00 00 00 00
53: 00
54: 66 66 66 66 66 66 2e data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
5b: 0f 1f 84 00 00 00 00
62: 00
63: 66 66 66 66 66 66 2e data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
6a: 0f 1f 84 00 00 00 00
71: 00
72: 66 66 66 66 66 2e 0f data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
79: 1f 84 00 00 00 00 00

0000000000000080 <kvm_is_zone_device_page>:
80: f3 0f 1e fa endbr64
....

When kvm_main.o is built with gcc, the two functions do not appear to
be part of the object file, but I don't see why not.

Attaching 0x6BC495CE-kvm_main-gcc.o and 0x6BC495CE-kvm_main-clang.o

>> 2 mm/kmsan/report.o: warning: objtool: kmsan_report+0x0: call to __fentry__() with UACCESS enabled
>
> mm/kasan/Makefil:CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
>
> How?!?!
>
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_warning+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_poison_alloca+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_n+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_8+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_4+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_2+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_1+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_n+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_8+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_4+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_2+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_1+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_instrument_asm_store+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_get_context_state+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_chain_origin+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/hooks.o: warning: objtool: kmsan_unpoison_memory+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/hooks.o: warning: objtool: kmsan_unpoison_entry_regs+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/hooks.o: warning: objtool: kmsan_copy_to_user+0x0: call to __fentry__() with UACCESS enabled
>
> Something seems to be going badly wrong there... notrace is being lost.

It's kmsan, not kasan!

I'll send a patch to copy the relevant Makefile bits from kasan to kmsan:
--- a/mm/kmsan/Makefile
+++ b/mm/kmsan/Makefile
@@ -16,6 +16,14 @@ CC_FLAGS_KMSAN_RUNTIME += -DDISABLE_BRANCH_PROFILING

CFLAGS_REMOVE.o = $(CC_FLAGS_FTRACE)

+# Disable ftrace to avoid recursion.
+CFLAGS_REMOVE_core.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_hooks.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_instrumentation.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
+
CFLAGS_core.o := $(CC_FLAGS_KMSAN_RUNTIME)
CFLAGS_hooks.o := $(CC_FLAGS_KMSAN_RUNTIME)
CFLAGS_init.o := $(CC_FLAGS_KMSAN_RUNTIME)

Arnd

Attachment: 0x1A63498B-config.gz
Description: application/gzip

Attachment: 0x1A63498B-traps.o.gz
Description: application/gzip

Attachment: 0x2D9ADB20-config.gz
Description: application/gzip

Attachment: 0x2D9ADB20-vmwgfx_msg.o.gz
Description: application/gzip

Attachment: 0x6BC495CE-kvm_main-clang.o
Description: application/object

Attachment: 0x6BC495CE-kvm_main-gcc.o
Description: application/object

Attachment: 0x6BC495CE-config
Description: Binary data