Re: [PATCH] vmlinux.lds.h: Force-align ELF notes section to four bytes

[Tianyi Liu]

On Tue, Apr 11, 2023 at 17:00 , Josh Poimboeuf wrote:
> On Tue, Feb 14, 2023 at 02:33:02PM +0800, Tianyi Liu wrote:
> > > LLVM_OBJCOPY=objcopy pahole -J --btf_gen_floats -j
> > > --skip_encoding_btf_inconsistent_proto --btf_gen_optimized
> > > .tmp_vmlinux.btf
> > > btf_encoder__encode: btf__dedup failed!
> > > Failed to encode BTF
> > >
> > > Thanks,
> > >
> >
> > I encountered the same problem when building a new kernel and I found some
> > reasons for the error.
> >
> > In short, enabling CONFIG_X86_KERNEL_IBT will change the order of records in
> > .notes section. In addition, due to historical problems, the alignment of
> > records in the .notes section is not unified, which leads to the inability of
> > gelf_getnote() to read the records after the wrong one.
> 
> Alexandre, Tianyi, are you still seeing this issue with the latest
> dwarves?  If so can you confirm the below patch fixes it?
> 

Josh, first of all, thank you very much for your help. However, this patch
doesn't work in my environment. I am using gcc 12.2.1 20230201.
After compiling, when I use readelf -S to view ELF sections,
the align of .notes section is still 8:

$ readelf -S .tmp_vmlinux.btf
[20] .notes            NOTE             ffffffff8250b570  0170b570
     0000000000000084  0000000000000000   A       0     0     8

> Apparently the latest dwarves release fixes it on Fedora Rawhide [1],
> does anybody know if there a specific dwarves and/or libbpf change for
> this?
> 

It has been fixed in dwarves[1], but it may just be a coincidence.

> [1] https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2346#note_1348057786
> 
> ---8<---
> 
> From: Josh Poimboeuf <jpoimboe@xxxxxxxxxx>
> Subject: [PATCH] vmlinux.lds.h: Force-align ELF notes section to four bytes
> 
> When tooling reads ELF notes, it assumes each note entry is aligned to
> the value listed in the .note section header's sh_addralign field.
> 
> The kernel-created ELF notes in the .note.Linux and .note.Xen sections
> are aligned to 4 bytes.  This causes the toolchain to set those
> sections' sh_addralign values to 4.
> 
> On the other hand, the GCC-created .note.gnu.property section has an
> sh_addralign value of 8 for some reason, despite being based on struct
> Elf32_Nhdr which only needs 4-byte alignment.
> 
> When the mismatched input sections get linked together into the vmlinux
> .notes output section, the higher alignment "wins", resulting in an
> sh_addralign of 8, which confuses tooling.  For example:
> 
>   $ readelf -n .tmp_vmlinux.btf
>   ...
>   readelf: .tmp_vmlinux.btf: Warning: note with invalid namesz and/or descsz found at offset 0x170
>   readelf: .tmp_vmlinux.btf: Warning:  type: 0x4, namesize: 0x006e6558, descsize: 0x00008801, alignment: 8
> 
> In this case readelf thinks there's alignment padding where there is
> none, so it starts reading an ELF note in the middle.
> 
> With newer toolchains (e.g., latest Fedora Rawhide), a similar mismatch
> triggers a build failure when combined with CONFIG_X86_KERNEL_IBT:
> 
>   btf_encoder__encode: btf__dedup failed!
>   Failed to encode BTF
>   libbpf: failed to find '.BTF' ELF section in vmlinux
>   FAILED: load BTF from vmlinux: No data available
>   make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 255
> 
> Fix it by forcing the .notes section input and output alignments to 4 to
> match the kernel's note entry alignments.
> 
> Note this doesn't break the 8-byte-aligned .note.gnu.property entries
> because their internal data representations fill the entire 8-byte
> alignment boundary, so there's no padding between entries to be
> misinterpreted.  And there's only a single entry in that section anyway.
> 
> Reported-by: Daniel Xu <dxu@xxxxxxxxx>
> Debugged-by: Tianyi Liu <i.pear@xxxxxxxxxxx>
> Signed-off-by: Josh Poimboeuf <jpoimboe@xxxxxxxxxx>
> ---
>  include/asm-generic/vmlinux.lds.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index d1f57e4868ed..1c7c87c9ae71 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -894,7 +894,7 @@
>   */
>  #define NOTES                                                           \
>          /DISCARD/ : { *(.note.GNU-stack) }                              \
> -       .notes : AT(ADDR(.notes) - LOAD_OFFSET) {                       \
> +       .notes ALIGN(4) : AT(ADDR(.notes) - LOAD_OFFSET) SUBALIGN(4) {  \
>                  BOUNDED_SECTION_BY(.note.*, _notes)                     \
>          } NOTES_HEADERS                                                 \
>          NOTES_HEADERS_RESTORE
> --
> 2.39.2

[1] https://github.com/acmel/dwarves/commit/a9498899109d3be14f17abbc322a8f55a1067bee