Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

From: Sedat Dilek
Date: Sat Feb 06 2021 - 02:00:40 EST

On Sat, Feb 6, 2021 at 7:26 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
>
> On Sat, Feb 6, 2021 at 6:53 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> >
> > On Sat, Feb 6, 2021 at 6:44 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > >
> > > On Sat, Feb 6, 2021 at 4:34 AM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > > >
> > > > On Fri, Feb 5, 2021 at 10:54 PM Yonghong Song <yhs@xxxxxx> wrote:
> > > > >
> > > > >
> > > > >
> > > > > On 2/5/21 12:31 PM, Sedat Dilek wrote:
> > > > > > On Fri, Feb 5, 2021 at 9:03 PM Yonghong Song <yhs@xxxxxx> wrote:
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On 2/5/21 11:24 AM, Arnaldo Carvalho de Melo wrote:
> > > > > >>> Em Fri, Feb 05, 2021 at 11:10:08AM -0800, Yonghong Song escreveu:
> > > > > >>>> On 2/5/21 11:06 AM, Sedat Dilek wrote:
> > > > > >>>>> On Fri, Feb 5, 2021 at 7:53 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > > > > >>>>> Grepping through linux.git/tools I guess some BTF tools/libs need to
> > > > > >>>>> know what BTF_INT_UNSIGNED is?
> > > > > >>>
> > > > > >>>> BTF_INT_UNSIGNED needs kernel support. Maybe to teach pahole to
> > > > > >>>> ignore this for now until kernel infrastructure is ready.
> > > > > >>>
> > > > > >>> Yeah, I thought about doing that.
> > > > > >>>
> > > > > >>>> Not sure whether this information will be useful or not
> > > > > >>>> for BTF. This needs to be discussed separately.
> > > > > >>>
> > > > > >>> Maybe search for the rationale for its introduction in DWARF.
> > > > > >>
> > > > > >> In LLVM, we have:
> > > > > >> uint8_t BTFEncoding;
> > > > > >> switch (Encoding) {
> > > > > >> case dwarf::DW_ATE_boolean:
> > > > > >> BTFEncoding = BTF::INT_BOOL;
> > > > > >> break;
> > > > > >> case dwarf::DW_ATE_signed:
> > > > > >> case dwarf::DW_ATE_signed_char:
> > > > > >> BTFEncoding = BTF::INT_SIGNED;
> > > > > >> break;
> > > > > >> case dwarf::DW_ATE_unsigned:
> > > > > >> case dwarf::DW_ATE_unsigned_char:
> > > > > >> BTFEncoding = 0;
> > > > > >> break;
> > > > > >>
> > > > > >> I think DW_ATE_unsigned can be ignored in pahole since
> > > > > >> the default encoding = 0. A simple comment is enough.
> > > > > >>
> > > > > >
> > > > > > Yonghong Son, do you have a patch/diff for me?
> > > > >
> > > > > Looking at error message from log:
> > > > >
> > > > > LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
> > > > > .tmp_vmlinux.btf
> > > > > [115] INT DW_ATE_unsigned_1 Error emitting BTF type
> > > > > Encountered error while encoding BTF.
> > > > >
> > > > > Not exactly what is the root cause. Maybe bt->bit_size is not
> > > > > encoded correctly. Could you put vmlinux (in the above it is
> > > > > .tmp_vmlinux.btf) somewhere, I or somebody else can investigate
> > > > > and provide a proper fix.
> > > > >
> > > >
> > > > [ TO: Masahiro ]
> > > >
> > > > Thanks for taking care Yonghong - hope this is your first name, if not
> > > > I am sorry.
> > > > In case of mixing my first and last name you will make me female -
> > > > Dilek is a Turkish female first name :-).
> > > > So, in some cultures you need to be careful.
> > > >
> > > > Anyway... back to business and facts.
> > > >
> > > > Out of frustration I killed my last build via `make distclean`.
> > > > The whole day I tested diverse combination of GCC-10 and LLVM-12
> > > > together with BTF Kconfigs, selfmade pahole, etc.
> > > >
> > > > I will do ne run with some little changes:
> > > >
> > > > #1: Pass LLVM_IAS=1 to make (means use Clang's Integrated ASsembler -
> > > > as per Nick this leads to the same error - should be unrelated)
> > > > #2: I did: DEBUG_INFO_COMPRESSED y -> n
> > > >
> > > > #2 I did in case you need vmlinux and I have to upload - I will
> > > > compress the resulting vmlinux with ZSTD.
> > > > You need vmlinux or .tmp_vmlinux.btf file?
> > > > Nick was not allowed from his company to download from a Dropbox link.
> > > > So, as an alternative I can offer GoogleDrive...
> > > > ...or bomb into your INBOX :-).
> > > >
> > > > Now, why I CCed Masahiro:
> > > >
> > > > In case of ERRORs when running `scripts/link-vmlinux.sh` above files
> > > > will be removed.
> > > >
> > > > Last, I found a hack to bypass this - means to keep these files (I
> > > > need to check old emails).
> > > >
> > > > Masahiro, you see a possibility to have a way to keep these files in
> > > > case of ERRORs without doing hackery?
> > > >
> > > > From a previous post in this thread:
> > > >
> > > > + info BTF .btf.vmlinux.bin.o
> > > > + [ != silent_ ]
> > > > + printf %-7s %s\n BTF .btf.vmlinux.bin.o
> > > > BTF .btf.vmlinux.bin.o
> > > > + LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
> > > > [2] INT long unsigned int Error emitting BTF type
> > > > Encountered error while encoding BTF.
> > > > + llvm-objcopy --only-section=.BTF --set-section-flags
> > > > .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
> > > > ...
> > > > + info BTFIDS vmlinux
> > > > + [ != silent_ ]
> > > > + printf %-7s %s\n BTFIDS vmlinux
> > > > BTFIDS vmlinux
> > > > + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
> > > > FAILED: load BTF from vmlinux: Invalid argument
> > > > + on_exit
> > > > + [ 255 -ne 0 ]
> > > > + cleanup
> > > > + rm -f .btf.vmlinux.bin.o
> > > > + rm -f .tmp_System.map
> > > > + rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
> > > > .tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
> > > > .tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
> > > > 2.o
> > > > + rm -f System.map
> > > > + rm -f vmlinux
> > > > + rm -f vmlinux.o
> > > > make[3]: *** [Makefile:1166: vmlinux] Error 255
> > > >
> > > > ^^^ Look here.
> > > >
> > >
> > > With this diff:
> > >
> > > $ git diff scripts/link-vmlinux.sh
> > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> > > index eef40fa9485d..40f1b6aae553 100755
> > > --- a/scripts/link-vmlinux.sh
> > > +++ b/scripts/link-vmlinux.sh
> > > @@ -330,7 +330,7 @@ vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
> > > # fill in BTF IDs
> > > if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
> > > info BTFIDS vmlinux
> > > - ${RESOLVE_BTFIDS} vmlinux
> > > + ##${RESOLVE_BTFIDS} vmlinux
> > > fi
> > >
> > > if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
> > >
> > > This files are kept - not removed:
> > >
> > > $ LC_ALL=C ll .*btf* vmlinux vmlinux.o
> > > -rwxr-xr-x 1 dileks dileks 31M Feb 6 06:37 .btf.vmlinux.bin.o
> > > -rwxr-xr-x 1 dileks dileks 348M Feb 6 06:37 .tmp_vmlinux.btf
> > > -rwxr-xr-x 1 dileks dileks 348M Feb 6 06:37 vmlinux
> > > -rw-r--r-- 1 dileks dileks 344M Feb 6 06:37 vmlinux.o
> > >
> > > Pleas let me know where to upload - Dropbox or GoogleDrive or
> > > elsewhere and give me a link.
> > >
> >
> >
> > WOW, ZSTD is great :-).
> >
> > $ zstd -19 -T0 -v vmlinux
> > *** zstd command line interface 64-bits v1.4.8, by Yann Collet ***
> > Note: 2 physical core(s) detected
> > vmlinux : 22.71% (364466016 => 82784801 bytes, vmlinux.zst)
> >
> > $ du -m vmlinux*
> > 348 vmlinux
> > 79 vmlinux.zst
> >
>
> Dropbox link:
> https://www.dropbox.com/sh/kvyh8ps7na0r1h5/AABfyNfDZ2bESse_bo4h05fFa?dl=0
>
> I hope this is public available.
>

Inspecting vmlinux with llvm-dwarf:

$ /opt/llvm-toolchain/bin/llvm-dwarfdump vmlinux | grep DW_AT_name |
grep DW_ATE_ | sort | uniq
DW_AT_name ("DW_ATE_signed_1")
DW_AT_name ("DW_ATE_signed_16")
DW_AT_name ("DW_ATE_signed_32")
DW_AT_name ("DW_ATE_signed_64")
DW_AT_name ("DW_ATE_signed_8")
DW_AT_name ("DW_ATE_unsigned_1")
DW_AT_name ("DW_ATE_unsigned_128")
DW_AT_name ("DW_ATE_unsigned_16")
DW_AT_name ("DW_ATE_unsigned_24")
DW_AT_name ("DW_ATE_unsigned_32")
DW_AT_name ("DW_ATE_unsigned_40")
DW_AT_name ("DW_ATE_unsigned_64")
DW_AT_name ("DW_ATE_unsigned_8")

- Sedat -