Re: Update15.x/kcfi to LLVM 15.0.3 (ZSTD + DWARFv5)

[Nick Desaulniers]

On Wed, Oct 19, 2022 at 10:44 PM Sedat Dilek (DHL Supply Chain)
<sedat.dilek@xxxxxxx> wrote:
>
> -----Ursprüngliche Nachricht-----
> Von: Sami Tolvanen <samitolvanen@xxxxxxxxxx>
> Gesendet: Dienstag, 18. Oktober 2022 18:00
> An: Sedat Dilek (DHL Supply Chain) <sedat.dilek@xxxxxxx>
> Betreff: Re: Update15.x/kcfi to LLVM 15.0.3
>
> On Tue, Oct 18, 2022 at 4:54 AM Sedat Dilek (DHL Supply Chain) <sedat.dilek@xxxxxxx> wrote:
> >
> [ CC Nick + Fangrui ]
>
> You happen to know or use the ZSTD cmake-option (see [1] and [2]) - came in with changes post-15.0.2?
>
> AFAICS Fangrui Song had some patches to use ZSTD debug-compression with DWARFv5.
>
> Unsure, if that combination is now possible in v15.0.3 or is a feature of LLVM-16.
>
> [3] says:
>
> [ lib/Kconfig.debug ]
>
> config DEBUG_INFO_COMPRESSED
>         bool "Compressed debugging information"
>         depends on $(cc-option,-gz=zlib)
>         depends on $(ld-option,--compress-debug-sections=zlib)
>         help
>           Compress the debug information using zlib.  Requires GCC 5.0+ or Clang
>           5.0+, binutils 2.26+, and zlib.
>
>           Users of dpkg-deb via scripts/package/builddeb may find an increase in
>           size of their debug .deb packages with this config set, due to the
>           debug info being compressed with zlib, then the object files being
>           recompressed with a different compression scheme. But this is still
>           preferable to setting $KDEB_COMPRESS to "none" which would be even
>           larger.
>
> I am interested in the equivalent changes to cc-option/ld-option with ZSTD (usable with DWARFv5).

Good idea.  It looks like there's still a blocker to using -gz=zstd with clang:
$ clang -gz=zstd -x c - < /dev/null
clang-16: warning: cannot compress debug sections (zstd not enabled)
[-Wdebug-compression-unavailable]

https://github.com/facebook/zstd/issues/3271

Attaching a WIP patch.

>
> Thanks.
>
> Best regards,
> -Sedat-
>
> [1] https://github.com/llvm/llvm-project/commit/6fba7854a2f0b6b3899bb156c1a0c4ae35c96e24
> [2] https://github.com/llvm/llvm-project/commit/b4840279846e1eea44c3dca575395a90c9d77ca0
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/Kconfig.debug#n315
> [4] https://metadata.ftp-master.debian.org/changelogs//main/l/llvm-toolchain-15/llvm-toolchain-15_15.0.3-1~exp1_changelog
>
>
>


-- 
Thanks,
~Nick Desaulniers
From c9ccf84c1a3fc742123dcc6b7789206506190260 Mon Sep 17 00:00:00 2001
From: Nick Desaulniers <ndesaulniers@xxxxxxxxxx>
Date: Thu, 20 Oct 2022 09:54:22 -0700
Subject: [PATCH] WIP: Makefile.debug: support for -gz=zstd

Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
uses zstd.

TODO: measurements

Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
Suggested-by: Sedat Dilek (DHL Supply Chain) <sedat.dilek@xxxxxxx>
Signed-off-by: Nick Desaulniers <ndesaulniers@xxxxxxxxxx>
---
 lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
 scripts/Makefile.debug |  4 ++++
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 3fc7abffc7aa..4085ac77dc12 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
+choice
+	prompt "Compressed Debug information"
+	depends on DEBUG_KERNEL
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_UNCOMPRESSED.
+
+config DEBUG_INFO_UNCOMPRESSED
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
+	  16.0+, binutils 2.40+, and zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
index 332c486f705f..8ac3379d2255 100644
--- a/scripts/Makefile.debug
+++ b/scripts/Makefile.debug
@@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)
-- 
2.38.0.135.g90850a2211-goog