Re: [PATCH 2/4] kbuild: avoid too many execution of scripts/pahole-flags.sh

[Masahiro Yamada]

On Wed, Oct 18, 2023 at 4:57 AM Andrii Nakryiko
<andrii.nakryiko@xxxxxxxxx> wrote:
>
> On Tue, Oct 17, 2023 at 3:38 AM Masahiro Yamada <masahiroy@xxxxxxxxxx> wrote:
> >
> > scripts/pahole-flags.sh is executed so many times.
> >
> > You can check how many times it is invoked during the build, as follows:
> >
> >   $ cat <<EOF >> scripts/pahole-flags.sh
> >   > echo "scripts/pahole-flags.sh was executed" >&2
> >   > EOF
> >
> >   $ make -s
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >     [ lots of repeated lines suppressed... ]
> >
> > This scripts is exectuted more than 20 times during the kernel build
> > because PAHOLE_FLAGS is a recursively expanded variable and exported
> > to sub-processes.
> >
> > With the GNU Make >= 4.4, it is executed more than 60 times because
> > exported variables are also passed to other $(shell ) invocations.
> > Without careful coding, it is known to cause an exponential fork
> > explosion. [1]
> >
> > The use of $(shell ) in an exported recursive variable is likely wrong
> > because $(shell ) is always evaluated due to the 'export' keyword, and
> > the evaluation can occur multiple times by the nature of recursive
> > variables.
> >
> > Convert the shell script to a Makefile, which is included only when
> > CONFIG_DEBUG_INFO_BTF=y.
> >
> > [1]: https://savannah.gnu.org/bugs/index.php?64746
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx>
> > ---
> >
> >  Makefile                |  4 +---
> >  scripts/Makefile.btf    | 19 +++++++++++++++++++
> >  scripts/pahole-flags.sh | 30 ------------------------------
> >  3 files changed, 20 insertions(+), 33 deletions(-)
> >  create mode 100644 scripts/Makefile.btf
> >  delete mode 100755 scripts/pahole-flags.sh
> >
> > diff --git a/Makefile b/Makefile
> > index fed9a6cc3665..eaddec67e5e1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -513,8 +513,6 @@ LZ4         = lz4c
> >  XZ             = xz
> >  ZSTD           = zstd
> >
> > -PAHOLE_FLAGS   = $(shell PAHOLE=$(PAHOLE) $(srctree)/scripts/pahole-flags.sh)
>
> What if we just used := here? Wouldn't it avoid unnecessary multiple executions?


Yeah, := is less silly than =.


But, I do not like to run the script for non-build targets
such as 'make clean', 'make help', etc.

Also, when building with CONFIG_DEBUG_INFO_BTF=n,
the shell is forked to compute PAHOLE_FLAGS,
which we know are unnecessary.





--
Best Regards

Masahiro Yamada