Re: [PATCH] kbuild: generate modules.builtin

From: Kay Sievers
Date: Wed Jul 15 2009 - 13:52:34 EST

Next message: Scott James Remnant: "Re: [PATCH] kbuild: generate modules.builtin"
Previous message: Kamalesh Babulal: "Re: [PATCH] percpu: add dummy pcpu_lpage_remapped() for !CONFIG_SMP"
In reply to: Scott James Remnant: "Re: [PATCH] kbuild: generate modules.builtin"
Next in thread: Andreas Robinson: "Re: [PATCH] kbuild: generate modules.builtin"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Sam, any objections, or a chance to take this through your tree? It
solves an old problem with module-init-tools and we would like to
teach m-i-t about it, which means it should be in -next at least.

Thanks,
Kay

On Wed, Jul 15, 2009 at 18:39, Scott James Remnant<scott@xxxxxxxxxx> wrote:
> Still would definitely like - adding linux-modules to the Cc
>
> On Fri, 2009-07-03 at 16:45 +0200, Michal Marek wrote:
>
>> On Tue, May 26, 2009 at 05:25:40PM +0200, Michal Marek wrote:
>> > Known issues (found when comparing results with allyesconfig and
>> > allmodconfig):
>> > Â* ALSA makefiles do some substitutions on the CONFIG_*
>> > Â Âvariables sometimes, which breaks the logic if the alsa modules are
>> > Â Âbuilt-in. Patch sent to alsa-devel.
>> > Â* samples/kobject/*.o could be compiled built-in, but doesn't show up
>> > Â Âin modules.builtin (and not even in vmlinux.o). Patch sent to Greg to
>> > Â Âmake it module-only.
>>
>> These two are merged already.
>>
>>
>> > Â* net/{8021q/vlan_core,ethernet/pe2,ipv6/inet6_hashtables}.o are always
>> > Â Âbuilt-in, even though the corresponding config option is tristate:
>> > Â Âobj-$(subst m,y,$(CONFIG_IPX)) += pe2.o
>> > Â ÂThe result is that modules.builtin can contain three built-in
>> > Â Â"modules" that can't possibly exist as modules. No fix for this so far,
>> > Â Âas it is not a big issue IMO.
>>
>> I hope this one is not a problem.
>>
>> I rebased the patch on top of current kbuild-next.git. What do you think
>> about it? Do you agree on the modules.builtin file name and format? I'd
>> like to add support for this to module-init-tools.
>>
>>
>> Subject: [PATCH] kbuild: generate modules.builtin
>>
>> To make it easier for tools like mkinitrd to detect whether a needed
>> module is missing or whether it is compiled into the kernel, install a
>> modules.builtin file listing all modules built into the kernel. This is
>> done by generating an alternate config file with all tristate =y options
>> set to =Y and reading the makefiles with this config included. The built
>> in modules then appear in obj-Y.
>>
>> Signed-off-by: Michal Marek <mmarek@xxxxxxx>
>> ---
>> Â.gitignore Â Â Â Â Â Â Â Â Â| Â Â1 +
>> ÂMakefile Â Â Â Â Â Â Â Â Â Â| Â 14 +++++++++---
>> Âscripts/Kbuild.include Â Â Â| Â Â6 +++++
>> Âscripts/Makefile.lib Â Â Â Â| Â Â5 +++-
>> Âscripts/Makefile.modbuiltin | Â 48 +++++++++++++++++++++++++++++++++++++++++++
>> Âscripts/kconfig/confdata.c Â| Â 48 ++++++++++++++++++++++++++++++++++++------
>> Â6 files changed, 110 insertions(+), 12 deletions(-)
>> Âcreate mode 100644 scripts/Makefile.modbuiltin
>>
>> diff --git a/.gitignore b/.gitignore
>> index cecb3b0..d8c1e79 100644
>> --- a/.gitignore
>> +++ b/.gitignore
>> @@ -22,6 +22,7 @@
>> Â*.lst
>> Â*.symtypes
>> Â*.order
>> +modules.builtin
>> Â*.elf
>> Â*.bin
>> Â*.gz
>> diff --git a/Makefile b/Makefile
>> index 46e1c9d..c59e167 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -871,6 +871,9 @@ $(sort $(vmlinux-init) $(vmlinux-main)) $(vmlinux-lds): $(vmlinux-dirs) ;
>> ÂPHONY += $(vmlinux-dirs)
>> Â$(vmlinux-dirs): prepare scripts
>> Â Â Â $(Q)$(MAKE) $(build)=$@
>> +ifdef CONFIG_MODULES
>> + Â Â $(Q)$(MAKE) $(modbuiltin)=$@
>> +endif
>>
>> Â# Build the kernel release string
>> Â#
>> @@ -1126,6 +1129,7 @@ all: modules
>> ÂPHONY += modules
>> Âmodules: $(vmlinux-dirs) $(if $(KBUILD_BUILTIN),vmlinux)
>> Â Â Â $(Q)$(AWK) '!x[$$0]++' $(vmlinux-dirs:%=$(objtree)/%/modules.order) > $(objtree)/modules.order
>> + Â Â $(Q)$(AWK) '!x[$$0]++' $(vmlinux-dirs:%=$(objtree)/%/modules.builtin) > $(objtree)/modules.builtin
>> Â Â Â @$(kecho) ' ÂBuilding modules, stage 2.';
>> Â Â Â $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
>> Â Â Â $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_modbuild
>> @@ -1154,7 +1158,7 @@ _modinst_:
>> Â Â Â Â Â Â Â rm -f $(MODLIB)/build ; \
>> Â Â Â Â Â Â Â ln -s $(objtree) $(MODLIB)/build ; \
>> Â Â Â fi
>> - Â Â @cp -f $(objtree)/modules.order $(MODLIB)/
>> + Â Â @cp -f $(objtree)/modules.{order,builtin} $(MODLIB)/
>> Â Â Â $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst
>>
>> Â# This depmod is only for convenience to give the initial
>> @@ -1217,8 +1221,9 @@ clean: archclean $(clean-dirs)
>> Â Â Â Â Â Â Â $ -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
>> Â Â Â Â Â Â Â -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
>> Â Â Â Â Â Â Â -o -name '*.symtypes' -o -name 'modules.order' \
>> - Â Â Â Â Â Â -o -name 'Module.markers' -o -name '.tmp_*.o.*' \
>> - Â Â Â Â Â Â -o -name '*.gcno' $ -type f -print | xargs rm -f
>> + Â Â Â Â Â Â -o -name 'modules.builtin' -o -name 'Module.markers' \
>> + Â Â Â Â Â Â -o -name '.tmp_*.o.*' -o -name '*.gcno' \) \
>> + Â Â Â Â Â Â -type f -print | xargs rm -f
>>
>> Â# mrproper - Delete all generated files, including .config
>> Â#
>> @@ -1416,7 +1421,8 @@ $(clean-dirs):
>> Âclean: Â Â Â rm-dirs := $(MODVERDIR)
>> Âclean: rm-files := $(KBUILD_EXTMOD)/Module.symvers \
>> Â Â Â Â Â Â Â Â Â Â $(KBUILD_EXTMOD)/Module.markers \
>> - Â Â Â Â Â Â Â Â Â $(KBUILD_EXTMOD)/modules.order
>> + Â Â Â Â Â Â Â Â Â $(KBUILD_EXTMOD)/modules.order \
>> + Â Â Â Â Â Â Â Â Â $(KBUILD_EXTMOD)/modules.builtin
>> Âclean: $(clean-dirs)
>> Â Â Â $(call cmd,rmdirs)
>> Â Â Â $(call cmd,rmfiles)
>> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
>> index c29be8f..1cded3b 100644
>> --- a/scripts/Kbuild.include
>> +++ b/scripts/Kbuild.include
>> @@ -143,6 +143,12 @@ ld-option = $(call try-run,\
>> Â# $(Q)$(MAKE) $(build)=dir
>> Âbuild := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.build obj
>>
>> +###
>> +# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.modbuiltin obj=
>> +# Usage:
>> +# $(Q)$(MAKE) $(modbuiltin)=dir
>> +modbuiltin := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.modbuiltin obj
>> +
>> Â# Prefix -I with $(srctree) if it is not an absolute path.
>> Â# skip if -I has no parameter
>> Âaddtree = $(if $(patsubst -I%,%,$(1)), \
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index 7a77787..3525b29 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -37,6 +37,8 @@ modorder Â Â:= $(patsubst %/,%/modules.order, $(filter %/, $(obj-y)) $(obj-m:.o=.ko
>>
>> Â__subdir-y Â := $(patsubst %/,%,$(filter %/, $(obj-y)))
>> Âsubdir-y Â Â += $(__subdir-y)
>> +__subdir-Y Â := $(patsubst %/,%,$(filter %/, $(obj-Y)))
>> +subdir-Y Â Â += $(__subdir-Y)
>> Â__subdir-m Â := $(patsubst %/,%,$(filter %/, $(obj-m)))
>> Âsubdir-m Â Â += $(__subdir-m)
>> Âobj-y Â Â Â Â Â Â Â Â:= $(patsubst %/, %/built-in.o, $(obj-y))
>> @@ -44,7 +46,7 @@ obj-m Â Â Â Â Â Â Â := $(filter-out %/, $(obj-m))
>>
>> Â# Subdirectories we need to descend into
>>
>> -subdir-ym Â Â:= $(sort $(subdir-y) $(subdir-m))
>> +subdir-ym Â Â:= $(sort $(subdir-y) $(subdir-Y) $(subdir-m))
>>
>> Â# if $(foo-objs) exists, foo.o is a composite object
>> Âmulti-used-y := $(sort $(foreach m,$(obj-y), $(if $(strip $($(m:.o=-objs)) $($(m:.o=-y))), $(m))))
>> @@ -76,6 +78,7 @@ always Â Â Â Â Â Â Â:= $(addprefix $(obj)/,$(always))
>> Âtargets Â Â Â Â Â Â Â:= $(addprefix $(obj)/,$(targets))
>> Âmodorder Â Â := $(addprefix $(obj)/,$(modorder))
>> Âobj-y Â Â Â Â Â Â Â Â:= $(addprefix $(obj)/,$(obj-y))
>> +obj-Y Â Â Â Â Â Â Â Â:= $(addprefix $(obj)/,$(obj-Y))
>> Âobj-m Â Â Â Â Â Â Â Â:= $(addprefix $(obj)/,$(obj-m))
>> Âlib-y Â Â Â Â Â Â Â Â:= $(addprefix $(obj)/,$(lib-y))
>> Âsubdir-obj-y := $(addprefix $(obj)/,$(subdir-obj-y))
>> diff --git a/scripts/Makefile.modbuiltin b/scripts/Makefile.modbuiltin
>> new file mode 100644
>> index 0000000..55e1885
>> --- /dev/null
>> +++ b/scripts/Makefile.modbuiltin
>> @@ -0,0 +1,48 @@
>> +# ==========================================================================
>> +# Generating modules.builtin
>> +# ==========================================================================
>> +
>> +src := $(obj)
>> +
>> +PHONY := __modbuiltin
>> +__modbuiltin:
>> +
>> +# Read auto2.conf which sets tristate variables to 'Y' instead of 'y'
>> +# That way, we get the list of built-in modules in obj-Y
>> +-include include/config/auto2.conf
>> +
>> +include scripts/Kbuild.include
>> +
>> +# The filename Kbuild has precedence over Makefile
>> +kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
>> +kbuild-file := $(if $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Kbuild,$(kbuild-dir)/Makefile)
>> +include $(kbuild-file)
>> +
>> +include scripts/Makefile.lib
>> +
>> +modbuiltin-subdirs := $(patsubst %,%/modules.builtin, $(subdir-ym))
>> +modbuiltin-mods Â Â:= $(filter %.ko, $(obj-Y:.o=.ko))
>> +modbuiltin-target Â:= $(obj)/modules.builtin
>> +
>> +__modbuiltin: $(modbuiltin-target) $(subdir-ym)
>> + Â Â @:
>> +
>> +modbuiltin-cmds = Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â\
>> + Â Â for m in $(modbuiltin-mods); do echo kernel/$$m; done; Â\
>> + Â Â cat /dev/null $(modbuiltin-subdirs);
>> +
>> +$(modbuiltin-target): $(subdir-ym)
>> + Â Â $(Q)($(modbuiltin-cmds)) > $@
>> +
>> +# Descending
>> +# ---------------------------------------------------------------------------
>> +
>> +PHONY += $(subdir-ym)
>> +$(subdir-ym):
>> + Â Â $(Q)$(MAKE) $(modbuiltin)=$@
>> +
>> +
>> +# Declare the contents of the .PHONY variable as phony. ÂWe keep that
>> +# information in a variable se we can use it in if_changed and friends.
>> +
>> +.PHONY: $(PHONY)
>> diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c
>> index a04da34..8e2c166 100644
>> --- a/scripts/kconfig/confdata.c
>> +++ b/scripts/kconfig/confdata.c
>> @@ -672,12 +672,27 @@ out:
>> Â Â Â return res;
>> Â}
>>
>> +int fprintf2(FILE *f1, FILE *f2, const char *fmt, ...)
>> +{
>> + Â Â va_list ap;
>> + Â Â int res;
>> +
>> + Â Â va_start(ap, fmt);
>> + Â Â vfprintf(f1, fmt, ap);
>> + Â Â va_end(ap);
>> + Â Â va_start(ap, fmt);
>> + Â Â res = vfprintf(f2, fmt, ap);
>> + Â Â va_end(ap);
>> +
>> + Â Â return res;
>> +}
>> +
>> Âint conf_write_autoconf(void)
>> Â{
>> Â Â Â struct symbol *sym;
>> Â Â Â const char *str;
>> Â Â Â const char *name;
>> - Â Â FILE *out, *out_h;
>> + Â Â FILE *out, *out2, *out_h;
>> Â Â Â time_t now;
>> Â Â Â int i, l;
>>
>> @@ -692,16 +707,23 @@ int conf_write_autoconf(void)
>> Â Â Â if (!out)
>> Â Â Â Â Â Â Â return 1;
>>
>> + Â Â out2 = fopen(".tmpconfig2", "w");
>> + Â Â if (!out2) {
>> + Â Â Â Â Â Â fclose(out);
>> + Â Â Â Â Â Â return 1;
>> + Â Â }
>> +
>> Â Â Â out_h = fopen(".tmpconfig.h", "w");
>> Â Â Â if (!out_h) {
>> Â Â Â Â Â Â Â fclose(out);
>> + Â Â Â Â Â Â fclose(out2);
>> Â Â Â Â Â Â Â return 1;
>> Â Â Â }
>>
>> Â Â Â sym = sym_lookup("KERNELVERSION", 0);
>> Â Â Â sym_calc_value(sym);
>> Â Â Â time(&now);
>> - Â Â fprintf(out, "#\n"
>> + Â Â fprintf2(out, out2, "#\n"
>> Â Â Â Â Â Â Â Â Â Â"# Automatically generated make config: don't edit\n"
>> Â Â Â Â Â Â Â Â Â Â"# Linux kernel version: %s\n"
>> Â Â Â Â Â Â Â Â Â Â"# %s"
>> @@ -726,45 +748,51 @@ int conf_write_autoconf(void)
>> Â Â Â Â Â Â Â Â Â Â Â case no:
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â Â Â Â Â case mod:
>> - Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out, "CONFIG_%s=m\n", sym->name);
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf2(out, out2, "CONFIG_%s=m\n",
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â sym->name);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out_h, "#define CONFIG_%s_MODULE 1\n", sym->name);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â Â Â Â Â case yes:
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out, "CONFIG_%s=y\n", sym->name);
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out2, "CONFIG_%s=%c\n", sym->name,
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â sym->type == S_BOOLEAN ? 'y' : 'Y');
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out_h, "#define CONFIG_%s 1\n", sym->name);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â case S_STRING:
>> Â Â Â Â Â Â Â Â Â Â Â str = sym_get_string_value(sym);
>> - Â Â Â Â Â Â Â Â Â Â fprintf(out, "CONFIG_%s=\"", sym->name);
>> + Â Â Â Â Â Â Â Â Â Â fprintf2(out, out2, "CONFIG_%s=\"", sym->name);
>> Â Â Â Â Â Â Â Â Â Â Â fprintf(out_h, "#define CONFIG_%s \"", sym->name);
>> Â Â Â Â Â Â Â Â Â Â Â while (1) {
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â l = strcspn(str, "\"\\");
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â if (l) {
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fwrite(str, l, 1, out);
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fwrite(str, l, 1, out2);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fwrite(str, l, 1, out_h);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â str += l;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â if (!*str)
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â break;
>> - Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out, "\\%c", *str);
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf2(out, out2, "\\%c", *str);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out_h, "\\%c", *str);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â str++;
>> Â Â Â Â Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â Â Â Â Â fputs("\"\n", out);
>> + Â Â Â Â Â Â Â Â Â Â fputs("\"\n", out2);
>> Â Â Â Â Â Â Â Â Â Â Â fputs("\"\n", out_h);
>> Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â case S_HEX:
>> Â Â Â Â Â Â Â Â Â Â Â str = sym_get_string_value(sym);
>> Â Â Â Â Â Â Â Â Â Â Â if (str[0] != '0' || (str[1] != 'x' && str[1] != 'X')) {
>> - Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out, "CONFIG_%s=%s\n", sym->name, str);
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf2(out, out2, "CONFIG_%s=%s\n",
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â sym->name, str);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â fprintf(out_h, "#define CONFIG_%s 0x%s\n", sym->name, str);
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â case S_INT:
>> Â Â Â Â Â Â Â Â Â Â Â str = sym_get_string_value(sym);
>> - Â Â Â Â Â Â Â Â Â Â fprintf(out, "CONFIG_%s=%s\n", sym->name, str);
>> + Â Â Â Â Â Â Â Â Â Â fprintf2(out, out2, "CONFIG_%s=%s\n", sym->name, str);
>> Â Â Â Â Â Â Â Â Â Â Â fprintf(out_h, "#define CONFIG_%s %s\n", sym->name, str);
>> Â Â Â Â Â Â Â Â Â Â Â break;
>> Â Â Â Â Â Â Â default:
>> @@ -772,6 +800,7 @@ int conf_write_autoconf(void)
>> Â Â Â Â Â Â Â }
>> Â Â Â }
>> Â Â Â fclose(out);
>> + Â Â fclose(out2);
>> Â Â Â fclose(out_h);
>>
>> Â Â Â name = getenv("KCONFIG_AUTOHEADER");
>> @@ -779,6 +808,11 @@ int conf_write_autoconf(void)
>> Â Â Â Â Â Â Â name = "include/linux/autoconf.h";
>> Â Â Â if (rename(".tmpconfig.h", name))
>> Â Â Â Â Â Â Â return 1;
>> + Â Â name = getenv("KCONFIG_AUTOCONFIG2");
>> + Â Â if (!name)
>> + Â Â Â Â Â Â name = "include/config/auto2.conf";
>> + Â Â if (rename(".tmpconfig2", name))
>> + Â Â Â Â Â Â return 1;
>> Â Â Â name = conf_get_autoconfig_name();
>> Â Â Â /*
>> Â Â Â Â* This must be the last step, kbuild has a dependency on auto.conf
> --
> Scott James Remnant
> scott@xxxxxxxxxx
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Scott James Remnant: "Re: [PATCH] kbuild: generate modules.builtin"
Previous message: Kamalesh Babulal: "Re: [PATCH] percpu: add dummy pcpu_lpage_remapped() for !CONFIG_SMP"
In reply to: Scott James Remnant: "Re: [PATCH] kbuild: generate modules.builtin"
Next in thread: Andreas Robinson: "Re: [PATCH] kbuild: generate modules.builtin"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]