Re: [PATCH] ARM: add boot image dependencies not to generate invalid images

From: Masahiro Yamada
Date: Fri Jul 10 2015 - 12:41:47 EST


Hi Russel,


2015-07-10 19:22 GMT+09:00 Russell King - ARM Linux <linux@xxxxxxxxxxxxxxxx>:
> On Mon, Jul 06, 2015 at 09:37:04PM +0900, Masahiro Yamada wrote:
>> [3] Then, re-build "all" and "uImage" simultaneously.
>> You will get an invalid uImage at random.
>> $ make -s -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
>> Image Name: Linux-4.2.0-rc1-00008-g1c4c715-d
>> Created: Mon Jul 6 17:52:22 2015
>> Image Type: ARM Linux Kernel Image (uncompressed)
>> Data Size: 26768 Bytes = 26.14 kB = 0.03 MB
>> Load Address: 80208000
>> Entry Point: 80208000
>
> At no point in the above do I see an attempt to rebuild init/main.o,
> which there should be as you touched the corresponding .c file -
> because you're hiding that output with -s. Please show what's going
> on without using -s.

Please see the log attached at the end of this message.


>> "make uImage" could descend into arch/arm/boot/Makefile before
>> "make zImage" is completed because arch/arm/Makefile describes no
>> dependency among boot targets.
>
> The uImage target should depend on vmlinux, which should force the
> rebuild of init/main.o, and relink of the top-level vmlinux file
> before decending into arch/arm/boot/Makefile for the
> arch/arm/boot/uImage target.
>
> That makefile contains the dependencies required to order things
> correctly - the arch/arm/boot/uImage target depends on
> arch/arm/boot/zImage, which in turn depends on
> arch/arm/boot/compressed/vmlinux, and then arch/arm/boot/Image.
>
> In other words, arch/arm/boot/Makefile deals with the dependencies
> between the targets it's responsible for building itself.

No, you do not understand what is happening in here.


The dependencies among targets such as Image, zImage, uImage,
are fully described in arch/arm/boot/Makefile, but it is not enough.

Because arch/arm/boot/Makefile is _not_ included from the top-level Makefile,
the top-level build cannot know the dependency between zImage and uImage.

arch/arm/Makefile, which is included from the top Makefile,
only describes that zImage depends on vmlinux,
and uImage depends on vmlinux as well.
But, no dependency between zImage and uImage is written in arch/arm/Makefile.

Consequently, we run make with the parallel option, first Kbuild
updates vmlinux,
and then two different threads descends into arch/arm/boot/Makefile
almost at the same time, one for updating zImage and the other for uImage.

The is a race between the two threads.

While one thread is re-generating zImage and also uImage on top of that,
the other thread tries to re-generate zImage independently.

zImage is overwritten by the slower thread, and then uImage is
re-generated based on the broken zImage.

See my detailed build log below:

Please note:
"Kernel: arch/arm/boot/zImage is ready"
is displayed twice,
the first one is shown before the uImage log, and the second one is
after uImage.

zImage is correct, but uImage is extremely small when this problem happens.



$ git describe
v4.2-rc1-62-gc4b5fd3


$ touch init/main.c
$ make -j8 ARCH=arm UIMAGE_LOADADDR=0x80208000 all uImage
CHK include/config/kernel.release
CHK include/generated/uapi/linux/version.h
CHK include/generated/utsrelease.h
make[1]: `include/generated/mach-types.h' is up to date.
CHK include/generated/timeconst.h
CHK include/generated/bounds.h
CHK include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
CC init/main.o
CHK include/generated/compile.h
LD init/built-in.o
LINK vmlinux
LD vmlinux.o
MODPOST vmlinux.o
GEN .version
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
LD init/built-in.o
KSYM .tmp_kallsyms1.o
KSYM .tmp_kallsyms2.o
LD vmlinux
SORTEX vmlinux
SYSMAP System.map
OBJCOPY arch/arm/boot/Image
Building modules, stage 2.
Kernel: arch/arm/boot/Image is ready
GZIP arch/arm/boot/compressed/piggy.gzip
AS arch/arm/boot/compressed/piggy.gzip.o
Kernel: arch/arm/boot/Image is ready
GZIP arch/arm/boot/compressed/piggy.gzip
LD arch/arm/boot/compressed/vmlinux
OBJCOPY arch/arm/boot/zImage
Kernel: arch/arm/boot/zImage is ready
UIMAGE arch/arm/boot/uImage
Image Name: Linux-4.2.0-rc1-00062-gc4b5fd3-d
Created: Sat Jul 11 01:22:22 2015
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 26472 Bytes = 25.85 kB = 0.03 MB
Load Address: 80208000
Entry Point: 80208000
Image arch/arm/boot/uImage is ready
MODPOST 192 modules
AS arch/arm/boot/compressed/piggy.gzip.o
LD arch/arm/boot/compressed/vmlinux
OBJCOPY arch/arm/boot/zImage
Kernel: arch/arm/boot/zImage is ready
$ LANG=C ls -l arch/arm/boot/
total 19636
-rwxrwxr-x 1 masahiro masahiro 13766656 Jul 11 01:22 Image
-rw-rw-r-- 1 masahiro masahiro 3137 Jul 3 01:17 Makefile
drwxrwxr-x 2 masahiro masahiro 4096 Jul 3 01:17 bootp
drwxrwxr-x 2 masahiro masahiro 4096 Jul 11 01:22 compressed
drwxrwxr-x 3 masahiro masahiro 155648 Jul 11 00:58 dts
-rw-rw-r-- 1 masahiro masahiro 1648 Jul 3 01:17 install.sh
-rw-rw-r-- 1 masahiro masahiro 26536 Jul 11 01:22 uImage
-rwxrwxr-x 1 masahiro masahiro 6135048 Jul 11 01:22 zImage







--
Best Regards
Masahiro Yamada
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/