Re: [arm64] kernel boot slowdown in v5.10.19 -> v5.10.42 update

[Vijay Balakrishna]

On 1/28/2022 10:03 AM, Vijay Balakrishna wrote:
> On 1/28/2022 4:06 AM, Catalin Marinas wrote:
> > On Mon, Jan 24, 2022 at 03:03:48PM -0800, Vijay Balakrishna wrote:
> > > We noticed 150ms kernel boot slowdown back in June, 2021, when moving 
> > > from
> > > v5.10.19 to v5.10.42.  This on a 8GB SoC.  Only recently we investigated
> > > this issue and found the regression is introduced by a change in 
> > > map_mem()
> > > (paging_init() -> map_mem() -> __map_memblock(), in particular "map 
> > > all the
> > > memory banks" for loop) by patch
> > >
> > > 2687275a5843d1089687f08fc64eb3f3b026a169
> > > arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
> > >
> > > above is a follow up to
> > >
> > > 0a30c53573b07d5561457e41fb0ab046cd857da5
> > > arm64: mm: Move reserve_crashkernel() into mem_init())
> > >
> > > which deferred crashkernel reservation into mem_init().
> > >
> > > The ~150ms slowdown disappears on booting without "crashkernel=.." on 
> > > kernel
> > > command-line.
> >
> > Is CONFIG_RODATA_FULL_DEFAULT_ENABLED set in your config file? It may
> > cause the same slowdown. I suspect it's because we end up mapping the
> > RAM at the page granularity so more loops and slightly higher TLB
> > pressure. Not sure we can do much about >
>
> Catalin,
>
> Thanks for your reply.
>
> CONFIG_RODATA_FULL_DEFAULT_ENABLED is not set in our config for boot 
> time performance reasons.

From following patches, the main source of deferring crash kernel 
reservation is for Raspberry Pi 4, which has no IOMMU.

1a8e1cef7603e218339ac63cb3178b25554524e5
arm64: use both ZONE_DMA and ZONE_DMA32

8424ecdde7df99d5426e1a1fd9f0fb36f4183032
arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges

0a30c53573b07d5561457e41fb0ab046cd857da5
arm64: mm: Move reserve_crashkernel() into mem_init())

2687275a5843d1089687f08fc64eb3f3b026a169
arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required

However above changes introduced slow boot (regression) for platforms 
with IOMMU, for example as I mentioned earlier ~150ms for us.  We don't 
have following configs set for our SoC --

# CONFIG_ZONE_DMA is not set
# CONFIG_ZONE_DMA32 is not set

Can we restore crash kernel reservation (no deferring to mem_init()) for 
platforms with no DMA memory zones?

I can submit a patch.

Thanks,
Vijay