Re: [PATCH v3 1/5] mm/memory_hotplug: introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers

[Sumanth Korikkar]

On Mon, Nov 27, 2023 at 04:16:18PM +0100, David Hildenbrand wrote:
> On 27.11.23 09:20, Sumanth Korikkar wrote:
> > Introduce  MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers to
> > prepare the transition of memory to and from a physically accessible
> > state. This enhancement is crucial for implementing the "memmap on
> > memory" feature for s390 in a subsequent patch.
> > 
> > Platforms such as x86 can support physical memory hotplug via ACPI. When
> > there is physical memory hotplug, ACPI event leads to the memory
> > addition with the following callchain:
> > acpi_memory_device_add()
> >    -> acpi_memory_enable_device()
> >       -> __add_memory()
> > 
> > After this, the hotplugged memory is physically accessible, and altmap
> > support prepared, before the "memmap on memory" initialization in
> > memory_block_online() is called.
> > 
> > On s390, memory hotplug works in a different way. The available hotplug
> > memory has to be defined upfront in the hypervisor, but it is made
> > physically accessible only when the user sets it online via sysfs,
> > currently in the MEM_GOING_ONLINE notifier. This is too late and "memmap
> > on memory" initialization is performed before calling MEM_GOING_ONLINE
> > notifier.
> > 
> > During the memory hotplug addition phase, altmap support is prepared and
> > during the memory onlining phase s390 requires memory to be physically
> > accessible and then subsequently initiate the "memmap on memory"
> > initialization process.
> > 
> > The memory provider will handle new MEM_PREPARE_ONLINE /
> > MEM_FINISH_OFFLINE notifications and make the memory accessible.
> > 
> > The mhp_flag MHP_OFFLINE_INACCESSIBLE is introduced and is relevant when
> > used along with MHP_MEMMAP_ON_MEMORY, because the altmap cannot be
> > written (e.g., poisoned) when adding memory -- before it is set online.
> > This allows for adding memory with an altmap that is not currently made
> > available by a hypervisor. When onlining that memory, the hypervisor can
> > be instructed to make that memory accessible via the new notifiers and
> > the onlining phase will not require any memory allocations, which is
> > helpful in low-memory situations.
> > 
> > All architectures ignore unknown memory notifiers.  Therefore, the
> > introduction of these new notifiers does not result in any functional
> > modifications across architectures.
> > 
> > Suggested-by: Gerald Schaefer <gerald.schaefer@xxxxxxxxxxxxx>
> > Suggested-by: David Hildenbrand <david@xxxxxxxxxx>
> > Signed-off-by: Sumanth Korikkar <sumanthk@xxxxxxxxxxxxx>
> > ---
...
> >   int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> > -			      struct zone *zone)
> > +			      struct zone *zone, bool mhp_off_inaccessible)
> >   {
> >   	unsigned long end_pfn = pfn + nr_pages;
> >   	int ret, i;
> > @@ -1092,7 +1109,14 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> >   	ret = kasan_add_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
> >   	if (ret)
> >   		return ret;
> > -
> > +	/*
> > +	 * Memory block is accessible at this stage and hence poison the struct
> > +	 * pages now.  If the memory block is accessible during memory hotplug
> > +	 * addition phase, then page poisining is already performed in
> > +	 * sparse_add_section().
> > +	 */
> > +	if (mhp_off_inaccessible)
> > +		page_init_poison_with_resched(pfn, nr_pages);
> 
> Can you elaborate why a simple page_init_poison() as for
> sparse_add_section() is insufficient?
>
Looks like cond_resched() is not needed. page_init_poison() could be
performed similar to when adding new memory in sparse_add_section().
IIUC, As memory is onlined in memory block granuality, this
cond_resched() wouldnt be needed then.

Thanks