Re: [PATCH 3/3] dax/kmem: Always enroll hotplugged memory for memmap_on_memory

[David Hildenbrand]

On 13.07.23 17:40, Verma, Vishal L wrote:
> On Thu, 2023-07-13 at 17:23 +0200, David Hildenbrand wrote:
> > On 13.07.23 17:15, Verma, Vishal L wrote:
> > > On Thu, 2023-07-13 at 09:23 +0200, David Hildenbrand wrote:
> > > > On 13.07.23 08:45, Verma, Vishal L wrote:
> > > > > I'm taking a shot at implementing the splitting internally in
> > > > > memory_hotplug.c. The caller (kmem) side does become trivial with this
> > > > > approach, but there's a slight complication if I don't have the module
> > > > > param override (patch 1 of this series).
> > > > >
> > > > > The kmem diff now looks like:
> > > > >
> > > > >       diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
> > > > >       index 898ca9505754..8be932f63f90 100644
> > > > >       --- a/drivers/dax/kmem.c
> > > > >       +++ b/drivers/dax/kmem.c
> > > > >       @@ -105,6 +105,8 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax)
> > > > >               data->mgid = rc;
> > > > >        
> > > > >               for (i = 0; i < dev_dax->nr_range; i++) {
> > > > >       +               mhp_t mhp_flags = MHP_NID_IS_MGID | MHP_MEMMAP_ON_MEMORY |
> > > > >       +                                 MHP_SPLIT_MEMBLOCKS;
> > > > >                       struct resource *res;
> > > > >                       struct range range;
> > > > >        
> > > > >       @@ -141,7 +143,7 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax)
> > > > >                        * this as RAM automatically.
> > > > >                        */
> > > > >                       rc = add_memory_driver_managed(data->mgid, range.start,
> > > > >       -                               range_len(&range), kmem_name, MHP_NID_IS_MGID);
> > > > >       +                               range_len(&range), kmem_name, mhp_flags);
> > > > >        
> > > > >                       if (rc) {
> > > > >                               dev_warn(dev, "mapping%d: %#llx-%#llx memory add failed\n",
> > > > >       
> > > >
> > > > Why do we need the MHP_SPLIT_MEMBLOCKS?
> > >
> > > I thought we still wanted either an opt-in or opt-out for the kmem
> > > driver to be able to do memmap_on_memory, in case there were
> > > performance implications or the lack of 1GiB PUDs. I haven't
> > > implemented that yet, but I was thinking along the lines of a sysfs
> > > knob exposed by kmem, that controls setting of this new
> > > MHP_SPLIT_MEMBLOCKS flag.
> >
> > Why is MHP_MEMMAP_ON_MEMORY not sufficient for that?
> Ah I see what you mean now - knob just controls MHP_MEMMAP_ON_MEMORY,
> and memory_hotplug is free to split to memblocks if it needs to to
> satisfy that.

And if you don't want memmap holes in a larger area you're adding (for 
example to runtime-allocate 1 GiB pages), simply check the size your 
adding, and if it's, say, less than 1 G, don't set the flag.

But that's probably a corner case use case not worth considering for now.

> That sounds reasonable. Let me give this a try and see if I run into
> anything else. Thanks David!

Sure!

--
Cheers,

David / dhildenb