Re: [PATCH V8 38/44] memremap_pages: Define pgmap_mk_{readwrite|noaccess}() calls

[Dan Williams]

On Fri, Feb 4, 2022 at 9:56 PM Ira Weiny <ira.weiny@xxxxxxxxx> wrote:
>
> On Fri, Feb 04, 2022 at 04:27:38PM -0800, Dan Williams wrote:
> > On Fri, Feb 4, 2022 at 4:25 PM Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
> > >
> > > On Fri, Feb 4, 2022 at 4:19 PM Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
> > > >
> > > > On Fri, Feb 4, 2022 at 4:10 PM Ira Weiny <ira.weiny@xxxxxxxxx> wrote:
> > > > >
> > > > > On Fri, Feb 04, 2022 at 10:35:59AM -0800, Dan Williams wrote:
> > > > > > On Thu, Jan 27, 2022 at 9:55 AM <ira.weiny@xxxxxxxxx> wrote:
> > > > > > >
> > > > >
> > > > > [snip]
> > > > >
> > > > > I'll address the other comments later but wanted to address the idea below.
> > > > >
> > > > > > > diff --git a/include/linux/sched.h b/include/linux/sched.h
> > > > > > > index f5b2be39a78c..5020ed7e67b7 100644
> > > > > > > --- a/include/linux/sched.h
> > > > > > > +++ b/include/linux/sched.h
> > > > > > > @@ -1492,6 +1492,13 @@ struct task_struct {
> > > > > > >         struct callback_head            l1d_flush_kill;
> > > > > > >  #endif
> > > > > > >
> > > > > > > +#ifdef CONFIG_DEVMAP_ACCESS_PROTECTION
> > > > > > > +       /*
> > > > > > > +        * NOTE: pgmap_prot_count is modified within a single thread of
> > > > > > > +        * execution.  So it does not need to be atomic_t.
> > > > > > > +        */
> > > > > > > +       u32                             pgmap_prot_count;
> > > > > > > +#endif
> > > > > >
> > > > > > It's not at all clear why the task struct needs to be burdened with
> > > > > > this accounting. Given that a devmap instance is needed to manage page
> > > > > > protections, why not move the nested protection tracking to a percpu
> > > > > > variable relative to an @pgmap arg? Something like:
> > > > > >
> > > > > > void __pgmap_mk_readwrite(struct dev_pagemap *pgmap)
> > > > > > {
> > > > > >        migrate_disable();
> > > > > >        preempt_disable();
> > > > >
> > > > > Why burden threads like this?  kmap_local_page() is perfectly able to migrate
> > > > > or be preempted?
> > > > >
> > > > > I think this is way to restrictive.
> > > >
> > > > kmap_local_page() holds migrate_disable() over the entire mapping, so
> > > > we're only talking about preempt_disable(). I tend to think that
> > > > bloating task_struct for something that is rarely used "kmap on dax
> > > > pmem pages" is not the right tradeoff.
> > >
> > > Now, I can see an argument that promoting kmap_local_page() to
> > > preempt_disable() could cause problems, but I'd like help confirming
> > > that before committing to extending task_struct.
> >
> > ...as I say that it occurs to me that the whole point of
> > kmap_local_page() is to be better than kmap_atomic() and this undoes
> > that. I'd at least like that documented as the reason that task_struct
> > needs to carry a new field.
>
> I'll try and update the commit message but kmap_local_page() only disables
> migrate on a highmem system.

Right, but that still means that the code in question is prepared for
migrate_disable(). Instead I would justify the task_struct expansion
on the observation that moving the enable tracker to percpu would
require pre-empt disable and promote kmap_local_page() to
kmap_atomic() level of restriction. Otherwise if there was a way to
avoid task_struct expansion with only migrate_disable() and not
preempt_disable I think it would be worth it, but nothing comes to
mind at the moment.

> The devmap/PKS use case is specifically not
> supported on highmem systems.  Mainly because on highmem systems
> kmap_local_page() actually creates a new mapping which is not covered by PKS
> anyway.
>
> So for the devmap/PKS use case kmap_local_page() is defined as:
>
>  static inline void *kmap_local_page(struct page *page)
>  {
> +       pgmap_mk_readwrite(page);
>         return page_address(page);
>  }
>
> ...for the linear mapping.  I'll try and update the commit message with this
> detail.

Yeah, add the kmap_atomic() observation, not the !highmem note.