Re: [PATCH 2/2] mm: memcg/slab: pre-allocate obj_cgroups for slab caches with SLAB_ACCOUNT

[Roman Gushchin]

On Thu, Nov 12, 2020 at 11:23:03AM -0500, Johannes Weiner wrote:
> On Tue, Nov 10, 2020 at 11:57:53AM -0800, Roman Gushchin wrote:
> > In general it's unknown in advance if a slab page will contain
> > accounted objects or not. In order to avoid memory waste, an
> > obj_cgroup vector is allocated dynamically when a need to account
> > of a new object arises. Such approach is memory efficient, but
> > requires an expensive cmpxchg() to set up the memcg/objcgs pointer,
> > because an allocation can race with a different allocation on another
> > cpu.
> > 
> > But in some common cases it's known for sure that a slab page will
> > contain accounted objects: if the page belongs to a slab cache with a
> > SLAB_ACCOUNT flag set. It includes such popular objects like
> > vm_area_struct, anon_vma, task_struct, etc.
> > 
> > In such cases we can pre-allocate the objcgs vector and simple assign
> > it to the page without any atomic operations, because at this early
> > stage the page is not visible to anyone else.
> > 
> > Signed-off-by: Roman Gushchin <guro@xxxxxx>
> 
> That's a nice optimization!
> 
> Some comments inline:
> 
> > @@ -485,14 +485,20 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
> >   * set_page_objcgs - associate a page with a object cgroups vector
> >   * @page: a pointer to the page struct
> >   * @objcgs: a pointer to the object cgroups vector
> > + * @atomic: save the value atomically
> >   *
> >   * Atomically associates a page with a vector of object cgroups.
> >   */
> >  static inline bool set_page_objcgs(struct page *page,
> > -					struct obj_cgroup **objcgs)
> > +				   struct obj_cgroup **objcgs, bool atomic)
> 
> bool parameters make callsites pretty hard to understand - unless the
> function interface obviously has two modes (read vs write etc.), which
> isn't the case here.
> 
> > -	return !cmpxchg(&page->memcg_data, 0, (unsigned long)objcgs |
> > -			MEMCG_DATA_OBJCGS);
> > +	unsigned long memcg_data = (unsigned long) objcgs | MEMCG_DATA_OBJCGS;
> > +
> > +	if (atomic)
> > +		return !cmpxchg(&page->memcg_data, 0, memcg_data);
> > +
> > +	page->memcg_data = memcg_data;
> > +	return true;
> >  }
> >  #else
> >  static inline struct obj_cgroup **page_objcgs(struct page *page)
> > @@ -506,7 +512,7 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
> >  }
> >  
> >  static inline bool set_page_objcgs(struct page *page,
> > -					struct obj_cgroup **objcgs)
> > +				   struct obj_cgroup **objcgs, bool atomic)
> >  {
> >  	return true;
> >  }
> > diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> > index 69a2893a6455..37bffd336235 100644
> > --- a/mm/memcontrol.c
> > +++ b/mm/memcontrol.c
> > @@ -2874,7 +2874,7 @@ static void commit_charge(struct page *page, struct mem_cgroup *memcg)
> >  
> >  #ifdef CONFIG_MEMCG_KMEM
> >  int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s,
> > -				 gfp_t gfp)
> > +				 gfp_t gfp, bool atomic)
> >  {
> >  	unsigned int objects = objs_per_slab_page(s, page);
> >  	void *vec;
> > @@ -2884,7 +2884,7 @@ int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s,
> >  	if (!vec)
> >  		return -ENOMEM;
> >  
> > -	if (!set_page_objcgs(page, vec))
> > +	if (!set_page_objcgs(page, vec, atomic))
> >  		kfree(vec);
> >  	else
> >  		kmemleak_not_leak(vec);
> 
> The life of page->memcg_data and this optimization could use a central
> comment somewhere, because it's hard to understand what's going on
> from the code alone. This function here seems like a good place?
> 
> I don't see a way to eliminate the bool on the allocation function,
> but maybe it could be more descriptive. Maybe bool slab_account?
> 
> set_page_objcgs() can be inlined at this point. It made some sense to
> abstract away the atomics with setter and matching getter, but with a
> non-atomic mode, inlining makes things clearer and allows for better
> in-place documentation in the sole callsite.

I agree, inlining makes sense here.

I'd prefer not to spill details about SLAB_ACCOUNT into memcg_alloc_page_obj_cgroups().
What's important there is only whether we have an exclusive access to the page or not.

How about the version below?

Thanks!

--