Re: High kmalloc-32 slab cache consumption with 10k containers

From: Michal Hocko
Date: Wed Apr 07 2021 - 07:54:53 EST


On Mon 05-04-21 11:18:48, Bharata B Rao wrote:
> Hi,
>
> When running 10000 (more-or-less-empty-)containers on a bare-metal Power9
> server(160 CPUs, 2 NUMA nodes, 256G memory), it is seen that memory
> consumption increases quite a lot (around 172G) when the containers are
> running. Most of it comes from slab (149G) and within slab, the majority of
> it comes from kmalloc-32 cache (102G)

Is this 10k cgroups a testing enviroment or does anybody really use that
in production? I would be really curious to hear how that behaves when
those containers are not idle. E.g. global memory reclaim iterating over
10k memcgs will likely be very visible. I do remember playing with
similar setups few years back and the overhead was very high.
--
Michal Hocko
SUSE Labs