Re: [PATCH v5 1/1] mm: report per-page metadata information

[David Hildenbrand]

On 02.11.23 16:43, Wei Xu wrote:
> On Wed, Nov 1, 2023 at 7:58 PM Pasha Tatashin <pasha.tatashin@xxxxxxxxxx> wrote:
> > On Wed, Nov 1, 2023 at 7:40 PM Wei Xu <weixugc@xxxxxxxxxx> wrote:
> > > On Wed, Nov 1, 2023 at 4:08 PM Sourav Panda <souravpanda@xxxxxxxxxx> wrote:
> > > > Adds a new per-node PageMetadata field to
> > > > /sys/devices/system/node/nodeN/meminfo
> > > > and a global PageMetadata field to /proc/meminfo. This information can
> > > > be used by users to see how much memory is being used by per-page
> > > > metadata, which can vary depending on build configuration, machine
> > > > architecture, and system use.
> > > >
> > > > Per-page metadata is the amount of memory that Linux needs in order to
> > > > manage memory at the page granularity. The majority of such memory is
> > > > used by "struct page" and "page_ext" data structures. In contrast to
> > > > most other memory consumption statistics, per-page metadata might not
> > > > be included in MemTotal. For example, MemTotal does not include memblock
> > > > allocations but includes buddy allocations. While on the other hand,
> > > > per-page metadata would include both memblock and buddy allocations.
> > >
> > > I expect that the new PageMetadata field in meminfo should help break
> > > down the memory usage of a system (MemUsed, or MemTotal - MemFree),
> > > similar to the other fields in meminfo.
> > >
> > > However, given that PageMetadata includes per-page metadata allocated
> > > from not only the buddy allocator, but also the memblock allocations,
> > > and MemTotal doesn't include memory reserved by memblock allocations,
> > > I wonder how a user can actually use this new PageMetadata to break
> > > down the system memory usage.  BTW, it is not robust to assume that
> > > all memblock allocations are for per-page metadata.
> >
> > Hi Wei,
> >
> > > Here are some ideas to address this problem:
> > >
> > > - Only report the buddy allocations for per-page medata in PageMetadata, or
> >
> > Making PageMetadata not to contain all per-page memory but just some
> > is confusing, especially right after boot it would always be 0, as all
> > struct pages are all coming from memblock during boot, yet we know we
> > have allocated tons of memory for struct pages.
> >
> > > - Report per-page metadata in two separate fields in meminfo, one for
> > > buddy allocations and another for memblock allocations, or
> >
> > This is also going to be confusing for the users, it is really
> > implementation detail which allocator was used to allocate struct
> > pages, and having to trackers is not going to improve things.
> >
> > > - Change MemTotal/MemUsed to include the memblock reserved memory as well.
> >
> > I think this is the right solution for an existing bug: MemTotal
> > should really include memblock reserved memory.
>
> Adding reserved memory to MemTotal is a cleaner approach IMO as well.
> But it changes the semantics of MemTotal, which may have compatibility
> issues.

I object.

--
Cheers,

David / dhildenb