Re: [PATCH] kernel: fs: drop_caches: add dds drop_caches_count

From: Nag Avadhanam
Date: Tue Feb 16 2016 - 13:37:26 EST


On Tue, 16 Feb 2016, Vladimir Davydov wrote:

On Tue, Feb 16, 2016 at 02:58:04AM +0000, Nag Avadhanam (nag) wrote:
We have a class of platforms that are essentially swap-less embedded
systems that have limited memory resources (2GB and less).

There is a need to implement early alerts (before the OOM killer kicks in)
based on the current memory usage so admins can take appropriate steps (do
not initiate provisioning operations but support existing services,
de-provision certain services, etc. based on the extent of memory usage in
the system) .

There is also a general need to let end users know the available memory so
they can determine if they can enable new services (helps in planning).

These two depend upon knowing approximate (accurate within few 10s of MB)
memory usage within the system. We want to alert admins before system
exhibits any thrashing behaviors.

Have you considered using /proc/kpageflags for counting such pages? It
should already export all information about memory pages you might need,
e.g. which pages are mapped, which are anonymous, which are inactive,
basically all page flags and even more. Moreover, you can even determine
the set of pages that are really read/written by processes - see
/sys/kernel/mm/page_idle/bitmap. On such a small machine scanning the
whole pfn range should be pretty cheap, so you might find this API
acceptable.

Thanks Vladimir. I came across the pagmemap interface sometime ago. I
was not sure if its mainstream. I think this should allow userspace VM scan (scans might take a bit longer). Will try it.

We could avoid the scans altogether.

The need plainly put is, inform the admins of these swapless embedded systems of the available memory.

If we can reliably and efficiently maintain counts of file pages (inactive and active) mapped into the address spaces of active user space processes, this need can be met. "Mapped" of /proc/meminfo does not seem to be a direct fit for this purpose (I need to understand this better). If I know for sure "Mapped" does not count device and the kernel pages mapped into the user space, then I can employ it gainfully for this need.

(Cached - Shmem - <mapped file/binary pages of active processes>) gives me
reclaimable file pages. If I can determine that then I can add that to MemFree and determine the available memory.

Thanks,
nag


Thanks,
Vladimir