Re: [PATCH v2 0/2] Add a new scheme to support demotion on tiered memory system

[Huang, Ying]

Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx> writes:

> On 12/23/2021 9:07 AM, Huang, Ying wrote:
>> Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx> writes:
>> 
>>> Hi,
>>>
>>> Now on tiered memory system with different memory types, the reclaim path in
>>> shrink_page_list() already support demoting pages to slow memory node instead
>>> of discarding the pages. However, at that time the fast memory node memory
>>> wartermark is already tense, which will increase the memory allocation latency
>>> during page demotion. So a new method from user space demoting cold pages
>>> proactively will be more helpful.
>>>
>>> We can rely on the DAMON in user space to help to monitor the cold memory on
>>> fast memory node, and demote the cold pages to slow memory node proactively to
>>> keep the fast memory node in a healthy state.
>>>
>>> This patch set introduces a new scheme named DAMOS_DEMOTE to support this feature,
>>> and works well from my testing. Any comments are welcome. Thanks.
>> As a performance optimization patch, it's better to provide some
>> test
>> results.
>
> Actually this is a functional patch, which adds a new scheme for
> DAMON. And I think it is too early to measure the performance for the
> real workload, and more work need to do for DAMON used on tiered
> memory system (like supporting promotion scheme later).

I don't think you provide any new functionality except the performance
influence.

And I think proactive demotion itself can show some performance benefit
already.  Just like we can find the performance benefit in the proactive
reclaim patchset as below.

https://lore.kernel.org/lkml/20211019150731.16699-1-sj@xxxxxxxxxx/

>> Another question is why we shouldn't do this in user space?  With DAMON,
>> it's possible to export cold memory regions information to the user
>> space, then we can use move_pages() to migrate them from DRAM to PMEM.
>> What's the problem of that?
>
> IMO this is the purpose of introducing scheme for DAMON, and you can
> check more in the Documentation/admin-guide/mm/damon/usage.rst.
>
> "
> Schemes
> -------
>
> For usual DAMON-based data access aware memory management
> optimizations, users
> would simply want the system to apply a memory management action to a memory
> region of a specific access pattern.  DAMON receives such formalized
> operation
> schemes from the user and applies those to the target processes.
> "

For proactive reclaim, we haven't a user space ABI to reclaim a page of
a process from memory to disk.  So it appears necessary to add a kernel
module to do that.

But for proactive demotion, we already have a user space ABI
(move_pages()) to demote a page of a process from DRAM to PMEM.  What
prevents you to do all these in the user space?

And, I found there are MADV_XXX schemes too.  Where the user space ABIs
are available already.  TBH, I don't know why we need these given there
are already user space ABIs.  Maybe this is a question for SeongJae too.

Best Regards,
Huang, Ying