Re: [PATCH] vmscan: memcg needs may_swap (Re: [patch] vmscan: rename sc.may_swap to may_unmap)

[Minchan Kim]

Hi,




On Mon, Mar 30, 2009 at 8:45 AM, KOSAKI Motohiro
<kosaki.motohiro@xxxxxxxxxxxxxx> wrote:
>> On Fri, 27 Mar 2009 15:19:26 +0900
>> Daisuke Nishimura <nishimura@xxxxxxxxxxxxxxxxx> wrote:
>>
>> > Added
>> > ÂCc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
>> > ÂCc: Balbir Singh <balbir@xxxxxxxxxx>
>> >
>> > I'm sorry for replying to a very old mail.
>> >
>> > > @@ -1713,7 +1713,7 @@ unsigned long try_to_free_mem_cgroup_pag
>> > > Â{
>> > > Â struct scan_control sc = {
>> > > Â Â Â Â Â .may_writepage = !laptop_mode,
>> > > - Â Â Â Â .may_swap = 1,
>> > > + Â Â Â Â .may_unmap = 1,
>> > > Â Â Â Â Â .swap_cluster_max = SWAP_CLUSTER_MAX,
>> > > Â Â Â Â Â .swappiness = swappiness,
>> > > Â Â Â Â Â .order = 0,
>> > > @@ -1723,7 +1723,7 @@ unsigned long try_to_free_mem_cgroup_pag
>> > > Â struct zonelist *zonelist;
>> > >
>> > > Â if (noswap)
>> > > - Â Â Â Â sc.may_swap = 0;
>> > > + Â Â Â Â sc.may_unmap = 0;
>> > >
>> > > Â sc.gfp_mask = (gfp_mask & GFP_RECLAIM_MASK) |
>> > > Â Â Â Â Â Â Â Â Â (GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK);
>> > IIUC, memcg had used may_swap as a flag for "we need to use swap?" as the name indicate.
>> >
>> > Because, when mem+swap hits the limit, trying to swapout pages is meaningless
>> > as it doesn't change mem+swap usage.
>> >
>> Good catch...sigh, I missed this disussion.
>>
>>
>>
>> > What do you think of this patch?
>> > ===
>> > From: Daisuke Nishimura <nishimura@xxxxxxxxxxxxxxxxx>
>> >
>> > vmscan-rename-scmay_swap-to-may_unmap.patch removed may_swap flag,
>> > but memcg had used it as a flag for "we need to use swap?", as the
>> > name indicate.
>> >
>> > And in current implementation, memcg cannot reclaim mapped file caches
>> > when mem+swap hits the limit.
>> >
>> When mem+swap hits the limit, swap-out anonymous page doesn't reduce the
>> amount of usage of mem+swap, so, swap-out should be avoided.
>>
>> > re-introduce may_swap flag and handle it at shrink_page_list.
>> >
>> > This patch doesn't influence any scan_control users other than memcg.
>> >
>>
>>
>> > Signed-off-by: Daisuke Nishimura <nishimura@xxxxxxxxxxxxxxxxx>
>>
>> Seems good,
>> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
>>
>> But hum....Maybe this lru scan work in the same way as the case
>> of !total_swap_pages. (means don't scan anon LRU.)
>> revisit this later.
>
> Well, How about following patch?
>
> So, I have to agree my judgement of may_unmap was wrong.
> You explain memcg can use may_swap instead may_unmap. and I think
> other may_unmap user (zone_reclaim and shrink_all_list) can convert
> may_unmap code to may_swap.
>
> IOW, Nishimura-san, you explain we can remove the branch of the may_unmap
> from shrink_page_list().
> it's really good job. thanks!
>
>
> ========
> Subject: vmswan: reintroduce sc->may_swap
>
> vmscan-rename-scmay_swap-to-may_unmap.patch removed may_swap flag,
> but memcg had used it as a flag for "we need to use swap?", as the
> name indicate.
>
> And in current implementation, memcg cannot reclaim mapped file caches
> when mem+swap hits the limit.
>
> re-introduce may_swap flag and handle it at get_scan_ratio().
> This patch doesn't influence any scan_control users other than memcg.
>
> Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@xxxxxxxxxxxxxx>
> Signed-off-by: Daisuke Nishimura <nishimura@xxxxxxxxxxxxxxxxx>
> --
> Âmm/vmscan.c | Â 12 ++++++++++--
> Â1 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 3be6157..00ea4a1 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -63,6 +63,9 @@ struct scan_control {
> Â Â Â Â/* Can mapped pages be reclaimed? */
> Â Â Â Âint may_unmap;
>
> + Â Â Â /* Can pages be swapped as part of reclaim? */
> + Â Â Â int may_swap;
> +

Sorry for too late response.
I don't know memcg well.

The memcg managed to use may_swap well with global page reclaim until now.
I think that was because may_swap can represent both meaning.
Do we need each variables really ?

How about using union variable ?
---

struct scan_control {
  /* Incremented by the number of inactive pages that were scanned */
  unsigned long nr_scanned;
...
   union {
    int may_swap; /* memcg: Cap pages be swapped as part of reclaim? */
    int may_unmap /* global: Can mapped pages be reclaimed? */
  };



> Â Â Â Â/* This context's SWAP_CLUSTER_MAX. If freeing memory for
> Â Â Â Â * suspend, we effectively ignore SWAP_CLUSTER_MAX.
> Â Â Â Â * In this context, it doesn't matter that we scan the
> @@ -1379,7 +1382,7 @@ static void get_scan_ratio(struct zone *zone, struct scan_control *sc,
> Â Â Â Âstruct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc);
>
> Â Â Â Â/* If we have no swap space, do not bother scanning anon pages. */
> - Â Â Â if (nr_swap_pages <= 0) {
> + Â Â Â if (!sc->may_swap || (nr_swap_pages <= 0)) {
> Â Â Â Â Â Â Â Âpercent[0] = 0;
> Â Â Â Â Â Â Â Âpercent[1] = 100;
> Â Â Â Â Â Â Â Âreturn;
> @@ -1695,6 +1698,7 @@ unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
> Â Â Â Â Â Â Â Â.may_writepage = !laptop_mode,
> Â Â Â Â Â Â Â Â.swap_cluster_max = SWAP_CLUSTER_MAX,
> Â Â Â Â Â Â Â Â.may_unmap = 1,
> + Â Â Â Â Â Â Â .may_swap = 1,
> Â Â Â Â Â Â Â Â.swappiness = vm_swappiness,
> Â Â Â Â Â Â Â Â.order = order,
> Â Â Â Â Â Â Â Â.mem_cgroup = NULL,
> @@ -1714,6 +1718,7 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *mem_cont,
> Â Â Â Âstruct scan_control sc = {
> Â Â Â Â Â Â Â Â.may_writepage = !laptop_mode,
> Â Â Â Â Â Â Â Â.may_unmap = 1,
> + Â Â Â Â Â Â Â .may_swap = 1,
> Â Â Â Â Â Â Â Â.swap_cluster_max = SWAP_CLUSTER_MAX,
> Â Â Â Â Â Â Â Â.swappiness = swappiness,
> Â Â Â Â Â Â Â Â.order = 0,
> @@ -1723,7 +1728,7 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *mem_cont,
> Â Â Â Âstruct zonelist *zonelist;
>
> Â Â Â Âif (noswap)
> - Â Â Â Â Â Â Â sc.may_unmap = 0;
> + Â Â Â Â Â Â Â sc.may_swap = 0;
>
> Â Â Â Âsc.gfp_mask = (gfp_mask & GFP_RECLAIM_MASK) |
> Â Â Â Â Â Â Â Â Â Â Â Â(GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK);
> @@ -1763,6 +1768,7 @@ static unsigned long balance_pgdat(pg_data_t *pgdat, int order)
> Â Â Â Âstruct scan_control sc = {
> Â Â Â Â Â Â Â Â.gfp_mask = GFP_KERNEL,
> Â Â Â Â Â Â Â Â.may_unmap = 1,
> + Â Â Â Â Â Â Â .may_swap = 1,
> Â Â Â Â Â Â Â Â.swap_cluster_max = SWAP_CLUSTER_MAX,
> Â Â Â Â Â Â Â Â.swappiness = vm_swappiness,
> Â Â Â Â Â Â Â Â.order = order,
> @@ -2109,6 +2115,7 @@ unsigned long shrink_all_memory(unsigned long nr_pages)
> Â Â Â Âstruct scan_control sc = {
> Â Â Â Â Â Â Â Â.gfp_mask = GFP_KERNEL,
> Â Â Â Â Â Â Â Â.may_unmap = 0,
> + Â Â Â Â Â Â Â .may_swap = 1,
> Â Â Â Â Â Â Â Â.swap_cluster_max = nr_pages,
> Â Â Â Â Â Â Â Â.may_writepage = 1,
> Â Â Â Â Â Â Â Â.isolate_pages = isolate_pages_global,
> @@ -2289,6 +2296,7 @@ static int __zone_reclaim(struct zone *zone, gfp_t gfp_mask, unsigned int order)
> Â Â Â Âstruct scan_control sc = {
> Â Â Â Â Â Â Â Â.may_writepage = !!(zone_reclaim_mode & RECLAIM_WRITE),
> Â Â Â Â Â Â Â Â.may_unmap = !!(zone_reclaim_mode & RECLAIM_SWAP),
> + Â Â Â Â Â Â Â .may_swap = 1,
> Â Â Â Â Â Â Â Â.swap_cluster_max = max_t(unsigned long, nr_pages,
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂSWAP_CLUSTER_MAX),
> Â Â Â Â Â Â Â Â.gfp_mask = gfp_mask,
>
>
>
>
>



-- 
Kinds regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/