Re: [Regression] 3.19-rc3 : memcg: Hang in mount memcg

From: Suzuki K. Poulose
Date: Fri Jan 23 2015 - 10:00:12 EST


On 22/01/15 13:45, Johannes Weiner wrote:
On Wed, Jan 21, 2015 at 04:39:55PM +0000, Will Deacon wrote:
On Mon, Jan 19, 2015 at 12:51:27PM +0000, Suzuki K. Poulose wrote:
On 10/01/15 08:55, Vladimir Davydov wrote:
The problem is that the memory cgroup controller takes a css reference
per each charged page and does not reparent charged pages on css
offline, while cgroup_mount/cgroup_kill_sb expect all css references to
offline cgroups to be gone soon, restarting the syscall if the ref count
!= 0. As a result, if you create a memory cgroup, charge some page cache
to it, and then remove it, unmount/mount will hang forever.

May be, we should kill the ref counter to the memory controller root in
cgroup_kill_sb only if there is no children at all, neither online nor
offline.


Still reproducible on 3.19-rc5 with the same setup.

Yeah, I'm seeing the same failure on my setup too.

From git bisect, the last good commit is :

commit 8df0c2dcf61781d2efa8e6e5b06870f6c6785735
Author: Pranith Kumar <bobby.prani@xxxxxxxxx>
Date: Wed Dec 10 15:42:28 2014 -0800

slab: replace smp_read_barrier_depends() with lockless_dereference()

So that points at 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")
as the offending commit.

With b2052564e66d ("mm: memcontrol: continue cache reclaim from
offlined groups"), page cache can pin an old css and its ancestors
indefinitely, making that hang in a second mount() very likely.

However, swap entries have also been doing that for quite a while now,
and as Vladimir pointed out, the same is true for kernel memory. This
latest change just makes this existing bug easier to trigger.

I think we have to update the lifetime rules to reflect reality here:
memory and swap lifetime is indefinite, so once the memory controller
is used, it has state that is independent from whether its mounted or
not. We can support an identical remount, but have to fail mounting
with new parameters that would change the behavior of the controller.

Suzuki, Will, could you give the following patch a shot?



Tejun, would that route be acceptable to you?

Thanks

---
From c5e88d02d185c52748df664aa30a2c5f8949b0f7 Mon Sep 17 00:00:00 2001
From: Johannes Weiner <hannes@xxxxxxxxxxx>
Date: Thu, 22 Jan 2015 08:16:31 -0500
Subject: [patch] kernel: cgroup: prevent mount hang due to memory controller
lifetime



Don't offline the controller root as long as there are any children,
dead or alive. A remount will no longer wait for these old references
to drain, it will simply mount the persistent controller state again.

Reported-by: "Suzuki K. Poulose" <Suzuki.Poulose@xxxxxxx>
Reported-by: Will Deacon <will.deacon@xxxxxxx>
Signed-off-by: Johannes Weiner <hannes@xxxxxxxxxxx>
This one fixes the issue.

Tested-by : Suzuki K. Poulose <suzuki.poulose@xxxxxxx>

Thanks
Suzuki



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/