[PATCH v3] migrate_pages: Avoid blocking for IO in MIGRATE_SYNC_LIGHT

[Douglas Anderson]

The MIGRATE_SYNC_LIGHT mode is intended to block for things that will
finish quickly but not for things that will take a long time. Exactly
how long is too long is not well defined, but waits of tens of
milliseconds is likely non-ideal.

When putting a Chromebook under memory pressure (opening over 90 tabs
on a 4GB machine) it was fairly easy to see delays waiting for some
locks in the kcompactd code path of > 100 ms. While the laptop wasn't
amazingly usable in this state, it was still limping along and this
state isn't something artificial. Sometimes we simply end up with a
lot of memory pressure.

Putting the same Chromebook under memory pressure while it was running
Android apps (though not stressing them) showed a much worse result
(NOTE: this was on a older kernel but the codepaths here are similar).
Android apps on ChromeOS currently run from a 128K-block,
zlib-compressed, loopback-mounted squashfs disk. If we get a page
fault from something backed by the squashfs filesystem we could end up
holding a folio lock while reading enough from disk to decompress 128K
(and then decompressing it using the somewhat slow zlib algorithms).
That reading goes through the ext4 subsystem (because it's a loopback
mount) before eventually ending up in the block subsystem. This extra
jaunt adds extra overhead. Without much work I could see cases where
we ended up blocked on a folio lock for over a second. With more
extreme memory pressure I could see up to 25 seconds.

We considered adding a timeout in the case of MIGRATE_SYNC_LIGHT for
the two locks that were seen to be slow [1] and that generated much
discussion. After discussion, it was decided that we should avoid
waiting for the two locks during MIGRATE_SYNC_LIGHT if they were being
held for IO. We'll continue with the unbounded wait for the more full
SYNC modes.

With this change, I couldn't see any slow waits on these locks with my
previous testcases.

NOTE: The reason I stated digging into this originally isn't because
some benchmark had gone awry, but because we've received in-the-field
crash reports where we have a hung task waiting on the page lock
(which is the equivalent code path on old kernels). While the root
cause of those crashes is likely unrelated and won't be fixed by this
patch, analyzing those crash reports did point out these very long
waits seemed like something good to fix. With this patch we should no
longer hang waiting on these locks, but presumably the system will
still be in a bad shape and hang somewhere else.

[1] https://lore.kernel.org/r/20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid

Suggested-by: Matthew Wilcox <willy@xxxxxxxxxxxxx>
Cc: Mel Gorman <mgorman@xxxxxxxxxxxxxxxxxxx>
Cc: Hillf Danton <hdanton@xxxxxxxx>
Cc: Gao Xiang <hsiangkao@xxxxxxxxxxxxxxxxx>
Signed-off-by: Douglas Anderson <dianders@xxxxxxxxxxxx>
---
Most of the actual code in this patch came from emails written by
Matthew Wilcox and I just cleaned the code up to get it to compile.
I'm happy to set authorship to him if he would like, but for now I've
credited him with Suggested-by.

This patch has changed pretty significantly between versions, so
adding a link to previous versions to help anyone needing to find the
history:
v1 - https://lore.kernel.org/r/20230413182313.RFC.1.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid
v2 - https://lore.kernel.org/r/20230421221249.1616168-1-dianders@xxxxxxxxxxxx/

Changes in v3:
- Combine patches for buffers and folios.
- Use buffer_uptodate() and folio_test_uptodate() instead of timeout.

Changes in v2:
- Keep unbounded delay in "SYNC", delay with a timeout in "SYNC_LIGHT".
- Also add a timeout for locking of buffers.

 mm/migrate.c | 49 ++++++++++++++++++++++++++-----------------------
 1 file changed, 26 insertions(+), 23 deletions(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index db3f154446af..4a384eb32917 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -698,37 +698,32 @@ static bool buffer_migrate_lock_buffers(struct buffer_head *head,
 							enum migrate_mode mode)
 {
 	struct buffer_head *bh = head;
+	struct buffer_head *failed_bh;
 
-	/* Simple case, sync compaction */
-	if (mode != MIGRATE_ASYNC) {
-		do {
-			lock_buffer(bh);
-			bh = bh->b_this_page;
-
-		} while (bh != head);
-
-		return true;
-	}
-
-	/* async case, we cannot block on lock_buffer so use trylock_buffer */
 	do {
 		if (!trylock_buffer(bh)) {
-			/*
-			 * We failed to lock the buffer and cannot stall in
-			 * async migration. Release the taken locks
-			 */
-			struct buffer_head *failed_bh = bh;
-			bh = head;
-			while (bh != failed_bh) {
-				unlock_buffer(bh);
-				bh = bh->b_this_page;
-			}
-			return false;
+			if (mode == MIGRATE_ASYNC)
+				goto unlock;
+			if (mode == MIGRATE_SYNC_LIGHT && !buffer_uptodate(bh))
+				goto unlock;
+			lock_buffer(bh);
 		}
 
 		bh = bh->b_this_page;
 	} while (bh != head);
+
 	return true;
+
+unlock:
+	/* We failed to lock the buffer and cannot stall. */
+	failed_bh = bh;
+	bh = head;
+	while (bh != failed_bh) {
+		unlock_buffer(bh);
+		bh = bh->b_this_page;
+	}
+
+	return false;
 }
 
 static int __buffer_migrate_folio(struct address_space *mapping,
@@ -1162,6 +1157,14 @@ static int migrate_folio_unmap(new_page_t get_new_page, free_page_t put_new_page
 		if (current->flags & PF_MEMALLOC)
 			goto out;
 
+		/*
+		 * In "light" mode, we can wait for transient locks (eg
+		 * inserting a page into the page table), but it's not
+		 * worth waiting for I/O.
+		 */
+		if (mode == MIGRATE_SYNC_LIGHT && !folio_test_uptodate(src))
+			goto out;
+
 		folio_lock(src);
 	}
 	locked = true;
-- 
2.40.1.495.gc816e09b53d-goog