Re: [PATCH v2 1/4] mm/filemap: Add folio_lock_timeout()

[Matthew Wilcox]

On Wed, Apr 26, 2023 at 11:09:18AM +0100, Mel Gorman wrote:
> On Tue, Apr 25, 2023 at 07:19:48AM -0700, Doug Anderson wrote:
> > On Mon, Apr 24, 2023 at 6:09???PM Hillf Danton <hdanton@xxxxxxxx> wrote:
> > > Take a look at another case of lock wait [1].
> > >
> > > [1] https://lore.kernel.org/lkml/CAHk-=wgyL9OujQ72er7oXt_VsMeno4bMKCTydBT1WSaagZ_5CA@xxxxxxxxxxxxxx/
> > 
> > So is this an explicit NAK on this approach, then? It still feels
> > worthwhile to me given the current kcompactd design where there is a
> > single thread that's in charge of going through and cleaning up all of
> > memory. Any single pags isn't _that_ important for kcompactd to deal
> > with and it's nice not to block the whole task's ability to make
> > progress. kcompactd is already very much designed in this model (which
> > is why SYNC_LIGHT exists in the first place) and that's why my patch
> > series was relatively simple/short. That being said, if people really
> > don't think I should pursue this then I won't send another version and
> > we can drop it.
> 
> I don't consider it to be an explicit NAK but lets
> cc Linus because it's a valid question. Linus, the patch is
> https://lore.kernel.org/lkml/20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid/
> asnd it's adding folio_lock_timeout which in older terms is a
> lock_page_timout. The intended use is kcompactd doing out-of-line
> compaction (like kswapd does out-of-line reclaim) to try lock a page in
> MIGRATE_SYNC_LIGHT mode but if it cannot be locked quickly then give up
> and move on to another migration candidate. The MIGRATE_SYNC_LIGHT is
> expected to incur some delays while trying to make forward progress and
> the overall problem is that kcompactd can sometimes stall for many seconds
> and sometimes minutes on one page.
> 
> The reason I don't consider this patch a NAK candidate is that this is not
> conditional locking as such because no special action is taken if the lock
> cannot be acquired. In the referenced mail, I think the context for the IO
> NOWAIT stuff is "try lock and if that fails, delegate the work to an async
> context". That is not necessarily a universal win and it's potentially
> complex. It's not a universal win because it's unknown how long it would
> take to acquire the lock and it may be a short enough period to be cheaper
> than the setup_for_async+context_switch+completion handler. If that happens
> often enough in a short window then delegation may be slower overall than
> doing the work synchronously. It's potentially complex because the setup
> for async handling and completion needs code that must be maintained.
> 
> The kcompactd case using folio_lock_timeout is different. If the lock
> fails, it's not being explicitly delegated to another context, the page
> is simply ignored and kcompactd moves on. Fair enough, another context
> may end up migrating the same page in direct compaction or kcompactd
> at a later time but there is no complex setup for that and it's not
> explicit delegation. It's vaguely similar to how shrink_folio_list()
> calls folio_trylock and if that fails, keep the page on the LRU for a
> future attempt with the main difference being that some time is spent on
> trylock. This is *also* not necessarily a universal win because kcompactd
> could find a suitable migration candidate quicker by a plain trylock but
> that's what MIGRATE_ASYNC is for, MIGRATE_SYNC_LIGHT is expected to delay
> for short periods of time when MIGRATE_ASYNC fails and the problem being
> solved is the folio lock taking minutes to acquire.

I'm not generally a fan of lock-with-timeout approaches.  I think the
rationale for this one makes sense, but we're going to see some people
try to use this for situations where it doesn't make sense.  I almost
wonder if we shouldn't spin rather than sleep on this lock, since the
window of time we're willing to wait is so short.  I'm certainly not
willing to NAK this patch since it's clearly fixing a real problem.

Hm.  If the problem is that we want to wait for the lock unless the
lock is being held for I/O, we can actually tell that in the caller.

	if (folio_test_uptodate(folio))
		folio_lock(folio);
	else
		folio_trylock(folio);

(the folio lock isn't held for writeback, just taken and released;
if the folio is uptodate, the folio lock should only be taken for a
short time; if it's !uptodate then it's probably being read)