Need help tracking down a bug in the bio-FOLL_PIN patches

[David Howells]

Hi Jens, Christoph,

I need some help tracking down a bug in the patches that make the bio using
page pinning or no pinning using iov_iter_extract_pages().  The bug causes
seemingly random memory corruption once the "block: Convert
bio_iov_iter_get_pages to use iov_iter_extract_pages" patch is applied.

The bug was detected by a syzbot special:

	https://lore.kernel.org/r/000000000000b0b3c005f3a09383@xxxxxxxxxx/

The basic test body creates/opens a file, truncates it, opens it again
O_DIRECT and then uses sendfile to copy from the file to itself, causing the
file to extend as it goes.  I've added a reduced testcase below.  Note that
the problem only seems to occur if several instances of the test are run in
parallel.  After a few iterations, random memory corruption start showing up
and I see things like:

syz-direct-send[6095]: segfault at 0 ip 0000000000000000 sp 00007ffc81488b28 error 14 in syz-direct-sendfile[400000+1000] likely on CPU 0 (core 0, socket 0)
Code: Unable to access opcode bytes at 0xffffffffffffffd6.
BUG: Bad rss-counter state mm:00000000d5d20a39 type:MM_FILEPAGES val:2
BUG: Bad rss-counter state mm:00000000d5d20a39 type:MM_ANONPAGES val:2
BUG: non-zero pgtables_bytes on freeing mm: 8192

The bug goes away if the file is not truncated, O_DIRECT is not used or two
different files are used.

I've investigated the splice and iov_iter code and looked at what sendfile()
is doing in this case:

 (1) sendfile creates buffer pages and adds them into a pipe, does an, in this
     case, DIO read into those pages, then calls the fs write_iter to write
     the data to the file.

 (2) iov_iter_extract_pages() does not get refs/pins on the pages extracted
     from an ITER_PIPE iterator - but it shouldn't need to as the pipe holds
     the refs.  These pages are passed to DIO read - this op is synchronous,
     so any bios associated with it should be complete.

 (3) I enabled the page_ref tracepoints and added a page flag to limit it to
     pages allocated by append_pipe().  This shows the buffer pipe pages being
     added and I made it dump the list of them in __bio_release_pages() (which
     I made non-optional in bio_release_pages()).

 (4) I added some extra page_ref_set tracepoints with weird "val" parameters
     to add markers into the log.

 (5) I added a tracepoint to trace the lifetime of a bio struct and a flag to
     turn on the tracing, set when the pageflag added in (3) is seen.  Most of
     the time I can see the bio being destroyed in the correct order with
     regard to the splice code, though occasionally there's a bit missing.

 (6) Substituting a fixed preallocated page for the page coming out of the
     pipe in iter_file_splice_write() doesn't get rid of the bug:

	-	array[n].bv_page = buf->page;
	+	array[n].bv_page = splice_tmp;

 (7) Getting an extra ref on the buffer pipe page and deliberately leaking it
     gets rid of the problem.

 (8) Substituting a fixed preallocated page for the page sent to the DIO read
     in iov_iter_extract_pipe_pages() gets rid of the problem.  The pages
     going through the pipe seem to passed to write_iter with no issues.

 (9) I've tried instrumenting kmap() and co. to catch debug-marked pages being
     accessed after they've been released, but didn't see anything.  This
     might not catch if DMA is doing the corrupting.

(10) On the notion that DMA might do the corrupting, I've tried adding a
     permanent ref on the pages, adding them to a list and scanning them
     occasionally - but that doesn't catch anything.

(11) KASAN doesn't spot anything interesting - which might also suggest
     DMA-based corruption.  But since we're dealing with the contents of
     pages, not the page structs themselves (I think), I'm not sure kasan
     would see spot anything.

I'm wondering if the apparent interaction with sendfile/splice is actually a
red herring and that the page turnover that that induces is having an effect.

One thing I don't see is how commenting out ftruncate() should cause the
problem to go away if it's something to do with the splice buffer pipe -
though I guess ftruncate() would release a bunch of pages.

Here's an excerpt from a trace of something I'd expect to see:

   page_ref_set: pfn=0x10e38c flags=debug_mark count=1 mapcount=0 mapping=0 mt=0 val=777
   page_ref_set: pfn=0x10e38c flags=debug_mark count=1 mapcount=0 mapping=0 mt=0 val=666
   bio: bio=00038d84 ADD-PG I=10e38c
   bio: bio=00038d84 END-IO I=0
   page_ref_set: pfn=0x10e38c flags=debug_mark count=1 mapcount=0 mapping=0 mt=0 val=623
   bio: bio=00038d84 UNINIT I=0
   bio_endio: bio=00038d84 iomap_dio_bio_end_io+0x0/0xec
   bio: bio=00038d84 REL-PG I=0
   page_ref_set: pfn=0x10e38c flags=debug_mark count=1 mapcount=0 mapping=0 mt=0 val=980
   bio: bio=00038d84 FREE   I=0
   bio: bio=00038d84 UNINIT I=0
   page_ref_set: pfn=0x10e38c flags=debug_mark count=1 mapcount=0 mapping=0 mt=0 val=888
   page_ref_mod_and_test: pfn=0x10e38c flags=debug_mark count=0 mapcount=0 mapping=0 mt=0 val=-1 ret=1

The weird val=N codes on page_ref_set lines are:
	777 - The page iov_iter_extract_pipe_pages() got from append_pipe()
	666 - __bio_add_page() adding a page
	623 - bio_endio() logging a page
	98n - __bio_release_pages() logging the nth page
	888 - iter_file_splice_write() adding page to array[]

But occasionally I'm seeing something like:

   page_ref_set: pfn=0x1102df flags=debug_mark count=1 mapcount=0 mapping=0000000000000000 mt=0 val=777
   page_ref_set: pfn=0x1102df flags=debug_mark count=1 mapcount=0 mapping=0000000000000000 mt=0 val=666
   bio: bio=0000e514 ADD-PG I=1102df
   page_ref_set: pfn=0x1102df flags=debug_mark count=1 mapcount=0 mapping=0000000000000000 mt=0 val=888
   page_ref_mod_and_test: pfn=0x1102df flags=debug_mark count=0 mapcount=0 mapping=0000000000000000 mt=0 val=-1 ret=1

though I'm not sure why.  Could it be an attempt to read beyond the EOF?  I
don't see the bio being torn down, but the page is passed to
iter_file_splice_write() and released, despite for all I know still with
outstanding I/O pending.  Another possibility is that the bio flag got
cleared.

David
---
#define _GNU_SOURCE 
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/sendfile.h>
#include <sys/wait.h>

#define file_size 0x800
#define send_size 0x1dd00
#define repeat_count 1000

static char buffer[send_size];

int main(int argc, char *argv[])
{
	int in, out, i, wt;

	if (argc != 2 || !argv[1][0]) {
		fprintf(stderr, "Usage: %s <file>\n", argv[0]);
		exit(2);
	}

	for (i = 0; i < repeat_count; i++) {
		switch (fork()) {
		case -1:
			perror("fork");
			exit(1);
		case 0:
			out = creat(argv[1], 0666);
			if (out < 0) {
				perror(argv[1]);
				exit(1);
			}

			if (ftruncate(out, file_size) < 0) {
				perror("ftruncate");
				exit(1);
			}

			if (lseek(out, 0x200, SEEK_SET) < 0) {
				perror("lseek");
				exit(1);
			}

			in = open(argv[1], O_RDONLY | O_DIRECT | O_NOFOLLOW);
			if (in < 0) {
				perror("open");
				exit(1);
			}

			if (sendfile(out, in, NULL, send_size) < 0) {
				perror("sendfile");
				exit(1);
			}
			exit(0);

		default:
			if (wait(&wt) < 0) {
				perror("wait");
				exit(1);
			}
			break;
		}
	}

	exit(0);
}