> What we need is to detect the situation where someone is linearly
> walking through a file which is preposterously too large to cache,
> and just start dropping it.
...
> The tricky part is designing the algorithm which decides when to
> pull the trigger.
I did a variation of this in SunOS years ago. It did not have the
"big file" wrinkle you are suggesting, it worked like
if (we are sequential AND
we are running low on memory AND
file size > 256K) {
invalidate the pages behind me
}
You use the same data structures which turn on read ahead to mean
sequential access, that's obvious.
What this didn't fix was when you read a monstor file into memory and
then didn't do anything with it. That would fill the page cache and
the above alg would keep you from thrashing the machine but didn't
flush the stale memory.
If I were to do it again, I'd maintain stats in the inode about
access pattern, # of pages in ram for the inode, time of last I/O,
time of last page fault or read. Then when memory is getting tight
you do a
foreach i (ALL INODES) {
unless (i.accesspat == SEQ) continue;
unless ((now() - i.pagefault) > STALE_TIME) continue;
unless ((now() - i.io) > STALE_TIME) continue;
flush_pages();
}
You want to be a lot more clever than that because you'd like to have
fudging in favor of the clean pages vs dirty pages, you more or less
end up wanting to go through the loop more than once, getting more and
more eager as you are more and more desparate for ram.
-- --- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Tue Oct 15 2002 - 22:00:27 EST