Re: [PATCH] O_STREAMING - flag for optimal streaming I/O

From: Larry McVoy (
Date: Tue Oct 08 2002 - 15:37:38 EST

> What we need is to detect the situation where someone is linearly
> walking through a file which is preposterously too large to cache,
> and just start dropping it.


> The tricky part is designing the algorithm which decides when to
> pull the trigger.

I did a variation of this in SunOS years ago. It did not have the
"big file" wrinkle you are suggesting, it worked like

        if (we are sequential AND
            we are running low on memory AND
            file size > 256K) {
                    invalidate the pages behind me

You use the same data structures which turn on read ahead to mean
sequential access, that's obvious.

What this didn't fix was when you read a monstor file into memory and
then didn't do anything with it. That would fill the page cache and
the above alg would keep you from thrashing the machine but didn't
flush the stale memory.

If I were to do it again, I'd maintain stats in the inode about
access pattern, # of pages in ram for the inode, time of last I/O,
time of last page fault or read. Then when memory is getting tight
you do a

        foreach i (ALL INODES) {
                unless (i.accesspat == SEQ) continue;
                unless ((now() - i.pagefault) > STALE_TIME) continue;
                unless ((now() - > STALE_TIME) continue;

You want to be a lot more clever than that because you'd like to have
fudging in favor of the clean pages vs dirty pages, you more or less
end up wanting to go through the loop more than once, getting more and
more eager as you are more and more desparate for ram.

Larry McVoy            	 lm at  
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

This archive was generated by hypermail 2b29 : Tue Oct 15 2002 - 22:00:27 EST