Re: [PATCH] O_STREAMING - flag for optimal streaming I/O

From: kernel@street-vision.com
Date: Wed Oct 09 2002 - 11:30:33 EST


>
> Robert Love wrote:
> >
> > ...
> >
> > Andrew, any experience on one vs. the other?
>
> I'd say that if you were designing a new application which
> streams large amount of data then yes, you would design it
> to use O_DIRECT. You would instantiate a separate IO worker
> thread and a message passing mechanism so that thread would
> pump your data for you, and would peform your readahead, etc.
>
> If your filesystem supports O_DIRECT, of course. Not all do.
>
> The strength of O_STREAMING is that you can take an existing,
> working, megahuge application and make it play better with the
> VM by changing a single line of code. No big redesign needed.

I think it will be a really big win on quite low speed machines streaming
at eg set top box speeds, without much memory (eg your Tivo).
O_DIRECT needs at least 4M reads to overcome lack of readahead,
so is only really useful if you are doing really big stuff (eg
uncompressed video). It is possible that O_STREAMING will be just as
good as this however, becasue there are latency issues with O_DIRECT
(you really need aio or multiple threads to sustain good readahead,
one test of mine needed 8 threads for optimal performance). I have
a bunch of things I could retest with O_STREAMING and large readahead.

O_DIRECT is always going to win for random access stuff though. It is
good to have a choice.

Justin
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Oct 15 2002 - 22:00:31 EST