Re: [PATCH] O_STREAMING - flag for optimal streaming I/O

From: Robert Love (rml@tech9.net)
Date: Tue Oct 08 2002 - 14:17:16 EST


On Tue, 2002-10-08 at 15:05, Chris Wedgwood wrote:

> > In other words, this flag pretty much disables the pagecache for
> > this mapping, although we happily keep it around for write-behind
> > and read-ahead. But once the data is behind us and safe to kill, we
> > do. It is manual drop-behind.
>
> OK. What might use this though? What applications might want to
> disable the page-cache but still use write-behind?

Streaming I/O wants read-ahead. Filesystems themselves implement the
write-behind and we do not want to circumvent so much of the kernel.

The point of O_STREAMING is one change: drop pages in the pagecache
behind our current position, that are free-able, because we know we will
never want them. Its a hint from the application saying "I will never
revisit this so dump it".

O_DIRECT is a much bigger can of worms. You lose a lot of what the
kernel provides. You have to do things in block-sized chunks. Etc.
etc.

> > O_DIRECT has a lot of semantics, one of which is to attempt to
> > minimize cache effects.
>
> It depends on the OS. Some OS are broken and treat O_DIRECT as a
> hint, Linux and IRIX know it's a *requirement*.

Yep. Linux treats most "hints" (e.g. madvise) as a requirement - it
fails if it cannot do it. That is against the spec most of the time,
but oh well...

> > O_STREAMING would be for your TiVo or network audio streamer. Any
> > file I/O that is inherently sequential and access-once. No point
> > trashing the pagecache with its data - but otherwise the behavior is
> > normal.
>
> Actually, this sounds perfect for O_DIRECT. But I don't know much
> about streaming video.
>
> Since you only want the data once, why use the page-cache at all and
> needlessly copy? Certainly, the requirements for O_DIRECT are not
> that hard to meet or implement.
>
> Don't get me wrong, I'm not saying this is a bad thing at all. The
> patch is small and elegant so it's hard to object; I'm just trying to
> understand where in practice I would use this over O_DIRECT.

Shrug. I do not have much experience with O_DIRECT. I suspect the
synchronous nature and the requirement of aligned buffers is not ideal.

With O_STREAMING you can simply set the flag and use your normal I/O and
normal interfaces and have a field day.

Andrew, any experience on one vs. the other?

        Robert Love

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Oct 15 2002 - 22:00:27 EST