On Wed Sep 19, 2001 at 01:05:06AM -0600, Eric W. Biederman wrote:
> Simon Kirby <sim@netnation.com> writes:
>
> > I've always wondered why it's not possible to do this:
> >
> > fd = open("an_actual_file",O_RDONLY);
> > fcntl(fd,F_SETFL,O_NONBLOCK);
> > r = read(fd,buf,4096);
> >
> > And actually have read return -1 and errno == EWOULDBLOCK/EAGAIN if the
> > block requested is not already cached.
> >
> > Wouldn't this be the ideal interface for daemons of all types that want
> > to stay single-threaded and still offer useful performance when the
> > working set doesn't fit in cache? It works with sockets, so why not
> > with files?
>
> Besides the SUS or the POSIX specs...
> What would cause the data to be read in if read just checks the caches?
> With sockets the other side is clearing pushing or pulling the data. With
> files there is no other side...
>
> This is resolveable just not currently implemented.
I was actually playing around with this a few days back, since this would have
the advantage of 1) maximizing utilization of the bottleneck device and 2)
leave the app free to update a user interface etc while the kernel was busy
reading/writing.
There are two problems with a dd type app. First the dd algorithm is
fundamentally serial -- read from A, stop, write to B, stop, rinse, repeat.
Second, it leaves the thread stuck in D state forever. For example, try
something like
dd if=/dev/sda of=/dev/sdb bs=1M
(assuming these devices are not holding important stuff) and let it run for a
bit and then hit ^C. Then wait for a minute or two while the kernel finishes
flushing everything before dd finally exits... So to get sane stop behavior,
you have to minimize the buffer size (and add many minutes to the copy time),
while to maximize performance, you have to maximize the buffer size (goodbye
user friendly).
So are there no single threaded semantics by which one can copy from device
A to device B without blocking (thereby serializing the copy and leaving
the app unresponsive)? I'd really hate to have to use threads to do this,
-Erik
-- Erik B. Andersen email: andersee@debian.org, formerly of Lineo --This message was written using 73% post-consumer electrons--
This archive was generated by hypermail 2b29 : Sun Sep 23 2001 - 21:00:29 EST