Streaming disk I/O: don't use raw, limit bufs per device/partition

Donald Becker (becker@tidalwave.net)
Sat, 28 Aug 1999 18:31:21 -0400 (EDT)

>On Mon, 23 Aug 1999, Peter Monta wrote:
>>
>> > I think without some buffer-cache usage limiting, or buffering disabling,
>> > Linux is actually UNSUITABLE for streaming applications which do run
>> > concurrently with other apps.
>>
>> I agree; the raw-io system will make this much more pleasant.
>
>Why do you think so?
>
>IO is IO, whether it is raw or not. And it will eat up resources from
>everybody else trying to do IO.
>
>I think this is a case of believing in magic cures and supernatural
>beings.

We solved a similar problem by limiting the buffering per device/partition to
a subset of the buffer cache (75%, tunable), rather than hacking in raw
devices.

This worked extremely well for our application, a parallel file system.
The system had following features:
Multiple disk partitions. (Note 1)
Bulk I/O tended to be on just one partition at a time.
Most other processes were active in partitions without bulk I/O.

Note 1: This approach has no benefit on single-partition installations.
However most bulk I/O users do have multiple partitions, and typically have
multiple disks.

Donald Becker

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/