In linux.kernel, you wrote:
> Helge Hafting (helgehaf@aitel.hist.no) wrote:
>> Erik Hensema wrote:
>>> In all kernels I've tested writes to disk are delayed a long time even when
>>> there's no need to do so.
>>>
>> Short answer - it is supposed to do that!
>>
>>> A very simple test shows this: on an otherwise idle system, create a tar of
>>> a NFS-mounted filesystem to a local disk. The kernel starts writing out the
>>> data after 30 seconds, while a slow and steady stream would be much nicer
>>> to the system, I think.
Agreed. We have a cluster which is writing on average something like
20 Megs/sec/node. We had to lower the write threshold from 30% to 0%,
because with the constant writing, linux will buffer it for 30 secs,
fill up RAM, try to empty the write-cache, stall, wash, rinse,
repeat. Because it was being filled up at roughly the rate it was
being emptied, once it got 30% behind, there was no catching up, so
the realtime system would lose data. Ouch.
>> You're wrong then. There's no need for a slow steady stream, why do
>> you want that. Of course you can set up cron to run sync at
>> regular (short) intervals to achieve this.
Last time I checked, cron had 1 minute resolution.
> I see that. However, I don't see why the kernel is writing out data
> as agressively as it does now. Delaying a write for 30 seconds isn't the
> problem: the aggressive writes are. Since the disks are otherwise idle, the
> kernel can gently start writing out the dirty cache. No need to try and
> write 40 MB in 1 sec when you can write 10 MB/sec in 4 seconds.
>
> [...]
>
>> For more detailed information, read a book about how filesystems and
>> disk caching works.
>
> I'm just reporting what's happening to me in practice, I don't really care
> about what should happen in theory.
Exactly.
Helge's comment about /tmp files and rewriting files multiple times:
in real life, how often does this happen? How often do you overwrite
one file many times in 30 seconds? The occasional 20 kilobyte /tmp
file perhaps, but I doubt it matters in real life. In real life, when
writing to disk constantly (not just scientific applications - I
believe this happens in the real world too!), waiting for 30 seconds
is a liability!
-- TimC -- http://astronomy.swin.edu.au/staff/tconnors/White dwarf seeks red giant star - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Mon Mar 31 2003 - 22:00:31 EST