2.4.13-pre5-aa1 O_DIRECT drastic HIGHMEM performance hit

From: Marvin Justice (Mjustice@boxxtech.com)
Date: Thu Oct 25 2001 - 11:57:08 EST


We're looking into setting up an HD video editing solution with the
following hardware:

dual 1GHz P3, 2GB RAM
Serverworks HeSL based board
dual U160 Adaptec SCSI card
5 68gb Fujitsu drives on each SCSI channel

The 10 drives are software raid0 striped with a 68k chunk size. For an
initial benchmark I got a copy of lmdd and modified it to open files
using O_DIRECT (also hacked it to work on W2K with FILE_FLAG_NO_BUFFER).
Here are the average results in MB/sec for 2.4.13-pre5-aa1 streaming
chunks of data 4MB at a time to RAM where they are immediately
discarded:

                     w r
no O_DIRECT 136 132
O_DIRECT 111 96

I experimented around a bit and discovered that increasing
KIO_MAX_ATOMIC_IO from 512 to 4096 gave a significant performance boost
for the O_DIRECT read case:
                     w r
no O_DIRECT 117 124
O_DIRECT 111 165

Next I recompiled without HIGHMEM support and was quite suprised
(numbers are with KIO_MAX_ATOMIC_IO=4096)

                     w r
no O_DIRECT 138 125
O_DIRECT 221 182

XFS shows similar behavior: (XFS barfs if you change KIO_MAX_ATOMIC_IO
from 512)

                           w r
O_DIRECT w/ HIGHMEM 114 146
O_DIRECT w/o HIGHMEM 218 248

(Incidentally, W2k reads and writes at a smooth 255 MB/sec on identical
hardware.)

Are we stuck with a low mem configuration or are there workarounds that
would allow us to stick with the initial 2GB of RAM and still get ~200
MB/sec.
 

--------------------------
Marvin Justice
Software Developer
BOXX Technologies, Inc.
www.boxxtech.com
mjustice@boxxtech.com
(V) (512)225-6318
(F) (512)835-0434
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Wed Oct 31 2001 - 21:00:28 EST