On Tue, May 23, 2006 at 06:41:36PM -0700, fitzboy wrote:
I read online in multiple places that the largest allocation groups should get is 4g,
Thats not correct (for a few years now).
I was also thinking that the more AGs the better since I do a lot of parallel reads/writes... granted it doesn't change the file system all that much (the file only grows or get existing blocks get modified), so I am not sure if the number of AGs matter, does it?
Yes, it can matter. For large extents like you have here, AGs
introduce a discontinuity that you'd otherwise not have.
Sorry, I meant that moving the Inode size to 2k (over 256bytes) gave me a sizeable increase in performance... I assume that is because the extent map can be smaller now (since blocks are much larger, less blocks to keep track of). Of course, ideal would be to have InodeSize be large and blocksize to be 32k... but I hit the limits on both...
It means that more extents/btree records fit inline in the inode,
as theres more space available after the stat data. 2k is your
best choice for inode size, stick with that.
- Preallocate the space in the file - i.e. before running the
dd you can do an "xfs_io -c 'resvsp 0 2t' /mnt/array/disk1/xxx"
(preallocates 2 terabytes) and then overwrite that. Yhis will
give you an optimal layout.
I tried this a couple of times, but it seemed to wedge the machine... I would do: 1) touch a file (just to create it), 2) do the above command
Oh, use the -f (create) option and you won't need a touch.
which would then show effect in du, but the file size was still 0 3) I then opened that file (without O_TRUNC or O_APPEND) and started to write out to it. It would work fine for a few minutes but after about 5 or 7GB the machine would freeze... nothing in syslog, only a brief message on console about come cpu state being bad...
Hmm - I'd be interested to hear if that happens with a recent
kernel.
- Your extent map is fairly large, the 2.6.17 kernel will have
some improvements in the way the memory management is done here
which may help you a bit too.
we have plenty of memory on the machines, shouldn't be an issue... I am a little cautious about moving to a new kernel though...
Its not the amount of memory that was the issue here, its more the
way we were using it that was a problem for kernels of the vintage
you're using here. You will definately see better performance in
a 2.6.17 kernel with that large extent map.
cheers.