how many hash bits did you try? 13? you might consider trying even more,
say 15 or 16. benchmarking has shown that the page hash function is
stable for any bit size between 11 and 16 (i didn't try others), so
varying it, as Doug's patch does, won't degenerate the hash.
> Adding 4 extra bits to the dcache hash was worth 2 full minutes; 12
> minutes total build time.
>
> This was absolutely obvious from the readprofile(8) traces, where
> d_lookup() was massively dominating the system CPU time (I had made both
> the page and buffer hash lookup functions into externs at this point so
> that they would all be profiled separately, not as inlines). d_lookup
> was using about 4 times as much CPU as the nearest other functions
> (which iirc were the page fault COW handler and file read() actors, both
> of which perform large copies).
readprofile traces i've taken show mostly the same thing, although the
page fault handler is by an order of magnitude the highest CPU user
(find_vma() is still up there, too). increasing the efficiency of the
dcache hash will buy probably the biggest performance improvement of any
hash modifications. i've also found that changing the x-or's in the
dcache hash function to addition operations improves the bucket size
distribution and buys a bit more improvement.
> Shrinking the dcaches excessively in this case will simply masaccre the
> performance.
actually, that's not strictly true. shrinking the dcache early will
improve the lookup efficiency of the hash, i've found almost by two times.
there's probably compromise here, so, if you're not _too_ aggressive about
shrinking it (but more aggressive than shrink_dcache**() is now), you may
find even more performance improvement.
my preliminary analysis of the inode hash table is that it is also too
small. find_inode() seems to be used mostly for unsuccessful searches
these days, and the hash table size is so small that the buckets contain
long long lists of inodes, making the unsuccessful searches very
expensive.
- Chuck Lever
-- corporate: <chuckl@netscape.com> personal: <chucklever@netscape.net> or <cel@monkey.org>The Linux Scalability project: http://www.citi.umich.edu/projects/citi-netscape/
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/