Re: [PATCH] ext2 statfs improvement for block and inode free count

[Badari Pulavarty]

On Wed, 2007-07-18 at 20:18 -0700, Andrew Morton wrote:
> On Fri, 13 Jul 2007 18:36:54 -0700 Badari Pulavarty <pbadari@xxxxxxxxxx> wrote:
> 
> > More statfs() improvements for ext2. ext2 already maintains
> > percpu counters for free blocks and inodes. Derive free
> > block count and inode count by summing up percpu counters,
> > instead of counting up all the groups in the filesystem
> > each time.
> > 
> 
> hm, another speedup patch with no measurements which demonstrate its
> benefit.

In my setups (4 & 8-way), I didn't measure any significant performance
improvements (in any reasonable workload). I see some decent
improvements on cooked-up (1 million stats) tests :(
> 
> > 
> > Signed-off-by: Badari Pulavarty <pbadari@xxxxxxxxxx>
> > Acked-by: Andreas Dilger <adilger@xxxxxxxxxxxxx>
> > 
> >  fs/ext2/super.c |    4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > Index: linux-2.6.22/fs/ext2/super.c
> > ===================================================================
> > --- linux-2.6.22.orig/fs/ext2/super.c	2007-07-13 20:06:38.000000000 -0700
> > +++ linux-2.6.22/fs/ext2/super.c	2007-07-13 20:06:51.000000000 -0700
> > @@ -1136,12 +1136,12 @@ static int ext2_statfs (struct dentry * 
> >  	buf->f_type = EXT2_SUPER_MAGIC;
> >  	buf->f_bsize = sb->s_blocksize;
> >  	buf->f_blocks = le32_to_cpu(es->s_blocks_count) - overhead;
> > -	buf->f_bfree = ext2_count_free_blocks(sb);
> > + 	buf->f_bfree = percpu_counter_sum(&sbi->s_freeblocks_counter);
> >  	buf->f_bavail = buf->f_bfree - le32_to_cpu(es->s_r_blocks_count);
> >  	if (buf->f_bfree < le32_to_cpu(es->s_r_blocks_count))
> >  		buf->f_bavail = 0;
> >  	buf->f_files = le32_to_cpu(es->s_inodes_count);
> > -	buf->f_ffree = ext2_count_free_inodes(sb);
> > + 	buf->f_ffree = percpu_counter_sum(&sbi->s_freeinodes_counter);
> >  	buf->f_namelen = EXT2_NAME_LEN;
> >  	fsid = le64_to_cpup((void *)es->s_uuid) ^
> >  	       le64_to_cpup((void *)es->s_uuid + sizeof(u64));
> > 
> 
> Well there's a tradeoff here.  At large CPU counts, percpu_counter_sum()
> becomes quite expensive - it takes a global lock and then goes off fishing
> in every CPU's percpu_alloced memory.
> 
> So there is some value of (num_online_cpus / sb->s_groups_count) at which
> this change becomes a loss.  Where does that value lie?

Yes. I debated long time whether I should submit this or not - due to
very reason. Old code wasn't holding any locks. I don't have any high
count CPU machine (>8way) with me. I will request for time on one.

> 
> Bear in mind that the global lock in percpu_counter_sum() will tilt the
> scales quite a bit.

Noticed that too. I added WARN_ON() to see if percpu sum doesn't match
computed sum. I saw few stacks in a 24 hour run of fsx runs. 

Thanks,
Badari

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/