Re: [PATCH v5 1/2] kernfs: use hashed mutex and spinlock in place of global ones.

[Imran Khan]

Hello Tejun,

Thanks for your feedback.

On 9/2/22 4:22 am, Tejun Heo wrote:
> On Sun, Feb 06, 2022 at 12:09:24PM +1100, Imran Khan wrote:
>> +/*
>> + * NR_KERNFS_LOCK_BITS determines size (NR_KERNFS_LOCKS) of hash
>> + * table of locks.
>> + * Having a small hash table would impact scalability, since
>> + * more and more kernfs_node objects will end up using same lock
>> + * and having a very large hash table would waste memory.
>> + *
>> + * At the moment size of hash table of locks is being set based on
>> + * the number of CPUs as follows:
>> + *
>> + * NR_CPU      NR_KERNFS_LOCK_BITS      NR_KERNFS_LOCKS
>> + *   1                  1                       2
>> + *  2-3                 2                       4
>> + *  4-7                 4                       16
>> + *  8-15                6                       64
>> + *  16-31               8                       256
>> + *  32 and more         10                      1024
>> + */
>> +#ifdef CONFIG_SMP
>> +#define NR_KERNFS_LOCK_BITS (2 * (ilog2(NR_CPUS < 32 ? NR_CPUS : 32)))
>> +#else
>> +#define NR_KERNFS_LOCK_BITS     1
>> +#endif
>> +
>> +#define NR_KERNFS_LOCKS     (1 << NR_KERNFS_LOCK_BITS)
> 
> I have a couple questions:
> 
> * How did you come up with the above numbers? Are they based on some
>   experimentation? It'd be nice if the comment explains why the numbers are
>   like that.
> 

I did some testing using different number of CPUs with qemu and the
above numbers were collected from there. It may not be optimum but this
is what I could think of while doing those internal tests.

Do you think it may be better to make this configurable via Kconfig.

> * IIRC, we split these locks to per kernfs instance recently as a way to
>   mitigate lock contention occurring across kernfs instances. I don't think
>   it's beneficial to keep these hashed locks separate. It'd be both simpler
>   and less contended to double one shared hashtable than splitting the table
>   into two separate half sized ones. So, maybe switch to global hashtables
>   and increase the size?

I have removed the hash table from kernfs_root. Doubling the size was
not giving any improvements so I have kept the hash table size same as
before. This change is present in v6 of patch at [1]

Thanks
 -- Imran

[1]:
https://lore.kernel.org/lkml/20220214120322.2402628-1-imran.f.khan@xxxxxxxxxx/


> 
> Thanks.
>