Re: [git pull] cpus4096 fixes

From: Mike Travis
Date: Mon Jul 28 2008 - 15:04:49 EST


Linus Torvalds wrote:
>
> On Mon, 28 Jul 2008, Mike Travis wrote:
>> Sorry, I didn't know that was the protocol. And yes, the clever idea of
>> compacting the memory is a good one (wish I would have thought of it... ;-)
>> But, and it's a big but, if you really have 4096 cpus present (not NR_CPUS,
>> but nr_cpu_ids), then 2MB is pretty much chump change.
>
> Umm. Yes, it's chump change, but if you compile a kernel to be generic,
> and you actually only have a few CPU's, it's no longer chump change.

The 2Mb's of initdata is released, I just meant that if you really have 4k
cpus in the system, you'll probably have 4k * [2 .. 32 Gb (or more?)] of
memory. The Nahalem memory limit is (iirc) 44 bits.

Originally, I only had the constant for cpu(0) but since it _was_ originally
a constant (alibi, rvalue only), then it might be thought that it's valid to
use any cpu# before setup_per_cpu_areas is called.

>
>> But I'll redo the patch again.
>
> Here's a trivial setup, that is even tested. It's _small_ too.
>
> /* cpu_bit_bitmap[0] is empty - so we can back into it */
> #define MASK_DECLARE_1(x) [x+1][0] = 1ul << (x)
> #define MASK_DECLARE_2(x) MASK_DECLARE_1(x), MASK_DECLARE_1(x+1)
> #define MASK_DECLARE_4(x) MASK_DECLARE_2(x), MASK_DECLARE_2(x+2)
> #define MASK_DECLARE_8(x) MASK_DECLARE_4(x), MASK_DECLARE_4(x+4)
>
> static const unsigned long cpu_bit_bitmap[BITS_PER_LONG+1][BITS_TO_LONGS(NR_CPUS)] = {
> MASK_DECLARE_8(0), MASK_DECLARE_8(8),
> MASK_DECLARE_8(16), MASK_DECLARE_8(24),
> #if BITS_PER_LONG > 32
> MASK_DECLARE_8(32), MASK_DECLARE_8(40),
> MASK_DECLARE_8(48), MASK_DECLARE_8(56),
> #endif
> };
>
> static inline const cpumask_t *get_cpu_mask(unsigned int nr)
> {
> const unsigned long *p = cpu_bit_bitmap[1 + nr % BITS_PER_LONG];
> p -= nr / BITS_PER_LONG;
> return (const cpumask_t *)p;
> }
>
> that should be all you need to do.

Very cool, thanks!!

>
> Honesty in advertizing: my "testing" was some trivial user-space harness,
> maybe I had some bug in it. But at least it's not _horribly_ wrong.
>
> And yes, this has the added optimization from Viro of overlapping the
> cpumask_t's internally too, rather than making them twice the size. So
> with 4096 CPU's, this should result 32.5kB of static const data.
>
> Linus

Don't worry, I'll beat it to death... ;-) [and try not to screw up the
acknowledgments this time... ;-)]

Thanks again,
Mike
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/