Hi,
here comes the benchmark I used for the NUMA scheduler test. Would
be interesting to know whether it is useful to any other NUMA
developer...
Regards,
Erich
PS: it uses a 'time' command that understands the --format option,
e.g. GNU time 1.7. Change it in the main script, if it doesn't
work for you.
On Sunday 06 October 2002 18:51, Erich Focht wrote:
> Hi,
>
> here are some results from testing various versions and approaches
> to a NUMA scheduler. I used the numa_test benchmark which I'll post
> in a separate email. It runs in parallel N tasks doing the same job:
> access randomly a large array. As the array is large enough not to
> fit into cache, this is very memory latency sensitive. Also it is
> memory bandwidth sensitive. To emulate a real multi-user environment, the
> jobs are disturbed by a short load peak. This is simulated by a call
> to "hackbench" 3 seconds after the tasks were started. The performance
> of the user tasks is depending very much on where they are scheduled
> and are CPU hoggers such that the user times are quite independent of
> the non-scheduler part of the underlying kernel. The elapsed times
> are depending on "hackbench" which actually blocks the machine for the
> time it is running. Hackbench is depending on the underlying kernel
> and one should compare "elapsed_time - hackbench_time".
>
> The test machine is a 16 CPU NEC Azusa with Itanium processors and
> four nodes. The tested schedulers are:
>
> A: O(1) scheduler in 2.5.39
> B: O(1) scheduler with task steal limited to only one task (node
> affine scheduler with CONFIG_NUMA_SCHED=n) under 2.4.18
> C: Michael Hohnbaum's "simple NUMA scheduler" under 2.5.39
> D: pooling NUMA scheduler, no initial load balancing, idle pool_delay
> set to 0, under 2.4.18
> E: node affine scheduler with initial load balancing and static homenode
> F: node affine scheduler without initial load balancing and dynamic
> homenode selection (homenode selected where most of the memory is
> allocated).
>
> As I'm rewriting the node affine scheduler to be more modular, I'll
> redo the tests for cases D, E, F on top of 2.5.X kernels soon.
>
> The results are summarized in the tables below. A set of outputs (for
> N=8, 16, 32) is attached. They show clearly why the node affine
> scheduler beats them all: The initial load balancing is node-aware,
> the task-stealing too. Sometimes the node affine force is not large
> enough to bring the task back to the homenode, but it is almost always
> good enough.
>
> The (F) solution with dynamically determined homenode show that the
> initial load balancing is crucial, as the equal node balance is not
> strongly enforced dynamically. So the optimal solution is probably
> (F) with initial load balancing.
>
>
> Average user time (U) and total user time (TU). Minimum per row should
> be considered.
> ----------------------------------------------------------------------
> Scheduler: A B C D E F
> N=4 U 28.12 30.77 33.00 - 27.20 30.29
> TU 112.55 123.13 132.08 - 108.88 121.25
> N=8 U 30.47 31.39 31.65 30.76 28.67 30.08
> TU 243.86 251.27 253.30 246.23 229.51 240.75
> N=16 U 36.42 33.64 32.18 32.27 31.50 32.83
> TU 582.91 538.49 515.11 516.53 504.17 525.59
> N=32 U 38.69 34.83 34.05 33.76 33.89 34.11
> TU 1238.4 1114.9 1090.1 1080.8 1084.9 1091.9
> N=64 U 39.73 34.73 34.23 - (33.32) 34.98
> TU 2543.4 2223.4 2191.7 - (2133) 2239.5
>
>
> Elapsed time (E), hackbench time (H). Diferences between 2.4.18 and
> 2.5.39 based kernels due to "hackbench", which performs differently.
> Compare E-H within a row, but don't take it too seriously.
> --------------------------------------------------------------------
> Scheduler: A B C D E F
> N=4 E 37.33 37.96 48.31 - 28.14 35.91
> H 9.98 1.49 10.65 - 1.99 1.43
> N=8 E 46.17 39.50 42.53 39.72 30.28 38.28
> H 9.64 1.86 7.27 2.07 2.33 1.86
> N=16 E 47.21 44.67 49.66 42.97 36.98 42.51
> H 5.90 4.69 2.93 5.178 5.56 5.94
> N=32 E 88.60 79.92 80.34 78.35 76.84 77.38
> H 6.29 5.23 2.85 4.51 5.29 4.28
> N=64 E 167.10 147.16 150.59 - (133.9) 148.94
> H 5.96 4.67 3.10 - (-) 6.86
>
> (The E:N=64 results are without hackbench disturbance.)
>
> Best regards,
> Erich
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Mon Oct 07 2002 - 22:00:55 EST