Re: [PATCH] sched: Tweak default dynamic preempt mode selection

From: Mike Galbraith
Date: Tue Nov 09 2021 - 06:00:55 EST


On Tue, 2021-11-09 at 09:52 +0000, Valentin Schneider wrote:
> On 09/11/21 06:30, Mike Galbraith wrote:
> > Not seeing your v2 land yet, I grabbed my mallet and had a go at goal
> > reconciliation over morning java.  Non-lovely result seems to work.
> >
>
> Yeah so I went down a debatable path, gave up on that and started something
> different, and gave up on that because it was late :-)
>
> Now interestingly my second attempt is pretty close to what you have
> below.

Well that's a shame, because while it seems to function, it also puts
PREEMPT_DYNAMIC in a pretty darn similar spot to the one PREEMPT_RT was
in. Drat.

> > sched, Kconfig: Fix preemption model selection
> >
> > Switch PREEMPT_DYNAMIC/PREEMPT_RT dependency around so PREEMPT_RT
> > can be selected during the initial preemption model selection.
> > Further, since PREEMPT_DYNAMIC requires PREEMPT, make it depend
> > upon it instead of selecting it, and add a menu to allow selection
> > of the boot time behavior, this to allow arches that do not support
> > PREEMPT_DYNAMIC to retain their various configs untouched.
> >
>
> Have some nits below, but otherwise where I stand right now I think it's
> the least ugly way of tackling this :)

We're supposed to be going for _least_ ugly? Oh ;) This is really a
job for someone who knows their way around Kconfig-land, but since I'm
not hearing "Yawn, here ya go", it gets whacked with a mallet until one
of us gives up.

I'm looking at two straight up choice sets, one for those who have and
want PREEMPT_DYNAMIC, and another for the rest of us. Unfortunately,
there's no "else", so I end up with this mess, which would likely be
better served by "source foo/bar".

---
kernel/Kconfig.preempt | 90 +++++++++++++++++++++++++++++--------------------
1 file changed, 54 insertions(+), 36 deletions(-)

--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -2,11 +2,41 @@

choice
prompt "Preemption Model"
- default PREEMPT_NONE_BEHAVIOUR
+ default PREEMPT_STATIC

-config PREEMPT_NONE_BEHAVIOUR
+config PREEMPT_STATIC
+ bool "Preemption behaviour defined at build"
+
+config PREEMPT_DYNAMIC
+ bool "Preemption behaviour defined on boot"
+ depends on HAVE_PREEMPT_DYNAMIC && !ARCH_NO_PREEMPT
+ select PREEMPT
+ select PREEMPTION
+ select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
+ help
+ This option allows to define the preemption model on the kernel
+ command line parameter and thus override the default preemption
+ model defined during compile time.
+
+ The feature is primarily interesting for Linux distributions which
+ provide a pre-built kernel binary to reduce the number of kernel
+ flavors they offer while still offering different usecases.
+
+ The runtime overhead is negligible with HAVE_STATIC_CALL_INLINE enabled
+ but if runtime patching is not available for the specific architecture
+ then the potential overhead should be considered.
+
+ Interesting if you want the same pre-built kernel should be used for
+ both Server and Desktop workloads.
+endchoice
+
+if PREEMPT_STATIC
+choice
+ prompt "Preemption Flavor"
+ default PREEMPT_NONE
+
+config PREEMPT_NONE
bool "No Forced Preemption (Server)"
- select PREEMPT_NONE if !PREEMPT_DYNAMIC
help
This is the traditional Linux preemption model, geared towards
throughput. It will still provide good latencies most of the
@@ -18,10 +48,9 @@ config PREEMPT_NONE_BEHAVIOUR
raw processing power of the kernel, irrespective of scheduling
latencies.

-config PREEMPT_VOLUNTARY_BEHAVIOUR
+config PREEMPT_VOLUNTARY
bool "Voluntary Kernel Preemption (Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT_VOLUNTARY if !PREEMPT_DYNAMIC
help
This option reduces the latency of the kernel by adding more
"explicit preemption points" to the kernel code. These new
@@ -37,10 +66,11 @@ config PREEMPT_VOLUNTARY_BEHAVIOUR

Select this if you are building a kernel for a desktop system.

-config PREEMPT_BEHAVIOUR
+config PREEMPT
bool "Preemptible Kernel (Low-Latency Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT
+ select PREEMPTION
+ select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
help
This option reduces the latency of the kernel by making
all kernel code (that is not executing in a critical section)
@@ -58,7 +88,7 @@ config PREEMPT_BEHAVIOUR

config PREEMPT_RT
bool "Fully Preemptible Kernel (Real-Time)"
- depends on EXPERT && ARCH_SUPPORTS_RT && !PREEMPT_DYNAMIC
+ depends on EXPERT && ARCH_SUPPORTS_RT
select PREEMPTION
help
This option turns the kernel into a real-time kernel by replacing
@@ -74,17 +104,26 @@ config PREEMPT_RT
require real-time guarantees.

endchoice
+endif # PREEMPT_STATIC

-config PREEMPT_NONE
+if PREEMPT_DYNAMIC
+config PREEMPT
bool

-config PREEMPT_VOLUNTARY
- bool
+choice
+ prompt "Boot Time Preemption Flavor"
+ default PREEMPT_NONE_BEHAVIOR

-config PREEMPT
- bool
- select PREEMPTION
- select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
+config PREEMPT_NONE_BEHAVIOR
+ bool "No Forced Preemption (Server)"
+
+config PREEMPT_VOLUNTARY_BEHAVIOR
+ bool "Voluntary Kernel Preemption (Desktop)"
+
+config PREEMPT_BEHAVIOR
+ bool "Preemptible Kernel (Low-Latency Desktop)"
+endchoice
+endif # PREEMPT_DYNAMIC

config PREEMPT_COUNT
bool
@@ -93,27 +132,6 @@ config PREEMPTION
bool
select PREEMPT_COUNT

-config PREEMPT_DYNAMIC
- bool "Preemption behaviour defined on boot"
- depends on HAVE_PREEMPT_DYNAMIC
- select PREEMPT
- default y
- help
- This option allows to define the preemption model on the kernel
- command line parameter and thus override the default preemption
- model defined during compile time.
-
- The feature is primarily interesting for Linux distributions which
- provide a pre-built kernel binary to reduce the number of kernel
- flavors they offer while still offering different usecases.
-
- The runtime overhead is negligible with HAVE_STATIC_CALL_INLINE enabled
- but if runtime patching is not available for the specific architecture
- then the potential overhead should be considered.
-
- Interesting if you want the same pre-built kernel should be used for
- both Server and Desktop workloads.
-
config SCHED_CORE
bool "Core Scheduling for SMT"
depends on SCHED_SMT