Re: Modifying isolcpus, nohz_full, and rcu_nocb kernel parameters at runtime

From: Waiman Long
Date: Tue Dec 12 2023 - 15:13:30 EST


On 12/12/23 09:04, Pandruvada, Srinivas wrote:
Hi Fredric,
On Tue, 2023-12-12 at 14:27 +0100, Frederic Weisbecker wrote:
On Fri, Dec 08, 2023 at 09:18:53AM -0500, Gianfranco Dutka wrote:
The isolcpus, nohz_full and rcu_nocbs are boot-time kernel
parameters. I am in the process of improving dynamic CPU
isolation at runtime. Right now, we are able to do
isolcpus=domain with the isolated cpuset partition functionality.
Other aspects of CPU isolation are being looked at with the goal
of reducing the gap of what one can do at boot time versus what
can be done at run time. It will certain take time to reach that
goal.

Cheers,
Longman

Thank you Waiman for the response. It would seem that getting
similar
functionality through cgroups/cpusets is the only option at the
moment. Is it
completely out of the question to possibly patch the kernel to
modify these
parameters at runtime? Or would that entail a significant change
that might
not be so trivial to accomplish? For instance, the solution
wouldn’t be as
simple as patching the kernel to make these writeable and then
calling the
same functions which run at boot-time when these parameters are
originally
written?
As for nohz_full (which implies rcu_nocb), it's certainly possible to
make it
tunable at runtime via cpusets. If people really want it, I'm willing
to help.

We have a case for dynamically isolating CPUs at run time.

https://lore.kernel.org/lkml/ZNM5qoUSCdBwNTuH@chenyu5-mobl2/

It was suggested by Vincent to use house keeping cpumask for solving
unnecessary wake ups on isolated CPUs. Can this house keeping cpu mask
and type be updated at runtime?

The house keeping cpumasks are statically set up at boot time. The cpuset code will have an internal isolated_cpus cpumask that lists all the CPUs that are in isolated partitions. It will also expose it with a new cpuset_cpu_is_isolated() function. The code is currently in the cgroup tree and will be merged into the upcoming v6.8 kernel. As the new isolated_cpus mask can be changed at run time, some synchronization may be needed depending on how it is being used. So the cpumask itself will not be exported, new API can be provided to copy out the cpumask if needed.

Cheers,
Longman