Re: [PATCH v1] KVM: isolation: retain initial mask for kthread VM worker

[Marcelo Tosatti]

On Tue, Oct 05, 2021 at 01:25:52PM +0200, Paolo Bonzini wrote:
> On 05/10/21 12:58, Marcelo Tosatti wrote:
> > > There are other effects of cgroups (e.g. memory accounting) than just the
> > > cpumask;
> > 
> > Is kvm-nx-hpage using significant amounts of memory?
> 
> No, that was just an example (and not a good one indeed, because
> kvm-nx-hpage is not using a substantial amount of either memory or CPU).
> But for example vhost also uses cgroup_attach_task_all, so it should have
> the same issue with SCHED_FIFO?

Yes. Would need to fix vhost as well.

> 
> > > Why doesn't the scheduler move the task to a CPU that is not being hogged by
> > > vCPU SCHED_FIFO tasks?
> > Because cpuset placement is enforced:
> 
> Yes, but I would expect the parent cgroup to include both isolated CPUs (for
> the vCPU threads) and non-isolated housekeeping vCPUs (for the QEMU I/O
> thread).  

Yes, the parent, but why would that matter? If you are in a child
cpuset, you are restricted to the child cpuset mask (and not the
parents).

> The QEMU I/O thread is not hogging the CPU 100% of the time, and
> therefore the nx-recovery thread should be able to run on that CPU.

Yes, but:

1) The cpumask of the parent thread is not inherited 

	set_cpus_allowed_ptr(task, housekeeping_cpumask(HK_FLAG_KTHREAD));

On __kthread_create_on_node should fail (because its cgroup, the one
inherited from QEMU, contains only isolated CPUs).

(The QEMU I/O thread runs on an isolated CPU, and is moved by libvirt
to HK-cgroup as mentioned before).

2) What if kernel threads that should be pinned to non-isolated CPUs are created
from vcpus? 



> 
> Thanks,
> 
> Paolo
> 
> > CPUSET(7)                            Linux Programmer's Manual                           CPUSET(7)
> > 
> >         Cpusets are integrated with the sched_setaffinity(2) scheduling affinity mechanism and  the
> >         mbind(2)  and set_mempolicy(2) memory-placement mechanisms in the kernel.  Neither of these
> >         mechanisms let a process make use of a CPU or memory node  that  is  not  allowed  by  that
> >         process's  cpuset.   If  changes  to a process's cpuset placement conflict with these other
> >         mechanisms, then cpuset placement is enforced even if it means overriding these other mech‐
> >         anisms.   The kernel accomplishes this overriding by silently restricting the CPUs and mem‐
> >         ory nodes requested by these other mechanisms to those allowed by  the  invoking  process's
> >         cpuset.   This  can  result in these other calls returning an error, if for example, such a
> >         call ends up requesting an empty set of  CPUs  or  memory  nodes,  after  that  request  is
> >         restricted to the invoking process's cpuset.
> > 
> > 
> 
>