Re: [RFC 19/19] s390/facilities: enable AP facilities needed by guest

From: Tony Krowiak
Date: Tue Dec 05 2017 - 10:01:55 EST


On 12/05/2017 09:04 AM, Cornelia Huck wrote:
On Tue, 5 Dec 2017 08:52:57 +0100
Harald Freudenberger <freude@xxxxxxxxxxxxxxxxxx> wrote:

On 12/02/2017 02:30 AM, Tony Krowiak wrote:
I agree with your suggestion that defining a new CPU model feature is probably
the best way to resolve this issue. The question is, should we define a single
feature indicating whether AP instructions are installed and set features bits
for the guest based on whether or not they are set in the linux host, or should
we define additional CPU model features for turning features bits on and off?
I guess it boils down to what behavior is expected for the AP bus running on
the linux guest. Here is a rundown of the facilities bits associated with AP
and how they affect the behavior of the AP bus:

* STFLE.12 indicates whether the AP query function is available. If this bit
is not set, then the AP bus scan will only test domains 0-15. For example,
if adapters 4, 5, and 6 and domains 12 and 71 (0x47) are installed, then AP
queues 04.0047, 05.0047 and 06.0047 will not be made available.
STFLE 12 is the indication for Query AP Configuration Information (QCI) available.
* STFLE.15 indicates whether the AP facilities test function is available. If
this bit is not set, then the CEX4, CEX5 and CEX6 device drivers discovered
by the AP bus scan will not get bound to any AP device drivers. Since the
AP matrix model supports only CEX4 and greater, no devices will be bound
to any driver for a guest.
This T-Bit extension to the TAPQ subfunction is a must have. When kvm only
supports CEX4 and upper then this bit could also act as the indicator for
AP instructions available. Of course if you want to implement pure virtual
full simulated AP without any real AP hardware on the host this bit can't
be the indicator.
It would probably make sense to group these two together. Or is there
any advantage in supporting only a part of it?
After thinking about this a little more, I've come to the conclusion that
all of this might be moot for the following reasons:

* If STFLE.12 is not set for the linux host, then AP bus scan running on
the host will not detect any domains with a domain number higher than 15,
so no AP queues with a queue index higher than 15 will be available to
bind to the vfio_ap_matrix driver. Consequently, no domain higher than
15 can be assigned to any guest. In this case, the AP bus scan running on
the guest will never detect a domain higher than 15, regardless of the
setting of STFLE.12 for the guest.

* If STFLE.15 is not set for the linux host, then then there will be no
CEX4, CEX5 or CEX6 queues available to bind to the vfio_ap_matrix
driver, so no AP adapters or domains can be assigned to any KVM guest.

The bottom line is the STFLE bit settings for the linux host will control
what APs are available to the KVM guest. Since STFLE.15 controls whether
any CEX4,5 or 6 devices are even available, I think this bit can be
combined into the feature that indicates whether AP is available. As long
as AP instructions are available on the linux host, I'm not sure whether
STFLE.12 needs a feature at all.

* STFLE.65 indicates whether AP interrupts are available. If this bit is not
set, then the AP bus will use polling instead of using interrupt handlers
to process AP events.
So, does this indicate "adapter interrupts for AP" only? If so, we
should keep this separate and only enable it when we have the gisa etc.
ready.
Yes, this indicates AP interrupts only. The plan is to enable this when
GISA is available and we can implement interrupt processing.