Re: [PATCH v4 01/14] docs: gunyah: Introduce Gunyah Hypervisor

[Elliot Berman]

On 9/28/2022 8:43 PM, Bagas Sanjaya wrote:
> On Wed, Sep 28, 2022 at 12:56:20PM -0700, Elliot Berman wrote:
> > diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
> > new file mode 100644
> > index 000000000000..959f451caccd
> > --- /dev/null
> > +++ b/Documentation/virt/gunyah/index.rst
> > @@ -0,0 +1,114 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +=================
> > +Gunyah Hypervisor
> > +=================
> > +
> > +.. toctree::
> > +   :maxdepth: 1
> > +
> > +   message-queue
> > +
> > +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
> > +a higher CPU privilege level. It does not depend on any lower-privileged operating system
> > +for its core functionality. This increases its security and can support a much smaller
> > +trusted computing base than a Type-2 hypervisor.
> > +
> > +Gunyah is an open source hypervisor. The source repo is available at
> > +https://github.com/quic/gunyah-hypervisor.
> > +
> > +Gunyah provides these following features.
> > +
> > +- Scheduling:
> > +
> > +  A scheduler for virtual CPUs (vCPUs) on physical CPUs and enables time-sharing
> > +  of the CPUs. Gunyah supports two models of scheduling:
> > +
> > +    1. "Behind the back" scheduling in which Gunyah hypervisor schedules vCPUS on its own
> > +    2. "Proxy" scheduling in which a delegated VM can donate part of one of its vCPU slice
> > +       to another VM's vCPU via a hypercall.
> > +
> > +- Memory Management:
> > +
> > +  APIs handling memory, abstracted as objects, limiting direct use of physical
> > +  addresses. Memory ownership and usage tracking of all memory under its control.
> > +  Memory partitioning between VMs is a fundamental security feature.
> > +
> > +- Interrupt Virtualization:
> > +
> > +  Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled
> > +  in the hypervisor and routed to the assigned VM.
> > +
> > +- Inter-VM Communication:
> > +
> > +  There are several different mechanisms provided for communicating between VMs.
> > +
> > +- Virtual platform:
> > +
> > +  Architectural devices such as interrupt controllers and CPU timers are directly provided
> > +  by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI.
> > +
> > +- Device Virtualization:
> > +
> > +  Para-virtualization of devices is supported using inter-VM communication.
> > +
> > +Architectures supported
> > +=======================
> > +AArch64 with a GIC
> > +
> > +Resources and Capabilities
> > +==========================
> > +
> > +Some services or resources provided by the Gunyah hypervisor are described to a virtual machine by
> > +capability IDs. For instance, inter-VM communication is performed with doorbells and message queues.
> > +Gunyah allows access to manipulate that doorbell via the capability ID. These devices are described
> > +in Linux as a struct gunyah_resource.
> > +
> > +High level management of these resources is performed by the resource manager VM. RM informs a
> > +guest VM about resources it can access through either the device tree or via guest-initiated RPC.
> > +
> > +For each virtual machine, Gunyah maintains a table of resources which can be accessed by that VM.
> > +An entry in this table is called a "capability" and VMs can only access resources via this
> > +capability table. Hence, virtual Gunyah devices are referenced by a "capability IDs" and not a
> > +"resource IDs". A VM can have multiple capability IDs mapping to the same resource. If 2 VMs have
> > +access to the same resource, they may not be using the same capability ID to access that resource
> > +since the tables are independent per VM.
> > +
> > +Resource Manager
> > +================
> > +
> > +The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor.
> > +It provides policy enforcement aspects of the virtualization system. The resource manager can
> > +be treated as an extension of the Hypervisor but is separated to its own partition to ensure
> > +that the hypervisor layer itself remains small and secure and to maintain a separation of policy
> > +and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines.
> > +
> > +Communication with the resource manager from each guest VM happens with message-queue.rst. Details
> > +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
> > +
> > +::
> > +
> > +  +-------+   +--------+   +--------+
> > +  |  RM   |   |  VM_A  |   |  VM_B  |
> > +  +-.-.-.-+   +---.----+   +---.----+
> > +    | |           |            |
> > +  +-.-.-----------.------------.----+
> > +  | | \==========/             |    |
> > +  |  \========================/     |
> > +  |            Gunyah               |
> > +  +---------------------------------+
> > +
> > +The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager.
> > +
> > +The resource manager provides the following features:
> > +
> > +- VM lifecycle management: allocating a VM, starting VMs, destruction of VMs
> > +- VM access control policy, including memory sharing and lending
> > +- Interrupt routing configuration
> > +- Forwarding of system-level events (e.g. VM shutdown) to owner VM
> > +
> > +When booting a virtual machine which uses a devicetree, resource manager overlays a
> > +/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM,
> > +how to communicate with resource manager, and basic description and capabilities of
> > +this VM. See Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a description
> > +of this node.
>
> The documentation LGTM.
>
> > diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
> > new file mode 100644
> > index 000000000000..e130f124ed52
> > --- /dev/null
> > +++ b/Documentation/virt/gunyah/message-queue.rst
> > <snipped>...
> > +The diagram below shows how message queue works. A typical configuration involves
> > +2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message
> > +queue 2 allows VM_B to send messages to VM_A.
> > +
> > +1. VM_A sends a message of up to 1024 bytes in length. It raises a hypercall
> > +   with the message to inform the hypervisor to add the message to
> > +   message queue 1's queue.
> > +2. Gunyah raises the corresponding interrupt for VM_B when any of these happens:
> > +   a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case.
> > +   b. Explicility with gh_msgq_push command from VM_A.
> > +   c. Message queue has reached a threshold depth.
> > +3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer.
> > +
>
> The nested list above should be separated with blank lines to be
> rendered properly:
>
> ---- >8 ----
>
> diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
> index e130f124ed525a..afaad99db215e6 100644
> --- a/Documentation/virt/gunyah/message-queue.rst
> +++ b/Documentation/virt/gunyah/message-queue.rst
> @@ -20,9 +20,11 @@ queue 2 allows VM_B to send messages to VM_A.
>      with the message to inform the hypervisor to add the message to
>      message queue 1's queue.
>   2. Gunyah raises the corresponding interrupt for VM_B when any of these happens:
> +
>      a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case.
>      b. Explicility with gh_msgq_push command from VM_A.
>      c. Message queue has reached a threshold depth.
> +
>   3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer.
>   
>   For VM_B to send a message to VM_A, the process is identical, except that hypercalls
>
> Thanks.

Thanks! Applied for next version.