Re: [PATCH v4 00/12] KVM: mm: fd-based approach for supporting KVM guest private memory

[Nakajima, Jun]

> On Jan 28, 2022, at 8:47 AM, Steven Price <steven.price@xxxxxxx> wrote:
> 
> On 18/01/2022 13:21, Chao Peng wrote:
>> This is the v4 of this series which try to implement the fd-based KVM
>> guest private memory. The patches are based on latest kvm/queue branch
>> commit:
>> 
>>  fea31d169094 KVM: x86/pmu: Fix available_event_types check for
>>               REF_CPU_CYCLES event
>> 
>> Introduction
>> ------------
>> In general this patch series introduce fd-based memslot which provides
>> guest memory through memory file descriptor fd[offset,size] instead of
>> hva/size. The fd can be created from a supported memory filesystem
>> like tmpfs/hugetlbfs etc. which we refer as memory backing store. KVM
>> and the the memory backing store exchange callbacks when such memslot
>> gets created. At runtime KVM will call into callbacks provided by the
>> backing store to get the pfn with the fd+offset. Memory backing store
>> will also call into KVM callbacks when userspace fallocate/punch hole
>> on the fd to notify KVM to map/unmap secondary MMU page tables.
>> 
>> Comparing to existing hva-based memslot, this new type of memslot allows
>> guest memory unmapped from host userspace like QEMU and even the kernel
>> itself, therefore reduce attack surface and prevent bugs.
>> 
>> Based on this fd-based memslot, we can build guest private memory that
>> is going to be used in confidential computing environments such as Intel
>> TDX and AMD SEV. When supported, the memory backing store can provide
>> more enforcement on the fd and KVM can use a single memslot to hold both
>> the private and shared part of the guest memory. 
> 
> This looks like it will be useful for Arm's Confidential Compute
> Architecture (CCA) too - in particular we need a way of ensuring that
> user space cannot 'trick' the kernel into accessing memory which has
> been delegated to a realm (i.e. protected guest), and a memfd seems like
> a good match.

Good to hear that it will be useful for ARM’s CCA as well.

> 
> Some comments below.
> 
>> mm extension
>> ---------------------
>> Introduces new F_SEAL_INACCESSIBLE for shmem and new MFD_INACCESSIBLE
>> flag for memfd_create(), the file created with these flags cannot read(),
>> write() or mmap() etc via normal MMU operations. The file content can
>> only be used with the newly introduced memfile_notifier extension.
> 
> For Arm CCA we are expecting to seed the realm with an initial memory
> contents (e.g. kernel and initrd) which will then be measured before
> execution starts. The 'obvious' way of doing this with a memfd would be
> to populate parts of the memfd then seal it with F_SEAL_INACCESSIBLE.

As far as I understand, we have the same problem with TDX, where a guest TD (Trust Domain) starts in private memory. We seed the private memory typically with a guest firmware, and the initial image (plaintext) is copied to somewhere in QEMU memory (from disk, for example) for that purpose; this location is not associated with the target GPA.

Upon a (new) ioctl from QEMU, KVM requests the TDX Module to copy the pages to private memory (by encrypting) specifying the target GPA, using a TDX interface function (TDH.MEM.PAGE.ADD). The actual pages for the private memory is allocated by the callbacks provided by the backing store during the “copy” operation.

We extended the existing KVM_MEMORY_ENCRYPT_OP (ioctl) for the above. 

> 
> However as things stand it's not possible to set the INACCESSIBLE seal
> after creating a memfd (F_ALL_SEALS hasn't been updated to include it).
> 
> One potential workaround would be for arm64 to provide a custom KVM
> ioctl to effectively memcpy() into the guest's protected memory which
> would only be accessible before the guest has started. The drawback is
> that it requires two copies of the data during guest setup.

So, the guest pages are not encrypted in the realm?

I think you could do the same thing, i.e. KVM copies the pages to the realm, where pages are allocated by the backing store. But, yes, it will have two copies of the data at that time unless encrypted. .

> 
> Do you think things could be relaxed so the F_SEAL_INACCESSIBLE flag
> could be set after a memfd has been created (and partially populated)?
> 

I think F_SEAL_INACCESSIBLE could be deferred to the point where measurement of the initial image is done (we call “build-time” measurement in TDX). For example, if we add a callback to activate F_SEAL_INACCESSIBLE and KVM calls it before such the measurement time, does that work for you?

--- 
Jun