Re: [PATCH kernel v2] x86/compressed/64: reduce #VC nesting for intercepted CPUID for SEV-SNP guest

[Ingo Molnar]

* Alexey Kardashevskiy <aik@xxxxxxx> wrote:

>  arch/x86/include/asm/svm.h   | 14 ++++++++++++++
>  arch/x86/kernel/sev-shared.c |  5 +++--

Doesn't build on x86-64 allmodconfig:

  arch/x86/kernel/sev-shared.c:442:75: error: ‘ghcb’ undeclared (first use in this function)
  arch/x86/kernel/sev-shared.c:442:81: error: ‘ctxt’ undeclared (first use in this function)

Not sure how this was supposed to work - there's no 'ghcb' passed in to
snp_cpuid_postprocess(). Does this patch have a dependency perhaps, that
I missed?

For the next version please also pick up the edited changelog I've done,
see below.

Thanks,

	Ingo

====================>
From: Alexey Kardashevskiy <aik@xxxxxxx>
Subject: [PATCH] x86/sev: Reduce #VC nesting for intercepted CPUID for SEV-SNP guest, to fix nesting crash

For certain intercepts an SNP guest uses the GHCB protocol to talk to
the hypervisor from the #VC handler. The protocol requires a shared page so
there is one per vCPU. In case NMI arrives in a middle of #VC or the NMI
handler triggers a #VC, there is another "backup" GHCB page which stores
the content of the first one while SVM_VMGEXIT_NMI_COMPLETE is sent.
The vc_raw_handle_exception() handler manages main and backup GHCB pages
via __sev_get_ghcb/__sev_put_ghcb.

This works fine for #VC and occasional NMIs. This does not work so fine if
the #VC handler causes intercept + another #VC, if NMI arrives during
the second #VC, there are no more pages for SVM_VMGEXIT_NMI_COMPLETE.
The problem place is the #VC CPUID handler. Running perf in the SNP guest
crashes with:

  Kernel panic - not syncing: Unable to handle #VC exception! GHCB and Backup GHCB are already in use

  vc_raw_handle_exception #1: exit_code 72 (CPUID) eax d ecx 1

We lock the main GHCB and while it is locked we get to
snp_cpuid_postprocess() which executes "rdmsr" of MSR_IA32_XSS==0xda0 which
triggers:

  vc_raw_handle_exception #2: exit_code 7c (MSR) ecx da0

Here we lock the backup ghcb.

And then PMC NMI comes which cannot complete as there is no GHCB page left
to use:

  CPU: 5 PID: 566 Comm: touch Not tainted 6.5.0-rc2-aik-ad9c-g7413e71d3dcf-dirty #27
  Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown unknown
  Call Trace:
   <NMI>
   dump_stack_lvl+0x44/0x60
   panic+0x222/0x310
   ____sev_get_ghcb+0x21e/0x220
   __sev_es_nmi_complete+0x28/0xf0
   exc_nmi+0x1ac/0x1c0
   end_repeat_nmi+0x16/0x67
  ...
   </NMI>
   <TASK>
   vc_raw_handle_exception+0x9e/0x2c0
   kernel_exc_vmm_communication+0x4d/0xa0
   asm_exc_vmm_communication+0x31/0x60
  RIP: 0010:snp_cpuid+0x2ad/0x420

Add a helper similar to rdmsr_safe() for making a direct hypercall in the SEV-ES
environment. Use the new helper instead of the raw "rdmsr" to avoid the extra #VC event.

Fixes: ee0bfa08a345 ("x86/compressed/64: Add support for SEV-SNP CPUID table in #VC handlers")
Signed-off-by: Alexey Kardashevskiy <aik@xxxxxxx>
Signed-off-by: Ingo Molnar <mingo@xxxxxxxxxx>
Acked-by: Tom Lendacky <thomas.lendacky@xxxxxxx>
Link: https://lore.kernel.org/r/20230926040526.957240-1-aik@xxxxxxx