Re: [PATCH] x86/asm: Remove the __iomem annotation of movdir64b()'s dst argument

[Yuan Yao]

On Fri, Jan 05, 2024 at 11:12:19AM +1300, Kai Huang wrote:
> Commit e56d28df2f66 ("x86/virt/tdx: Configure global KeyID on all
> packages") causes below sparse check warning:
>
>   arch/x86/virt/vmx/tdx/tdx.c:683:27: warning: incorrect type in argument 1 (different address spaces)
>   arch/x86/virt/vmx/tdx/tdx.c:683:27:    expected void [noderef] __iomem *dst
>   arch/x86/virt/vmx/tdx/tdx.c:683:27:    got void *
>
> The reason is TDX must use the MOVDIR64B instruction to convert TDX
> private memory (which is normal RAM but not MMIO) back to normal.  The
> TDX code uses existing movdir64b() helper to do that, but the first
> argument @dst of movdir64b() is annotated with __iomem.
>
> When movdir64b() was firstly introduced in commit 0888e1030d3e
> ("x86/asm: Carve out a generic movdir64b() helper for general usage"),
> it didn't have the __iomem annotation.  But this commit also introduced
> the same "incorrect type" sparse warning because the iosubmit_cmds512(),
> which was the solo caller of movdir64b(), has the __iomem annotation.
>
> This was later fixed by commit 6ae58d871319 ("x86/asm: Annotate
> movdir64b()'s dst argument with __iomem").  That fix was reasonable
> because until TDX code the movdir64b() was only used to move data to
> MMIO location, as described by the commit message:
>
>   ... The current usages send a 64-bytes command descriptor to an MMIO
>   location (portal) on a device for consumption. When future usages for
>   the MOVDIR64B instruction warrant a separate variant of a memory to
>   memory operation, the argument annotation can be revisited.
>
> Now TDX code uses MOVDIR64B to move data to normal memory so it's time
> to revisit.
>
> The SDM says the destination of MOVDIR64B is "memory location specified
> in a general register", thus it's more reasonable that movdir64b() does
> not have the __iomem annotation on the @dst.
>
> Remove the __iomem annotation from the @dst argument of movdir64b() to
> fix the sparse warning in TDX code.  Similar to memset_io(), introduce a
> new movdir64b_io() to cover the case where the destination is an MMIO
> location, and change the solo caller iosubmit_cmds512() to use the new
> movdir64b_io().
>
> In movdir64b_io() explicitly use __force in the type casting otherwise
> there will be below sparse warning:
>
>   warning: cast removes address space '__iomem' of expression
>
> Fixes: e56d28df2f66 ("x86/virt/tdx: Configure global KeyID on all packages")
> Reported-by: kernel test robot <lkp@xxxxxxxxx>
> Closes: https://lore.kernel.org/oe-kbuild-all/202312311924.tGjsBIQD-lkp@xxxxxxxxx/
> Signed-off-by: Kai Huang <kai.huang@xxxxxxxxx>

Reviewed-by: Yuan Yao <yuan.yao@xxxxxxxxx>

> ---
>  arch/x86/include/asm/io.h            | 2 +-
>  arch/x86/include/asm/special_insns.h | 9 +++++++--
>  2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
> index 76238842406a..de2dc9837f11 100644
> --- a/arch/x86/include/asm/io.h
> +++ b/arch/x86/include/asm/io.h
> @@ -379,7 +379,7 @@ static inline void iosubmit_cmds512(void __iomem *dst, const void *src,
>  	const u8 *end = from + count * 64;
>
>  	while (from < end) {
> -		movdir64b(dst, from);
> +		movdir64b_io(dst, from);
>  		from += 64;
>  	}
>  }
> diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h
> index d6cd9344f6c7..f661277e52d6 100644
> --- a/arch/x86/include/asm/special_insns.h
> +++ b/arch/x86/include/asm/special_insns.h
> @@ -224,10 +224,10 @@ static inline void serialize(void)
>  }
>
>  /* The dst parameter must be 64-bytes aligned */
> -static inline void movdir64b(void __iomem *dst, const void *src)
> +static inline void movdir64b(void *dst, const void *src)
>  {
>  	const struct { char _[64]; } *__src = src;
> -	struct { char _[64]; } __iomem *__dst = dst;
> +	struct { char _[64]; } *__dst = dst;
>
>  	/*
>  	 * MOVDIR64B %(rdx), rax.
> @@ -245,6 +245,11 @@ static inline void movdir64b(void __iomem *dst, const void *src)
>  		     :  "m" (*__src), "a" (__dst), "d" (__src));
>  }
>
> +static inline void movdir64b_io(void __iomem *dst, const void *src)
> +{
> +	movdir64b((void __force *)dst, src);
> +}
> +
>  /**
>   * enqcmds - Enqueue a command in supervisor (CPL0) mode
>   * @dst: destination, in MMIO space (must be 512-bit aligned)
>
> base-commit: 83e1bdc94f32dcf52dfcd2025acc7a2b9376b1e8
> --
> 2.43.0
>