Introduction
============
Arm has implemented memory coloring in hardware, and the feature is called
Memory Tagging Extensions (MTE). It works by embedding a 4 bit tag in bits
59..56 of a pointer, and storing this tag to a reserved memory location.
When the pointer is dereferenced, the hardware compares the tag embedded in
the pointer (logical tag) with the tag stored in memory (allocation tag).
The relation between memory and where the tag for that memory is stored is
static.
The memory where the tags are stored have been so far unaccessible to Linux.
This series aims to change that, by adding support for using the tag storage
memory only as data memory; tag storage memory cannot be itself tagged.
Implementation
==============
The series is based on v6.5-rc3 with these two patches cherry picked:
- mm: Call arch_swap_restore() from unuse_pte():
https://lore.kernel.org/all/20230523004312.1807357-3-pcc@xxxxxxxxxx/
- arm64: mte: Simplify swap tag restoration logic:
https://lore.kernel.org/all/20230523004312.1807357-4-pcc@xxxxxxxxxx/
The above two patches are queued for the v6.6 merge window:
https://lore.kernel.org/all/20230702123821.04e64ea2c04dd0fdc947bda3@xxxxxxxxxxxxxxxxxxxx/
The entire series, including the above patches, can be cloned with:
$ git clone https://gitlab.arm.com/linux-arm/linux-ae.git \
-b arm-mte-dynamic-carveout-rfc-v1
On the arm64 architecture side, an extension is being worked on that will
clarify how MTE tag storage reuse should behave. The extension will be
made public soon.
On the Linux side, MTE tag storage reuse is accomplished with the
following changes:
1. The tag storage memory is exposed to the memory allocator as a new
migratetype, MIGRATE_METADATA. It behaves similarly to MIGRATE_CMA, with
the restriction that it cannot be used to allocate tagged memory (tag
storage memory cannot be tagged). On tagged page allocation, the
corresponding tag storage is reserved via alloc_contig_range().
2. mprotect(PROT_MTE) is implemented by changing the pte prot to
PAGE_METADATA_NONE. When the page is next accessed, a fault is taken and
the corresponding tag storage is reserved.
3. When the code tries to copy tags to a page which doesn't have the tag
storage reserved, the tags are copied to an xarray and restored in
set_pte_at(), when the page is eventually mapped with the tag storage
reserved.
arch/arm64/Kconfig | 13 +
arch/arm64/include/asm/assembler.h | 10 +
arch/arm64/include/asm/memory_metadata.h | 49 ++
arch/arm64/include/asm/mte-def.h | 16 +-
arch/arm64/include/asm/mte.h | 40 +-
arch/arm64/include/asm/mte_tag_storage.h | 36 ++
arch/arm64/include/asm/page.h | 5 +-
arch/arm64/include/asm/pgtable-prot.h | 2 +
arch/arm64/include/asm/pgtable.h | 33 +-
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/elfcore.c | 14 +-
arch/arm64/kernel/hibernate.c | 46 +-
arch/arm64/kernel/mte.c | 31 +-
arch/arm64/kernel/mte_tag_storage.c | 667 +++++++++++++++++++++++
arch/arm64/kernel/setup.c | 7 +
arch/arm64/kvm/arm.c | 6 +-
arch/arm64/lib/mte.S | 30 +-
arch/arm64/mm/copypage.c | 26 +
arch/arm64/mm/fault.c | 35 +-
arch/arm64/mm/mteswap.c | 113 +++-
fs/proc/meminfo.c | 8 +
fs/proc/page.c | 1 +
include/asm-generic/Kbuild | 1 +
include/asm-generic/memory_metadata.h | 50 ++
include/linux/gfp.h | 10 +
include/linux/gfp_types.h | 14 +-
include/linux/huge_mm.h | 6 +
include/linux/kernel-page-flags.h | 1 +
include/linux/migrate_mode.h | 1 +
include/linux/mm.h | 12 +-
include/linux/mmzone.h | 26 +-
include/linux/page-flags.h | 1 +
include/linux/pgtable.h | 19 +
include/linux/sched.h | 2 +-
include/linux/sched/mm.h | 13 +
include/linux/vm_event_item.h | 5 +
include/linux/vmstat.h | 2 +
include/trace/events/mmflags.h | 5 +-
mm/Kconfig | 5 +
mm/compaction.c | 52 +-
mm/huge_memory.c | 109 ++++
mm/internal.h | 7 +
mm/khugepaged.c | 7 +
mm/memory.c | 180 +++++-
mm/mempolicy.c | 7 +
mm/migrate.c | 6 +
mm/mm_init.c | 23 +-
mm/mprotect.c | 46 ++
mm/page_alloc.c | 136 ++++-
mm/page_isolation.c | 19 +-
mm/page_owner.c | 3 +-
mm/shmem.c | 14 +-
mm/show_mem.c | 4 +
mm/swapfile.c | 4 +
mm/vmscan.c | 3 +
mm/vmstat.c | 13 +-
56 files changed, 1834 insertions(+), 161 deletions(-)
create mode 100644 arch/arm64/include/asm/memory_metadata.h
create mode 100644 arch/arm64/include/asm/mte_tag_storage.h
create mode 100644 arch/arm64/kernel/mte_tag_storage.c
create mode 100644 include/asm-generic/memory_metadata.h