[PATCH v2 00/29] virtio-mem: Big Block Mode (BBM)

From: David Hildenbrand
Date: Thu Nov 12 2020 - 08:38:35 EST


@Andrew, can we have an ack for patch #27, so that one can go via
the vhost tree for 5.11?

---

virtio-mem currently only supports device block sizes that span at most
a single Linux memory block. For example, gigantic pages in the hypervisor
result on x86-64 in a device block size of 1 GiB - when the Linux memory
block size is 128 MiB, we cannot support such devices (we fail loading the
driver). Of course, we want to support any device block size in any Linux
VM.

Bigger device block sizes will become especially important once supporting
VFIO in QEMU - each device block has to be mapped separately, and the
maximum number of mappings for VFIO is 64k. So we usually want blocks in
the gigabyte range when wanting to grow the VM big.

Patch #1 - #10 are cleanups and optimizations
Patch #11 - #24 are refactorings to prepare for BBM
Patch #25 - #29 implement BBM, including one mm/memory_hotplug extension

This series is based on latest linus/master and can be found at:
git@xxxxxxxxxx:davidhildenbrand/linux.git virtio-mem-bbm-v2

v1 -> v2:
- Code wise, the only bigger change is using an union for sbm/bbm state
- Reworked some subjects/patch descriptions
- Reshuffled patches to make reviweing easier, and to have
cleanups+optimizations before all refactorings
- "virtio-mem: more precise calculation in
virtio_mem_mb_state_prepare_next_mb()"
-- Changed subject
-- Avoid two local variables

David Hildenbrand (29):
virtio-mem: determine nid only once using memory_add_physaddr_to_nid()
virtio-mem: more precise calculation in
virtio_mem_mb_state_prepare_next_mb()
virtio-mem: simplify MAX_ORDER - 1 / pageblock_order handling
virtio-mem: drop rc2 in virtio_mem_mb_plug_and_add()
virtio-mem: use "unsigned long" for nr_pages when fake
onlining/offlining
virtio-mem: factor out calculation of the bit number within the
subblock bitmap
virtio-mem: print debug messages from virtio_mem_send_*_request()
virtio-mem: factor out fake-offlining into virtio_mem_fake_offline()
virtio-mem: factor out handling of fake-offline pages in memory
notifier
virtio-mem: retry fake-offlining via alloc_contig_range() on
ZONE_MOVABLE
virtio-mem: generalize check for added memory
virtio-mem: generalize virtio_mem_owned_mb()
virtio-mem: generalize virtio_mem_overlaps_range()
virtio-mem: drop last_mb_id
virtio-mem: don't always trigger the workqueue when offlining memory
virtio-mem: generalize handling when memory is getting onlined
deferred
virito-mem: document Sub Block Mode (SBM)
virtio-mem: memory block states are specific to Sub Block Mode (SBM)
virito-mem: subblock states are specific to Sub Block Mode (SBM)
virtio-mem: nb_sb_per_mb and subblock_size are specific to Sub Block
Mode (SBM)
virtio-mem: memory block ids are specific to Sub Block Mode (SBM)
virito-mem: existing (un)plug functions are specific to Sub Block Mode
(SBM)
virtio-mem: memory notifier callbacks are specific to Sub Block Mode
(SBM)
virtio-mem: factor out adding/removing memory from Linux
virtio-mem: Big Block Mode (BBM) memory hotplug
virtio-mem: allow to force Big Block Mode (BBM) and set the big block
size
mm/memory_hotplug: extend offline_and_remove_memory() to handle more
than one memory block
virtio-mem: Big Block Mode (BBM) - basic memory hotunplug
virtio-mem: Big Block Mode (BBM) - safe memory hotunplug

drivers/virtio/virtio_mem.c | 1789 +++++++++++++++++++++++++----------
mm/memory_hotplug.c | 105 +-
2 files changed, 1376 insertions(+), 518 deletions(-)

--
2.26.2