Re: [PATCH v5 0/5] Unify NUMA implementation between ARM64 & RISC-V

[Palmer Dabbelt]

On Mon, 11 Jan 2021 11:31:11 PST (-0800), atishp@xxxxxxxxxxxxxx wrote:
> On Sat, Jan 9, 2021 at 12:51 PM Palmer Dabbelt <palmer@xxxxxxxxxxx> wrote:
> > On Sun, 13 Dec 2020 17:02:19 PST (-0800), atishp@xxxxxxxxxxxxxx wrote:
> > > On Wed, Nov 18, 2020 at 4:39 PM Atish Patra <atish.patra@xxxxxxx> wrote:
> > >>
> > >> This series attempts to move the ARM64 numa implementation to common
> > >> code so that RISC-V can leverage that as well instead of reimplementing
> > >> it again.
> > >>
> > >> RISC-V specific bits are based on initial work done by Greentime Hu [1] but
> > >> modified to reuse the common implementation to avoid duplication.
> > >>
> > >> [1] https://lkml.org/lkml/2020/1/10/233
> > >>
> > >> This series has been tested on qemu with numa enabled for both RISC-V & ARM64.
> > >> It would be great if somebody can test it on numa capable ARM64 hardware platforms.
> > >> This patch series doesn't modify the maintainers list for the common code (arch_numa)
> > >> as I am not sure if somebody from ARM64 community or Greg should take up the
> > >> maintainership. Ganapatrao was the original author of the arm64 version.
> > >> I would be happy to update that in the next revision once it is decided.
> > >>
> > >> # numactl --hardware
> > >> available: 2 nodes (0-1)
> > >> node 0 cpus: 0 1 2 3
> > >> node 0 size: 486 MB
> > >> node 0 free: 470 MB
> > >> node 1 cpus: 4 5 6 7
> > >> node 1 size: 424 MB
> > >> node 1 free: 408 MB
> > >> node distances:
> > >> node   0   1
> > >>   0:  10  20
> > >>   1:  20  10
> > >> # numactl -show
> > >> policy: default
> > >> preferred node: current
> > >> physcpubind: 0 1 2 3 4 5 6 7
> > >> cpubind: 0 1
> > >> nodebind: 0 1
> > >> membind: 0 1
> > >>
> > >> The patches are also available at
> > >> https://github.com/atishp04/linux/tree/5.11_numa_unified_v5
> > >>
> > >> For RISC-V, the following qemu series is a pre-requisite(already available in upstream)
> > >> https://patchwork.kernel.org/project/qemu-devel/list/?series=303313
> > >>
> > >> Testing:
> > >> RISC-V:
> > >> Tested in Qemu and 2 socket OmniXtend FPGA.
> > >>
> > >> ARM64:
> > >> 2 socket kunpeng920 (4 nodes around 250G a node)
> > >> Tested-by: Jonathan Cameron <Jonathan.Cameron@xxxxxxxxxx>
> > >>
> > >> Changes from v4->v5:
> > >> 1. Added by Acked-by & Reviewed-by tags.
> > >> 2. Swapped patch 1 & 2 in v4 version.
> > >>
> > >> Changes from v3->v4:
> > >> 1. Removed redundant duplicate header.
> > >> 2. Added Reviewed-by tags.
> > >>
> > >> Changes from v2->v3:
> > >> 1. Added Acked-by/Reviewed-by tags.
> > >> 2. Replaced asm/acpi.h with linux/acpi.h
> > >> 3. Defined arch_acpi_numa_init as static.
> > >>
> > >> Changes from v1->v2:
> > >> 1. Replaced ARM64 specific compile time protection with ACPI specific ones.
> > >> 2. Dropped common pcibus_to_node changes. Added required changes in RISC-V.
> > >> 3. Fixed few typos.
> > >>
> > >> Atish Patra (4):
> > >> arm64, numa: Change the numa init functions name to be generic
> > >> numa: Move numa implementation to common code
> > >> riscv: Separate memory init from paging init
> > >> riscv: Add numa support for riscv64 platform
> > >>
> > >> Greentime Hu (1):
> > >> riscv: Add support pte_protnone and pmd_protnone if
> > >> CONFIG_NUMA_BALANCING
> > >>
> > >> arch/arm64/Kconfig                            |  1 +
> > >> arch/arm64/include/asm/numa.h                 | 48 +----------------
> > >> arch/arm64/kernel/acpi_numa.c                 | 12 -----
> > >> arch/arm64/mm/Makefile                        |  1 -
> > >> arch/arm64/mm/init.c                          |  4 +-
> > >> arch/riscv/Kconfig                            | 31 ++++++++++-
> > >> arch/riscv/include/asm/mmzone.h               | 13 +++++
> > >> arch/riscv/include/asm/numa.h                 |  8 +++
> > >> arch/riscv/include/asm/pci.h                  | 14 +++++
> > >> arch/riscv/include/asm/pgtable.h              | 21 ++++++++
> > >> arch/riscv/kernel/setup.c                     | 11 +++-
> > >> arch/riscv/kernel/smpboot.c                   | 12 ++++-
> > >> arch/riscv/mm/init.c                          | 10 +++-
> > >> drivers/base/Kconfig                          |  6 +++
> > >> drivers/base/Makefile                         |  1 +
> > >> .../mm/numa.c => drivers/base/arch_numa.c     | 27 ++++++++--
> > >> include/asm-generic/numa.h                    | 52 +++++++++++++++++++
> > >> 17 files changed, 200 insertions(+), 72 deletions(-)
> > >> create mode 100644 arch/riscv/include/asm/mmzone.h
> > >> create mode 100644 arch/riscv/include/asm/numa.h
> > >> rename arch/arm64/mm/numa.c => drivers/base/arch_numa.c (96%)
> > >> create mode 100644 include/asm-generic/numa.h
> > >>
> > >> --
> > >> 2.25.1
> > >>
> > >>
> > >> _______________________________________________
> > >> linux-riscv mailing list
> > >> linux-riscv@xxxxxxxxxxxxxxxxxxx
> > >> http://lists.infradead.org/mailman/listinfo/linux-riscv
> > >
> > > Hey Palmer,
> > > I did not see this series in for-next. Let me know if you need
> > > anything else to be done for this series.
> >
> > Sorry about that.  It's on for-next, with Randy's comment addressed.  There was
> > one merge conflict: we don't have resource_init() in for-next yet (which I
> > think means I missed something else).
>
> resource_init is changed to init_resource and moved to setup.c in the
> following patch which was merged in 5.11 MW.
> 00ab027a3b82 RISC-V: Add kernel image sections to the resource tree

Ah, great, for some reason I thought we hadn't merged those yet.

> > IDK if that's necessary for the NUMA
> > stuff, I just dropped it.  I haven't tested this yet.