[GIT PULL] x86/core changes for v6.9

From: Ingo Molnar
Date: Mon Mar 11 2024 - 09:48:35 EST


Linus,

Please pull the latest x86/core git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86-core-2024-03-11

# HEAD: 35ce64922c8263448e58a2b9e8d15a64e11e9b2d x86/idle: Select idle routine only once

Core x86 changes for v6.9:

- The biggest change is the rework of the percpu code,
to support the 'Named Address Spaces' GCC feature,
by Uros Bizjak:

- This allows C code to access GS and FS segment relative
memory via variables declared with such attributes,
which allows the compiler to better optimize those accesses
than the previous inline assembly code.

- The series also includes a number of micro-optimizations
for various percpu access methods, plus a number of
cleanups of %gs accesses in assembly code.

- These changes have been exposed to linux-next testing for
the last ~5 months, with no known regressions in this area.

- Fix/clean up __switch_to()'s broken but accidentally
working handling of FPU switching - which also generates
better code.

- Propagate more RIP-relative addressing in assembly code,
to generate slightly better code.

- Rework the CPU mitigations Kconfig space to be less idiosyncratic,
to make it easier for distros to follow & maintain these options.

- Rework the x86 idle code to cure RCU violations and
to clean up the logic.

- Clean up the vDSO Makefile logic.

- Misc cleanups and fixes.

MERGE NOTES:

Please note that there's three merge commits in this branch, which is
higher than is typical in x86 topic trees. This happened due to the
unusually long testing lifecycle of the percpu changes that involved 3
merge windows, which generated a longer history and various interactions
with other WIP core x86 changes that we felt better about to carry in a
single tree. [ Should there be any conceptual problems with the percpu
changes that we missed, then we can still unwind these dependencies in a
relatively straightforward fashion, so there's no merge pressure on you. ]

This tree also has a new conflict with recent upstream in
arch/x86/entry/entry.S, due to a recent x86/urgent VERW related fix.

The interaction is between:

# upstream (x86/urgent)
baf8361e5455 ("x86/bugs: Add asm helpers for executing VERW")

.. which added the new mds_verw_sel VERW method, and this commit in x86/core:

# x86/core:
4461438a8405 x86/retpoline: Ensure default return thunk isn't used at runtime

.. which added the __warn_thunk method.

The conflict is contextual only, it's fine to put the new __warn_thunk line
anywhere in entry.S, as long as it's outside the other section blocks.
Putting it last in the file is the most straightforward resolution.

Thanks,

Ingo

------------------>
Arnd Bergmann (1):
x86/alternatives: Move apply_relocation() out of init section

Breno Leitao (10):
x86/bugs: Rename CONFIG_GDS_FORCE_MITIGATION => CONFIG_MITIGATION_GDS_FORCE
x86/bugs: Rename CONFIG_CPU_IBPB_ENTRY => CONFIG_MITIGATION_IBPB_ENTRY
x86/bugs: Rename CONFIG_CALL_DEPTH_TRACKING => CONFIG_MITIGATION_CALL_DEPTH_TRACKING
x86/bugs: Rename CONFIG_PAGE_TABLE_ISOLATION => CONFIG_MITIGATION_PAGE_TABLE_ISOLATION
x86/bugs: Rename CONFIG_RETPOLINE => CONFIG_MITIGATION_RETPOLINE
x86/bugs: Rename CONFIG_SLS => CONFIG_MITIGATION_SLS
x86/bugs: Rename CONFIG_CPU_UNRET_ENTRY => CONFIG_MITIGATION_UNRET_ENTRY
x86/bugs: Rename CONFIG_CPU_IBRS_ENTRY => CONFIG_MITIGATION_IBRS_ENTRY
x86/bugs: Rename CONFIG_CPU_SRSO => CONFIG_MITIGATION_SRSO
x86/bugs: Rename CONFIG_RETHUNK => CONFIG_MITIGATION_RETHUNK

Daniel Micay (1):
x86/vdso: Move vDSO to mmap region

Ingo Molnar (2):
x86/callthunks: Mark apply_relocation() as __init_or_module
x86/vdso/kbuild: Group non-standard build attributes and primary object file rules together

Josh Poimboeuf (2):
x86/retpoline: Ensure default return thunk isn't used at runtime
x86/vdso: Fix rethunk patching for vdso-image-{32,64}.o

Kees Cook (1):
x86: Increase brk randomness entropy for 64-bit systems

Linus Torvalds (1):
x86/fpu: Clean up FPU switching in the middle of task switching

Masahiro Yamada (4):
x86/vdso: Consolidate targets and clean-files
x86/vdso: Simplify obj-y addition
x86/vdso: Use $(addprefix ) instead of $(foreach )
x86/vdso: Use CONFIG_COMPAT_32 to specify vdso32

Nadav Amit (1):
x86/percpu: Use compiler segment prefix qualifier

Thomas Gleixner (6):
sched/idle: Conditionally handle tick broadcast in default_idle_call()
x86/idle: Sanitize X86_BUG_AMD_E400 handling
x86/idle: Clean up idle selection
x86/idle: Cleanup idle_setup()
x86/idle: Let prefer_mwait_c1_over_halt() return bool
x86/idle: Select idle routine only once

Uros Bizjak (23):
x86/percpu: Enable named address spaces with known compiler version
x86/percpu: Use C for percpu read/write accessors
x86/percpu: Disable named address spaces for KASAN
x86/percpu: Rewrite arch_raw_cpu_ptr() to be easier for compilers to optimize
x86/percpu: Use C for arch_raw_cpu_ptr(), to improve code generation
x86/percpu: Use the correct asm operand modifier in percpu_stable_op()
x86/percpu: Correct PER_CPU_VAR() usage to include symbol and its addend
x86/percpu, xen: Correct PER_CPU_VAR() usage to include symbol and its addend
x86/percpu: Introduce %rip-relative addressing to PER_CPU_VAR()
x86/percpu: Introduce const-qualified const_pcpu_hot to micro-optimize code generation
x86/percpu: Return correct variable from current_top_of_stack()
x86/percpu: Define PER_CPU_VAR macro also for !__ASSEMBLY__
x86/callthunks: Handle %rip-relative relocations in call thunk template
x86/callthunks: Fix and unify call thunks assembly snippets
x86/acpi: Use %rip-relative addressing in wakeup_64.S
x86/percpu: Declare const_pcpu_hot as extern const variable
x86/smp: Move the call to smp_processor_id() after the early exit in native_stop_other_cpus()
x86/smp: Use atomic_try_cmpxchg in native_stop_other_cpus()
x86/callthunks: Correct calculation of dest address in is_callthunk()
x86/percpu: Fix "const_pcpu_hot" version generation failure
x86/traps: Use current_top_of_stack() helper in traps.c
x86/percpu: Avoid sparse warning with cast to named address space
x86/percpu: Use %RIP-relative address in untagged_addr()


Documentation/admin-guide/hw-vuln/spectre.rst | 8 +-
Documentation/admin-guide/kernel-parameters.txt | 4 +-
Documentation/arch/x86/pti.rst | 6 +-
arch/x86/Kconfig | 45 ++++--
arch/x86/Makefile | 11 +-
arch/x86/boot/compressed/ident_map_64.c | 4 +-
arch/x86/configs/i386_defconfig | 2 +-
arch/x86/entry/calling.h | 74 ++++++++-
arch/x86/entry/entry.S | 4 +
arch/x86/entry/entry_32.S | 2 +-
arch/x86/entry/entry_64.S | 8 +-
arch/x86/entry/thunk_32.S | 34 +----
arch/x86/entry/thunk_64.S | 33 ----
arch/x86/entry/vdso/Makefile | 57 +++----
arch/x86/entry/vdso/vma.c | 57 +------
arch/x86/include/asm/current.h | 9 +-
arch/x86/include/asm/disabled-features.h | 10 +-
arch/x86/include/asm/elf.h | 1 -
arch/x86/include/asm/fpu/sched.h | 10 +-
arch/x86/include/asm/linkage.h | 16 +-
arch/x86/include/asm/nospec-branch.h | 55 +++----
arch/x86/include/asm/percpu.h | 191 ++++++++++++++++++------
arch/x86/include/asm/pgalloc.h | 2 +-
arch/x86/include/asm/pgtable-3level.h | 2 +-
arch/x86/include/asm/pgtable.h | 18 +--
arch/x86/include/asm/pgtable_64.h | 3 +-
arch/x86/include/asm/preempt.h | 2 +-
arch/x86/include/asm/processor-flags.h | 2 +-
arch/x86/include/asm/processor.h | 5 +-
arch/x86/include/asm/pti.h | 2 +-
arch/x86/include/asm/static_call.h | 2 +-
arch/x86/include/asm/text-patching.h | 2 +
arch/x86/include/asm/uaccess_64.h | 11 +-
arch/x86/kernel/acpi/wakeup_64.S | 24 +--
arch/x86/kernel/alternative.c | 23 ++-
arch/x86/kernel/asm-offsets.c | 2 +-
arch/x86/kernel/callthunks.c | 32 +++-
arch/x86/kernel/cpu/amd.c | 2 +-
arch/x86/kernel/cpu/bugs.c | 48 +++---
arch/x86/kernel/cpu/common.c | 5 +-
arch/x86/kernel/dumpstack.c | 2 +-
arch/x86/kernel/ftrace.c | 3 +-
arch/x86/kernel/head_32.S | 4 +-
arch/x86/kernel/head_64.S | 4 +-
arch/x86/kernel/kprobes/opt.c | 2 +-
arch/x86/kernel/ldt.c | 8 +-
arch/x86/kernel/process.c | 99 +++++-------
arch/x86/kernel/process_32.c | 7 +-
arch/x86/kernel/process_64.c | 7 +-
arch/x86/kernel/smp.c | 10 +-
arch/x86/kernel/static_call.c | 2 +-
arch/x86/kernel/sys_x86_64.c | 7 -
arch/x86/kernel/traps.c | 4 +-
arch/x86/kernel/vmlinux.lds.S | 11 +-
arch/x86/kvm/mmu/mmu.c | 2 +-
arch/x86/kvm/mmu/mmu_internal.h | 2 +-
arch/x86/kvm/svm/svm.c | 2 +-
arch/x86/kvm/svm/vmenter.S | 4 +-
arch/x86/kvm/vmx/vmx.c | 2 +-
arch/x86/lib/Makefile | 2 +-
arch/x86/lib/cmpxchg16b_emu.S | 12 +-
arch/x86/lib/cmpxchg8b_emu.S | 30 ++--
arch/x86/lib/retpoline.S | 41 +++--
arch/x86/mm/Makefile | 2 +-
arch/x86/mm/debug_pagetables.c | 4 +-
arch/x86/mm/dump_pagetables.c | 4 +-
arch/x86/mm/pgtable.c | 4 +-
arch/x86/mm/tlb.c | 10 +-
arch/x86/net/bpf_jit_comp.c | 4 +-
arch/x86/net/bpf_jit_comp32.c | 2 +-
arch/x86/purgatory/Makefile | 2 +-
arch/x86/xen/xen-asm.S | 10 +-
include/linux/compiler-gcc.h | 2 +-
include/linux/compiler.h | 2 +-
include/linux/cpu.h | 2 +
include/linux/indirect_call_wrapper.h | 2 +-
include/linux/module.h | 2 +-
include/linux/objtool.h | 2 +-
include/linux/pti.h | 2 +-
include/linux/tick.h | 3 +
include/net/netfilter/nf_tables_core.h | 2 +-
include/net/tc_wrapper.h | 2 +-
kernel/sched/idle.c | 21 +++
kernel/time/Kconfig | 5 +
kernel/trace/ring_buffer.c | 2 +-
net/netfilter/Makefile | 2 +-
net/netfilter/nf_tables_core.c | 6 +-
net/netfilter/nft_ct.c | 4 +-
net/netfilter/nft_lookup.c | 2 +-
net/sched/sch_api.c | 2 +-
scripts/Makefile.lib | 8 +-
scripts/Makefile.vmlinux_o | 2 +-
scripts/generate_rust_target.rs | 2 +-
scripts/mod/modpost.c | 2 +-
tools/arch/x86/include/asm/disabled-features.h | 10 +-
tools/objtool/arch/x86/special.c | 2 +-
tools/objtool/check.c | 4 +-
97 files changed, 669 insertions(+), 563 deletions(-)