diff options
| author | Ard Biesheuvel <ardb@kernel.org> | 2022-07-01 13:10:45 +0200 |
|---|---|---|
| committer | Will Deacon <will@kernel.org> | 2022-07-01 15:19:07 +0100 |
| commit | 0aaa68532e9da5cd6b1383e1535a5526253e359f (patch) | |
| tree | 25ed4fb29514509bdb8e05458c302e05a8c4840f /arch/arm64/mm | |
| parent | bdbcd22d491212c266589892f0818c65a2bc4704 (diff) | |
arm64: mm: fix booting with 52-bit address space
Joey reports that booting 52-bit VA capable builds on 52-bit VA capable
CPUs is broken since commit 0d9b1ffefabe ("arm64: mm: make vabits_actual
a build time constant if possible"). This is due to the fact that the
primary CPU reads the vabits_actual variable before it has been
assigned.
The reason for deferring the assignment of vabits_actual was that we try
to perform as few stores to memory as we can with the MMU and caches
off, due to the cache coherency issues it creates.
Since __cpu_setup() [which is where the read of vabits_actual occurs] is
also called on the secondary boot path, we cannot just read the CPU ID
registers directly, given that the size of the VA space is decided by
the capabilities of the primary CPU. So let's read vabits_actual only on
the secondary boot path, and read the CPU ID registers directly on the
primary boot path, by making it a function parameter of __cpu_setup().
To ensure that all users of vabits_actual (including kasan_early_init())
observe the correct value, move the assignment of vabits_actual back
into asm code, but still defer it to after the MMU and caches have been
enabled.
Cc: Will Deacon <will@kernel.org>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Fixes: 0d9b1ffefabe ("arm64: mm: make vabits_actual a build time constant if possible")
Reported-by: Joey Gouly <joey.gouly@arm.com>
Co-developed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20220701111045.2944309-1-ardb@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
Diffstat (limited to 'arch/arm64/mm')
| -rw-r--r-- | arch/arm64/mm/init.c | 15 | ||||
| -rw-r--r-- | arch/arm64/mm/proc.S | 5 |
2 files changed, 4 insertions, 16 deletions
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 1faa6760895e..339ee84e5a61 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -265,20 +265,7 @@ early_param("mem", early_mem); void __init arm64_memblock_init(void) { - s64 linear_region_size; - -#if VA_BITS > 48 - if (cpuid_feature_extract_unsigned_field( - read_sysreg_s(SYS_ID_AA64MMFR2_EL1), - ID_AA64MMFR2_LVA_SHIFT)) - vabits_actual = VA_BITS; - - /* make the variable visible to secondaries with the MMU off */ - dcache_clean_inval_poc((u64)&vabits_actual, - (u64)&vabits_actual + sizeof(vabits_actual)); -#endif - - linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual); + s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual); /* * Corner case: 52-bit VA capable systems running KVM in nVHE mode may diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 605c6640f94b..9eb490effb7f 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -397,6 +397,8 @@ SYM_FUNC_END(idmap_kpti_install_ng_mappings) * * Initialise the processor for turning the MMU on. * + * Input: + * x0 - actual number of VA bits (ignored unless VA_BITS > 48) * Output: * Return in x0 the value of the SCTLR_EL1 register. */ @@ -466,8 +468,7 @@ SYM_FUNC_START(__cpu_setup) tcr_clear_errata_bits tcr, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52 - ldr_l x9, vabits_actual - sub x9, xzr, x9 + sub x9, xzr, x0 add x9, x9, #64 tcr_set_t1sz tcr, x9 #else |