arm64: kvm: Fix kvm teardown for systems using the extended idmap

If memory is located above 1<<VA_BITS, kvm adds an extra level to its page
tables, merging the runtime tables and boot tables that contain the idmap.
This lets us avoid the trampoline dance during initialisation.

This also means there is no trampoline page mapped, so
__cpu_reset_hyp_mode() can't call __kvm_hyp_reset() in this page. The good
news is the idmap is still mapped, so we don't need the trampoline page.
The bad news is we can't call it directly as the idmap is above
HYP_PAGE_OFFSET, so its address is masked by kvm_call_hyp.

Add a function __extended_idmap_trampoline which will branch into
__kvm_hyp_reset in the idmap, change kvm_hyp_reset_entry() to return
this address if __kvm_cpu_uses_extended_idmap(). In this case
__kvm_hyp_reset() will still switch to the boot tables (which are the
merged tables that were already in use), and branch into the idmap (where
it already was).

This fixes boot failures on these systems, where we fail to execute the
missing trampoline page when tearing down kvm in init_subsystems():
[    2.508922] kvm [1]: 8-bit VMID
[    2.512057] kvm [1]: Hyp mode initialized successfully
[    2.517242] kvm [1]: interrupt-controller@e1140000 IRQ13
[    2.522622] kvm [1]: timer IRQ3
[    2.525783] Kernel panic - not syncing: HYP panic:
[    2.525783] PS:200003c9 PC:0000007ffffff820 ESR:86000005
[    2.525783] FAR:0000007ffffff820 HPFAR:00000000003ffff0 PAR:0000000000000000
[    2.525783] VCPU:          (null)
[    2.525783]
[    2.547667] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W       4.6.0-rc5+ #1
[    2.555137] Hardware name: Default string Default string/Default string, BIOS ROD0084E 09/03/2015
[    2.563994] Call trace:
[    2.566432] [<ffffff80080888d0>] dump_backtrace+0x0/0x240
[    2.571818] [<ffffff8008088b24>] show_stack+0x14/0x20
[    2.576858] [<ffffff80083423ac>] dump_stack+0x94/0xb8
[    2.581899] [<ffffff8008152130>] panic+0x10c/0x250
[    2.586677] [<ffffff8008152024>] panic+0x0/0x250
[    2.591281] SMP: stopping secondary CPUs
[    3.649692] SMP: failed to stop secondary CPUs 0-2,4-7
[    3.654818] Kernel Offset: disabled
[    3.658293] Memory Limit: none
[    3.661337] ---[ end Kernel panic - not syncing: HYP panic:
[    3.661337] PS:200003c9 PC:0000007ffffff820 ESR:86000005
[    3.661337] FAR:0000007ffffff820 HPFAR:00000000003ffff0 PAR:0000000000000000
[    3.661337] VCPU:          (null)
[    3.661337]

Reported-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>

1 files changed, 23 insertions, 7 deletions

diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 4062e6dd4cc1..b1ad730e1567 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -135,12 +135,28 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 
 extern char __hyp_idmap_text_start[];
 
-phys_addr_t kvm_hyp_reset_entry(void)
+unsigned long kvm_hyp_reset_entry(void)
 {
-	unsigned long offset;
-
-	offset = (unsigned long)__kvm_hyp_reset
-		 - ((unsigned long)__hyp_idmap_text_start & PAGE_MASK);
-
-	return TRAMPOLINE_VA + offset;
+	if (!__kvm_cpu_uses_extended_idmap()) {
+		unsigned long offset;
+
+		/*
+		 * Find the address of __kvm_hyp_reset() in the trampoline page.
+		 * This is present in the running page tables, and the boot page
+		 * tables, so we call the code here to start the trampoline
+		 * dance in reverse.
+		 */
+		offset = (unsigned long)__kvm_hyp_reset
+			 - ((unsigned long)__hyp_idmap_text_start & PAGE_MASK);
+
+		return TRAMPOLINE_VA + offset;
+	} else {
+		/*
+		 * KVM is running with merged page tables, which don't have the
+		 * trampoline page mapped. We know the idmap is still mapped,
+		 * but can't be called into directly. Use
+		 * __extended_idmap_trampoline to do the call.
+		 */
+		return (unsigned long)kvm_ksym_ref(__extended_idmap_trampoline);
+	}
 }