summaryrefslogtreecommitdiff
path: root/arch/riscv
AgeCommit message (Collapse)Author
2023-02-17riscv, bpf: Add bpf trampoline support for RV64Pu Lehui
BPF trampoline is the critical infrastructure of the BPF subsystem, acting as a mediator between kernel functions and BPF programs. Numerous important features, such as using BPF program for zero overhead kernel introspection, rely on this key component. We can't wait to support bpf trampoline on RV64. The related tests have passed, as well as the test_verifier with no new failure ceses. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Björn Töpel <bjorn@rivosinc.com> Acked-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/bpf/20230215135205.1411105-5-pulehui@huaweicloud.com
2023-02-17riscv, bpf: Add bpf_arch_text_poke support for RV64Pu Lehui
Implement bpf_arch_text_poke for RV64. For call scenario, to make BPF trampoline compatible with the kernel and BPF context, we follow the framework of RV64 ftrace to reserve 4 nops for BPF programs as function entry, and use auipc+jalr instructions for function call. However, since auipc+jalr call instruction is non-atomic operation, we need to use stop-machine to make sure instructions patching in atomic context. Also, we use auipc+jalr pair and need to patch in stop-machine context for jump scenario. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Björn Töpel <bjorn@rivosinc.com> Acked-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/bpf/20230215135205.1411105-4-pulehui@huaweicloud.com
2023-02-17riscv, bpf: Factor out emit_call for kernel and bpf contextPu Lehui
The current emit_call function is not suitable for kernel function call as it store return value to bpf R0 register. We can separate it out for common use. Meanwhile, simplify judgment logic, that is, fixed function address can use jal or auipc+jalr, while the unfixed can use only auipc+jalr. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Björn Töpel <bjorn@rivosinc.com> Acked-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/bpf/20230215135205.1411105-3-pulehui@huaweicloud.com
2023-02-17riscv: Extend patch_text for multiple instructionsPu Lehui
Extend patch_text for multiple instructions. This is the preparaiton for multiple instructions text patching in riscv BPF trampoline, and may be useful for other scenario. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Björn Töpel <bjorn@rivosinc.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Acked-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/bpf/20230215135205.1411105-2-pulehui@huaweicloud.com
2023-02-15RISC-V: move some stray __RISCV_INSN_FUNCS definitions from kprobesHeiko Stuebner
The __RISCV_INSN_FUNCS originally declared riscv_insn_is_* functions inside the kprobes implementation. This got moved into a central header in commit ec5f90877516 ("RISC-V: Move riscv_insn_is_* macros into a common header"). Though it looks like I overlooked two of them, so fix that. FENCE itself is an instruction defined directly by its own opcode, while the created riscv_isn_is_system function covers all instructions defined under the SYSTEM opcode. Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Link: https://lore.kernel.org/r/20230113211955.3534431-1-heiko@sntech.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15Merge patch series "riscv: Optimize function trace"Palmer Dabbelt
guoren@kernel.org <guoren@kernel.org> says: From: Guo Ren <guoren@linux.alibaba.com> The previous ftrace detour implementation fc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") contain three problems. - The most horrible bug is preemption panic which found by Andy [1]. Let's disable preemption for ftrace first, and Andy could continue the ftrace preemption work. - The "-fpatchable-function-entry= CFLAG" wasted code size !RISCV_ISA_C. - The ftrace detour implementation wasted code size. - When livepatching, the trampoline (ftrace_regs_caller) would not return to <func_prolog+12> but would rather jump to the new function. So, "REG_L ra, -SZREG(sp)" would not run and the original return address would not be restored. The kernel is likely to hang or crash as a result. (Found by Evgenii Shatokhin [4]) [Palmer: The first three patches in this series are pretty concrete fixes, so I'm pulling them ahead of the rest of the series.] * b4-shazam-merge: riscv: ftrace: Reduce the detour code size to half riscv: ftrace: Remove wasted nops for !RISCV_ISA_C riscv: ftrace: Fixup panic by disabling preemption Link: https://lore.kernel.org/r/20230112090603.1295340-1-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15riscv: ftrace: Reduce the detour code size to halfGuo Ren
Use a temporary register to reduce the size of detour code from 16 bytes to 8 bytes. The previous implementation is from 'commit afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT")'. Before the patch: <func_prolog>: 0: REG_S ra, -SZREG(sp) 4: auipc ra, ? 8: jalr ?(ra) 12: REG_L ra, -SZREG(sp) (func_boddy) After the patch: <func_prolog>: 0: auipc t0, ? 4: jalr t0, ?(t0) (func_boddy) This patch not just reduces the size of detour code, but also fixes an important issue: An Ftrace callback registered with FTRACE_OPS_FL_IPMODIFY flag can actually change the instruction pointer, e.g. to "replace" the given kernel function with a new one, which is needed for livepatching, etc. In this case, the trampoline (ftrace_regs_caller) would not return to <func_prolog+12> but would rather jump to the new function. So, "REG_L ra, -SZREG(sp)" would not run and the original return address would not be restored. The kernel is likely to hang or crash as a result. This can be easily demonstrated if one tries to "replace", say, cmdline_proc_show() with a new function with the same signature using instruction_pointer_set(&fregs->regs, new_func_addr) in the Ftrace callback. Link: https://lore.kernel.org/linux-riscv/20221122075440.1165172-1-suagrfillet@gmail.com/ Link: https://lore.kernel.org/linux-riscv/d7d5730b-ebef-68e5-5046-e763e1ee6164@yadro.com/ Co-developed-by: Song Shuai <suagrfillet@gmail.com> Signed-off-by: Song Shuai <suagrfillet@gmail.com> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Evgenii Shatokhin <e.shatokhin@yadro.com> Reviewed-by: Evgenii Shatokhin <e.shatokhin@yadro.com> Link: https://lore.kernel.org/r/20230112090603.1295340-4-guoren@kernel.org Cc: stable@vger.kernel.org Fixes: 10626c32e382 ("riscv/ftrace: Add basic support") Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15riscv: ftrace: Remove wasted nops for !RISCV_ISA_CGuo Ren
When CONFIG_RISCV_ISA_C=n, -fpatchable-function-entry=8 would generate more nops than we expect. Because it treat nop opcode as 0x00000013 instead of 0x0001. Dump of assembler code for function dw_pcie_free_msi: 0xffffffff806fce94 <+0>: sd ra,-8(sp) 0xffffffff806fce98 <+4>: auipc ra,0xff90f 0xffffffff806fce9c <+8>: jalr -684(ra) # 0xffffffff8000bbec <ftrace_caller> 0xffffffff806fcea0 <+12>: ld ra,-8(sp) 0xffffffff806fcea4 <+16>: nop /* wasted */ 0xffffffff806fcea8 <+20>: nop /* wasted */ 0xffffffff806fceac <+24>: nop /* wasted */ 0xffffffff806fceb0 <+28>: nop /* wasted */ 0xffffffff806fceb4 <+0>: addi sp,sp,-48 0xffffffff806fceb8 <+4>: sd s0,32(sp) 0xffffffff806fcebc <+8>: sd s1,24(sp) 0xffffffff806fcec0 <+12>: sd s2,16(sp) 0xffffffff806fcec4 <+16>: sd s3,8(sp) 0xffffffff806fcec8 <+20>: sd ra,40(sp) 0xffffffff806fcecc <+24>: addi s0,sp,48 Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230112090603.1295340-3-guoren@kernel.org Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15riscv: ftrace: Fixup panic by disabling preemptionAndy Chiu
In RISCV, we must use an AUIPC + JALR pair to encode an immediate, forming a jump that jumps to an address over 4K. This may cause errors if we want to enable kernel preemption and remove dependency from patching code with stop_machine(). For example, if a task was switched out on auipc. And, if we changed the ftrace function before it was switched back, then it would jump to an address that has updated 11:0 bits mixing with previous XLEN:12 part. p: patched area performed by dynamic ftrace ftrace_prologue: p| REG_S ra, -SZREG(sp) p| auipc ra, 0x? ------------> preempted ... change ftrace function ... p| jalr -?(ra) <------------- switched back p| REG_L ra, -SZREG(sp) func: xxx ret Fixes: afc76b8b8011 ("riscv: Using PATCHABLE_FUNCTION_ENTRY instead of MCOUNT") Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230112090603.1295340-2-guoren@kernel.org Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15Merge tag 'kvm-riscv-6.3-1' of https://github.com/kvm-riscv/linux into HEADPaolo Bonzini
KVM/riscv changes for 6.3 - Fix wrong usage of PGDIR_SIZE to check page sizes - Fix privilege mode setting in kvm_riscv_vcpu_trap_redirect() - Redirect illegal instruction traps to guest - SBI PMU support for guest
2023-02-14Merge patch series "Remove toolchain dependencies for Zicbom"Palmer Dabbelt
Conor Dooley <conor@kernel.org> says: From: Conor Dooley <conor.dooley@microchip.com> I've yoinked patch 1 from Drew's series adding support for Zicboz & attached two more patches here that remove the need for, and then drop the toolchain support checks for Zicbom. The goal is to remove the need for checking the presence of toolchain Zicbom support in the work being done to support non instruction based CMOs [1]. I've tested compliation on a number of different configurations with the Zicbom config option enabled. The important ones to call out I guess are: - clang/llvm 14 w/ LLVM=1 which doesn't support Zicbom atm. - gcc 11 w/ binutils 2.37 which doesn't support Zicbom atm either. - clang/llvm 15 w/ LLVM=1 BUT with binutils 2.37's ld. This is the configuration that prompted adding the LD checks as cc/as supports Zicbom, but ld doesn't [2]. - gcc 12 w/ binutils 2.39 & clang 15 w/ LLVM=1, both of these supported Zicbom before and still do. I also checked building the THEAD errata etc with CONFIG_RISCV_ISA_ZICBOM disabled, and there were no build issues there either. * b4-shazam-merge: RISC-V: remove toolchain version checks for Zicbom RISC-V: replace cbom instructions with an insn-def RISC-V: insn-def: Add I-type insn-def Link: https://lore.kernel.org/r/20230108163356.3063839-1-conor@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14RISC-V: remove toolchain version checks for ZicbomConor Dooley
Commit b8c86872d1dc ("riscv: fix detection of toolchain Zicbom support") fixed building on systems where Zicbom was supported by the compiler/assembler but not by the linker in an easily backportable manner. Now that the we have insn-defs for the 3 instructions, toolchain support is no longer required for Zicbom. Stop emitting "_zicbom" in -march when Zicbom is enabled & drop the version checks entirely. Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20230108163356.3063839-4-conor@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14RISC-V: replace cbom instructions with an insn-defConor Dooley
Using the cbom instructions directly in ALT_CMO_OP, requires toolchain support for the instructions. Using an insn-def will allow for removal of toolchain version checks in the build system & simplification of the proposed [1] function-based CMO scheme. Link: https://lore.kernel.org/linux-riscv/fb3b34ae-e35e-4dc2-a8f4-19984a2f58a8@app.fastmail.com/ [1] Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20230108163356.3063839-3-conor@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14RISC-V: insn-def: Add I-type insn-defAndrew Jones
CBO instructions use the I-type of instruction format where the immediate is used to identify the CBO instruction type. Add I-type instruction encoding support to insn-def. Signed-off-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Heiko Stuebner <heiko@sntech.de> Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20230108163356.3063839-2-conor@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14Merge patch series "dt-bindings: Add a cpu-capacity property for RISC-V"Palmer Dabbelt
Conor Dooley <conor@kernel.org> says: From: Conor Dooley <conor.dooley@microchip.com> Ever since RISC-V starting using generic arch topology code, the code paths for cpu-capacity have been there but there's no binding defined to actually convey the information. Defining the same property as used on arm seems to be the only logical thing to do, so do it. [Palmer: This is on top of the fix required to make it work, which itself wasn't merged until late in the 6.2 cycle and thus pulls in various other fixes.] * b4-shazam-merge: dt-bindings: riscv: add a capacity-dmips-mhz cpu property dt-bindings: arm: move cpu-capacity to a shared loation riscv: Move call to init_cpu_topology() to later initialization stage riscv/kprobe: Fix instruction simulation of JALR riscv: fix -Wundef warning for CONFIG_RISCV_BOOT_SPINWAIT MAINTAINERS: add an IRC entry for RISC-V RISC-V: fix compile error from deduplicated __ALTERNATIVE_CFG_2 dt-bindings: riscv: fix single letter canonical order dt-bindings: riscv: fix underscore requirement for multi-letter extensions riscv: uaccess: fix type of 0 variable on error in get_user() riscv, kprobes: Stricter c.jr/c.jalr decoding Link: https://lore.kernel.org/r/20230104180513.1379453-1-conor@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14riscv: Fix Zbb alternative IDsSamuel Holland
Commit 4bf8860760d9 ("riscv: cpufeature: extend riscv_cpufeature_patch_func to all ISA extensions") switched ISA extension alternatives to use the RISCV_ISA_EXT_* macros instead of CPUFEATURE_*. This was mismerged when applied on top of the Zbb series, so the Zbb alternatives referenced the wrong errata ID values. Fixes: 9daca9a5b9ac ("Merge patch series "riscv: improve boot time isa extensions handling"") Signed-off-by: Samuel Holland <samuel@sholland.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Guo Ren <guoren@kernel.org> Tested-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230212021534.59121-3-samuel@sholland.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14riscv: Fix early alternative patchingSamuel Holland
Now that the text to patch is located using a relative offset from the alternative entry, the text address should be computed without applying the kernel mapping offset, both before and after VM setup. Fixes: 8d23e94a4433 ("riscv: switch to relative alternative entries") Signed-off-by: Samuel Holland <samuel@sholland.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Jisheng Zhang <jszhang@kernel.org> Tested-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230212021534.59121-2-samuel@sholland.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-14RISC-V: re-order Kconfig selects alphanumericallyConor Dooley
Selects should be sorted alphanumerically, and were tidied up originally by Palmer in commit e8c7ef7d5819 ("RISC-V: Sort select statements alphanumerically") since then, things have gotten out of order again. Fish RMK's original script out of commit b1b3f49ce460 ("ARM: config: sort select statements alphanumerically") and do some spring cleaning. Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Acked-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20221219172836.134709-1-conor@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-13clocksource/drivers/riscv: Get rid of clocksource_arch_init() callbackLad Prabhakar
Having a clocksource_arch_init() callback always sets vdso_clock_mode to VDSO_CLOCKMODE_ARCHTIMER if GENERIC_GETTIMEOFDAY is enabled, this is required for the riscv-timer. This works for platforms where just riscv-timer clocksource is present. On platforms where other clock sources are available we want them to register with vdso_clock_mode set to VDSO_CLOCKMODE_NONE. On the Renesas RZ/Five SoC OSTM block can be used as clocksource [0], to avoid multiple clock sources being registered as VDSO_CLOCKMODE_ARCHTIMER move setting of vdso_clock_mode in the riscv-timer driver instead of doing this in clocksource_arch_init() callback as done similarly for ARM/64 architecture. [0] drivers/clocksource/renesas-ostm.c Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> Tested-by: Samuel Holland <samuel@sholland.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Samuel Holland <samuel@sholland.org> Link: https://lore.kernel.org/r/20221229224601.103851-1-prabhakar.mahadev-lad.rj@bp.renesas.com Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2023-02-13RISC-V: time: initialize hrtimer based broadcast clock event deviceConor Dooley
Similarly to commit 022eb8ae8b5e ("ARM: 8938/1: kernel: initialize broadcast hrtimer based clock event device"), RISC-V needs to initiate hrtimer based broadcast clock event device before C3STOP can be used. Otherwise, the introduction of C3STOP for the RISC-V arch timer in commit 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend") leaves us without any broadcast timer registered. This prevents the kernel from entering oneshot mode, which breaks timer behaviour, for example clock_nanosleep(). A test app that sleeps each cpu for 6, 5, 4, 3 ms respectively, HZ=250 & C3STOP enabled, the sleep times are rounded up to the next jiffy: == CPU: 1 == == CPU: 2 == == CPU: 3 == == CPU: 4 == Mean: 7.974992 Mean: 7.976534 Mean: 7.962591 Mean: 3.952179 Std Dev: 0.154374 Std Dev: 0.156082 Std Dev: 0.171018 Std Dev: 0.076193 Hi: 9.472000 Hi: 10.495000 Hi: 8.864000 Hi: 4.736000 Lo: 6.087000 Lo: 6.380000 Lo: 4.872000 Lo: 3.403000 Samples: 521 Samples: 521 Samples: 521 Samples: 521 Link: https://lore.kernel.org/linux-riscv/YzYTNQRxLr7Q9JR0@spud/ Fixes: 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend") Suggested-by: Samuel Holland <samuel@sholland.org> Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Samuel Holland <samuel@sholland.org> Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230103141102.772228-2-apatel@ventanamicro.com Signed-off-by: Daniel Lezcano <daniel.lezcano@kernel.org>
2023-02-10Merge tag 'riscv-for-linus-6.2-rc8' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux Pull RISC-V fixes from Palmer Dabbelt: "This is a little bigger that I'd hope for this late in the cycle, but they're all pretty concrete fixes and the only one that's bigger than a few lines is pmdp_collapse_flush() (which is almost all boilerplate/comment). It's also all bug fixes for issues that have been around for a while. So I think it's not all that scary, just bad timing. - avoid partial TLB fences for huge pages, which are disallowed by the ISA - avoid missing a frame when dumping stacks - avoid misaligned accesses (and possibly overflows) in kprobes - fix a race condition in tracking page dirtiness" * tag 'riscv-for-linus-6.2-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux: riscv: Fixup race condition on PG_dcache_clean in flush_icache_pte riscv: kprobe: Fixup misaligned load text riscv: stacktrace: Fix missing the first frame riscv: mm: Implement pmdp_collapse_flush for THP
2023-02-09mm, arch: add generic implementation of pfn_valid() for FLATMEMMike Rapoport (IBM)
Every architecture that supports FLATMEM memory model defines its own version of pfn_valid() that essentially compares a pfn to max_mapnr. Use mips/powerpc version implemented as static inline as a generic implementation of pfn_valid() and drop its per-architecture definitions. [rppt@kernel.org: fix the generic pfn_valid()] Link: https://lkml.kernel.org/r/Y9lg7R1Yd931C+y5@kernel.org Link: https://lkml.kernel.org/r/20230129124235.209895-5-rppt@kernel.org Signed-off-by: Mike Rapoport (IBM) <rppt@kernel.org> Acked-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Guo Ren <guoren@kernel.org> [csky] Acked-by: Huacai Chen <chenhuacai@loongson.cn> [LoongArch] Acked-by: Stafford Horne <shorne@gmail.com> [OpenRISC] Acked-by: Michael Ellerman <mpe@ellerman.id.au> [powerpc] Reviewed-by: David Hildenbrand <david@redhat.com> Tested-by: Conor Dooley <conor.dooley@microchip.com> Cc: Brian Cain <bcain@quicinc.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Dinh Nguyen <dinguyen@kernel.org> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Greg Ungerer <gerg@linux-m68k.org> Cc: Helge Deller <deller@gmx.de> Cc: Huacai Chen <chenhuacai@kernel.org> Cc: Matt Turner <mattst88@gmail.com> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: Michal Simek <monstr@monstr.eu> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Richard Weinberger <richard@nod.at> Cc: Rich Felker <dalias@libc.org> Cc: Russell King <linux@armlinux.org.uk> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Vineet Gupta <vgupta@kernel.org> Cc: WANG Xuerui <kernel@xen0n.name> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-02-09riscv: Fixup race condition on PG_dcache_clean in flush_icache_pteGuo Ren
In commit 588a513d3425 ("arm64: Fix race condition on PG_dcache_clean in __sync_icache_dcache()"), we found RISC-V has the same issue as the previous arm64. The previous implementation didn't guarantee the correct sequence of operations, which means flush_icache_all() hasn't been called when the PG_dcache_clean was set. That would cause a risk of page synchronization. Fixes: 08f051eda33b ("RISC-V: Flush I$ when making a dirty page executable") Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230127035306.1819561-1-guoren@kernel.org Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-09riscv: kprobe: Fixup misaligned load textGuo Ren
The current kprobe would cause a misaligned load for the probe point. This patch fixup it with two half-word loads instead. Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported") Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/linux-riscv/878rhig9zj.fsf@all.your.base.are.belong.to.us/ Reported-by: Bjorn Topel <bjorn.topel@gmail.com> Reviewed-by: Björn Töpel <bjorn@kernel.org> Link: https://lore.kernel.org/r/20230204063531.740220-1-guoren@kernel.org Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-07RISC-V: KVM: Increment firmware pmu eventsAtish Patra
KVM supports firmware events now. Invoke the firmware event increment function from appropriate places. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Support firmware eventsAtish Patra
SBI PMU extension defines a set of firmware events which can provide useful information to guests about the number of SBI calls. As hypervisor implements the SBI PMU extension, these firmware events correspond to ecall invocations between VS->HS mode. All other firmware events will always report zero if monitored as KVM doesn't implement them. This patch adds all the infrastructure required to support firmware events. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Implement perf support without samplingAtish Patra
RISC-V SBI PMU & Sscofpmf ISA extension allows supporting perf in the virtualization enviornment as well. KVM implementation relies on SBI PMU extension for the most part while trapping & emulating the CSRs read for counter access. This patch doesn't have the event sampling support yet. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Implement trap & emulate for hpmcountersAtish Patra
As the KVM guests only see the virtual PMU counters, all hpmcounter access should trap and KVM emulates the read access on behalf of guests. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Disable all hpmcounter access for VS/VU modeAtish Patra
Any guest must not get access to any hpmcounter including cycle/instret without any checks. We achieve that by disabling all the bits except TM bit in hcounteren. However, instret and cycle access for guest user space can be enabled upon explicit request (via ONE REG) or on first trap from VU mode to maintain ABI requirement in the future. This patch doesn't support that as ONE REG interface is not settled yet. Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Make PMU functionality depend on SscofpmfAtish Patra
The privilege mode filtering feature must be available in the host so that the host can inhibit the counters while the execution is in HS mode. Otherwise, the guests may have access to critical guest information. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Add SBI PMU extension supportAtish Patra
SBI PMU extension allows KVM guests to configure/start/stop/query about the PMU counters in virtualized enviornment as well. In order to allow that, KVM implements the entire SBI PMU extension. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Add skeleton support for perfAtish Patra
This patch only adds barebone structure of perf implementation. Most of the function returns zero at this point and will be implemented fully in the future. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Modify SBI extension handler to return SBI error codeAtish Patra
Currently, the SBI extension handle is expected to return Linux error code. The top SBI layer converts the Linux error code to SBI specific error code that can be returned to guest invoking the SBI calls. This model works as long as SBI error codes have 1-to-1 mappings between them. However, that may not be true always. This patch attempts to disassociate both these error codes by allowing the SBI extension implementation to return SBI specific error codes as well. The extension will continue to return the Linux error specific code which will indicate any problem *with* the extension emulation while the SBI specific error will indicate the problem *of* the emulation. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Suggested-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Return correct code for hsm stop functionAtish Patra
According to the SBI specification, the stop function can only return error code SBI_ERR_FAILED. However, currently it returns -EINVAL which will be mapped SBI_ERR_INVALID_PARAM. Return an linux error code that maps to SBI_ERR_FAILED i.e doesn't map to any other SBI error code. While EACCES is not the best error code to describe the situation, it is close enough and will be replaced with SBI error codes directly anyways. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Define a probe function for SBI extension data structuresAtish Patra
Currently the probe function just checks if an SBI extension is registered or not. However, the extension may not want to advertise itself depending on some other condition. An additional extension specific probe function will allow extensions to decide if they want to be advertised to the caller or not. Any extension that does not require additional dependency checks can avoid implementing this function. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: Improve SBI PMU extension related definitionsAtish Patra
This patch fixes/improve few minor things in SBI PMU extension definition. 1. Align all the firmware event names. 2. Add macros for bit positions in cache event ID & ops. The changes were small enough to combine them together instead of creating 1 liner patches. Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Redirect illegal instruction traps to guestAndy Chiu
The M-mode redirects an unhandled illegal instruction trap back to S-mode. However, KVM running in HS-mode terminates the VS-mode software when it receives illegal instruction trap. Instead, KVM should redirect the illegal instruction trap back to VS-mode, and let VS-mode trap handler decide the next step. This futher allows guest kernel to implement on-demand enabling of vector extension for a guest user space process upon first-use. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07RISC-V: KVM: Fix privilege mode setting in kvm_riscv_vcpu_trap_redirect()Anup Patel
The kvm_riscv_vcpu_trap_redirect() should set guest privilege mode to supervisor mode because guest traps/interrupts are always handled in virtual supervisor mode. Fixes: 9f7013265112 ("RISC-V: KVM: Handle MMIO exits for VCPU") Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-07KVM: RISC-V: Fix wrong usage of PGDIR_SIZE to check page sizesAlexandre Ghiti
At the moment, riscv only supports PMD and PUD hugepages. For sv39, PGDIR_SIZE == PUD_SIZE but not for sv48 and sv57. So fix this by changing PGDIR_SIZE into PUD_SIZE. Fixes: 9d05c1fee837 ("RISC-V: KVM: Implement stage2 page table programming") Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Anup Patel <anup@brainfault.org>
2023-02-06Merge 6.2-rc7 into tty-nextGreg Kroah-Hartman
We need the tty/serial fixes in here as well. Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-02-04efi: Discover BTI support in runtime services regionsArd Biesheuvel
Add the generic plumbing to detect whether or not the runtime code regions were constructed with BTI/IBT landing pads by the firmware, permitting the OS to enable enforcement when mapping these regions into the OS's address space. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Kees Cook <keescook@chromium.org>
2023-02-03Merge tag 'riscv-for-linus-6.2-rc7' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux Pull RISC-V fixes from Palmer Dabbelt: - A build fix to avoid static branches in cpu_relax(), which greatly inflates the jump tables and breaks at least CONFIG_CC_OPTIMIZE_FOR_SIZE=y. - A fix for a kernel panic when probing impossible instruction positions. - A fix to disable unwind tables, which are enabled by default for GCC-13 and result in unhandled relocations in modules. * tag 'riscv-for-linus-6.2-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux: riscv: disable generation of unwind tables riscv: kprobe: Fixup kernel panic when probing an illegal position riscv: Fix build with CONFIG_CC_OPTIMIZE_FOR_SIZE=y
2023-02-02mm: remove __HAVE_ARCH_PTE_SWP_EXCLUSIVEDavid Hildenbrand
__HAVE_ARCH_PTE_SWP_EXCLUSIVE is now supported by all architectures that support swp PTEs, so let's drop it. Link: https://lkml.kernel.org/r/20230113171026.582290-27-david@redhat.com Signed-off-by: David Hildenbrand <david@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-02-02riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVEDavid Hildenbrand
Let's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the offset. This reduces the maximum swap space per file: on 32bit to 16 GiB (was 32 GiB). Note that this bit does not conflict with swap PMDs and could also be used in swap PMD context later. While at it, mask the type in __swp_entry(). Link: https://lkml.kernel.org/r/20230113171026.582290-20-david@redhat.com Signed-off-by: David Hildenbrand <david@redhat.com> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-02-02riscv: stacktrace: Fix missing the first frameLiu Shixin
When running kfence_test, I found some testcases failed like this: # test_out_of_bounds_read: EXPECTATION FAILED at mm/kfence/kfence_test.c:346 Expected report_matches(&expect) to be true, but is false not ok 1 - test_out_of_bounds_read The corresponding call-trace is: BUG: KFENCE: out-of-bounds read in kunit_try_run_case+0x38/0x84 Out-of-bounds read at 0x(____ptrval____) (32B right of kfence-#10): kunit_try_run_case+0x38/0x84 kunit_generic_run_threadfn_adapter+0x12/0x1e kthread+0xc8/0xde ret_from_exception+0x0/0xc The kfence_test using the first frame of call trace to check whether the testcase is succeed or not. Commit 6a00ef449370 ("riscv: eliminate unreliable __builtin_frame_address(1)") skip first frame for all case, which results the kfence_test failed. Indeed, we only need to skip the first frame for case (task==NULL || task==current). With this patch, the call-trace will be: BUG: KFENCE: out-of-bounds read in test_out_of_bounds_read+0x88/0x19e Out-of-bounds read at 0x(____ptrval____) (1B left of kfence-#7): test_out_of_bounds_read+0x88/0x19e kunit_try_run_case+0x38/0x84 kunit_generic_run_threadfn_adapter+0x12/0x1e kthread+0xc8/0xde ret_from_exception+0x0/0xc Fixes: 6a00ef449370 ("riscv: eliminate unreliable __builtin_frame_address(1)") Signed-off-by: Liu Shixin <liushixin2@huawei.com> Tested-by: Samuel Holland <samuel@sholland.org> Link: https://lore.kernel.org/r/20221207025038.1022045-1-liushixin2@huawei.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-01riscv: mm: Implement pmdp_collapse_flush for THPMayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic pmdp_collapse_flush() which will further use flush_tlb_range() to shoot-down stale TLB entries. Unfortunately, the generic pmdp_collapse_flush() only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs. Provide a RISC-V specific pmdp_collapse_flush() which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification. Fixes: e88b333142e4 ("riscv: mm: add THP support on 64-bit") Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com> Link: https://lore.kernel.org/r/20230130074815.1694055-1-mchitale@ventanamicro.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-01riscv: disable generation of unwind tablesAndreas Schwab
GCC 13 will enable -fasynchronous-unwind-tables by default on riscv. In the kernel, we don't have any use for unwind tables yet, so disable them. More importantly, the .eh_frame section brings relocations (R_RISC_32_PCREL, R_RISCV_SET{6,8,16}, R_RISCV_SUB{6,8,16}) into modules that we are not prepared to handle. Signed-off-by: Andreas Schwab <schwab@suse.de> Link: https://lore.kernel.org/r/mvmzg9xybqu.fsf@suse.de Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-01riscv: kprobe: Fixup kernel panic when probing an illegal positionGuo Ren
The kernel would panic when probed for an illegal position. eg: (CONFIG_RISCV_ISA_C=n) echo 'p:hello kernel_clone+0x16 a0=%a0' >> kprobe_events echo 1 > events/kprobes/hello/enable cat trace Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __do_sys_newfstatat+0xb8/0xb8 CPU: 0 PID: 111 Comm: sh Not tainted 6.2.0-rc1-00027-g2d398fe49a4d #490 Hardware name: riscv-virtio,qemu (DT) Call Trace: [<ffffffff80007268>] dump_backtrace+0x38/0x48 [<ffffffff80c5e83c>] show_stack+0x50/0x68 [<ffffffff80c6da28>] dump_stack_lvl+0x60/0x84 [<ffffffff80c6da6c>] dump_stack+0x20/0x30 [<ffffffff80c5ecf4>] panic+0x160/0x374 [<ffffffff80c6db94>] generic_handle_arch_irq+0x0/0xa8 [<ffffffff802deeb0>] sys_newstat+0x0/0x30 [<ffffffff800158c0>] sys_clone+0x20/0x30 [<ffffffff800039e8>] ret_from_syscall+0x0/0x4 ---[ end Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __do_sys_newfstatat+0xb8/0xb8 ]--- That is because the kprobe's ebreak instruction broke the kernel's original code. The user should guarantee the correction of the probe position, but it couldn't make the kernel panic. This patch adds arch_check_kprobe in arch_prepare_kprobe to prevent an illegal position (Such as the middle of an instruction). Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported") Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Reviewed-by: Björn Töpel <bjorn@kernel.org> Link: https://lore.kernel.org/r/20230201040604.3390509-1-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-01Merge patch series "riscv: improve boot time isa extensions handling"Palmer Dabbelt
Jisheng Zhang <jszhang@kernel.org> says: Generally, riscv ISA extensions are fixed for any specific hardware platform, so a hart's features won't change after booting, this chacteristic makes it straightforward to use a static branch to check a specific ISA extension is supported or not to optimize performance. However, some ISA extensions such as SVPBMT and ZICBOM are handled via. the alternative sequences. Basically, for ease of maintenance, we prefer to use static branches in C code, but recently, Samuel found that the static branch usage in cpu_relax() breaks building with CONFIG_CC_OPTIMIZE_FOR_SIZE[1]. As Samuel pointed out, "Having a static branch in cpu_relax() is problematic because that function is widely inlined, including in some quite complex functions like in the VDSO. A quick measurement shows this static branch is responsible by itself for around 40% of the jump table." Samuel's findings pointed out one of a few downsides of static branches usage in C code to handle ISA extensions detected at boot time: static branch's metadata in the __jump_table section, which is not discarded after ISA extensions are finalized, wastes some space. I want to try to solve the issue for all possible dynamic handling of ISA extensions at boot time. Inspired by Mark[2], this patch introduces riscv_has_extension_*() helpers, which work like static branches but are patched using alternatives, thus the metadata can be freed after patching. [1]https://lore.kernel.org/linux-riscv/20220922060958.44203-1-samuel@sholland.org/ [2]https://lore.kernel.org/linux-arm-kernel/20220912162210.3626215-8-mark.rutland@arm.com/ [3]https://lore.kernel.org/linux-riscv/20221130225614.1594256-1-heiko@sntech.de/ * b4-shazam-merge: riscv: remove riscv_isa_ext_keys[] array and related usage riscv: KVM: Switch has_svinval() to riscv_has_extension_unlikely() riscv: cpu_relax: switch to riscv_has_extension_likely() riscv: alternative: patch alternatives in the vDSO riscv: switch to relative alternative entries riscv: module: Add ADD16 and SUB16 rela types riscv: module: move find_section to module.h riscv: fpu: switch has_fpu() to riscv_has_extension_likely() riscv: introduce riscv_has_extension_[un]likely() riscv: cpufeature: extend riscv_cpufeature_patch_func to all ISA extensions riscv: hwcap: make ISA extension ids can be used in asm riscv: cpufeature: detect RISCV_ALTERNATIVES_EARLY_BOOT earlier riscv: move riscv_noncoherent_supported() out of ZICBOM probe Link: https://lore.kernel.org/r/20230128172856.3814-1-jszhang@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-31riscv: remove riscv_isa_ext_keys[] array and related usageJisheng Zhang
All users have switched to riscv_has_extension_*, remove unused definitions, vars and related setting code. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Heiko Stuebner <heiko@sntech.de> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230128172856.3814-14-jszhang@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
generated by cgit (git 2.34.1) at 2025-09-24 03:32:09 +0000