diff options
| author | Peter Zijlstra <peterz@infradead.org> | 2020-08-18 15:57:48 +0200 |
|---|---|---|
| committer | Ingo Molnar <mingo@kernel.org> | 2020-09-01 09:58:05 +0200 |
| commit | 452cddbff74b6a15b9354505671011700fe03710 (patch) | |
| tree | 75bd4702221b4ca983ef387cfd33f49262e050f0 /arch/x86/include/asm/static_call.h | |
| parent | c43a43e439e00ad2a4d98716895d961ade6bbbfc (diff) | |
static_call: Add static_call_cond()
Extend the static_call infrastructure to optimize the following common
pattern:
if (func_ptr)
func_ptr(args...)
For the trampoline (which is in effect a tail-call), we patch the
JMP.d32 into a RET, which then directly consumes the trampoline call.
For the in-line sites we replace the CALL with a NOP5.
NOTE: this is 'obviously' limited to functions with a 'void' return type.
NOTE: DEFINE_STATIC_COND_CALL() only requires a typename, as opposed
to a full function.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20200818135805.042977182@infradead.org
Diffstat (limited to 'arch/x86/include/asm/static_call.h')
| -rw-r--r-- | arch/x86/include/asm/static_call.h | 12 |
1 files changed, 9 insertions, 3 deletions
diff --git a/arch/x86/include/asm/static_call.h b/arch/x86/include/asm/static_call.h index 33469ae3612c..c37f11999d0c 100644 --- a/arch/x86/include/asm/static_call.h +++ b/arch/x86/include/asm/static_call.h @@ -20,15 +20,21 @@ * it does tail-call optimization on the call; since you cannot compute the * relative displacement across sections. */ -#define ARCH_DEFINE_STATIC_CALL_TRAMP(name, func) \ + +#define __ARCH_DEFINE_STATIC_CALL_TRAMP(name, insns) \ asm(".pushsection .static_call.text, \"ax\" \n" \ ".align 4 \n" \ ".globl " STATIC_CALL_TRAMP_STR(name) " \n" \ STATIC_CALL_TRAMP_STR(name) ": \n" \ - " .byte 0xe9 # jmp.d32 \n" \ - " .long " #func " - (. + 4) \n" \ + insns " \n" \ ".type " STATIC_CALL_TRAMP_STR(name) ", @function \n" \ ".size " STATIC_CALL_TRAMP_STR(name) ", . - " STATIC_CALL_TRAMP_STR(name) " \n" \ ".popsection \n") +#define ARCH_DEFINE_STATIC_CALL_TRAMP(name, func) \ + __ARCH_DEFINE_STATIC_CALL_TRAMP(name, ".byte 0xe9; .long " #func " - (. + 4)") + +#define ARCH_DEFINE_STATIC_CALL_NULL_TRAMP(name) \ + __ARCH_DEFINE_STATIC_CALL_TRAMP(name, "ret; nop; nop; nop; nop") + #endif /* _ASM_STATIC_CALL_H */ |