diff options
| author | Rik van Riel <riel@surriel.com> | 2025-07-02 13:52:54 -0400 |
|---|---|---|
| committer | Thomas Gleixner <tglx@linutronix.de> | 2025-07-06 11:57:39 +0200 |
| commit | 946a7281982530d333eaee62bd1726f25908b3a9 (patch) | |
| tree | 0b742ecc393a925eb41d7fbb9f787a2ada4d1168 | |
| parent | e0e9506523fea415e0d5abaa103fd67dc8a39696 (diff) | |
smp: Wait only if work was enqueued
Whenever work is enqueued for a remote CPU, smp_call_function_many_cond()
may need to wait for that work to be completed. However, if no work is
enqueued for a remote CPU, because the condition func() evaluated to false
for all CPUs, there is no need to wait.
Set run_remote only if work was enqueued on remote CPUs.
Document the difference between "work enqueued", and "CPU needs to be
woken up"
Suggested-by: Jann Horn <jannh@google.com>
Signed-off-by: Rik van Riel <riel@surriel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Yury Norov (NVIDIA) <yury.norov@gmail.com>
Link: https://lore.kernel.org/all/20250703203019.11331ac3@fangorn
| -rw-r--r-- | kernel/smp.c | 8 |
1 files changed, 7 insertions, 1 deletions
diff --git a/kernel/smp.c b/kernel/smp.c index 99d1fd0e9e0e..c5e1da7a88da 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -802,7 +802,6 @@ static void smp_call_function_many_cond(const struct cpumask *mask, /* Check if we need remote execution, i.e., any CPU excluding this one. */ if (cpumask_any_and_but(mask, cpu_online_mask, this_cpu) < nr_cpu_ids) { - run_remote = true; cfd = this_cpu_ptr(&cfd_data); cpumask_and(cfd->cpumask, mask, cpu_online_mask); __cpumask_clear_cpu(this_cpu, cfd->cpumask); @@ -816,6 +815,9 @@ static void smp_call_function_many_cond(const struct cpumask *mask, continue; } + /* Work is enqueued on a remote CPU. */ + run_remote = true; + csd_lock(csd); if (wait) csd->node.u_flags |= CSD_TYPE_SYNC; @@ -827,6 +829,10 @@ static void smp_call_function_many_cond(const struct cpumask *mask, #endif trace_csd_queue_cpu(cpu, _RET_IP_, func, csd); + /* + * Kick the remote CPU if this is the first work + * item enqueued. + */ if (llist_add(&csd->node.llist, &per_cpu(call_single_queue, cpu))) { __cpumask_set_cpu(cpu, cfd->cpumask_ipi); nr_cpus++; |