summaryrefslogtreecommitdiff
path: root/mm/page_alloc.c
diff options
context:
space:
mode:
authorVlastimil Babka <vbabka@suse.cz>2017-02-24 14:56:53 -0800
committerLinus Torvalds <torvalds@linux-foundation.org>2017-02-24 17:46:54 -0800
commit5104782011a12b04fe9cfaa6f1085bdcdedd79c4 (patch)
treeff56727123682eda3d8dc2eb0a9b602643cebecd /mm/page_alloc.c
parentdf76cee6bbeb2ed036f1622f63a99c28cecf6b30 (diff)
mm, page_alloc: don't check cpuset allowed twice in fast-path
Since commit 682a3385e773 ("mm, page_alloc: inline the fast path of the zonelist iterator") we replace a NULL nodemask with cpuset_current_mems_allowed in the fast path, so that get_page_from_freelist() filters nodes allowed by the cpuset via for_next_zone_zonelist_nodemask(). In that case it's pointless to additionaly check __cpuset_zone_allowed() in each iteration, which we can avoid by not adding ALLOC_CPUSET to alloc_flags in that scenario. This saves some cycles in the allocator fast path on systems with one or more non-root cpuset configured. In the slow path, ALLOC_CPUSET is reset according to __alloc_pages_slowpath(). Without configured cpusets, this code is disabled by a static key. Link: http://lkml.kernel.org/r/20170124150511.5710-2-vbabka@suse.cz Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Anshuman Khandual <khandual@linux.vnet.ibm.com> Acked-by: Michal Hocko <mhocko@suse.com> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Mel Gorman <mgorman@techsingularity.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'mm/page_alloc.c')
-rw-r--r--mm/page_alloc.c3
1 files changed, 2 insertions, 1 deletions
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 65876feb86f3..46c30fa26acd 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3903,9 +3903,10 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
if (cpusets_enabled()) {
*alloc_mask |= __GFP_HARDWALL;
- *alloc_flags |= ALLOC_CPUSET;
if (!ac->nodemask)
ac->nodemask = &cpuset_current_mems_allowed;
+ else
+ *alloc_flags |= ALLOC_CPUSET;
}
lockdep_trace_alloc(gfp_mask);