| Age | Commit message (Collapse) | Author |
|---|
|
As pointed out by Alexander Duyck, the DMA mapping done in page_pool needs
to use the DMA attribute DMA_ATTR_SKIP_CPU_SYNC.
As the principle behind page_pool keeping the pages mapped is that the
driver takes over the DMA-sync steps.
Reported-by: Alexander Duyck <alexander.duyck@gmail.com>
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
As pointed out by David Miller the current page_pool implementation
stores dma_addr_t in page->private.
This won't work on 32-bit platforms with 64-bit DMA addresses since the
page->private is an unsigned long and the dma_addr_t a u64.
A previous patch is adding dma_addr_t on struct page to accommodate this.
This patch adapts the page_pool related functions to use the newly added
struct for storing and retrieving DMA addresses from network drivers.
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
Fix the warning below by calling the ptr_ring_consume_bh,
which uses spin_[un]lock_bh.
[ 179.064300] ================================
[ 179.069073] WARNING: inconsistent lock state
[ 179.073846] 4.18.0-rc2+ #18 Not tainted
[ 179.078133] --------------------------------
[ 179.082907] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
[ 179.089637] swapper/21/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
[ 179.095478] 00000000963d1995 (&(&r->consumer_lock)->rlock){+.?.}, at:
__page_pool_empty_ring+0x61/0x100
[ 179.105988] {SOFTIRQ-ON-W} state was registered at:
[ 179.111443] _raw_spin_lock+0x35/0x50
[ 179.115634] __page_pool_empty_ring+0x61/0x100
[ 179.120699] page_pool_destroy+0x32/0x50
[ 179.125204] mlx5e_free_rq+0x38/0xc0 [mlx5_core]
[ 179.130471] mlx5e_close_channel+0x20/0x120 [mlx5_core]
[ 179.136418] mlx5e_close_channels+0x26/0x40 [mlx5_core]
[ 179.142364] mlx5e_close_locked+0x44/0x50 [mlx5_core]
[ 179.148509] mlx5e_close+0x42/0x60 [mlx5_core]
[ 179.153936] __dev_close_many+0xb1/0x120
[ 179.158749] dev_close_many+0xa2/0x170
[ 179.163364] rollback_registered_many+0x148/0x460
[ 179.169047] rollback_registered+0x56/0x90
[ 179.174043] unregister_netdevice_queue+0x7e/0x100
[ 179.179816] unregister_netdev+0x18/0x20
[ 179.184623] mlx5e_remove+0x2a/0x50 [mlx5_core]
[ 179.190107] mlx5_remove_device+0xe5/0x110 [mlx5_core]
[ 179.196274] mlx5_unregister_interface+0x39/0x90 [mlx5_core]
[ 179.203028] cleanup+0x5/0xbfc [mlx5_core]
[ 179.208031] __x64_sys_delete_module+0x16b/0x240
[ 179.213640] do_syscall_64+0x5a/0x210
[ 179.218151] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 179.224218] irq event stamp: 334398
[ 179.228438] hardirqs last enabled at (334398): [<ffffffffa511d8b7>]
rcu_process_callbacks+0x1c7/0x790
[ 179.239178] hardirqs last disabled at (334397): [<ffffffffa511d872>]
rcu_process_callbacks+0x182/0x790
[ 179.249931] softirqs last enabled at (334386): [<ffffffffa509732e>] irq_enter+0x5e/0x70
[ 179.259306] softirqs last disabled at (334387): [<ffffffffa509741c>] irq_exit+0xdc/0xf0
[ 179.268584]
[ 179.268584] other info that might help us debug this:
[ 179.276572] Possible unsafe locking scenario:
[ 179.276572]
[ 179.283877] CPU0
[ 179.286954] ----
[ 179.290033] lock(&(&r->consumer_lock)->rlock);
[ 179.295546] <Interrupt>
[ 179.298830] lock(&(&r->consumer_lock)->rlock);
[ 179.304550]
[ 179.304550] *** DEADLOCK ***
Fixes: ff7d6b27f894 ("page_pool: refurbish version of page_pool code")
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|
Need a fast page recycle mechanism for ndo_xdp_xmit API for returning
pages on DMA-TX completion time, which have good cross CPU
performance, given DMA-TX completion time can happen on a remote CPU.
Refurbish my page_pool code, that was presented[1] at MM-summit 2016.
Adapted page_pool code to not depend the page allocator and
integration into struct page. The DMA mapping feature is kept,
even-though it will not be activated/used in this patchset.
[1] http://people.netfilter.org/hawk/presentations/MM-summit2016/generic_page_pool_mm_summit2016.pdf
V2: Adjustments requested by Tariq
- Changed page_pool_create return codes, don't return NULL, only
ERR_PTR, as this simplifies err handling in drivers.
V4: many small improvements and cleanups
- Add DOC comment section, that can be used by kernel-doc
- Improve fallback mode, to work better with refcnt based recycling
e.g. remove a WARN as pointed out by Tariq
e.g. quicker fallback if ptr_ring is empty.
V5: Fixed SPDX license as pointed out by Alexei
V6: Adjustments requested by Eric Dumazet
- Adjust ____cacheline_aligned_in_smp usage/placement
- Move rcu_head in struct page_pool
- Free pages quicker on destroy, minimize resources delayed an RCU period
- Remove code for forward/backward compat ABI interface
V8: Issues found by kbuild test robot
- Address sparse should be static warnings
- Only compile+link when a driver use/select page_pool,
mlx5 selects CONFIG_PAGE_POOL, although its first used in two patches
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
