summaryrefslogtreecommitdiff
path: root/include/linux/dma-resv.h
AgeCommit message (Collapse)Author
2021-11-05Merge tag 'drm-misc-next-2021-10-14' of ↵Dave Airlie
git://anongit.freedesktop.org/drm/drm-misc into drm-next drm-misc-next for 5.16: UAPI Changes: Cross-subsystem Changes: Core Changes: - fbdev: Fix double-free, Remove unused scrolling acceleration - locking: improve logging for contented locks without backoff - dma-buf: Add dma_resv_for_each_fence iterator, and conversion of users Driver Changes: - nouveau: Various code style improvements - bridge: HPD improvements for lt9611uxc, eDP aux-bus support for ps8640, lvds-codec data-mapping selection support - panels: Vivax TPC-9150, Innolux G070Y2-T02, LOGIC Technologies LTTD800480070-L2RT, Sharp LS060T1SX01, Signed-off-by: Dave Airlie <airlied@redhat.com> From: Maxime Ripard <maxime@cerno.tech> Link: https://patchwork.freedesktop.org/patch/msgid/20211014120452.2wicnt6hobu3kbwb@gilmour
2021-10-11dma-resv: Fix dma_resv_get_fences and dma_resv_copy_fences after conversionTvrtko Ursulin
Cache the count of shared fences in the iterator to avoid dereferencing the dma_resv_object outside the RCU protection. Otherwise iterator and its users can observe an incosistent state which makes it impossible to use safely. Such as: <6> [187.517041] [IGT] gem_sync: executing <7> [187.536343] i915 0000:00:02.0: [drm:i915_gem_context_create_ioctl [i915]] HW context 1 created <7> [187.536793] i915 0000:00:02.0: [drm:i915_gem_context_create_ioctl [i915]] HW context 1 created <6> [187.551235] [IGT] gem_sync: starting subtest basic-many-each <1> [188.935462] BUG: kernel NULL pointer dereference, address: 0000000000000010 <1> [188.935485] #PF: supervisor write access in kernel mode <1> [188.935495] #PF: error_code(0x0002) - not-present page <6> [188.935504] PGD 0 P4D 0 <4> [188.935512] Oops: 0002 [#1] PREEMPT SMP NOPTI <4> [188.935521] CPU: 2 PID: 1467 Comm: gem_sync Not tainted 5.15.0-rc4-CI-Patchwork_21264+ #1 <4> [188.935535] Hardware name: /NUC6CAYB, BIOS AYAPLCEL.86A.0049.2018.0508.1356 05/08/2018 <4> [188.935546] RIP: 0010:dma_resv_get_fences+0x116/0x2d0 <4> [188.935560] Code: 10 85 c0 7f c9 be 03 00 00 00 e8 15 8b df ff eb bd e8 8e c6 ff ff eb b6 41 8b 04 24 49 8b 55 00 48 89 e7 8d 48 01 41 89 0c 24 <4c> 89 34 c2 e8 41 f2 ff ff 49 89 c6 48 85 c0 75 8c 48 8b 44 24 10 <4> [188.935583] RSP: 0018:ffffc900011dbcc8 EFLAGS: 00010202 <4> [188.935593] RAX: 0000000000000000 RBX: 00000000ffffffff RCX: 0000000000000001 <4> [188.935603] RDX: 0000000000000010 RSI: ffffffff822e343c RDI: ffffc900011dbcc8 <4> [188.935613] RBP: ffffc900011dbd48 R08: ffff88812d255bb8 R09: 00000000fffffffe <4> [188.935623] R10: 0000000000000001 R11: 0000000000000000 R12: ffffc900011dbd44 <4> [188.935633] R13: ffffc900011dbd50 R14: ffff888113d29cc0 R15: 0000000000000000 <4> [188.935643] FS: 00007f68d17e9700(0000) GS:ffff888277900000(0000) knlGS:0000000000000000 <4> [188.935655] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [188.935665] CR2: 0000000000000010 CR3: 000000012d0a4000 CR4: 00000000003506e0 <4> [188.935676] Call Trace: <4> [188.935685] i915_gem_object_wait+0x1ff/0x410 [i915] <4> [188.935988] i915_gem_wait_ioctl+0xf2/0x2a0 [i915] <4> [188.936272] ? i915_gem_object_wait+0x410/0x410 [i915] <4> [188.936533] drm_ioctl_kernel+0xae/0x140 <4> [188.936546] drm_ioctl+0x201/0x3d0 <4> [188.936555] ? i915_gem_object_wait+0x410/0x410 [i915] <4> [188.936820] ? __fget_files+0xc2/0x1c0 <4> [188.936830] ? __fget_files+0xda/0x1c0 <4> [188.936839] __x64_sys_ioctl+0x6d/0xa0 <4> [188.936848] do_syscall_64+0x3a/0xb0 <4> [188.936859] entry_SYSCALL_64_after_hwframe+0x44/0xae If the shared object has changed during the RCU unlocked period callers will correctly handle the restart on the next iteration. Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Fixes: 96601e8a4755 ("dma-buf: use new iterator in dma_resv_copy_fences") Fixes: d3c80698c9f5 ("dma-buf: use new iterator in dma_resv_get_fences v3") Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/4274 Cc: Christian König <christian.koenig@amd.com> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: linux-media@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linaro-mm-sig@lists.linaro.org Link: https://patchwork.freedesktop.org/patch/msgid/20211008095007.972693-1-tvrtko.ursulin@linux.intel.com Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Christian König <christian.koenig@amd.com>
2021-10-11Merge tag 'drm-intel-gt-next-2021-10-08' of ↵Dave Airlie
git://anongit.freedesktop.org/drm/drm-intel into drm-next UAPI Changes: - Add uAPI for using PXP protected objects Mesa changes: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8064 - Add PCI IDs and LMEM discovery/placement uAPI for DG1 Mesa changes: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11584 - Disable engine bonding on Gen12+ except TGL, RKL and ADL-S Cross-subsystem Changes: - Merges 'tip/locking/wwmutex' branch (core kernel tip) - "mei: pxp: export pavp client to me client bus" Core Changes: - Update ttm_move_memcpy for async use (Thomas) Driver Changes: - Enable GuC submission by default on DG1 (Matt B) - Add PXP (Protected Xe Path) support for Gen12 integrated (Daniele, Sean, Anshuman) See "drm/i915/pxp: add PXP documentation" for details! - Remove force_probe protection for ADL-S (Raviteja) - Add base support for XeHP/XeHP SDV (Matt R, Stuart, Lucas) - Handle DRI_PRIME=1 on Intel igfx + Intel dgfx hybrid graphics setup (Tvrtko) - Use Transparent Hugepages when IOMMU is enabled (Tvrtko, Chris) - Implement LMEM backup and restore for suspend / resume (Thomas) - Report INSTDONE_GEOM values in error state for DG2 (Matt R) - Add DG2-specific shadow register table (Matt R) - Update Gen11/Gen12/XeHP shadow register tables (Matt R) - Maintain backward-compatible nested batch behavior on TGL+ (Matt R) - Add new LRI reg offsets for DG2 (Akeem) - Initialize unused MOCS entries to device specific values (Ayaz) - Track and use the correct UC MOCS index on Gen12 (Ayaz) - Add separate MOCS table for Gen12 devices other than TGL/RKL (Ayaz) - Simplify the locking and eliminate some RCU usage (Daniel) - Add some flushing for the 64K GTT path (Matt A) - Mark GPU wedging on driver unregister unrecoverable (Janusz) - Major rework in the GuC codebase, simplify locking and add docs (Matt B) - Add DG1 GuC/HuC firmwares (Daniele, Matt B) - Remember to call i915_sw_fence_fini on guc_state.blocked (Matt A) - Use "gt" forcewake domain name for error messages instead of "blitter" (Matt R) - Drop now duplicate LMEM uAPI RFC kerneldoc section (Daniel) - Fix early tracepoints for requests (Matt A) - Use locked access to ctx->engines in set_priority (Daniel) - Convert gen6/gen7/gen8 read operations to fwtable (Matt R) - Drop gen11/gen12 specific mmio write handlers (Matt R) - Drop gen11 specific mmio read handlers (Matt R) - Use designated initializers for init/exit table (Kees) - Fix syncmap memory leak (Matt B) - Add pretty printing for buddy allocator state debug (Matt A) - Fix potential error pointer dereference in pinned_context() (Dan) - Remove IS_ACTIVE macro (Lucas) - Static code checker fixes (Nathan) - Clean up disabled warnings (Nathan) - Increase timeout in i915_gem_contexts selftests 5x for GuC submission (Matt B) - Ensure wa_init_finish() is called for ctx workaround list (Matt R) - Initialize L3CC table in mocs init (Sreedhar, Ayaz, Ram) - Get PM ref before accessing HW register (Vinay) - Move __i915_gem_free_object to ttm_bo_destroy (Maarten) - Deduplicate frequency dump on debugfs (Lucas) - Make wa list per-gt (Venkata) - Do not define dummy vma in stack (Venkata) - Take pinning into account in __i915_gem_object_is_lmem (Matt B, Thomas) - Do not report currently active engine when describing objects (Tvrtko) - Fix pdfdocs build error by removing nested grid from GuC docs (Akira) - Remove false warning from the rps worker (Tejas) - Flush buffer pools on driver remove (Janusz) - Fix runtime pm handling in i915_gem_shrink (Maarten) - Rework TTM object initialization slightly (Thomas) - Use fixed offset for PTEs location (Michal Wa) - Verify result from CTB (de)register action and improve error messages (Michal Wa) - Fix bug in user proto-context creation that leaked contexts (Matt B) - Re-use Gen11 forcewake read functions on Gen12 (Matt R) - Make shadow tables range-based (Matt R) - Ditch the i915_gem_ww_ctx loop member (Thomas, Maarten) - Use NULL instead of 0 where appropriate (Ville) - Rename pci/debugfs functions to respect file prefix (Jani, Lucas) - Drop guc_communication_enabled (Daniele) - Selftest fixes (Thomas, Daniel, Matt A, Maarten) - Clean up inconsistent indenting (Colin) - Use direction definition DMA_BIDIRECTIONAL instead of PCI_DMA_BIDIRECTIONAL (Cai) - Add "intel_" as prefix in set_mocs_index() (Ayaz) From: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/YWAO80MB2eyToYoy@jlahtine-mobl.ger.corp.intel.com Signed-off-by: Dave Airlie <airlied@redhat.com>
2021-10-07dma-buf: add dma_resv_for_each_fence v3Christian König
A simpler version of the iterator to be used when the dma_resv object is locked. v2: fix index check here as well v3: minor coding improvement, some documentation cleanup Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211006123609.2026-1-christian.koenig@amd.com
2021-10-06dma-buf: add dma_resv_for_each_fence_unlocked v8Christian König
Abstract the complexity of iterating over all the fences in a dma_resv object. The new loop handles the whole RCU and retry dance and returns only fences where we can be sure we grabbed the right one. v2: fix accessing the shared fences while they might be freed, improve kerneldoc, rename _cursor to _iter, add dma_resv_iter_is_exclusive, add dma_resv_iter_begin/end v3: restructor the code, move rcu_read_lock()/unlock() into the iterator, add dma_resv_iter_is_restarted() v4: fix NULL deref when no explicit fence exists, drop superflous rcu_read_lock()/unlock() calls. v5: fix typos in the documentation v6: fix coding error when excl fence is NULL v7: one more logic fix v8: fix index check in dma_resv_iter_is_exclusive() Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> (v7) Link: https://patchwork.freedesktop.org/patch/msgid/20211005113742.1101-2-christian.koenig@amd.com
2021-09-17kernel/locking: Add context to ww_mutex_trylock()Maarten Lankhorst
i915 will soon gain an eviction path that trylock a whole lot of locks for eviction, getting dmesg failures like below: BUG: MAX_LOCK_DEPTH too low! turning off the locking correctness validator. depth: 48 max: 48! 48 locks held by i915_selftest/5776: #0: ffff888101a79240 (&dev->mutex){....}-{3:3}, at: __driver_attach+0x88/0x160 #1: ffffc900009778c0 (reservation_ww_class_acquire){+.+.}-{0:0}, at: i915_vma_pin.constprop.63+0x39/0x1b0 [i915] #2: ffff88800cf74de8 (reservation_ww_class_mutex){+.+.}-{3:3}, at: i915_vma_pin.constprop.63+0x5f/0x1b0 [i915] #3: ffff88810c7f9e38 (&vm->mutex/1){+.+.}-{3:3}, at: i915_vma_pin_ww+0x1c4/0x9d0 [i915] #4: ffff88810bad5768 (reservation_ww_class_mutex){+.+.}-{3:3}, at: i915_gem_evict_something+0x110/0x860 [i915] #5: ffff88810bad60e8 (reservation_ww_class_mutex){+.+.}-{3:3}, at: i915_gem_evict_something+0x110/0x860 [i915] ... #46: ffff88811964d768 (reservation_ww_class_mutex){+.+.}-{3:3}, at: i915_gem_evict_something+0x110/0x860 [i915] #47: ffff88811964e0e8 (reservation_ww_class_mutex){+.+.}-{3:3}, at: i915_gem_evict_something+0x110/0x860 [i915] INFO: lockdep is turned off. Fixing eviction to nest into ww_class_acquire is a high priority, but it requires a rework of the entire driver, which can only be done one step at a time. As an intermediate solution, add an acquire context to ww_mutex_trylock, which allows us to do proper nesting annotations on the trylocks, making the above lockdep splat disappear. This is also useful in regulator_lock_nested, which may avoid dropping regulator_nesting_mutex in the uncontended path, so use it there. TTM may be another user for this, where we could lock a buffer in a fastpath with list locks held, without dropping all locks we hold. [peterz: rework actual ww_mutex_trylock() implementations] Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/YUBGPdDDjKlxAuXJ@hirez.programming.kicks-ass.net
2021-08-30dma-resv: Give the docs a do-overDaniel Vetter
Specifically document the new/clarified rules around how the shared fences do not have any ordering requirements against the exclusive fence. But also document all the things a bit better, given how central struct dma_resv to dynamic buffer management the docs have been very inadequat. - Lots more links to other pieces of the puzzle. Unfortunately ttm_buffer_object has no docs, so no links :-( - Explain/complain a bit about dma_resv_locking_ctx(). I still don't like that one, but fixing the ttm call chains is going to be horrible. Plus we want to plug in real slowpath locking when we do that anyway. - Main part of the patch is some actual docs for struct dma_resv. Overall I think we still have a lot of bad naming in this area (e.g. dma_resv.fence is singular, but contains the multiple shared fences), but I think that's more indicative of how the semantics and rules are just not great. Another thing that's real awkard is how chaining exclusive fences right now means direct dma_resv.exclusive_fence pointer access with an rcu_assign_pointer. Not so great either. v2: - Fix a pile of typos (Matt, Jason) - Hammer it in that breaking the rules leads to use-after-free issues around dma-buf sharing (Christian) Reviewed-by: Christian König <christian.koenig@amd.com> Cc: Jason Ekstrand <jason@jlekstrand.net> Cc: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: "Christian König" <christian.koenig@amd.com> Cc: linux-media@vger.kernel.org Cc: linaro-mm-sig@lists.linaro.org Link: https://patchwork.freedesktop.org/patch/msgid/20210805104705.862416-21-daniel.vetter@ffwll.ch
2021-06-29<linux/dma-resv.h>: correct a function name in kernel-docRandy Dunlap
Fix kernel-doc function name warning: ../include/linux/dma-resv.h:227: warning: expecting prototype for dma_resv_exclusive(). Prototype was for dma_resv_excl_fence() instead Fixes: 6edbd6abb783d ("dma-buf: rename and cleanup dma_resv_get_excl v3") Signed-off-by: Randy Dunlap <rdunlap@infradead.org> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: Christian König <christian.koenig@amd.com> Cc: linux-media@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: linaro-mm-sig@lists.linaro.org Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210628004012.6792-1-rdunlap@infradead.org
2021-06-06dma-buf: drop the _rcu postfix on function names v3Christian König
The functions can be called both in _rcu context as well as while holding the lock. v2: add some kerneldoc as suggested by Daniel v3: fix indentation Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-7-christian.koenig@amd.com
2021-06-06dma-buf: rename dma_resv_get_excl_rcu to _unlockedChristian König
That describes much better what the function is doing here. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-6-christian.koenig@amd.com
2021-06-06dma-buf: rename and cleanup dma_resv_get_list v2Christian König
When the comment needs to state explicitly that this is doesn't get a reference to the object then the function is named rather badly. Rename the function and use it in even more places. v2: use dma_resv_shared_list as new name Signed-off-by: Christian König <christian.koenig@amd.com> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-5-christian.koenig@amd.com
2021-06-06dma-buf: rename and cleanup dma_resv_get_excl v3Christian König
When the comment needs to state explicitly that this doesn't get a reference to the object then the function is named rather badly. Rename the function and use rcu_dereference_check(), this way it can be used from both rcu as well as lock protected critical sections. v2: improve kerneldoc as suggested by Daniel v3: use dma_resv_excl_fence as function name Signed-off-by: Christian König <christian.koenig@amd.com> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-4-christian.koenig@amd.com
2021-06-04dma-buf: cleanup dma-resv shared fence debugging a bit v2Christian König
Make that a function instead of inline. v2: improve the kerneldoc wording as suggested by Daniel Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-3-christian.koenig@amd.com
2020-07-29dma-buf: Use sequence counter with associated wound/wait mutexAhmed S. Darwish
A sequence counter write side critical section must be protected by some form of locking to serialize writers. If the serialization primitive is not disabling preemption implicitly, preemption has to be explicitly disabled before entering the sequence counter write side critical section. The dma-buf reservation subsystem uses plain sequence counters to manage updates to reservations. Writer serialization is accomplished through a wound/wait mutex. Acquiring a wound/wait mutex does not disable preemption, so this needs to be done manually before and after the write side critical section. Use the newly-added seqcount_ww_mutex_t instead: - It associates the ww_mutex with the sequence count, which enables lockdep to validate that the write side critical section is properly serialized. - It removes the need to explicitly add preempt_disable/enable() around the write side critical section because the write_begin/end() functions for this new data type automatically do this. If lockdep is disabled this ww_mutex lock association is compiled out and has neither storage size nor runtime overhead. Signed-off-by: Ahmed S. Darwish <a.darwish@linutronix.de> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://lkml.kernel.org/r/20200720155530.1173732-13-a.darwish@linutronix.de
2020-07-29dma-buf: Remove custom seqcount lockdep class keyAhmed S. Darwish
Commit 3c3b177a9369 ("reservation: add support for read-only access using rcu") introduced a sequence counter to manage updates to reservations. Back then, the reservation object initializer reservation_object_init() was always inlined. Having the sequence counter initialization inlined meant that each of the call sites would have a different lockdep class key, which would've broken lockdep's deadlock detection. The aforementioned commit thus introduced, and exported, a custom seqcount lockdep class key and name. The commit 8735f16803f00 ("dma-buf: cleanup reservation_object_init...") transformed the reservation object initializer to a normal non-inlined C function. seqcount_init(), which automatically defines the seqcount lockdep class key and must be called non-inlined, can now be safely used. Remove the seqcount custom lockdep class key, name, and export. Use seqcount_init() inside the dma reservation object initializer. Signed-off-by: Ahmed S. Darwish <a.darwish@linutronix.de> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://lkml.kernel.org/r/20200720155530.1173732-12-a.darwish@linutronix.de
2019-08-16dma-buf: Restore seqlock around dma_resv updatesChris Wilson
This reverts 67c97fb79a7f ("dma-buf: add reservation_object_fences helper") dd7a7d1ff2f1 ("drm/i915: use new reservation_object_fences helper") 0e1d8083bddb ("dma-buf: further relax reservation_object_add_shared_fence") 5d344f58da76 ("dma-buf: nuke reservation_object seq number") The scenario that defeats simply grabbing a set of shared/exclusive fences and using them blissfully under RCU is that any of those fences may be reallocated by a SLAB_TYPESAFE_BY_RCU fence slab cache. In this scenario, while keeping the rcu_read_lock we need to establish that no fence was changed in the dma_resv after a read (or full) memory barrier. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Acked-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20190814182401.25009-1-chris@chris-wilson.co.uk
2019-08-13dma-buf: rename reservation_object to dma_resvChristian König
Be more consistent with the naming of the other DMA-buf objects. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> Link: https://patchwork.freedesktop.org/patch/323401/