summaryrefslogtreecommitdiff
path: root/drivers/iommu
AgeCommit message (Collapse)Author
2017-08-15iommu/amd: Disable iommu only if amd_iommu=off is specifiedBaoquan He
It's ok to disable iommu early in normal kernel or in kdump kernel when amd_iommu=off is specified. While we should not disable it in kdump kernel when on-flight dma is still on-going. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Don't copy GCR3 table root pointerBaoquan He
When iommu is pre_enabled in kdump kernel, if a device is set up with guest translations (DTE.GV=1), then don't copy GCR3 table root pointer but move the device over to an empty guest-cr3 table and handle the faults in the PPR log (which answer them with INVALID). After all these PPR faults are recoverable for the device and we should not allow the device to change old-kernels data when we don't have to. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Allocate memory below 4G for dev table if translation pre-enabledBaoquan He
AMD pointed out it's unsafe to update the device-table while iommu is enabled. It turns out that device-table pointer update is split up into two 32bit writes in the IOMMU hardware. So updating it while the IOMMU is enabled could have some nasty side effects. The safe way to work around this is to always allocate the device-table below 4G, including the old device-table in normal kernel and the device-table used for copying the content of the old device-table in kdump kernel. Meanwhile we need check if the address of old device-table is above 4G because it might has been touched accidentally in corrupted 1st kernel. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Use is_attach_deferred call-backBaoquan He
Implement call-back is_attach_deferred and use it to defer the domain attach from iommu driver init to device driver init when iommu is pre-enabled in kdump kernel. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu: Add is_attach_deferred call-back to iommu-opsBaoquan He
This new call-back will be used to check if the domain attach need be deferred for now. If yes, the domain attach/detach will return directly. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Do sanity check for address translation and irq remap of old dev ↵Baoquan He
table entry Firstly split the dev table entry copy into address translation part and irq remapping part. Because these two parts could be enabled independently. Secondly do sanity check for address translation and irq remap of old dev table entry separately. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Copy old trans table from old kernelBaoquan He
Here several things need be done: - If iommu is pre-enabled in a normal kernel, just disable it and print warning. - If any one of IOMMUs is not pre-enabled in kdump kernel, just continue as it does in normal kernel. - If failed to copy dev table of old kernel, continue to proceed as it does in normal kernel. - Only if all IOMMUs are pre-enabled and copy dev table is done well, free the dev table allocated in early_amd_iommu_init() and make amd_iommu_dev_table point to the copied one. - Disable and Re-enable event/cmd buffer, install the copied DTE table to reg, and detect and enable guest vapic. - Flush all caches Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Add function copy_dev_tables()Baoquan He
Add function copy_dev_tables to copy the old DEV table entries of the panicked kernel to the new allocated device table. Since all iommus share the same device table the copy only need be done one time. Here add a new global old_dev_tbl_cpy to point to the newly allocated device table which the content of old device table will be copied to. Besides, we also need to: - Check whether all IOMMUs actually use the same device table with the same size - Verify that the size of the old device table is the expected size. - Reserve the old domain id occupied in 1st kernel to avoid touching the old io-page tables. Then on-flight DMA can continue looking it up. And also define MACRO DEV_DOMID_MASK to replace magic number 0xffffULL, it can be reused in copy_dev_tables(). Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Define bit fields for DTE particularlyBaoquan He
In AMD-Vi spec several bits of IO PTE fields and DTE fields are similar so that both of them can share the same MACRO definition. However defining them respectively can make code more read-able. Do it now. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15Revert "iommu/amd: Suppress IO_PAGE_FAULTs in kdump kernel"Baoquan He
This reverts commit 54bd63570484167cb13edf81e31fff107b879981. We still need the IO_PAGE_FAULT message to warn error after the issue of on-flight dma in kdump kernel is fixed. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Add several helper functionsBaoquan He
Move single iommu enabling codes into a wrapper function early_enable_iommu(). This can make later kdump change easier. And also add iommu_disable_command_buffer and iommu_disable_event_buffer for later usage. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/amd: Detect pre enabled translationBaoquan He
Add functions to check whether translation is already enabled in IOMMU. Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/qcom: Initialize secure page tableStanimir Varbanov
This basically gets the secure page table size, allocates memory for secure pagetables and passes the physical address to the trusted zone. Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org> Signed-off-by: Rob Clark <robdclark@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/qcom: Add qcom_iommuRob Clark
An iommu driver for Qualcomm "B" family devices which do implement the ARM SMMU spec, but not in a way that is compatible with how the arm-smmu driver is designed. It seems SMMU_SCR1.GASRAE=1 so the global register space is not accessible. This means it needs to get configuration from devicetree instead of setting it up dynamically. In the end, other than register definitions, there is not much code to share with arm-smmu (other than what has already been refactored out into the pgtable helpers). Signed-off-by: Rob Clark <robdclark@gmail.com> Tested-by: Riku Voipio <riku.voipio@linaro.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/arm-smmu: Split out register definesRob Clark
I want to re-use some of these for qcom_iommu, which has (roughly) the same context-bank registers. Signed-off-by: Rob Clark <robdclark@gmail.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/pamu: Add support for generic iommu-deviceJoerg Roedel
This patch adds a global iommu-handle to the pamu driver and initializes it at probe time. Also link devices added to the iommu to this handle. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/pamu: WARN when fsl_pamu_probe() is called more than onceJoerg Roedel
The function probes the PAMU hardware from device-tree specifications. It initializes global variables and can thus be only safely called once. Add a check that that prints a warning when its called more than once. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/pamu: Make driver depend on CONFIG_PHYS_64BITJoerg Roedel
Certain address calculations in the driver make the assumption that phys_addr_t and dma_addr_t are 64 bit wide. Force this by depending on CONFIG_PHYS_64BIT to be set. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu/pamu: Let PAMU depend on PCIJoerg Roedel
The driver does not compile when PCI is not selected, so make it depend on it. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-15iommu: Fix wrong freeing of iommu_device->devJoerg Roedel
The struct iommu_device has a 'struct device' embedded into it, not as a pointer, but the whole struct. In the conversion of the iommu drivers to use struct iommu_device it was forgotten that the relase function for that struct device simply calls kfree() on the pointer. This frees memory that was never allocated and causes memory corruption. To fix this issue, use a pointer to struct device instead of embedding the whole struct. This needs some updates in the iommu sysfs code as well as the Intel VT-d and AMD IOMMU driver. Reported-by: Sebastian Ott <sebott@linux.vnet.ibm.com> Fixes: 39ab9555c241 ('iommu: Add sysfs bindings for struct iommu_device') Cc: stable@vger.kernel.org # >= v4.11 Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-11iommu/arm-smmu: fix null-pointer dereference in arm_smmu_add_deviceArtem Savkov
Commit c54451a "iommu/arm-smmu: Fix the error path in arm_smmu_add_device" removed fwspec assignment in legacy_binding path as redundant which is wrong. It needs to be updated after fwspec initialisation in arm_smmu_register_legacy_master() as it is dereferenced later. Without this there is a NULL-pointer dereference panic during boot on some hosts. Signed-off-by: Artem Savkov <asavkov@redhat.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-10iommu: Finish making iommu_group support mandatoryRobin Murphy
Now that all the drivers properly implementing the IOMMU API support groups (I'm ignoring the etnaviv GPU MMUs which seemingly only do just enough to convince the ARM DMA mapping ops), we can remove the FIXME workarounds from the core code. In the process, it also seems logical to make the .device_group callback non-optional for drivers calling iommu_group_get_for_dev() - the current callers all implement it anyway, and it doesn't make sense for any future callers not to either. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-10iommu/tegra-gart: Add iommu_group supportRobin Murphy
As the last step to making groups mandatory, clean up the remaining drivers by adding basic support. Whilst it may not perfectly reflect the isolation capabilities of the hardware, using generic_device_group() should at least maintain existing behaviour with respect to the API. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Dmitry Osipenko <digetx@gmail.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-10iommu/tegra-smmu: Add iommu_group supportRobin Murphy
As the last step to making groups mandatory, clean up the remaining drivers by adding basic support. Whilst it may not perfectly reflect the isolation capabilities of the hardware (tegra_smmu_swgroup sounds suspiciously like something that might warrant representing at the iommu_group level), using generic_device_group() should at least maintain existing behaviour with respect to the API. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Tested-by: Mikko Perttunen <mperttunen@nvidia.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-10iommu/msm: Add iommu_group supportRobin Murphy
As the last step to making groups mandatory, clean up the remaining drivers by adding basic support. Whilst it may not perfectly reflect the isolation capabilities of the hardware, using generic_device_group() should at least maintain existing behaviour with respect to the API. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-04iommu/exynos: Remove custom platform device registration codeMarek Szyprowski
Commit 09515ef5ddad ("of/acpi: Configure dma operations at probe time for platform/amba/pci bus devices") postponed the moment of attaching IOMMU controller to its device, so there is no need to register IOMMU controllers very early, before all other devices in the system. This change gives us an opportunity to use standard platform device registration method also for Exynos SYSMMU controllers. Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-04iommu/omap: Use DMA-API for performing cache flushesJosue Albarran
The OMAP IOMMU driver was using ARM assembly code directly for flushing the MMU page table entries from the caches. This caused MMU faults on OMAP4 (Cortex-A9 based SoCs) as L2 caches were not handled due to the presence of a PL310 L2 Cache Controller. These faults were however not seen on OMAP5/DRA7 SoCs (Cortex-A15 based SoCs). The OMAP IOMMU driver is adapted to use the DMA Streaming API instead now to flush the page table/directory table entries from the CPU caches. This ensures that the devices always see the updated page table entries. The outer caches are now addressed automatically with the usage of the DMA API. Signed-off-by: Josue Albarran <j-albarran@ti.com> Acked-by: Suman Anna <s-anna@ti.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-04iommu/omap: Fix disabling of MMU upon a faultFernando Guzman Lugo
The IOMMU framework lets its client users be notified on a MMU fault and allows them to either handle the interrupt by dynamic reloading of an appropriate TLB/PTE for the offending fault address or to completely restart/recovery the device and its IOMMU. The OMAP remoteproc driver performs the latter option, and does so after unwinding the previous mappings. The OMAP IOMMU fault handler however disables the MMU and cuts off the clock upon a MMU fault at present, resulting in an interconnect abort during any subsequent operation that touches the MMU registers. So, disable the IP-level fault interrupts instead of disabling the MMU, to allow continued MMU register operations as well as to avoid getting interrupted again. Signed-off-by: Fernando Guzman Lugo <fernando.lugo@ti.com> [s-anna@ti.com: add commit description] Signed-off-by: Suman Anna <s-anna@ti.com> Signed-off-by: Josue Albarran <j-albarran@ti.com> Acked-by: Suman Anna <s-anna@ti.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-08-04iommu/exynos: prevent building on big-endian kernelsArnd Bergmann
Since we print the correct warning, an allmodconfig build is no longer clean but always prints it, which defeats compile-testing: drivers/iommu/exynos-iommu.c:58:2: error: #warning "revisit driver if we can enable big-endian ptes" [-Werror=cpp] This replaces the #warning with a dependency, moving warning text into a comment. Fixes: 1f59adb1766d ("iommu/exynos: Replace non-existing big-endian Kconfig option") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Reviewed-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-27iommu/rockchip: ignore isp mmu reset operationSimon Xue
ISP mmu can't support reset operation, it won't get the expected result when reset, but rest functions work normally. Add this patch as a WA for this issue. Signed-off-by: Simon Xue <xxm@rock-chips.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-27iommu/rockchip: add multi irqs supportSimon Xue
RK3368 vpu mmu have two irqs, this patch support multi irqs Signed-off-by: Simon Xue <xxm@rock-chips.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/amd: Fix schedule-while-atomic BUG in initialization codeJoerg Roedel
The register_syscore_ops() function takes a mutex and might sleep. In the IOMMU initialization code it is invoked during irq-remapping setup already, where irqs are disabled. This causes a schedule-while-atomic bug: BUG: sleeping function called from invalid context at kernel/locking/mutex.c:747 in_atomic(): 0, irqs_disabled(): 1, pid: 1, name: swapper/0 no locks held by swapper/0/1. irq event stamp: 304 hardirqs last enabled at (303): [<ffffffff818a87b6>] _raw_spin_unlock_irqrestore+0x36/0x60 hardirqs last disabled at (304): [<ffffffff8235d440>] enable_IR_x2apic+0x79/0x196 softirqs last enabled at (36): [<ffffffff818ae75f>] __do_softirq+0x35f/0x4ec softirqs last disabled at (31): [<ffffffff810c1955>] irq_exit+0x105/0x120 CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.13.0-rc2.1.el7a.test.x86_64.debug #1 Hardware name: PowerEdge C6145 /040N24, BIOS 3.5.0 10/28/2014 Call Trace: dump_stack+0x85/0xca ___might_sleep+0x22a/0x260 __might_sleep+0x4a/0x80 __mutex_lock+0x58/0x960 ? iommu_completion_wait.part.17+0xb5/0x160 ? register_syscore_ops+0x1d/0x70 ? iommu_flush_all_caches+0x120/0x150 mutex_lock_nested+0x1b/0x20 register_syscore_ops+0x1d/0x70 state_next+0x119/0x910 iommu_go_to_state+0x29/0x30 amd_iommu_enable+0x13/0x23 Fix it by moving the register_syscore_ops() call to the next initialization step, which runs with irqs enabled. Reported-by: Artem Savkov <asavkov@redhat.com> Tested-by: Artem Savkov <asavkov@redhat.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> Fixes: 2c0ae1720c09 ('iommu/amd: Convert iommu initialization to state machine') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu: Convert to using %pOF instead of full_nameRob Herring
Now that we have a custom printf format specifier, convert users of full_name to use %pOF instead. This is preparation to remove storing of the full path string for each node. Signed-off-by: Rob Herring <robh@kernel.org> Cc: Joerg Roedel <joro@8bytes.org> Cc: Heiko Stuebner <heiko@sntech.de> Cc: iommu@lists.linux-foundation.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-rockchip@lists.infradead.org Reviewed-by: Heiko Stuebner <heiko@sntech.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Clean up device trackingRobin Murphy
Get rid of now unused device tracking code. Future code should instead be able to use driver_for_each_device() for this purpose. This is a simplified version of the following patch from Robin [PATCH] iommu/ipmmu-vmsa: Clean up group allocation Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Replace local utlb code with fwspec idsMagnus Damm
Now when both 32-bit and 64-bit code inside the driver is using fwspec it is possible to replace the utlb handling with fwspec ids that get populated from ->of_xlate(). Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Use fwspec on both 32 and 64-bit ARMRobin Murphy
Consolidate the 32-bit and 64-bit code to make use of fwspec instead of archdata for the 32-bit ARM case. This is a simplified version of the fwspec handling code from Robin posted as [PATCH] iommu/ipmmu-vmsa: Convert to iommu_fwspec Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Consistent ->of_xlate() handlingMagnus Damm
The 32-bit ARM code gets updated to make use of ->of_xlate() and the code is shared between 64-bit and 32-bit ARM. The of_device_is_available() check gets dropped since it is included in of_iommu_xlate(). Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/ipmmu-vmsa: Use iommu_device_register()/unregister()Magnus Damm
Extend the driver to make use of iommu_device_register()/unregister() functions together with iommu_device_set_ops() and iommu_set_fwnode(). These used to be part of the earlier posted 64-bit ARM (r8a7795) series but it turns out that these days they are required on 32-bit ARM as well. Signed-off-by: Magnus Damm <damm+renesas@opensource.se> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/exynos: Replace non-existing big-endian Kconfig optionKrzysztof Kozlowski
Wrong Kconfig option was used when adding warning for untested big-endian capabilities. There is no CONFIG_BIG_ENDIAN option. Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/vt-d: Don't free parent pagetable of the PTE we're addingDavid Dillow
When adding a large scatterlist entry that covers more than the L3 superpage size (1GB) but has an alignment such that we must use L2 superpages (2MB) , we give dma_pte_free_level() a range that causes it to free the L3 pagetable we're about to populate. We fix this by telling dma_pte_free_pagetable() about the pagetable level we're about to populate to prevent freeing it. For example, mapping a scatterlist with entry lengths 854MB and 1194MB at IOVA 0xffff80000000 would, when processing the 2MB-aligned second entry, cause pfn_to_dma_pte() to create a L3 directory to hold L2 superpages for the mapping at IOVA 0xffffc0000000. We would previously call dma_pte_free_pagetable(domain, 0xffffc0000, 0xfffffffff), which would free the L3 directory pfn_to_dma_pte() just created for IO PFN 0xffffc0000. Telling dma_pte_free_pagetable() to retain the L3 directories while using L2 superpages avoids the erroneous free. Signed-off-by: David Dillow <dillow@google.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-26iommu/of: Handle PCI aliases properlyRobin Murphy
When a PCI device has DMA quirks, we need to ensure that an upstream IOMMU knows about all possible aliases, since the presence of a DMA quirk does not preclude the device still also emitting transactions (e.g. MSIs) on its 'real' RID. Similarly, the rules for bridge aliasing are relatively complex, and some bridges may only take ownership of transactions under particular transient circumstances, leading again to multiple RIDs potentially being seen at the IOMMU for the given device. Take all this into account in the OF code by translating every RID produced by the alias walk, not just whichever one comes out last. Happily, this also makes things tidy enough that we can reduce the number of both total lines of code, and confusing levels of indirection, by pulling the "iommus"/"iommu-map" parsing helpers back in-line again. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-25iommu/amd: Enable ga_log_intr when enabling guest_modeSuravee Suthikulpanit
IRTE[GALogIntr] bit should set when enabling guest_mode, which enables IOMMU to generate entry in GALog when IRTE[IsRun] is not set, and send an interrupt to notify IOMMU driver. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: stable@vger.kernel.org # v4.9+ Fixes: d98de49a53e48 ('iommu/amd: Enable vAPIC interrupt remapping mode by default') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-07-20iommu/io-pgtable: Sanitise map/unmap addressesRobin Murphy
It may be an egregious error to attempt to use addresses outside the range of the pagetable format, but that still doesn't mean we should merrily wreak havoc by silently mapping/unmapping whatever truncated portions of them might happen to correspond to real addresses. Add some up-front checks to sanitise our inputs so that buggy callers don't invite potential memory corruption. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-07-20iommu/arm-smmu: Fix the error path in arm_smmu_add_deviceVivek Gautam
fwspec->iommu_priv is available only after arm_smmu_master_cfg instance has been allocated. We shouldn't free it before that. Also it's logical to free the master cfg itself without checking for fwspec. Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org> [will: remove redundant assignment to fwspec] Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-07-20Revert "iommu/io-pgtable: Avoid redundant TLB syncs"Robin Murphy
The tlb_sync_pending flag was necessary for correctness in the Mediatek M4U driver, but since it offered a small theoretical optimisation for all io-pgtable users it was implemented as a high-level thing. However, now that some users may not be using a synchronising lock, there are several ways this flag can go wrong for them, and at worst it could result in incorrect behaviour. Since we've addressed the correctness issue within the Mediatek driver itself, and fixing the optimisation aspect to be concurrency-safe would be quite a headache (and impose extra overhead on every operation for the sake of slightly helping one case which will virtually never happen in typical usage), let's just retire it. This reverts commit 88492a4700360a086e55d8874ad786105a5e8b0f. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-07-20iommu/mtk: Avoid redundant TLB syncs locallyRobin Murphy
Under certain circumstances, the io-pgtable code may end up issuing two TLB sync operations without any intervening invalidations. This goes badly for the M4U hardware, since it means the second sync ends up polling for a non-existent operation to finish, and as a result times out and warns. The io_pgtable_tlb_* helpers implement a high-level optimisation to avoid issuing the second sync at all in such cases, but in order to work correctly that requires all pagetable operations to be serialised under a lock, thus is no longer applicable to all io-pgtable users. Since we're the only user actually relying on this flag for correctness, let's reimplement it locally to avoid the headache of trying to make the high-level version concurrency-safe for other users. CC: Yong Wu <yong.wu@mediatek.com> CC: Matthias Brugger <matthias.bgg@gmail.com> Tested-by: Yong Wu <yong.wu@mediatek.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-07-20iommu/arm-smmu: Reintroduce locking around TLB sync operationsWill Deacon
Commit 523d7423e21b ("iommu/arm-smmu: Remove io-pgtable spinlock") removed the locking used to serialise map/unmap calls into the io-pgtable code from the ARM SMMU driver. This is good for performance, but opens us up to a nasty race with TLB syncs because the TLB sync register is shared within a context bank (or even globally for stage-2 on SMMUv1). There are two cases to consider: 1. A CPU can be spinning on the completion of a TLB sync, take an interrupt which issues a subsequent TLB sync, and then report a timeout on return from the interrupt. 2. A CPU can be spinning on the completion of a TLB sync, but other CPUs can continuously issue additional TLB syncs in such a way that the backoff logic reports a timeout. Rather than fix this by spinning for completion of prior TLB syncs before issuing a new one (which may suffer from fairness issues on large systems), instead reintroduce locking around TLB sync operations in the ARM SMMU driver. Fixes: 523d7423e21b ("iommu/arm-smmu: Remove io-pgtable spinlock") Cc: Robin Murphy <robin.murphy@arm.com> Reported-by: Ray Jui <ray.jui@broadcom.com> Tested-by: Ray Jui <ray.jui@broadcom.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-07-18iommu/amd: Allow the AMD IOMMU to work with memory encryptionTom Lendacky
The IOMMU is programmed with physical addresses for the various tables and buffers that are used to communicate between the device and the driver. When the driver allocates this memory it is encrypted. In order for the IOMMU to access the memory as encrypted the encryption mask needs to be included in these physical addresses during configuration. The PTE entries created by the IOMMU should also include the encryption mask so that when the device behind the IOMMU performs a DMA, the DMA will be performed to encrypted memory. Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Borislav Petkov <bp@suse.de> Acked-by: Joerg Roedel <jroedel@suse.de> Cc: <iommu@lists.linux-foundation.org> Cc: Alexander Potapenko <glider@google.com> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@kernel.org> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Brijesh Singh <brijesh.singh@amd.com> Cc: Dave Young <dyoung@redhat.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Larry Woodman <lwoodman@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Matt Fleming <matt@codeblueprint.co.uk> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Radim Krčmář <rkrcmar@redhat.com> Cc: Rik van Riel <riel@redhat.com> Cc: Toshimitsu Kani <toshi.kani@hpe.com> Cc: kasan-dev@googlegroups.com Cc: kvm@vger.kernel.org Cc: linux-arch@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: linux-efi@vger.kernel.org Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/3053631ea25ba8b1601c351cb7c541c496f6d9bc.1500319216.git.thomas.lendacky@amd.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-07-12Merge tag 'iommu-updates-v4.13' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU updates from Joerg Roedel: "This update comes with: - Support for lockless operation in the ARM io-pgtable code. This is an important step to solve the scalability problems in the common dma-iommu code for ARM - Some Errata workarounds for ARM SMMU implemenations - Rewrite of the deferred IO/TLB flush code in the AMD IOMMU driver. The code suffered from very high flush rates, with the new implementation the flush rate is down to ~1% of what it was before - Support for amd_iommu=off when booting with kexec. The problem here was that the IOMMU driver bailed out early without disabling the iommu hardware, if it was enabled in the old kernel - The Rockchip IOMMU driver is now available on ARM64 - Align the return value of the iommu_ops->device_group call-backs to not miss error values - Preempt-disable optimizations in the Intel VT-d and common IOVA code to help Linux-RT - Various other small cleanups and fixes" * tag 'iommu-updates-v4.13' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (60 commits) iommu/vt-d: Constify intel_dma_ops iommu: Warn once when device_group callback returns NULL iommu/omap: Return ERR_PTR in device_group call-back iommu: Return ERR_PTR() values from device_group call-backs iommu/s390: Use iommu_group_get_for_dev() in s390_iommu_add_device() iommu/vt-d: Don't disable preemption while accessing deferred_flush() iommu/iova: Don't disable preempt around this_cpu_ptr() iommu/arm-smmu-v3: Add workaround for Cavium ThunderX2 erratum #126 iommu/arm-smmu-v3: Enable ACPI based HiSilicon CMD_PREFETCH quirk(erratum 161010701) iommu/arm-smmu-v3: Add workaround for Cavium ThunderX2 erratum #74 ACPI/IORT: Fixup SMMUv3 resource size for Cavium ThunderX2 SMMUv3 model iommu/arm-smmu-v3, acpi: Add temporary Cavium SMMU-V3 IORT model number definitions iommu/io-pgtable-arm: Use dma_wmb() instead of wmb() when publishing table iommu/io-pgtable: depend on !GENERIC_ATOMIC64 when using COMPILE_TEST with LPAE iommu/arm-smmu-v3: Remove io-pgtable spinlock iommu/arm-smmu: Remove io-pgtable spinlock iommu/io-pgtable-arm-v7s: Support lockless operation iommu/io-pgtable-arm: Support lockless operation iommu/io-pgtable: Introduce explicit coherency iommu/io-pgtable-arm-v7s: Refactor split_blk_unmap ...
2017-07-06Merge tag 'dma-mapping-4.13' of git://git.infradead.org/users/hch/dma-mappingLinus Torvalds
Pull dma-mapping infrastructure from Christoph Hellwig: "This is the first pull request for the new dma-mapping subsystem In this new subsystem we'll try to properly maintain all the generic code related to dma-mapping, and will further consolidate arch code into common helpers. This pull request contains: - removal of the DMA_ERROR_CODE macro, replacing it with calls to ->mapping_error so that the dma_map_ops instances are more self contained and can be shared across architectures (me) - removal of the ->set_dma_mask method, which duplicates the ->dma_capable one in terms of functionality, but requires more duplicate code. - various updates for the coherent dma pool and related arm code (Vladimir) - various smaller cleanups (me)" * tag 'dma-mapping-4.13' of git://git.infradead.org/users/hch/dma-mapping: (56 commits) ARM: dma-mapping: Remove traces of NOMMU code ARM: NOMMU: Set ARM_DMA_MEM_BUFFERABLE for M-class cpus ARM: NOMMU: Introduce dma operations for noMMU drivers: dma-mapping: allow dma_common_mmap() for NOMMU drivers: dma-coherent: Introduce default DMA pool drivers: dma-coherent: Account dma_pfn_offset when used with device tree dma: Take into account dma_pfn_offset dma-mapping: replace dmam_alloc_noncoherent with dmam_alloc_attrs dma-mapping: remove dmam_free_noncoherent crypto: qat - avoid an uninitialized variable warning au1100fb: remove a bogus dma_free_nonconsistent call MAINTAINERS: add entry for dma mapping helpers powerpc: merge __dma_set_mask into dma_set_mask dma-mapping: remove the set_dma_mask method powerpc/cell: use the dma_supported method for ops switching powerpc/cell: clean up fixed mapping dma_ops initialization tile: remove dma_supported and mapping_error methods xen-swiotlb: remove xen_swiotlb_set_dma_mask arm: implement ->dma_supported instead of ->set_dma_mask mips/loongson64: implement ->dma_supported instead of ->set_dma_mask ...
generated by cgit (git 2.34.1) at 2024-05-19 21:25:36 +0000