summaryrefslogtreecommitdiff
path: root/lib/logic_pio.c
AgeCommit message (Collapse)Author
2023-08-24minmax: add in_range() macroMatthew Wilcox (Oracle)
Patch series "New page table range API", v6. This patchset changes the API used by the MM to set up page table entries. The four APIs are: set_ptes(mm, addr, ptep, pte, nr) update_mmu_cache_range(vma, addr, ptep, nr) flush_dcache_folio(folio) flush_icache_pages(vma, page, nr) flush_dcache_folio() isn't technically new, but no architecture implemented it, so I've done that for them. The old APIs remain around but are mostly implemented by calling the new interfaces. The new APIs are based around setting up N page table entries at once. The N entries belong to the same PMD, the same folio and the same VMA, so ptep++ is a legitimate operation, and locking is taken care of for you. Some architectures can do a better job of it than just a loop, but I have hesitated to make too deep a change to architectures I don't understand well. One thing I have changed in every architecture is that PG_arch_1 is now a per-folio bit instead of a per-page bit when used for dcache clean/dirty tracking. This was something that would have to happen eventually, and it makes sense to do it now rather than iterate over every page involved in a cache flush and figure out if it needs to happen. The point of all this is better performance, and Fengwei Yin has measured improvement on x86. I suspect you'll see improvement on your architecture too. Try the new will-it-scale test mentioned here: https://lore.kernel.org/linux-mm/20230206140639.538867-5-fengwei.yin@intel.com/ You'll need to run it on an XFS filesystem and have CONFIG_TRANSPARENT_HUGEPAGE set. This patchset is the basis for much of the anonymous large folio work being done by Ryan, so it's received quite a lot of testing over the last few months. This patch (of 38): Determine if a value lies within a range more efficiently (subtraction + comparison vs two comparisons and an AND). It also has useful (under some circumstances) behaviour if the range exceeds the maximum value of the type. Convert all the conflicting definitions of in_range() within the kernel; some can use the generic definition while others need their own definition. Link: https://lkml.kernel.org/r/20230802151406.3735276-1-willy@infradead.org Link: https://lkml.kernel.org/r/20230802151406.3735276-2-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2021-02-17PCI: Fix pci_register_io_range() memory leakGeert Uytterhoeven
Kmemleak reports: unreferenced object 0xc328de40 (size 64): comm "kworker/1:1", pid 21, jiffies 4294938212 (age 1484.670s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 e0 d8 fc eb 00 00 00 00 ................ 00 00 10 fe 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<ad758d10>] pci_register_io_range+0x3c/0x80 [<2c7f139e>] of_pci_range_to_resource+0x48/0xc0 [<f079ecc8>] devm_of_pci_get_host_bridge_resources.constprop.0+0x2ac/0x3ac [<e999753b>] devm_of_pci_bridge_init+0x60/0x1b8 [<a895b229>] devm_pci_alloc_host_bridge+0x54/0x64 [<e451ddb0>] rcar_pcie_probe+0x2c/0x644 In case a PCI host driver's probe is deferred, the same I/O range may be allocated again, and be ignored, causing a memory leak. Fix this by (a) letting logic_pio_register_range() return -EEXIST if the passed range already exists, so pci_register_io_range() will free it, and by (b) making pci_register_io_range() not consider -EEXIST an error condition. Link: https://lore.kernel.org/r/20210202100332.829047-1-geert+renesas@glider.be Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2020-05-07logic_pio: Use _inX() and _outX()John Garry
Use _inX() and _outX(), which include memory barriers which may be overridden per arch. Reviewed-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Wei Xu <xuwei5@hisilicon.com>
2020-05-07logic_pio: Improve macro argument nameJohn Garry
Macro argument "bw" is used for building byte, word, and long-based functions. Use "bwl" instead, to include long. Reviewed-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Wei Xu <xuwei5@hisilicon.com>
2019-11-05lib: logic_pio: Enforce LOGIC_PIO_INDIRECT region ops are set at registrationJohn Garry
Since the only LOGIC_PIO_INDIRECT host (hisi-lpc) now sets the ops prior to registration, enforce this check for accessors ops at registration instead of in the IO port accessors to simplify and marginally optimise the code. A slight misalignment is also tidied. Also add myself as an author. Suggested-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Wei Xu <xuwei5@hisilicon.com>
2019-08-13lib: logic_pio: Add logic_pio_unregister_range()John Garry
Add a function to unregister a logical PIO range. Logical PIO space can still be leaked when unregistering certain LOGIC_PIO_CPU_MMIO regions, but this acceptable for now since there are no callers to unregister LOGIC_PIO_CPU_MMIO regions, and the logical PIO region allocation scheme would need significant work to improve this. Cc: stable@vger.kernel.org Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Wei Xu <xuwei5@hisilicon.com>
2019-08-13lib: logic_pio: Avoid possible overlap for unregistering regionsJohn Garry
The code was originally written to not support unregistering logical PIO regions. To accommodate supporting unregistering logical PIO regions, subtly modify LOGIC_PIO_CPU_MMIO region registration code, such that the "end" of the registered regions is the "end" of the last region, and not the sum of the sizes of all the registered regions. Cc: stable@vger.kernel.org Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Wei Xu <xuwei5@hisilicon.com>
2019-08-13lib: logic_pio: Fix RCU usageJohn Garry
The traversing of io_range_list with list_for_each_entry_rcu() is not properly protected by rcu_read_lock() and rcu_read_unlock(), so add them. These functions mark the critical section scope where the list is protected for the reader, it cannot be "reclaimed". Any updater - in this case, the logical PIO registration functions - cannot update the list until the reader exits this critical section. In addition, the list traversing used in logic_pio_register_range() does not need to use the rcu variant. This is because we are already using io_range_mutex to guarantee mutual exclusion from mutating the list. Cc: stable@vger.kernel.org Fixes: 031e3601869c ("lib: Add generic PIO mapping method") Signed-off-by: John Garry <john.garry@huawei.com> Signed-off-by: Wei Xu <xuwei5@hisilicon.com>
2018-03-21lib: Add generic PIO mapping methodZhichang Yuan
41f8bba7f555 ("of/pci: Add pci_register_io_range() and pci_pio_to_address()") added support for PCI I/O space mapped into CPU physical memory space. With that support, the I/O ranges configured for PCI/PCIe hosts on some architectures can be mapped to logical PIO and converted easily between CPU address and the corresponding logical PIO. Based on this, PCI I/O port space can be accessed via in/out accessors that use memory read/write. But on some platforms, there are bus hosts that access I/O port space with host-local I/O port addresses rather than memory addresses. Add a more generic I/O mapping method to support those devices. With this patch, both the CPU addresses and the host-local port can be mapped into the logical PIO space with different logical/fake PIOs. After this, all the I/O accesses to either PCI MMIO devices or host-local I/O peripherals can be unified into the existing I/O accessors defined in asm-generic/io.h and be redirected to the right device-specific hooks based on the input logical PIO. Tested-by: dann frazier <dann.frazier@canonical.com> Signed-off-by: Zhichang Yuan <yuanzhichang@hisilicon.com> Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com> Signed-off-by: John Garry <john.garry@huawei.com> [bhelgaas: remove -EFAULT return from logic_pio_register_range() per https://lkml.kernel.org/r/20180403143909.GA21171@ulmo, fix NULL pointer checking per https://lkml.kernel.org/r/20180403211505.GA29612@embeddedor.com] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>