| Age | Commit message (Collapse) | Author |
|---|
|
- Add pci_epc_function_is_valid() to avoid repeating common validation
checks (Damien Le Moal)
- Skip attempts to allocate from endpoint controller memory window if the
requested size is larger than the window (Damien Le Moal)
- Add and document pci_epc_mem_map() and pci_epc_mem_unmap() to handle
controller-specific size and alignment constraints, and add test cases to
the endpoint test driver (Damien Le Moal)
- Implement dwc pci_epc_ops.align_addr() so pci_epc_mem_map() can observe
DWC-specific alignment requirements (Damien Le Moal)
- Synchronously cancel command handler work in endpoint test before
cleaning up DMA and BARs (Damien Le Moal)
- Respect endpoint page size in dw_pcie_ep_align_addr() (Niklas Cassel)
- Use dw_pcie_ep_align_addr() in dw_pcie_ep_raise_msi_irq() and
dw_pcie_ep_raise_msix_irq() instead of open coding the equivalent (Niklas
Cassel)
- Remove superfluous 'return' from pci_epf_test_clean_dma_chan() (Wang
Jiang)
- Avoid NULL dereference if Modem Host Interface Endpoint lacks 'mmio' DT
property (Zhongqiu Han)
- Release PCI domain ID of Endpoint controller parent (not controller
itself) and before unregistering the controller, to avoid use-after-free
(Zijun Hu)
- Clear secondary (not primary) EPC in pci_epc_remove_epf() when removing
the secondary controller associated with an NTB (Zijun Hu)
- Fix pci_epc_map map_size kerneldoc (Rick Wertenbroek)
* pci/endpoint:
PCI: endpoint: Fix pci_epc_map map_size kerneldoc string
PCI: endpoint: Clear secondary (not primary) EPC in pci_epc_remove_epf()
PCI: endpoint: Fix PCI domain ID release in pci_epc_destroy()
PCI: endpoint: epf-mhi: Avoid NULL dereference if DT lacks 'mmio'
PCI: endpoint: Remove surplus return statement from pci_epf_test_clean_dma_chan()
PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()
PCI: endpoint: test: Synchronously cancel command handler work
PCI: dwc: endpoint: Implement the pci_epc_ops::align_addr() operation
PCI: endpoint: test: Use pci_epc_mem_map/unmap()
PCI: endpoint: Update documentation
PCI: endpoint: Introduce pci_epc_mem_map()/unmap()
PCI: endpoint: Improve pci_epc_mem_alloc_addr()
PCI: endpoint: Introduce pci_epc_function_is_valid()
|
|
- Update mediatek-gen3 DT binding to require the exact number of clocks for
each SoC (Fei Shao)
- Add qcom SAR2130P DT binding with an additional clock (Dmitry Baryshkov)
* pci/dt-bindings:
dt-bindings: PCI: snps,dw-pcie: Drop "#interrupt-cells" from example
dt-bindings: PCI: qcom,pcie-sm8550: Add SAR2130P compatible
dt-bindings: PCI: mediatek-gen3: Allow exact number of clocks only
|
|
- Add ACS quirk for Wangxun FF5xxx NICs, which don't advertise and ACS
capability but do isolate functions as though PCI_ACS_RR and PCI_ACS_CR
were set, so the functions can be in independent IOMMU groups (Mengyuan
Lou)
* pci/virtualization:
PCI: Add ACS quirk for Wangxun FF5xxx NICs
|
|
- Add and document TLP Processing Hints (TPH) support so drivers can enable
and disable TPH and the kernel can save/restore TPH configuration (Wei
Huang)
- Add TPH Steering Tag support so drivers can retrieve Steering Tag values
associated with specific CPUs via an ACPI _DSM to direct DMA writes
closer to their consumers (Wei Huang)
* pci/tph:
PCI/TPH: Add TPH documentation
PCI/TPH: Add Steering Tag support
PCI: Add TLP Processing Hints (TPH) support
|
|
- Detect some Thunderbolt chips that are built-in and hence 'trustworthy'
by a heuristic since the 'ExternalFacingPort' and 'usb4-host-interface'
ACPI properties are not quite enough (Esther Shimanovich)
* pci/thunderbolt:
PCI: Detect and trust built-in Thunderbolt chips
|
|
- Add resource_set_size() to set resource size when start has already been
set (Ilpo Järvinen)
- Add resource_set_range() helper to set both resource start and size (Ilpo
Järvinen)
- Use IS_ALIGNED() and resource_size() in quirk_s3_64M() instead of
open-coding them (Ilpo Järvinen)
- Add ALIGN_DOWN_IF_NONZERO() to avoid code duplication when distributing
resources across devices (Ilpo Järvinen)
- Improve pdev_sort_resources() warning message to be more specific (Ilpo
Järvinen)
* pci/resource:
PCI: Improve pdev_sort_resources() warning message
PCI: Add ALIGN_DOWN_IF_NONZERO() helper
PCI: Use align and resource helpers, and SZ_* in quirk_s3_64M()
PCI: Use resource_set_{range,size}() helpers
resource: Add resource set range and size helpers
|
|
- Add sysfs 'reset_subordinate' to reset hierarchy below bridge (Keith
Busch)
- Warn if we reset a running device where driver didn't register
pci_error_handlers notification callbacks (Keith Busch)
* pci/reset:
PCI: Warn if a running device is unaware of reset
PCI: Add 'reset_subordinate' to reset hierarchy below bridge
|
|
- Use of_platform_device_create() instead of of_platform_populate() to
create pwrctl platform devices so we can control it based on the child
nodes (Manivannan Sadhasivam)
- Create pwrctrl platform devices only if there's a relevant power supply
property (Manivannan Sadhasivam)
- Add device link from the pwrctl supplier to the PCI dev to ensure pwrctl
drivers are probed before the PCI dev driver; this avoids a race where
pwrctl could change device power state while the PCI driver was active
(Manivannan Sadhasivam)
- Find pwrctl device for removal with of_find_device_by_node() instead of
searching all children of the parent (Manivannan Sadhasivam)
- Rename 'pwrctl' to 'pwrctrl' to use the same 'ctrl' suffix as 'bwctrl'
and other PCI files to reduce confusion (Bjorn Helgaas)
* pci/pwrctl:
PCI/pwrctrl: Rename pwrctrl functions and structures
PCI/pwrctrl: Rename pwrctl files to pwrctrl
PCI/pwrctl: Remove pwrctl device without iterating over all children of pwrctl parent
PCI/pwrctl: Ensure that pwrctl drivers are probed before PCI client drivers
PCI/pwrctl: Create pwrctl device only if at least one power supply is present
PCI/pwrctl: Use of_platform_device_create() to create pwrctl devices
# Conflicts:
# drivers/pci/bus.c
# drivers/pci/remove.c
|
|
- Enable starfive controller runtime PM before probing host bridge (Mayank
Rana)
- Enable runtime power management for host bridges (Krishna chaitanya
chundru)
* pci/pm:
PCI: Enable runtime PM of the host bridge
PCI: starfive: Enable controller runtime PM before probing host bridge
|
|
- Use PCI bus addresses (not CPU addresses) in 'ranges' properties when
building dynamic DT nodes so systems where the PCI and CPU addresses
space differ work correctly (Andrea della Porta)
* pci/of:
PCI: of_property: Assign PCI instead of CPU bus address to dynamic PCI nodes
|
|
- Make pci_stop_dev() and pci_destroy_dev() concurrent safe (Keith Busch)
- Move __pci_walk_bus() mutex up into the caller, which avoids the need for
a parameter to control locking (Keith Busch)
- Simplify __pci_walk_bus() by making it recursive (Keith Busch)
- Unexport pci_walk_bus_locked(), which is only used internally by the PCI
core (Keith Busch)
* pci/locking:
PCI: Unexport pci_walk_bus_locked()
PCI: Convert __pci_walk_bus() to be recursive
PCI: Move __pci_walk_bus() mutex to where we need it
PCI: Make pci_destroy_dev() concurrent safe
PCI: Make pci_stop_dev() concurrent safe
|
|
- Add hotplug controller driver for Marvell OCTEON multi-function device
where function 0 has a management console interface to enable/disable and
provision various personalities for the other functions (Shijith Thotton)
* pci/hotplug-octeon:
PCI: hotplug: Add OCTEON PCI hotplug controller driver
|
|
- Remove unused cpcihp struct cpci_hp_controller_ops.hardware_test
(Guilherme Giacomo Simoes)
- Remove unused cpqphp struct ctrl_dbg.ctrl (Christophe JAILLET)
- Clean up cpqphp PCIBIOS_* return value confusion (Ilpo Järvinen)
- Use pci_bus_read_dev_vendor_id() instead of hand-coded presence detection
in cpqphp (Ilpo Järvinen)
- Simplify cpqphp enumeration, which is already simple-minded and doesn't
handle devices below hot-added bridges (Ilpo Järvinen)
- Retain a reference to the pci_bus for the lifetime of a pci_slot to avoid
a use-after-free when the thunderbolt driver resets USB4 host routers on
boot, causing hotplug remove/add of downstream docks or other devices
(Lukas Wunner)
* pci/hotplug:
PCI: Fix use-after-free of slot->bus on hot remove
PCI: cpqphp: Simplify PCI_ScanBusForNonBridge()
PCI: cpqphp: Use define to read class/revision dword
PCI: cpqphp: Use pci_bus_read_dev_vendor_id() to detect presence
PCI: cpqphp: Fix PCIBIOS_* return value confusion
PCI: cpqphp: Remove unused struct ctrl_dbg.ctrl
PCI: cpcihp: Remove unused struct cpci_hp_controller_ops.hardware_test
|
|
- Simplify pci_read_bridge_bases() logic (Ilpo Järvinen)
* pci/enumeration:
PCI: Simplify pci_read_bridge_bases() logic
PCI: Move struct pci_bus_resource into bus.c
PCI: Remove unused PCI_SUBTRACTIVE_DECODE
|
|
- Convert driver .remove_new() callbacks to .remove() again to finish the
conversion from returning 'int' to being 'void' (Sergio Paracuellos)
* pci/driver-remove:
PCI: acpiphp_ampere_altra: Switch back to struct platform_driver::remove()
PCI: controller: Switch back to struct platform_driver::remove()
|
|
- Export pcim_request_all_regions(), a managed interface to request all
BARs (Philipp Stanner)
- Replace pcim_iomap_regions_request_all() with pcim_request_all_regions(),
and pcim_iomap_table()[n] with pcim_iomap(n), in the following drivers:
ahci, crypto qat, crypto octeontx2, intel_th, iwlwifi, ntb idt, serial
rp2, ALSA korg1212 (Philipp Stanner)
- Remove the now unused pcim_iomap_regions_request_all() (Philipp Stanner)
- Export pcim_iounmap_region(), a managed interface to unmap and release a
PCI BAR (Philipp Stanner)
- Replace pcim_iomap_regions(mask) with pcim_iomap_region(n), and
pcim_iounmap_regions(mask) with pcim_iounmap_region(n), in the following
drivers: fpga dfl-pci, block mtip32xx, gpio-merrifield, cavium (Philipp
Stanner)
* pci/devm:
ethernet: cavium: Replace deprecated PCI functions
gpio: Replace deprecated PCI functions
fpga/dfl-pci.c: Replace deprecated PCI functions
PCI: Deprecate pcim_iounmap_regions()
PCI: Make pcim_iounmap_region() a public function
PCI: Remove pcim_iomap_regions_request_all()
ALSA: korg1212: Replace deprecated PCI functions
serial: rp2: Replace deprecated PCI functions
ntb: idt: Replace deprecated PCI functions
wifi: iwlwifi: replace deprecated PCI functions
intel_th: pci: Replace deprecated PCI functions
crypto: marvell - replace deprecated PCI functions
crypto: qat - replace deprecated PCI functions
ata: ahci: Replace deprecated PCI functions
PCI: Make pcim_request_all_regions() a public function
|
|
- Wait up to 1 second for DOE Busy bit to clear before writing a request to
the mailbox to avoid failures if the mailbox is still busy from a
previous transfer (Gregory Price)
* pci/doe:
PCI/DOE: Poll DOE Busy bit for up to 1 second in pci_doe_send_req()
|
|
- Add read/modify/write locking for Link Control 2, which is used to manage
Link speed (Ilpo Järvinen)
- Cache all supported Link speeds for use by the PCIe bandwidth controller
(Ilpo Järvinen)
- Extract the Link Bandwidth Management Status check into pcie_lbms_seen(),
where it can be shared between the bandwidth controller and quirks that
use it to help retrain failed links (Ilpo Järvinen)
- Re-add Link Bandwidth notification support with updates to address the
reasons it was previously reverted (Alexandru Gagniuc, Ilpo Järvinen)
- Add pcie_set_target_speed() and related functionality to manage PCIe Link
speed based on thermal constraints (Ilpo Järvinen)
- Add a thermal cooling driver to throttle PCIe Links via the existing
thermal management framework (Ilpo Järvinen)
- Add a userspace selftest for the PCIe bandwidth controller (Ilpo
Järvinen)
- Drop duplicate pcie_get_speed_cap(), pcie_get_width_cap() declarations
(Bjorn Helgaas)
* pci/bwctrl:
PCI: Drop duplicate pcie_get_speed_cap(), pcie_get_width_cap() declarations
selftests/pcie_bwctrl: Create selftests
thermal: Add PCIe cooling driver
PCI/bwctrl: Add pcie_set_target_speed() to set PCIe Link Speed
PCI/bwctrl: Re-add BW notification portdrv as PCIe BW controller
PCI: Abstract LBMS seen check into pcie_lbms_seen()
PCI: Refactor pcie_update_link_speed()
PCI: Store all PCIe Supported Link Speeds
PCI: Protect Link Control 2 Register with RMW locking
Documentation PCI: Reformat RMW ops documentation
|
|
- Disable ASPM L1 before touching L1 PM Substates to follow the spec
closer and avoid a CPU load timeout on some platforms (Ajay Agarwal)
- Set devices below Intel VMD to D0 before enabling ASPM L1 Substates
(Jian-Hong Pan)
* pci/aspm:
PCI: vmd: Set devices to D0 before enabling PM L1 Substates
PCI/ASPM: Add notes about enabling PCI-PM L1SS to pci_enable_link_state(_locked)
PCI/ASPM: Disable L1 before disabling L1 PM Substates
|
|
Rename pwrctrl functions and structures from "pwrctl" to "pwrctrl" to match
the similar file renames.
Link: https://lore.kernel.org/r/20241115214428.2061153-3-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Krzysztof Wilczyński <kw@linux.com>
|
|
To slightly reduce confusion between "pwrctl" (the power controller and
power sequencing framework) and "bwctrl" (the bandwidth controller),
rename "pwrctl" to "pwrctrl" so they use the same "ctrl" suffix.
Rename drivers/pci/pwrctl/ to drivers/pci/pwrctrl/, including the related
MAINTAINERS, include file (include/linux/pci-pwrctl.h), Makefile, and
Kconfig changes.
This is the minimal rename of files only. A subsequent commit will rename
functions and data structures.
Link: https://lore.kernel.org/r/20241115214428.2061153-2-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Krzysztof Wilczyński <kw@linux.com>
|
|
pwrctl parent
There is no need to iterate over all children of the pwrctl device parent
to remove the pwrctl device. Since the pwrctl device associated with the
PCI device can be found using of_find_device_by_node() API, use it directly
instead.
Any pwrctl devices lying around without getting associated with the PCI
devices will be removed once their parent device gets removed.
Link: https://lore.kernel.org/r/20241025-pci-pwrctl-rework-v2-5-568756156cbe@linaro.org
Tested-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Tested-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
|
|
As per the kernel device driver model, a pwrctl device is the supplier for
the PCI device, but the device link that enforces the supplier-consumer
relationship was previously created by the pwrctl driver. Therefore, the
driver model didn't prevent probing PCI client drivers before probing the
corresponding pwrctl drivers. This may lead to a race condition if the PCI
device was already powered on by the bootloader (before the pwrctl driver).
If the bootloader did not power on the PCI device, this wouldn't create any
problem as the pwrctl driver will be the one powering on the device, so the
PCI client driver always gets probed afterward. But if the device was
already powered on, then the device will be seen by the PCI core and the
PCI client driver may get probed before its pwrctl driver. This creates a
race condition as the pwrctl driver may change the device power state while
the device is being accessed by the client driver.
One such issue was already reported on the Qcom X13s platform with the WLAN
device and fixed with a hack in the WCN pwrseq driver by a9aaf1ff88a8
("power: sequencing: request the WLAN enable GPIO as-is").
A cleaner way to fix the above mentioned race condition is to ensure that
the pwrctl drivers are always probed before the client drivers.
If the PCI device is associated with a pwrctl platform device with a power
supply, add a device link between the PCI device and the pwrctl device
before device_attach() in pci_bus_add_device().
Note that there is no need to explicitly remove the device link as that
will be taken care of by the driver core when the PCI device gets removed.
Fixes: 4565d2652a37 ("PCI/pwrctl: Add PCI power control core code")
Fixes: 8fb18619d910 ("PCI/pwrctl: Create platform devices for child OF nodes of the port node")
Link: https://lore.kernel.org/r/20241025-pci-pwrctl-rework-v2-3-568756156cbe@linaro.org
Tested-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Tested-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[bhelgaas: squash fix from
https://lore.kernel.org/r/20241120062459.6371-1-manivannan.sadhasivam@linaro.org
for SPARCv9 issue reported by Jonathan Currier <dullfire@yahoo.com>]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
[kwilczynski: wrap code to 80 columns]
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Cc: stable+noautosel@kernel.org # Depends on power supply check
|
|
Currently, pwrctl devices are created if the corresponding PCI nodes are
defined in devicetree. But this is not correct, because not all PCI nodes
require pwrctl support. Pwrctl comes into the picture only when the device
requires kernel to manage its power state. This can be determined using the
power supply properties present in the devicetree node of the device.
Add of_pci_supply_present() to check whether the devicetree contains at
least one power supply property for a device. If one is present, create a
pwrctl device for that PCI node.
Suggested-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Fixes: 8fb18619d910 ("PCI/pwrctl: Create platform devices for child OF nodes of the port node")
Link: https://lore.kernel.org/r/20241025-pci-pwrctl-rework-v2-2-568756156cbe@linaro.org
Tested-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Tested-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[bhelgaas: rename of_pci_is_supply_present() to of_pci_supply_present() for
readability]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Cc: stable+noautosel@kernel.org # Depends on of_platform_device_create() rework
|
|
The of_platform_populate() API creates platform devices by descending
through the children of the parent node. But it provides no control over
the child nodes, which makes it difficult to add checks for the child
nodes in the future.
Use of_platform_device_create() and for_each_child_of_node_scoped() to make
it possible to add checks for each node before creating the platform
device.
Link: https://lore.kernel.org/r/20241025-pci-pwrctl-rework-v2-1-568756156cbe@linaro.org
Tested-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
Tested-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
|
|
Because some endpoint controllers have requirements on the alignment of
the controller physical memory address that must be used to map a RC PCI
address region, the map PCI start address is not necessarily the desired
PCI base address to be mapped. This can result in map_pci_addr being
lower than pci_addr as documented. This results in map_size covering the
range map_pci_addr..pci_addr+pci_size.
The old text had the pci_addr twice instead of map_pci_addr..pci_addr,
so replace the erroneous kerneldoc string to reflect the actual range.
Link: https://lore.kernel.org/r/20241114161032.3046202-1-rick.wertenbroek@gmail.com
Signed-off-by: Rick Wertenbroek <rick.wertenbroek@gmail.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
|
|
In addition to a primary endpoint controller, an endpoint function may be
associated with a secondary endpoint controller, epf->sec_epc, to provide
NTB (non-transparent bridge) functionality.
Previously, pci_epc_remove_epf() incorrectly cleared epf->epc instead of
epf->sec_epc when removing from the secondary endpoint controller.
Extend the epc->list_lock coverage and clear either epf->epc or
epf->sec_epc as indicated.
Link: https://lore.kernel.org/r/20241107-epc_rfc-v2-2-da5b6a99a66f@quicinc.com
Fixes: 63840ff53223 ("PCI: endpoint: Add support to associate secondary EPC with EPF")
Signed-off-by: Zijun Hu <quic_zijuhu@quicinc.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[mani: reworded subject and description]
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Cc: stable@vger.kernel.org
|
|
pci_epc_destroy() invokes pci_bus_release_domain_nr() to release the PCI
domain ID, but there are two issues:
- 'epc->dev' is passed to pci_bus_release_domain_nr() which was already
freed by device_unregister(), leading to a use-after-free issue.
- Domain ID corresponds to the EPC device parent, so passing 'epc->dev'
is also wrong.
Fix these issues by passing 'epc->dev.parent' to
pci_bus_release_domain_nr() and also do it before device_unregister().
Fixes: 0328947c5032 ("PCI: endpoint: Assign PCI domain number for endpoint controllers")
Signed-off-by: Zijun Hu <quic_zijuhu@quicinc.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/r/20241107-epc_rfc-v2-1-da5b6a99a66f@quicinc.com
[mani: reworded subject and description]
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Cc: stable@vger.kernel.org
|
|
If platform_get_resource_byname() fails and returns NULL because DT lacks
an 'mmio' property for the MHI endpoint, dereferencing res->start will
cause a NULL pointer access. Add a check to prevent it.
Fixes: 1bf5f25324f7 ("PCI: endpoint: Add PCI Endpoint function driver for MHI bus")
Link: https://lore.kernel.org/r/20241105120735.1240728-1-quic_zhonhan@quicinc.com
Signed-off-by: Zhongqiu Han <quic_zhonhan@quicinc.com>
[kwilczynski: error message update per the review feedback]
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Niklas Cassel <cassel@kernel.org>
|
|
pci_epf_test_clean_dma_chan()
Remove a surplus return statement from the void function that has been
added in the commit commit 8353813c88ef ("PCI: endpoint: Enable DMA
tests for endpoints with DMA capabilities").
Especially, as an empty return statements at the end of a void functions
serve little purpose.
This fixes the following checkpatch.pl script warning:
WARNING: void function return statements are not generally useful
#296: FILE: drivers/pci/endpoint/functions/pci-epf-test.c:296:
+ return;
+}
Link: https://lore.kernel.org/r/tencent_F250BEE2A65745A524E2EFE70CF615CA8F06@qq.com
Signed-off-by: Wang Jiang <jiangwang@kylinos.cn>
[kwilczynski: commit log]
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
|
|
Use the dw_pcie_ep_align_addr() function to calculate the alignment in
dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.
Link: https://lore.kernel.org/r/20241017132052.4014605-6-cassel@kernel.org
Link: https://lore.kernel.org/r/20241104205144.409236-2-cassel@kernel.org
Tested-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Niklas Cassel <cassel@kernel.org>
[kwilczynski: squashed patch that fixes memory map sizes]
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
|
|
6cf57be0f78e ("PCI: Add pcie_get_speed_cap() to find max supported link
speed") and c70b65fb7f12 ("PCI: Add pcie_get_width_cap() to find max
supported link width") added declarations to drivers/pci/pci.h.
576c7218a154 ("PCI: Export pcie_get_speed_cap and pcie_get_width_cap")
subsequently added duplicates to include/linux/pci.h.
Remove the originals from drivers/pci/pci.h. Both interfaces are used by
amdgpu, so they must be in include/linux/pci.h.
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Krzysztof Wilczyński <kw@linux.com>
|
|
Create selftests for PCIe BW control through the PCIe cooling device sysfs
interface.
First, the BW control selftest finds the PCIe Port to test with. By
default, the PCIe Port with the highest Link Speed is selected but
another PCIe Port can be provided with -d parameter.
The actual test steps the cur_state of the cooling device one-by-one
from max_state to what the cur_state was initially. The speed change
is confirmed by observing the current_link_speed for the corresponding
PCIe Port.
Link: https://lore.kernel.org/r/20241018144755.7875-10-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
|
Add a thermal cooling driver to provide path to access PCIe bandwidth
controller using the usual thermal interfaces.
A cooling device is instantiated for controllable PCIe Ports from the
bwctrl service driver.
If registering the cooling device fails, allow bwctrl's probe to succeed
regardless. As cdev in that case contains IS_ERR() pseudo "pointer", clean
that up inside the probe function so the remove side doesn't need to
suddenly make an odd looking IS_ERR() check.
The thermal side state 0 means no throttling, i.e., maximum supported PCIe
Link Speed.
Link: https://lore.kernel.org/r/20241018144755.7875-9-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
[bhelgaas: dropped data->cdev test per
https://lore.kernel.org/r/ZzRm1SJTwEMRsAr8@wunner.de]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: Rafael J. Wysocki <rafael@kernel.org> # From the cooling device interface perspective
|
|
Currently, PCIe Link Speeds are adjusted by custom code rather than in a
common function provided in PCI core. The PCIe bandwidth controller
(bwctrl) introduces an in-kernel API, pcie_set_target_speed(), to set PCIe
Link Speed.
Convert Target Speed quirk to use the new API. The Target Speed quirk runs
very early when bwctrl is not yet probed for a Port and can also run later
when bwctrl is already setup for the Port, which requires the per port
mutex (set_speed_mutex) to be only taken if the bwctrl setup is already
complete.
The new API is also intended to be used in an upcoming commit that adds a
thermal cooling device to throttle PCIe bandwidth when thermal thresholds
are reached.
The PCIe bandwidth control procedure is as follows. The highest speed
supported by the Port and the PCIe device which is not higher than the
requested speed is selected and written into the Target Link Speed in the
Link Control 2 Register. Then bandwidth controller retrains the PCIe Link.
Bandwidth Notifications enable the cur_bus_speed in the struct pci_bus to
keep track PCIe Link Speed changes. While Bandwidth Notifications should
also be generated when bandwidth controller alters the PCIe Link Speed, a
few platforms do not deliver LMBS interrupt after Link Training as
expected. Thus, after changing the Link Speed, bandwidth controller makes
additional read for the Link Status Register to ensure cur_bus_speed is
consistent with the new PCIe Link Speed.
Link: https://lore.kernel.org/r/20241018144755.7875-8-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
[bhelgaas: squash devm_mutex_init() error checking from
https://lore.kernel.org/r/20241030163139.2111689-1-andriy.shevchenko@linux.intel.com,
drop export of pcie_set_target_speed()]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
|
This mostly reverts the commit b4c7d2076b4e ("PCI/LINK: Remove bandwidth
notification"). An upcoming commit extends this driver building PCIe
bandwidth controller on top of it.
PCIe bandwidth notifications were first added in the commit e8303bb7a75c
("PCI/LINK: Report degraded links via link bandwidth notification") but
later had to be removed. The significant changes compared with the old
bandwidth notification driver include:
1) Don't print the notifications into kernel log, just keep the Link
Speed cached in struct pci_bus updated. While somewhat unfortunate,
the log spam was the source of complaints that eventually lead to
the removal of the bandwidth notifications driver (see the links
below for further information).
2) Besides the Link Bandwidth Management Interrupt, also enable Link
Autonomous Bandwidth Interrupt to cover the other source of bandwidth
changes.
3) Handle Link Speed updates robustly. Refresh the cached Link Speed
when enabling Bandwidth Notification Interrupts, and solve the race
between Link Speed read and LBMS/LABS update in
pcie_bwnotif_irq_thread().
4) Use concurrency safe LNKCTL RMW operations.
5) The driver is now called PCIe bwctrl (bandwidth controller) instead
of just bandwidth notifications because of increased scope and
functionality within the driver.
6) Coexist with the Target Link Speed quirk in pcie_failed_link_retrain().
Provide LBMS counting API for it.
7) Tweaks to variable/functions names for consistency and length reasons.
Bandwidth Notifications enable the cur_bus_speed in the struct pci_bus to
keep track PCIe Link Speed changes.
[bhelgaas: This is based on previous work by Alexandru Gagniuc
<mr.nuke.me@gmail.com>; see e8303bb7a75c ("PCI/LINK: Report degraded links
via link bandwidth notification")]
Link: https://lore.kernel.org/r/20241018144755.7875-7-ilpo.jarvinen@linux.intel.com
Link: https://lore.kernel.org/all/20190429185611.121751-1-helgaas@kernel.org/
Link: https://lore.kernel.org/linux-pci/20190501142942.26972-1-keith.busch@intel.com/
Link: https://lore.kernel.org/linux-pci/20200115221008.GA191037@google.com/
Suggested-by: Lukas Wunner <lukas@wunner.de> # Building bwctrl on top of bwnotif
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
[bhelgaas: squash fix to drop IRQF_ONESHOT and convert to hardirq handler:
https://lore.kernel.org/r/20241115165717.15233-1-ilpo.jarvinen@linux.intel.com]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Tested-by: Stefan Wahren <wahrenst@gmx.net>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
|
Wangxun FF5xxx NICs are similar to SFxxx, RP1000 and RP2000 NICs. They may
be multi-function devices, but they do not advertise an ACS capability.
But the hardware does isolate FF5xxx functions as though it had an ACS
capability and PCI_ACS_RR and PCI_ACS_CR were set in the ACS Control
register, i.e., all peer-to-peer traffic is directed upstream instead of
being routed internally.
Add ACS quirk for FF5xxx NICs in pci_quirk_wangxun_nic_acs() so the
functions can be in independent IOMMU groups.
Link: https://lore.kernel.org/r/E16053DB2B80E9A5+20241115024604.30493-1-mengyuanlou@net-swift.com
Signed-off-by: Mengyuan Lou <mengyuanlou@net-swift.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
|
|
When populating "ranges" property for a PCI bridge or endpoint,
of_pci_prop_ranges() incorrectly uses the CPU address of the resource. In
such PCI nodes, the window should instead be in PCI address space. Call
pci_bus_address() on the resource in order to obtain the PCI bus address.
[Previous discussion at:
https://lore.kernel.org/all/8b4fa91380fc4754ea80f47330c613e4f6b6592c.1724159867.git.andrea.porta@suse.com/]
Link: https://lore.kernel.org/r/20241108094256.28933-1-andrea.porta@suse.com
Fixes: 407d1a51921e ("PCI: Create device tree node for bridge")
Tested-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Andrea della Porta <andrea.porta@suse.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org
|
|
Add a PCI hotplug controller driver for the OCTEON PCIe device. The OCTEON
PCIe device is a multi-function device where function 0 serves as the PCI
hotplug controller.
There is an out-of-band management console interface to firmware running on
function 0 whereby an administrator can disable functions to save power or
enable them with one of several personalities (virtio-net, virtio-crypto,
NVMe, etc) for the other functions. Function 0 initiates hotplug events
handled by this driver when the other functions are enabled or disabled.
+--------------------------------+
| Root Port |
+--------------------------------+
|
PCIe
|
+---------------------------------------------------------------+
| OCTEON PCIe Multifunction Device |
+---------------------------------------------------------------+
| | | |
| | | |
+---------------------+ +----------------+ +-----+ +----------------+
| Function 0 | | Function 1 | | ... | | Function 7 |
| (Hotplug controller)| | (Hotplug slot) | | | | (Hotplug slot) |
+---------------------+ +----------------+ +-----+ +----------------+
|
|
+-------------------------+
| Controller Firmware |
+-------------------------+
The hotplug controller driver enables hotplugging of non-controller
functions within the same device. During probing, the driver removes
the non-controller functions and registers them as PCI hotplug slots.
These slots are added back by the driver, only upon request from the
device firmware.
The controller uses MSI-X interrupts to notify the host of hotplug
events initiated by the OCTEON firmware. Additionally, the driver
allows users to enable or disable individual functions via sysfs slot
entries, as provided by the PCI hotplug framework.
Link: https://lore.kernel.org/r/20241111134523.2796699-1-sthotton@marvell.com
Co-developed-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Shijith Thotton <sthotton@marvell.com>
[bhelgaas: use pci_info() when possible]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
|
|
If a reset is issued to a running device with a driver that didn't register
the notification callbacks, the driver may be unaware of this event and
have an inconsistent view of the device's state. Log a warning of this
event because there's nothing else indicating the event occured, which
could be confusing when debugging such situations.
Link: https://lore.kernel.org/r/20241025222755.3756162-2-kbusch@meta.com
Signed-off-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Amey Narkhede <ameynarkhede03@gmail.com>
Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
|
|
The "bus" and "cxl_bus" reset methods reset a device by asserting Secondary
Bus Reset on the bridge leading to the device. These only work if the
device is the only device below the bridge.
Add a sysfs 'reset_subordinate' attribute on bridges that can assert
Secondary Bus Reset regardless of how many devices are below the bridge.
This resets all the devices below a bridge in a single command, including
the locking and config space save/restore that reset methods normally do.
This may be the only way to reset devices that don't support other reset
methods (ACPI, FLR, PM reset, etc).
Link: https://lore.kernel.org/r/20241025222755.3756162-1-kbusch@meta.com
Signed-off-by: Keith Busch <kbusch@kernel.org>
[bhelgaas: commit log, add capable(CAP_SYS_ADMIN) check]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Amey Narkhede <ameynarkhede03@gmail.com>
|
|
The controller driver is the parent device of the PCIe host bridge,
PCI-PCI bridge and PCIe endpoint as shown below.
PCIe controller (Top level parent & parent of host bridge)
|
v
PCIe Host bridge (Parent of PCI-PCI bridge)
|
v
PCI-PCI bridge (Parent of endpoint driver)
|
v
PCIe endpoint driver
Now, when the controller device goes to runtime suspend, PM framework will
check the runtime PM state of the child device (host bridge) and will find
it to be disabled. So it will allow the parent (controller device) to go to
runtime suspend. Only if the child device's state was 'active' it will
prevent the parent to get suspended.
It is a property of the runtime PM framework that it can only follow
continuous dependency chains. That is, if there is a device with runtime
PM disabled in a dependency chain, runtime PM cannot be enabled for devices
below it and above it in that chain both at the same time.
Since runtime PM is disabled for host bridge, the state of the child
devices under the host bridge is not taken into account by PM framework for
the top level parent, PCIe controller. So the PM framework allows the
controller driver to enter runtime PM irrespective of the state of the
devices under the host bridge. And this causes the topology breakage and
also possible PM issues like controller driver going to runtime suspend
while the endpoint driver is doing transfers.
Because of the above, in order to enable runtime PM for a PCIe controller
device, one needs to ensure that runtime PM is enabled for all devices in
every dependency chain between it and any PCIe endpoint (as runtime PM is
enabled for PCIe endpoints).
This means that runtime PM needs to be enabled for the host bridge device,
which is present in all of these dependency chains.
After this change, the host bridge device will be runtime-suspended by the
runtime PM framework automatically after suspending its last child and it
will be runtime-resumed automatically before resuming its first child which
will allow the runtime PM framework to track dependencies between the host
bridge device and all of its descendants.
The PM framework expects parent runtime PM to be enabled before enabling
runtime PM of the child. Ensure pm_runtime_enable() is called for the
controller drivers before calling pci_host_probe().
Link: https://lore.kernel.org/r/20241111-runtime_pm-v7-2-9c164eefcd87@quicinc.com
Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
|
|
A PCI controller device, e.g., StarFive, is parent to PCI host bridge
device. We must enable runtime PM of the controller before enabling runtime
PM of the host bridge, which will happen in pci_host_probe(), to avoid this
warning:
pcie-starfive 940000000.pcie: Enabling runtime PM for inactive device with active children
Fix this issue by enabling StarFive controller device's runtime PM before
calling pci_host_probe() in plda_pcie_host_init().
Link: https://lore.kernel.org/r/20241111-runtime_pm-v7-1-9c164eefcd87@quicinc.com
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Mayank Rana <quic_mrana@quicinc.com>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
|
|
There's only one user of pci_walk_bus_locked(), and it's internal to the
PCI core. Unexport it and make it private to drivers/pci/.
Link: https://lore.kernel.org/r/20241022224851.340648-6-kbusch@meta.com
Signed-off-by: Keith Busch <kbusch@kernel.org>
[bhelgaas: move decl to drivers/pci/pci.h]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
|
|
The Target Speed quirk in pcie_failed_link_retrain() uses the presence of
LBMS bit as one of the triggering conditions, effectively monopolizing the
use of that bit. An upcoming change will introduce a PCIe bandwidth
controller which sets up an interrupt to track LBMS. As LBMS will be
cleared by the interrupt handler, the Target Speed quirk will no longer be
able to observe LBMS directly.
As a preparatory step for the change, extract the LBMS seen check from
pcie_failed_link_retrain() into a new function pcie_lmbs_seen().
Link: https://lore.kernel.org/r/20241018144755.7875-6-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
|
pcie_update_link_speed() is passed the Link Status register but not all
callers have that value at hand nor need the value.
Refactor pcie_update_link_speed() to include reading the Link Status
register and create __pcie_update_link_speed() which can be used by the
hotplug code that has the register value at hand beforehand (and needs the
value for other purposes).
Link: https://lore.kernel.org/r/20241018144755.7875-5-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
|
The PCIe bandwidth controller added by a subsequent commit will require
selecting PCIe Link Speeds that are lower than the Maximum Link Speed.
The struct pci_bus only stores max_bus_speed. Even if PCIe r6.1 sec 8.2.1
currently disallows gaps in supported Link Speeds, the Implementation Note
in PCIe r6.1 sec 7.5.3.18, recommends determining supported Link Speeds
using the Supported Link Speeds Vector in the Link Capabilities 2 Register
(when available) to "avoid software being confused if a future
specification defines Links that do not require support for all slower
speeds."
Reuse code in pcie_get_speed_cap() to add pcie_get_supported_speeds() to
query the Supported Link Speeds Vector of a PCIe device. The value is taken
directly from the Supported Link Speeds Vector or synthesized from the Max
Link Speed in the Link Capabilities Register when the Link Capabilities 2
Register is not available.
The Supported Link Speeds Vector in the Link Capabilities Register 2
corresponds to the bus below on Root Ports and Downstream Ports, whereas it
corresponds to the bus above on Upstream Ports and Endpoints (PCIe r6.1 sec
7.5.3.18):
Supported Link Speeds Vector - This field indicates the supported Link
speed(s) of the associated Port.
Add supported_speeds into the struct pci_dev that caches the
Supported Link Speeds Vector.
supported_speeds contains a set of Link Speeds only in the case where PCIe
Link Speed can be determined. Root Complex Integrated Endpoints do not have
a well-defined Link Speed because they do not implement either of the Link
Capabilities Registers, which is allowed by PCIe r6.1 sec 7.5.3 (the same
limitation applies to determining cur_bus_speed and max_bus_speed that are
PCI_SPEED_UNKNOWN in such case). This is of no concern from PCIe bandwidth
controller point of view because such devices are not attached into a PCIe
Root Port that could be controlled.
The supported_speeds field keeps the extra reserved zero at the least
significant bit to match the Link Capabilities 2 Register layout.
An attempt was made to store supported_speeds field into the struct pci_bus
as an intersection of both ends of the Link, however, the subordinate
struct pci_bus is not available early enough. The Target Speed quirk (in
pcie_failed_link_retrain()) can run either during initial scan or later,
requiring it to use the API provided by the PCIe bandwidth controller to
set the Target Link Speed in order to co-exist with the bandwidth
controller. When the Target Speed quirk is calling the bandwidth controller
during initial scan, the struct pci_bus is not yet initialized. As such,
storing supported_speeds into the struct pci_bus is not viable.
Suggested-by: Lukas Wunner <lukas@wunner.de>
Link: https://lore.kernel.org/r/20241018144755.7875-4-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
[bhelgaas: move pcie_get_supported_speeds() decl to drivers/pci/pci.h]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
|
The original implementation of __pci_walk_bus() chose a non-recursive walk,
presumably as a precaution on stack use. We do recursive bus walking in
other places though. For example:
pci_bus_resettable()
pci_stop_bus_device()
pci_remove_bus_device()
pci_bus_allocate_dev_resources()
So recursive pci bus walking is well tested and safe, and is easier to
follow.
Convert __pci_walk_bus() to be recursive to make it easier to introduce
finer grain locking in the future.
Link: https://lore.kernel.org/r/20241022224851.340648-5-kbusch@meta.com
Signed-off-by: Keith Busch <kbusch@kernel.org>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
|
|
Simplify __pci_walk_bus() by moving the pci_bus_sem mutex into
pci_walk_bus(), the only place it is needed, and removing the parameter
that told __pci_walk_bus() whether to acquire the mutex.
Link: https://lore.kernel.org/r/20241022224851.340648-4-kbusch@meta.com
Signed-off-by: Keith Busch <kbusch@kernel.org>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
|
|
Use an atomic flag instead of the racy check against the device's kobj
parent. We shouldn't be poking into device implementation details at this
level anyway.
Link: https://lore.kernel.org/r/20241022224851.340648-3-kbusch@meta.com
Signed-off-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
|
