summaryrefslogtreecommitdiff
path: root/Documentation/accelerators
diff options
context:
space:
mode:
authorFrederic Barrat <fbarrat@linux.vnet.ibm.com>2018-01-23 12:31:47 +0100
committerMichael Ellerman <mpe@ellerman.id.au>2018-01-27 20:02:24 +1100
commit00b96c0e3c54a1fff9e8b909f46a672e43ef9598 (patch)
tree1cc4d60fd76c6b65077f6684f72db5a68ad044d8 /Documentation/accelerators
parent741ddae6c4c19bf2212ca318777320ae4f76c714 (diff)
ocxl: Documentation
ocxl.rst gives a quick, high-level view of opencapi. Update ioctl-number.txt to reflect ioctl numbers being used by the ocxl driver Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com> [mpe: Fix up mixed whitespace as spotted by gregkh] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Diffstat (limited to 'Documentation/accelerators')
-rw-r--r--Documentation/accelerators/ocxl.rst160
1 files changed, 160 insertions, 0 deletions
diff --git a/Documentation/accelerators/ocxl.rst b/Documentation/accelerators/ocxl.rst
new file mode 100644
index 000000000000..4f7af841d935
--- /dev/null
+++ b/Documentation/accelerators/ocxl.rst
@@ -0,0 +1,160 @@
+========================================================
+OpenCAPI (Open Coherent Accelerator Processor Interface)
+========================================================
+
+OpenCAPI is an interface between processors and accelerators. It aims
+at being low-latency and high-bandwidth. The specification is
+developed by the `OpenCAPI Consortium <http://opencapi.org/>`_.
+
+It allows an accelerator (which could be a FPGA, ASICs, ...) to access
+the host memory coherently, using virtual addresses. An OpenCAPI
+device can also host its own memory, that can be accessed from the
+host.
+
+OpenCAPI is known in linux as 'ocxl', as the open, processor-agnostic
+evolution of 'cxl' (the driver for the IBM CAPI interface for
+powerpc), which was named that way to avoid confusion with the ISDN
+CAPI subsystem.
+
+
+High-level view
+===============
+
+OpenCAPI defines a Data Link Layer (DL) and Transaction Layer (TL), to
+be implemented on top of a physical link. Any processor or device
+implementing the DL and TL can start sharing memory.
+
+::
+
+ +-----------+ +-------------+
+ | | | |
+ | | | Accelerated |
+ | Processor | | Function |
+ | | +--------+ | Unit | +--------+
+ | |--| Memory | | (AFU) |--| Memory |
+ | | +--------+ | | +--------+
+ +-----------+ +-------------+
+ | |
+ +-----------+ +-------------+
+ | TL | | TLX |
+ +-----------+ +-------------+
+ | |
+ +-----------+ +-------------+
+ | DL | | DLX |
+ +-----------+ +-------------+
+ | |
+ | PHY |
+ +---------------------------------------+
+
+
+
+Device discovery
+================
+
+OpenCAPI relies on a PCI-like configuration space, implemented on the
+device. So the host can discover AFUs by querying the config space.
+
+OpenCAPI devices in Linux are treated like PCI devices (with a few
+caveats). The firmware is expected to abstract the hardware as if it
+was a PCI link. A lot of the existing PCI infrastructure is reused:
+devices are scanned and BARs are assigned during the standard PCI
+enumeration. Commands like 'lspci' can therefore be used to see what
+devices are available.
+
+The configuration space defines the AFU(s) that can be found on the
+physical adapter, such as its name, how many memory contexts it can
+work with, the size of its MMIO areas, ...
+
+
+
+MMIO
+====
+
+OpenCAPI defines two MMIO areas for each AFU:
+
+* the global MMIO area, with registers pertinent to the whole AFU.
+* a per-process MMIO area, which has a fixed size for each context.
+
+
+
+AFU interrupts
+==============
+
+OpenCAPI includes the possibility for an AFU to send an interrupt to a
+host process. It is done through a 'intrp_req' defined in the
+Transaction Layer, specifying a 64-bit object handle which defines the
+interrupt.
+
+The driver allows a process to allocate an interrupt and obtain its
+64-bit object handle, that can be passed to the AFU.
+
+
+
+char devices
+============
+
+The driver creates one char device per AFU found on the physical
+device. A physical device may have multiple functions and each
+function can have multiple AFUs. At the time of this writing though,
+it has only been tested with devices exporting only one AFU.
+
+Char devices can be found in /dev/ocxl/ and are named as:
+/dev/ocxl/<AFU name>.<location>.<index>
+
+where <AFU name> is a max 20-character long name, as found in the
+config space of the AFU.
+<location> is added by the driver and can help distinguish devices
+when a system has more than one instance of the same OpenCAPI device.
+<index> is also to help distinguish AFUs in the unlikely case where a
+device carries multiple copies of the same AFU.
+
+
+
+Sysfs class
+===========
+
+An ocxl class is added for the devices representing the AFUs. See
+/sys/class/ocxl. The layout is described in
+Documentation/ABI/testing/sysfs-class-ocxl
+
+
+
+User API
+========
+
+open
+----
+
+Based on the AFU definition found in the config space, an AFU may
+support working with more than one memory context, in which case the
+associated char device may be opened multiple times by different
+processes.
+
+
+ioctl
+-----
+
+OCXL_IOCTL_ATTACH:
+
+ Attach the memory context of the calling process to the AFU so that
+ the AFU can access its memory.
+
+OCXL_IOCTL_IRQ_ALLOC:
+
+ Allocate an AFU interrupt and return an identifier.
+
+OCXL_IOCTL_IRQ_FREE:
+
+ Free a previously allocated AFU interrupt.
+
+OCXL_IOCTL_IRQ_SET_FD:
+
+ Associate an event fd to an AFU interrupt so that the user process
+ can be notified when the AFU sends an interrupt.
+
+
+mmap
+----
+
+A process can mmap the per-process MMIO area for interactions with the
+AFU.