diff options
Diffstat (limited to 'Documentation/driver-api/media')
| -rw-r--r-- | Documentation/driver-api/media/camera-sensor.rst | 8 | ||||
| -rw-r--r-- | Documentation/driver-api/media/drivers/index.rst | 1 | ||||
| -rw-r--r-- | Documentation/driver-api/media/drivers/ipu6.rst | 190 | ||||
| -rw-r--r-- | Documentation/driver-api/media/drivers/zoran.rst | 2 | ||||
| -rw-r--r-- | Documentation/driver-api/media/maintainer-entry-profile.rst | 2 | ||||
| -rw-r--r-- | Documentation/driver-api/media/mc-core.rst | 67 | ||||
| -rw-r--r-- | Documentation/driver-api/media/tx-rx.rst | 44 | ||||
| -rw-r--r-- | Documentation/driver-api/media/v4l2-core.rst | 1 | ||||
| -rw-r--r-- | Documentation/driver-api/media/v4l2-jpeg.rst | 10 |
9 files changed, 276 insertions, 49 deletions
diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst index b4920b34cebc..c290833165e6 100644 --- a/Documentation/driver-api/media/camera-sensor.rst +++ b/Documentation/driver-api/media/camera-sensor.rst @@ -81,10 +81,10 @@ restart when the system is resumed. This requires coordination between the camera sensor and the rest of the camera pipeline. Bridge drivers are responsible for this coordination, and instruct camera sensors to stop and restart streaming by calling the appropriate subdev operations -(``.s_stream()``, ``.enable_streams()`` or ``.disable_streams()``). Camera -sensor drivers shall therefore **not** keep track of the streaming state to -stop streaming in the PM suspend handler and restart it in the resume handler. -Drivers should in general not implement the system PM handlers. +(``.enable_streams()`` or ``.disable_streams()``). Camera sensor drivers shall +therefore **not** keep track of the streaming state to stop streaming in the PM +suspend handler and restart it in the resume handler. Drivers should in general +not implement the system PM handlers. Camera sensor drivers shall **not** implement the subdev ``.s_power()`` operation, as it is deprecated. While this operation is implemented in some diff --git a/Documentation/driver-api/media/drivers/index.rst b/Documentation/driver-api/media/drivers/index.rst index c4123a16b5f9..7f6f3dcd5c90 100644 --- a/Documentation/driver-api/media/drivers/index.rst +++ b/Documentation/driver-api/media/drivers/index.rst @@ -26,6 +26,7 @@ Video4Linux (V4L) drivers vimc-devel zoran ccs/ccs + ipu6 Digital TV drivers diff --git a/Documentation/driver-api/media/drivers/ipu6.rst b/Documentation/driver-api/media/drivers/ipu6.rst new file mode 100644 index 000000000000..88f6498e74db --- /dev/null +++ b/Documentation/driver-api/media/drivers/ipu6.rst @@ -0,0 +1,190 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================== +Intel IPU6 Driver +================== + +Author: Bingbu Cao <bingbu.cao@intel.com> + +Overview +========= + +Intel IPU6 is the sixth generation of Intel Image Processing Unit used in some +Intel Chipsets such as Tiger Lake, Jasper Lake, Alder Lake, Raptor Lake and +Meteor Lake. IPU6 consists of two major systems: Input System (ISYS) and +Processing System (PSYS). IPU6 are visible on the PCI bus as a single device, it +can be found by ``lspci``: + +``0000:00:05.0 Multimedia controller: Intel Corporation Device xxxx (rev xx)`` + +IPU6 has a 16 MB BAR in PCI configuration Space for MMIO registers which is +visible for driver. + +Buttress +========= + +The IPU6 is connecting to the system fabric with Buttress which is enabling host +driver to control the IPU6, it also allows IPU6 access the system memory to +store and load frame pixel streams and any other metadata. + +Buttress mainly manages several system functionalities: power management, +interrupt handling, firmware authentication and global timer sync. + +ISYS and PSYS Power flow +------------------------ + +IPU6 driver initialize the ISYS and PSYS power up or down request by setting the +Buttress frequency control register for ISYS and PSYS +(``IPU6_BUTTRESS_REG_IS_FREQ_CTL`` and ``IPU6_BUTTRESS_REG_PS_FREQ_CTL``) in +function: + +.. c:function:: int ipu6_buttress_power(...) + +Buttress forwards the request to Punit, after Punit execute the power up flow, +Buttress indicates driver that ISYS or PSYS is powered up by updating the power +status registers. + +.. Note:: ISYS power up needs take place prior to PSYS power up, ISYS power down + needs take place after PSYS power down due to hardware limitation. + +Interrupt +--------- + +IPU6 interrupt can be generated as MSI or INTA, interrupt will be triggered when +ISYS, PSYS, Buttress event or error happen, driver can get the interrupt cause +by reading the interrupt status register ``BUTTRESS_REG_ISR_STATUS``, driver +clears the irq status and then calls specific ISYS or PSYS irq handler. + +.. c:function:: irqreturn_t ipu6_buttress_isr(int irq, ...) + +Security and firmware authentication +------------------------------------- + +To address the IPU6 firmware security concerns, the IPU6 firmware needs to +undergo an authentication process before it is allowed to executed on the IPU6 +internal processors. The IPU6 driver will work with Converged Security Engine +(CSE) to complete authentication process. The CSE is responsible of +authenticating the IPU6 firmware. The authenticated firmware binary is copied +into an isolated memory region. Firmware authentication process is implemented +by CSE following an IPC handshake with the IPU6 driver. There are some Buttress +registers used by the CSE and the IPU6 driver to communicate with each other via +IPC. + +.. c:function:: int ipu6_buttress_authenticate(...) + +Global timer sync +----------------- + +The IPU6 driver initiates a Hammock Harbor synchronization flow each time it +starts camera operation. The IPU6 will synchronizes an internal counter in the +Buttress with a copy of the SoC time, this counter maintains the up-to-date time +until camera operation is stopped. The IPU6 driver can use this time counter to +calibrate the timestamp based on the timestamp in response event from firmware. + +.. c:function:: int ipu6_buttress_start_tsc_sync(...) + +DMA and MMU +============ + +The IPU6 has its own scalar processor where the firmware run at and an internal +32-bit virtual address space. The IPU6 has MMU address translation hardware to +allow that scalar processors to access the internal memory and external system +memory through IPU6 virtual address. The address translation is based on two +levels of page lookup tables stored in system memory which are maintained by the +IPU6 driver. The IPU6 driver sets the level-1 page table base address to MMU +register and allows MMU to perform page table lookups. + +The IPU6 driver exports its own DMA operations. The IPU6 driver will update the +page table entries for each DMA operation and invalidate the MMU TLB after each +unmap and free. + +Firmware file format +==================== + +The IPU6 firmware is in Code Partition Directory (CPD) file format. The CPD +firmware contains a CPD header, several CPD entries and components. The CPD +component includes 3 entries - manifest, metadata and module data. Manifest and +metadata are defined by CSE and used by CSE for authentication. Module data is +specific to IPU6 which holds the binary data of firmware called package +directory. The IPU6 driver (``ipu6-cpd.c`` in particular) parses and validates +the CPD firmware file and gets the package directory binary data of the IPU6 +firmware, copies it to specific DMA buffer and sets its base address to Buttress +``FW_SOURCE_BASE`` register. Finally the CSE will do authentication for this +firmware binary. + + +Syscom interface +================ + +The IPU6 driver communicates with firmware via the Syscom ABI. Syscom is an +inter-processor communication mechanism between the IPU scalar processors and +the CPU. There are a number of resources shared between firmware and software. +A system memory region where the message queues reside, firmware can access the +memory region via the IPU MMU. The Syscom queues are FIFO fixed depth queues +with a configurable number of tokens (messages). There are also common IPU6 MMIO +registers where the queue read and write indices reside. Software and firmware +function as producer and consumer of tokens in the queues and update the write +and read indices separately when sending or receiving each message. + +The IPU6 driver must prepare and configure the number of input and output +queues, configure the count of tokens per queue and the size of per token before +initiating and starting the communication with firmware. Firmware and software +must use same configurations. The IPU6 Buttress has a number of firmware boot +parameter registers which can be used to store the address of configuration and +initialise the Syscom state, then driver can request firmware to start and run via +setting the scalar processor control status register. + +Input System +============ + +IPU6 input system consists of MIPI D-PHY and several CSI-2 receivers. It can +capture image pixel data from camera sensors or other MIPI CSI-2 output devices. + +D-PHYs and CSI-2 ports lane mapping +----------------------------------- + +The IPU6 integrates different D-PHY IPs on different SoCs, on Tiger Lake and +Alder Lake, IPU6 integrates MCD10 D-PHY, IPU6SE on Jasper Lake integrates JSL +D-PHY and IPU6EP on Meteor Lake integrates a Synopsys DWC D-PHY. There is an +adaptional layer between D-PHY and CSI-2 receiver controller which includes port +configuration, PHY wrapper or private test interfaces for D-PHY. There are 3 +D-PHY drivers ``ipu6-isys-mcd-phy.c``, ``ipu6-isys-jsl-phy.c`` and +``ipu6-isys-dwc-phy.c`` program the above 3 D-PHYs in IPU6. + +Different IPU6 versions have different D-PHY lanes mappings, On Tiger Lake, +there are 12 data lanes and 8 clock lanes, IPU6 support maximum 8 CSI-2 ports, +see the PPI mmapping in ``ipu6-isys-mcd-phy.c`` for more information. On Jasper +Lake and Alder Lake, D-PHY has 8 data lanes and 4 clock lanes, the IPU6 supports +maximum 4 CSI-2 ports. For Meteor Lake, D-PHY has 12 data lanes and 6 clock +lanes so IPU6 support maximum 6 CSI-2 ports. + +.. Note:: Each pair of CSI-2 two ports is a single unit that can share the data + lanes. For example, for CSI-2 port 0 and 1, CSI-2 port 0 support + maximum 4 data lanes, CSI-2 port 1 support maximum 2 data lanes, CSI-2 + port 0 with 2 data lanes can work together with CSI-2 port 1 with 2 + data lanes. If trying to use CSI-2 port 0 with 4 lanes, CSI-2 port 1 + will not be available as the 4 data lanes are shared by CSI-2 port 0 + and 1. The same applies to CSI ports 2/3, 4/5 and 7/8. + +ISYS firmware ABIs +------------------ + +The IPU6 firmware implements a series of ABIs for software access. In general, +software firstly prepares the stream configuration ``struct +ipu6_fw_isys_stream_cfg_data_abi`` and sends the configuration to firmware via +sending ``STREAM_OPEN`` command. Stream configuration includes input pins and +output pins, input pin ``struct ipu6_fw_isys_input_pin_info_abi`` defines the +resolution and data type of input source, output pin ``struct +ipu6_fw_isys_output_pin_info_abi`` defines the output resolution, stride and +frame format, etc. + +Once the driver gets the interrupt from firmware that indicates stream open +successfully, the driver will send the ``STREAM_START`` and ``STREAM_CAPTURE`` +command to request firmware to start capturing image frames. ``STREAM_CAPTURE`` +command queues the buffers to firmware with ``struct +ipu6_fw_isys_frame_buff_set``, software then waits for the interrupt and +response from firmware, ``PIN_DATA_READY`` means a buffer is ready on a specific +output pin and then software can return the buffer to user. + +.. Note:: See :ref:`Examples<ipu6_isys_capture_examples>` about how to do + capture by IPU6 ISYS driver. diff --git a/Documentation/driver-api/media/drivers/zoran.rst b/Documentation/driver-api/media/drivers/zoran.rst index b205e10c3154..3e05b7f0442a 100644 --- a/Documentation/driver-api/media/drivers/zoran.rst +++ b/Documentation/driver-api/media/drivers/zoran.rst @@ -222,7 +222,7 @@ The CCIR - I uses the PAL colorsystem, and is used in Great Britain, Hong Kong, Ireland, Nigeria, South Africa. The CCIR - N uses the PAL colorsystem and PAL frame size but the NTSC framerate, -and is used in Argentinia, Uruguay, an a few others +and is used in Argentina, Uruguay, an a few others We do not talk about how the audio is broadcast ! diff --git a/Documentation/driver-api/media/maintainer-entry-profile.rst b/Documentation/driver-api/media/maintainer-entry-profile.rst index ffc712a5f632..ad96a89ee916 100644 --- a/Documentation/driver-api/media/maintainer-entry-profile.rst +++ b/Documentation/driver-api/media/maintainer-entry-profile.rst @@ -116,7 +116,7 @@ CEC drivers ``cec-compliance`` .. [3] The ``v4l2-compliance`` also covers the media controller usage inside V4L2 drivers. -Other compilance tools are under development to check other parts of the +Other compliance tools are under development to check other parts of the subsystem. Those tests need to pass before the patches go upstream. diff --git a/Documentation/driver-api/media/mc-core.rst b/Documentation/driver-api/media/mc-core.rst index 2456950ce8ff..1d010bd7ec49 100644 --- a/Documentation/driver-api/media/mc-core.rst +++ b/Documentation/driver-api/media/mc-core.rst @@ -144,7 +144,8 @@ valid values are described at :c:func:`media_create_pad_link()` and Graph traversal ^^^^^^^^^^^^^^^ -The media framework provides APIs to iterate over entities in a graph. +The media framework provides APIs to traverse media graphs, locating connected +entities and links. To iterate over all entities belonging to a media device, drivers can use the media_device_for_each_entity macro, defined in @@ -159,31 +160,6 @@ the media_device_for_each_entity macro, defined in ... } -Drivers might also need to iterate over all entities in a graph that can be -reached only through enabled links starting at a given entity. The media -framework provides a depth-first graph traversal API for that purpose. - -.. note:: - - Graphs with cycles (whether directed or undirected) are **NOT** - supported by the graph traversal API. To prevent infinite loops, the graph - traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``, - currently defined as 16. - -Drivers initiate a graph traversal by calling -:c:func:`media_graph_walk_start()` - -The graph structure, provided by the caller, is initialized to start graph -traversal at the given entity. - -Drivers can then retrieve the next entity by calling -:c:func:`media_graph_walk_next()` - -When the graph traversal is complete the function will return ``NULL``. - -Graph traversal can be interrupted at any moment. No cleanup function call -is required and the graph structure can be freed normally. - Helper functions can be used to find a link between two given pads, or a pad connected to another pad through an enabled link (:c:func:`media_entity_find_link()`, :c:func:`media_pad_remote_pad_first()`, @@ -276,6 +252,45 @@ Subsystems should facilitate link validation by providing subsystem specific helper functions to provide easy access for commonly needed information, and in the end provide a way to use driver-specific callbacks. +Pipeline traversal +^^^^^^^^^^^^^^^^^^ + +Once a pipeline has been constructed with :c:func:`media_pipeline_start()`, +drivers can iterate over entities or pads in the pipeline with the +:c:macro:´media_pipeline_for_each_entity` and +:c:macro:´media_pipeline_for_each_pad` macros. Iterating over pads is +straightforward: + +.. code-block:: c + + media_pipeline_pad_iter iter; + struct media_pad *pad; + + media_pipeline_for_each_pad(pipe, &iter, pad) { + /* 'pad' will point to each pad in turn */ + ... + } + +To iterate over entities, the iterator needs to be initialized and cleaned up +as an additional steps: + +.. code-block:: c + + media_pipeline_entity_iter iter; + struct media_entity *entity; + int ret; + + ret = media_pipeline_entity_iter_init(pipe, &iter); + if (ret) + ...; + + media_pipeline_for_each_entity(pipe, &iter, entity) { + /* 'entity' will point to each entity in turn */ + ... + } + + media_pipeline_entity_iter_cleanup(&iter); + Media Controller Device Allocator API ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Documentation/driver-api/media/tx-rx.rst b/Documentation/driver-api/media/tx-rx.rst index 29d66a47b56e..0b8c9cde8ee4 100644 --- a/Documentation/driver-api/media/tx-rx.rst +++ b/Documentation/driver-api/media/tx-rx.rst @@ -49,12 +49,31 @@ Link frequency The :ref:`V4L2_CID_LINK_FREQ <v4l2-cid-link-freq>` control is used to tell the receiver the frequency of the bus (i.e. it is not the same as the symbol rate). -``.s_stream()`` callback -^^^^^^^^^^^^^^^^^^^^^^^^ +Drivers that do not have user-configurable link frequency should report it +through the ``.get_mbus_config()`` subdev pad operation, in the ``link_freq`` +field of struct v4l2_mbus_config, instead of through controls. + +Receiver drivers should use :c:func:`v4l2_get_link_freq` helper to obtain the +link frequency from the transmitter sub-device. + +``.enable_streams()`` and ``.disable_streams()`` callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The struct struct v4l2_subdev_video_ops->s_stream() callback is used by the -receiver driver to control the transmitter driver's streaming state. +The struct v4l2_subdev_pad_ops->enable_streams() and struct +v4l2_subdev_pad_ops->disable_streams() callbacks are used by the receiver driver +to control the transmitter driver's streaming state. These callbacks may not be +called directly, but by using ``v4l2_subdev_enable_streams()`` and +``v4l2_subdev_disable_streams()``. + +Stopping the transmitter +^^^^^^^^^^^^^^^^^^^^^^^^ +A transmitter stops sending the stream of images as a result of +calling the ``.disable_streams()`` callback. Some transmitters may stop the +stream at a frame boundary whereas others stop immediately, +effectively leaving the current frame unfinished. The receiver driver +should not make assumptions either way, but function properly in both +cases. CSI-2 transmitter drivers ------------------------- @@ -76,14 +95,15 @@ where * - link_freq - The value of the ``V4L2_CID_LINK_FREQ`` integer64 menu item. * - nr_of_lanes - - Number of data lanes used on the CSI-2 link. This can - be obtained from the OF endpoint configuration. + - Number of data lanes used on the CSI-2 link. * - 2 - Data is transferred on both rising and falling edge of the signal. * - bits_per_sample - Number of bits per sample. * - k - - 16 for D-PHY and 7 for C-PHY + - 16 for D-PHY and 7 for C-PHY. + +Information on whether D-PHY or C-PHY is used, and the value of ``nr_of_lanes``, can be obtained from the OF endpoint configuration. .. note:: @@ -122,13 +142,3 @@ device, so this should be only done when it is needed. Receiver drivers that do not need explicit LP-11 or LP-111 state setup are waived from calling the two callbacks. - -Stopping the transmitter -^^^^^^^^^^^^^^^^^^^^^^^^ - -A transmitter stops sending the stream of images as a result of -calling the ``.s_stream()`` callback. Some transmitters may stop the -stream at a frame boundary whereas others stop immediately, -effectively leaving the current frame unfinished. The receiver driver -should not make assumptions either way, but function properly in both -cases. diff --git a/Documentation/driver-api/media/v4l2-core.rst b/Documentation/driver-api/media/v4l2-core.rst index 58cba831ade5..ad987c34ad2a 100644 --- a/Documentation/driver-api/media/v4l2-core.rst +++ b/Documentation/driver-api/media/v4l2-core.rst @@ -26,3 +26,4 @@ Video4Linux devices v4l2-tuner v4l2-common v4l2-tveeprom + v4l2-jpeg diff --git a/Documentation/driver-api/media/v4l2-jpeg.rst b/Documentation/driver-api/media/v4l2-jpeg.rst new file mode 100644 index 000000000000..af3bc52f865b --- /dev/null +++ b/Documentation/driver-api/media/v4l2-jpeg.rst @@ -0,0 +1,10 @@ +.. SPDX-License-Identifier: GPL-2.0 + +V4L2 JPEG header related functions and data structures +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: include/media/v4l2-jpeg.h + :internal: + +.. kernel-doc:: drivers/media/v4l2-core/v4l2-jpeg.c + :export: |