diff options
Diffstat (limited to 'Documentation/driver-api/media/v4l2-subdev.rst')
| -rw-r--r-- | Documentation/driver-api/media/v4l2-subdev.rst | 198 |
1 files changed, 159 insertions, 39 deletions
diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst index 08ea2673b19e..13aec460e802 100644 --- a/Documentation/driver-api/media/v4l2-subdev.rst +++ b/Documentation/driver-api/media/v4l2-subdev.rst @@ -157,6 +157,9 @@ below. Using one or the other registration method only affects the probing process, the run-time bridge-subdevice interaction is in both cases the same. +Registering synchronous sub-devices +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + In the **synchronous** case a device (bridge) driver needs to register the :c:type:`v4l2_subdev` with the v4l2_device: @@ -175,10 +178,14 @@ You can unregister a sub-device using: :c:func:`v4l2_device_unregister_subdev <v4l2_device_unregister_subdev>` (:c:type:`sd <v4l2_subdev>`). - Afterwards the subdev module can be unloaded and :c:type:`sd <v4l2_subdev>`->dev == ``NULL``. +.. _media-registering-async-subdevs: + +Registering asynchronous sub-devices +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + In the **asynchronous** case subdevice probing can be invoked independently of the bridge driver availability. The subdevice driver then has to verify whether all the requirements for a successful probing are satisfied. This can include a @@ -190,58 +197,94 @@ performed using the :c:func:`v4l2_async_unregister_subdev` call. Subdevices registered this way are stored in a global list of subdevices, ready to be picked up by bridge drivers. -Bridge drivers in turn have to register a notifier object. This is -performed using the :c:func:`v4l2_async_nf_register` call. To -unregister the notifier the driver has to call -:c:func:`v4l2_async_nf_unregister`. The former of the two functions -takes two arguments: a pointer to struct :c:type:`v4l2_device` and a -pointer to struct :c:type:`v4l2_async_notifier`. +Drivers must complete all initialization of the sub-device before +registering it using :c:func:`v4l2_async_register_subdev`, including +enabling runtime PM. This is because the sub-device becomes accessible +as soon as it gets registered. + +Asynchronous sub-device notifiers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Bridge drivers in turn have to register a notifier object. This is performed +using the :c:func:`v4l2_async_nf_register` call. To unregister the notifier the +driver has to call :c:func:`v4l2_async_nf_unregister`. Before releasing memory +of an unregister notifier, it must be cleaned up by calling +:c:func:`v4l2_async_nf_cleanup`. Before registering the notifier, bridge drivers must do two things: first, the -notifier must be initialized using the :c:func:`v4l2_async_nf_init`. -Second, bridge drivers can then begin to form a list of subdevice descriptors -that the bridge device needs for its operation. Several functions are available -to add subdevice descriptors to a notifier, depending on the type of device and -the needs of the driver. - -:c:func:`v4l2_async_nf_add_fwnode_remote` and -:c:func:`v4l2_async_nf_add_i2c` are for bridge and ISP drivers for -registering their async sub-devices with the notifier. - -:c:func:`v4l2_async_register_subdev_sensor` is a helper function for -sensor drivers registering their own async sub-device, but it also registers a -notifier and further registers async sub-devices for lens and flash devices -found in firmware. The notifier for the sub-device is unregistered with the -async sub-device. - -These functions allocate an async sub-device descriptor which is of type struct -:c:type:`v4l2_async_subdev` embedded in a driver-specific struct. The &struct -:c:type:`v4l2_async_subdev` shall be the first member of this struct: +notifier must be initialized using the :c:func:`v4l2_async_nf_init`. Second, +bridge drivers can then begin to form a list of async connection descriptors +that the bridge device needs for its +operation. :c:func:`v4l2_async_nf_add_fwnode`, +:c:func:`v4l2_async_nf_add_fwnode_remote` and :c:func:`v4l2_async_nf_add_i2c` + +Async connection descriptors describe connections to external sub-devices the +drivers for which are not yet probed. Based on an async connection, a media data +or ancillary link may be created when the related sub-device becomes +available. There may be one or more async connections to a given sub-device but +this is not known at the time of adding the connections to the notifier. Async +connections are bound as matching async sub-devices are found, one by one. + +Asynchronous sub-device notifier for sub-devices +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A driver that registers an asynchronous sub-device may also register an +asynchronous notifier. This is called an asynchronous sub-device notifier and the +process is similar to that of a bridge driver apart from that the notifier is +initialised using :c:func:`v4l2_async_subdev_nf_init` instead. A sub-device +notifier may complete only after the V4L2 device becomes available, i.e. there's +a path via async sub-devices and notifiers to a notifier that is not an +asynchronous sub-device notifier. + +Asynchronous sub-device registration helper for camera sensor drivers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:c:func:`v4l2_async_register_subdev_sensor` is a helper function for sensor +drivers registering their own async connection, but it also registers a notifier +and further registers async connections for lens and flash devices found in +firmware. The notifier for the sub-device is unregistered and cleaned up with +the async sub-device, using :c:func:`v4l2_async_unregister_subdev`. + +Asynchronous sub-device notifier example +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These functions allocate an async connection descriptor which is of type struct +:c:type:`v4l2_async_connection` embedded in a driver-specific struct. The &struct +:c:type:`v4l2_async_connection` shall be the first member of this struct: .. code-block:: c - struct my_async_subdev { - struct v4l2_async_subdev asd; + struct my_async_connection { + struct v4l2_async_connection asc; ... }; - struct my_async_subdev *my_asd; + struct my_async_connection *my_asc; struct fwnode_handle *ep; ... - my_asd = v4l2_async_nf_add_fwnode_remote(¬ifier, ep, - struct my_async_subdev); + my_asc = v4l2_async_nf_add_fwnode_remote(¬ifier, ep, + struct my_async_connection); fwnode_handle_put(ep); - if (IS_ERR(asd)) - return PTR_ERR(asd); + if (IS_ERR(my_asc)) + return PTR_ERR(my_asc); + +Asynchronous sub-device notifier callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The V4L2 core will then use these connection descriptors to match asynchronously +registered subdevices to them. If a match is detected the ``.bound()`` notifier +callback is called. After all connections have been bound the .complete() +callback is called. When a connection is removed from the system the +``.unbind()`` method is called. All three callbacks are optional. -The V4L2 core will then use these descriptors to match asynchronously -registered subdevices to them. If a match is detected the ``.bound()`` -notifier callback is called. After all subdevices have been located the -.complete() callback is called. When a subdevice is removed from the -system the .unbind() method is called. All three callbacks are optional. +Drivers can store any type of custom data in their driver-specific +:c:type:`v4l2_async_connection` wrapper. If any of that data requires special +handling when the structure is freed, drivers must implement the ``.destroy()`` +notifier callback. The framework will call it right before freeing the +:c:type:`v4l2_async_connection`. Calling subdev operations ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -315,7 +358,7 @@ response to video node operations. This hides the complexity of the underlying hardware from applications. For complex devices, finer-grained control of the device than what the video nodes offer may be required. In those cases, bridge drivers that implement :ref:`the media controller API <media_controller>` may -opt for making the subdevice operations directly accessible from userpace. +opt for making the subdevice operations directly accessible from userspace. Device nodes named ``v4l-subdev``\ *X* can be created in ``/dev`` to access sub-devices directly. If a sub-device supports direct userspace configuration @@ -518,6 +561,83 @@ The :c:func:`v4l2_i2c_new_subdev` function will call :c:type:`i2c_board_info` structure using the ``client_type`` and the ``addr`` to fill it. +Centrally managed subdev active state +------------------------------------- + +Traditionally V4L2 subdev drivers maintained internal state for the active +device configuration. This is often implemented as e.g. an array of struct +v4l2_mbus_framefmt, one entry for each pad, and similarly for crop and compose +rectangles. + +In addition to the active configuration, each subdev file handle has a struct +v4l2_subdev_state, managed by the V4L2 core, which contains the try +configuration. + +To simplify the subdev drivers the V4L2 subdev API now optionally supports a +centrally managed active configuration represented by +:c:type:`v4l2_subdev_state`. One instance of state, which contains the active +device configuration, is stored in the sub-device itself as part of +the :c:type:`v4l2_subdev` structure, while the core associates a try state to +each open file handle, to store the try configuration related to that file +handle. + +Sub-device drivers can opt-in and use state to manage their active configuration +by initializing the subdevice state with a call to v4l2_subdev_init_finalize() +before registering the sub-device. They must also call v4l2_subdev_cleanup() +to release all the allocated resources before unregistering the sub-device. +The core automatically allocates and initializes a state for each open file +handle to store the try configurations and frees it when closing the file +handle. + +V4L2 sub-device operations that use both the :ref:`ACTIVE and TRY formats +<v4l2-subdev-format-whence>` receive the correct state to operate on through +the 'state' parameter. The state must be locked and unlocked by the +caller by calling :c:func:`v4l2_subdev_lock_state()` and +:c:func:`v4l2_subdev_unlock_state()`. The caller can do so by calling the subdev +operation through the :c:func:`v4l2_subdev_call_state_active()` macro. + +Operations that do not receive a state parameter implicitly operate on the +subdevice active state, which drivers can exclusively access by +calling :c:func:`v4l2_subdev_lock_and_get_active_state()`. The sub-device active +state must equally be released by calling :c:func:`v4l2_subdev_unlock_state()`. + +Drivers must never manually access the state stored in the :c:type:`v4l2_subdev` +or in the file handle without going through the designated helpers. + +While the V4L2 core passes the correct try or active state to the subdevice +operations, many existing device drivers pass a NULL state when calling +operations with :c:func:`v4l2_subdev_call()`. This legacy construct causes +issues with subdevice drivers that let the V4L2 core manage the active state, +as they expect to receive the appropriate state as a parameter. To help the +conversion of subdevice drivers to a managed active state without having to +convert all callers at the same time, an additional wrapper layer has been +added to v4l2_subdev_call(), which handles the NULL case by getting and locking +the callee's active state with :c:func:`v4l2_subdev_lock_and_get_active_state()`, +and unlocking the state after the call. + +The whole subdev state is in reality split into three parts: the +v4l2_subdev_state, subdev controls and subdev driver's internal state. In the +future these parts should be combined into a single state. For the time being +we need a way to handle the locking for these parts. This can be accomplished +by sharing a lock. The v4l2_ctrl_handler already supports this via its 'lock' +pointer and the same model is used with states. The driver can do the following +before calling v4l2_subdev_init_finalize(): + +.. code-block:: c + + sd->ctrl_handler->lock = &priv->mutex; + sd->state_lock = &priv->mutex; + +This shares the driver's private mutex between the controls and the states. + +Streams, multiplexed media pads and internal routing +---------------------------------------------------- + +A subdevice driver can implement support for multiplexed streams by setting +the V4L2_SUBDEV_FL_STREAMS subdev flag and implementing support for +centrally managed subdev active state, routing and stream based +configuration. + V4L2 sub-device functions and data structures --------------------------------------------- |