^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) Writing camera sensor drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) =============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) CSI-2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) Please see what is written on :ref:`MIPI_CSI_2`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) Handling clocks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) ---------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) Camera sensors have an internal clock tree including a PLL and a number of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) divisors. The clock tree is generally configured by the driver based on a few
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) input parameters that are specific to the hardware:: the external clock frequency
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) and the link frequency. The two parameters generally are obtained from system
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) firmware. No other frequencies should be used in any circumstances.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) The reason why the clock frequencies are so important is that the clock signals
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) come out of the SoC, and in many cases a specific frequency is designed to be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) used in the system. Using another frequency may cause harmful effects
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) elsewhere. Therefore only the pre-determined frequencies are configurable by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) user.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) Frame size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) There are two distinct ways to configure the frame size produced by camera
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) sensors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) Freely configurable camera sensor drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) Freely configurable camera sensor drivers expose the device's internal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) processing pipeline as one or more sub-devices with different cropping and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) scaling configurations. The output size of the device is the result of a series
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) of cropping and scaling operations from the device's pixel array's size.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) An example of such a driver is the smiapp driver (see drivers/media/i2c/smiapp).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) Register list based drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) ~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) Register list based drivers generally, instead of able to configure the device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) they control based on user requests, are limited to a number of preset
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) configurations that combine a number of different parameters that on hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) level are independent. How a driver picks such configuration is based on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) format set on a source pad at the end of the device's internal pipeline.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) Most sensor drivers are implemented this way, see e.g.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) drivers/media/i2c/imx319.c for an example.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) Frame interval configuration
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) ----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) There are two different methods for obtaining possibilities for different frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) intervals as well as configuring the frame interval. Which one to implement
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) depends on the type of the device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) Raw camera sensors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) ~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) Instead of a high level parameter such as frame interval, the frame interval is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) a result of the configuration of a number of camera sensor implementation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) specific parameters. Luckily, these parameters tend to be the same for more or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) less all modern raw camera sensors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) The frame interval is calculated using the following equation::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) frame interval = (analogue crop width + horizontal blanking) *
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) (analogue crop height + vertical blanking) / pixel rate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) The formula is bus independent and is applicable for raw timing parameters on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) large variety of devices beyond camera sensors. Devices that have no analogue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) crop, use the full source image size, i.e. pixel array size.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79) ``V4L2_CID_VBLANK``, respectively. The unit of these controls are lines. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) pixel rate is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) unit of that control is Hz.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) Register list based drivers need to implement read-only sub-device nodes for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) purpose. Devices that are not register list based need these to configure the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) device's internal processing pipeline.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) The first entity in the linear pipeline is the pixel array. The pixel array may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) be followed by other entities that are there to allow configuring binning,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) skipping, scaling or digital crop :ref:`v4l2-subdev-selections`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) USB cameras etc. devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) ~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) USB video class hardware, as well as many cameras offering a similar higher
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) level interface natively, generally use the concept of frame interval (or frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) rate) on device level in firmware or hardware. This means lower level controls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) implemented by raw cameras may not be used on uAPI (or even kAPI) to control the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) frame interval on these devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) Power management
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) ----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) Always use runtime PM to manage the power states of your device. Camera sensor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) drivers are in no way special in this respect: they are responsible for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) controlling the power state of the device they otherwise control as well. In
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) general, the device must be powered on at least when its registers are being
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) accessed and when it is streaming.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) Existing camera sensor drivers may rely on the old
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) :c:type:`v4l2_subdev_core_ops`->s_power() callback for bridge or ISP drivers to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) manage their power state. This is however **deprecated**. If you feel you need
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) to begin calling an s_power from an ISP or a bridge driver, instead please add
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) runtime PM support to the sensor driver you are using. Likewise, new drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) should not use s_power.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) ``drivers/media/i2c/smiapp/smiapp-core.c``. The two drivers work in both ACPI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) and DT based systems.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) Control framework
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) ~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) ``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) PM ``runtime_resume`` callback, as it has no way to figure out the power state
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) of the device. This is because the power state of the device is only changed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) after the power state transition has taken place. The ``s_ctrl`` callback can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) used to obtain device's power state after the power state transition:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) .. c:function::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) int pm_runtime_get_if_in_use(struct device *dev);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) The function returns a non-zero value if it succeeded getting the power count or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) runtime PM was disabled, in either of which cases the driver may proceed to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) access the device.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags