^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) .. include:: <isonum.txt>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) Qualcomm Camera Subsystem driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) ================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) Introduction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) This file documents the Qualcomm Camera Subsystem driver located under
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) drivers/media/platform/qcom/camss.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) The current version of the driver supports the Camera Subsystem found on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) Qualcomm MSM8916/APQ8016 and MSM8996/APQ8096 processors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) The driver implements V4L2, Media controller and V4L2 subdev interfaces.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) Camera sensor using V4L2 subdev interface in the kernel is supported.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) The driver is implemented using as a reference the Qualcomm Camera Subsystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) driver for Android as found in Code Aurora [#f1]_ [#f2]_.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) Qualcomm Camera Subsystem hardware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) ----------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) The Camera Subsystem hardware found on 8x16 / 8x96 processors and supported by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) the driver consists of:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) - 2 / 3 CSIPHY modules. They handle the Physical layer of the CSI2 receivers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) A separate camera sensor can be connected to each of the CSIPHY module;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) - 2 / 4 CSID (CSI Decoder) modules. They handle the Protocol and Application
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) layer of the CSI2 receivers. A CSID can decode data stream from any of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) CSIPHY. Each CSID also contains a TG (Test Generator) block which can generate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) artificial input data for test purposes;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) - ISPIF (ISP Interface) module. Handles the routing of the data streams from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) the CSIDs to the inputs of the VFE;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) - 1 / 2 VFE (Video Front End) module(s). Contain a pipeline of image processing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) hardware blocks. The VFE has different input interfaces. The PIX (Pixel) input
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) interface feeds the input data to the image processing pipeline. The image
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) processing pipeline contains also a scale and crop module at the end. Three
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) RDI (Raw Dump Interface) input interfaces bypass the image processing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) pipeline. The VFE also contains the AXI bus interface which writes the output
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) data to memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) Supported functionality
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) -----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) The current version of the driver supports:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) - Input from camera sensor via CSIPHY;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) - Generation of test input data by the TG in CSID;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) - RDI interface of VFE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) - Raw dump of the input data to memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) Supported formats:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) - YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) - MIPI RAW8 (8bit Bayer RAW - V4L2_PIX_FMT_SRGGB8 /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) V4L2_PIX_FMT_SGRBG8 / V4L2_PIX_FMT_SGBRG8 / V4L2_PIX_FMT_SBGGR8);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) - MIPI RAW10 (10bit packed Bayer RAW - V4L2_PIX_FMT_SBGGR10P /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) V4L2_PIX_FMT_SGBRG10P / V4L2_PIX_FMT_SGRBG10P / V4L2_PIX_FMT_SRGGB10P /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) V4L2_PIX_FMT_Y10P);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) - MIPI RAW12 (12bit packed Bayer RAW - V4L2_PIX_FMT_SRGGB12P /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) V4L2_PIX_FMT_SGBRG12P / V4L2_PIX_FMT_SGRBG12P / V4L2_PIX_FMT_SRGGB12P).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) - (8x96 only) MIPI RAW14 (14bit packed Bayer RAW - V4L2_PIX_FMT_SRGGB14P /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) V4L2_PIX_FMT_SGBRG14P / V4L2_PIX_FMT_SGRBG14P / V4L2_PIX_FMT_SRGGB14P).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) - (8x96 only) Format conversion of the input data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) Supported input formats:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) - MIPI RAW10 (10bit packed Bayer RAW - V4L2_PIX_FMT_SBGGR10P / V4L2_PIX_FMT_Y10P).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) Supported output formats:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) - Plain16 RAW10 (10bit unpacked Bayer RAW - V4L2_PIX_FMT_SBGGR10 / V4L2_PIX_FMT_Y10).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) - PIX interface of VFE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) - Format conversion of the input data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) Supported input formats:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) - YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) Supported output formats:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) - NV12/NV21 (two plane YUV 4:2:0 - V4L2_PIX_FMT_NV12 / V4L2_PIX_FMT_NV21);
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) - NV16/NV61 (two plane YUV 4:2:2 - V4L2_PIX_FMT_NV16 / V4L2_PIX_FMT_NV61).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) - (8x96 only) YUYV/UYVY/YVYU/VYUY (packed YUV 4:2:2 - V4L2_PIX_FMT_YUYV /
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) V4L2_PIX_FMT_UYVY / V4L2_PIX_FMT_YVYU / V4L2_PIX_FMT_VYUY).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) - Scaling support. Configuration of the VFE Encoder Scale module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) for downscalling with ratio up to 16x.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) - Cropping support. Configuration of the VFE Encoder Crop module.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) - Concurrent and independent usage of two (8x96: three) data inputs -
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) could be camera sensors and/or TG.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) Driver Architecture and Design
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) ------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) The driver implements the V4L2 subdev interface. With the goal to model the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) hardware links between the modules and to expose a clean, logical and usable
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) interface, the driver is split into V4L2 sub-devices as follows (8x16 / 8x96):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) - 2 / 3 CSIPHY sub-devices - each CSIPHY is represented by a single sub-device;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) - 2 / 4 CSID sub-devices - each CSID is represented by a single sub-device;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) - 2 / 4 ISPIF sub-devices - ISPIF is represented by a number of sub-devices
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) equal to the number of CSID sub-devices;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) - 4 / 8 VFE sub-devices - VFE is represented by a number of sub-devices equal to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) the number of the input interfaces (3 RDI and 1 PIX for each VFE).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) The considerations to split the driver in this particular way are as follows:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) - representing CSIPHY and CSID modules by a separate sub-device for each module
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) allows to model the hardware links between these modules;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) - representing VFE by a separate sub-devices for each input interface allows
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) to use the input interfaces concurrently and independently as this is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) supported by the hardware;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) - representing ISPIF by a number of sub-devices equal to the number of CSID
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) sub-devices allows to create linear media controller pipelines when using two
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) cameras simultaneously. This avoids branches in the pipelines which otherwise
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) will require a) userspace and b) media framework (e.g. power on/off
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) operations) to make assumptions about the data flow from a sink pad to a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) source pad on a single media entity.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) Each VFE sub-device is linked to a separate video device node.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) The media controller pipeline graph is as follows (with connected two / three
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) OV5645 camera sensors):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) .. _qcom_camss_graph:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) .. kernel-figure:: qcom_camss_graph.dot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) :alt: qcom_camss_graph.dot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) :align: center
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) Media pipeline graph 8x16
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) .. kernel-figure:: qcom_camss_8x96_graph.dot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) :alt: qcom_camss_8x96_graph.dot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) :align: center
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) Media pipeline graph 8x96
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) Implementation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) Runtime configuration of the hardware (updating settings while streaming) is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) not required to implement the currently supported functionality. The complete
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) configuration on each hardware module is applied on STREAMON ioctl based on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) the current active media links, formats and controls set.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) The output size of the scaler module in the VFE is configured with the actual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) compose selection rectangle on the sink pad of the 'msm_vfe0_pix' entity.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) The crop output area of the crop module in the VFE is configured with the actual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) crop selection rectangle on the source pad of the 'msm_vfe0_pix' entity.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) Documentation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) -------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) APQ8016 Specification:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) https://developer.qualcomm.com/download/sd410/snapdragon-410-processor-device-specification.pdf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) Referenced 2016-11-24.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) APQ8096 Specification:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) https://developer.qualcomm.com/download/sd820e/qualcomm-snapdragon-820e-processor-apq8096sge-device-specification.pdf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) Referenced 2018-06-22.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) References
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) .. [#f1] https://source.codeaurora.org/quic/la/kernel/msm-3.10/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) .. [#f2] https://source.codeaurora.org/quic/la/kernel/msm-3.18/
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags