^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) i.MX Video Capture Driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) =========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) Introduction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) handles the flow of image frames to and from capture devices and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) display devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) For image capture, the IPU contains the following internal subunits:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) - Image DMA Controller (IDMAC)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) - Camera Serial Interface (CSI)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) - Image Converter (IC)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) - Sensor Multi-FIFO Controller (SMFC)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) - Image Rotator (IRT)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) - Video De-Interlacing or Combining Block (VDIC)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) The IDMAC is the DMA controller for transfer of image frames to and from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) memory. Various dedicated DMA channels exist for both video capture and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) display paths. During transfer, the IDMAC is also capable of vertical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) image flip, 8x8 block transfer (see IRT description), pixel component
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) re-ordering (for example UYVY to YUYV) within the same colorspace, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) packed <--> planar conversion. The IDMAC can also perform a simple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) de-interlacing by interweaving even and odd lines during transfer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) (without motion compensation which requires the VDIC).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) The CSI is the backend capture unit that interfaces directly with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 buses.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) The IC handles color-space conversion, resizing (downscaling and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) upscaling), horizontal flip, and 90/270 degree rotation operations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) There are three independent "tasks" within the IC that can carry out
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) conversions concurrently: pre-process encoding, pre-process viewfinder,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) and post-processing. Within each task, conversions are split into three
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) sections: downsizing section, main section (upsizing, flip, colorspace
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) conversion, and graphics plane combining), and rotation section.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) The IPU time-shares the IC task operations. The time-slice granularity
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) is one burst of eight pixels in the downsizing section, one image line
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) in the main processing section, one image frame in the rotation section.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) The SMFC is composed of four independent FIFOs that each can transfer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) captured frames from sensors directly to memory concurrently via four
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) IDMAC channels.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) The IRT carries out 90 and 270 degree image rotation operations. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) rotation operation is carried out on 8x8 pixel blocks at a time. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) operation is supported by the IDMAC which handles the 8x8 block transfer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) along with block reordering, in coordination with vertical flip.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) The VDIC handles the conversion of interlaced video to progressive, with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) support for different motion compensation modes (low, medium, and high
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) motion). The deinterlaced output frames from the VDIC can be sent to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) IC pre-process viewfinder task for further conversions. The VDIC also
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) contains a Combiner that combines two image planes, with alpha blending
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) and color keying.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63) In addition to the IPU internal subunits, there are also two units
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) outside the IPU that are also involved in video capture on i.MX:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) - MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) interface. This is a Synopsys DesignWare core.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) - Two video multiplexers for selecting among multiple sensor inputs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) to send to a CSI.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) For more info, refer to the latest versions of the i.MX5/6 reference
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) manuals [#f1]_ and [#f2]_.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) Features
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) --------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) Some of the features of this driver include:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) - Many different pipelines can be configured via media controller API,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) that correspond to the hardware video capture pipelines supported in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) the i.MX.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) - Supports parallel, BT.565, and MIPI CSI-2 interfaces.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) - Concurrent independent streams, by configuring pipelines to multiple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) video capture interfaces using independent entities.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) - Scaling, color-space conversion, horizontal and vertical flip, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) image rotation via IC task subdevs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) - Many pixel formats supported (RGB, packed and planar YUV, partial
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) planar YUV).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) - The VDIC subdev supports motion compensated de-interlacing, with three
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) motion compensation modes: low, medium, and high motion. Pipelines are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) defined that allow sending frames to the VDIC subdev directly from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) CSI. There is also support in the future for sending frames to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) VDIC from memory buffers via a output/mem2mem devices.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) - Includes a Frame Interval Monitor (FIM) that can correct vertical sync
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) problems with the ADV718x video decoders.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) Topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) --------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) The following shows the media topologies for the i.MX6Q SabreSD and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) i.MX6Q SabreAuto. Refer to these diagrams in the entity descriptions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) in the next section.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) The i.MX5/6 topologies can differ upstream from the IPUv3 CSI video
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) multiplexers, but the internal IPUv3 topology downstream from there
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) is common to all i.MX5/6 platforms. For example, the SabreSD, with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) MIPI CSI-2 OV5640 sensor, requires the i.MX6 MIPI CSI-2 receiver. But
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) the SabreAuto has only the ADV7180 decoder on a parallel bt.656 bus, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) therefore does not require the MIPI CSI-2 receiver, so it is missing in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) its graph.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) .. _imx6q_topology_graph:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) .. kernel-figure:: imx6q-sabresd.dot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) :alt: Diagram of the i.MX6Q SabreSD media pipeline topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) :align: center
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) Media pipeline graph on i.MX6Q SabreSD
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) .. kernel-figure:: imx6q-sabreauto.dot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) :alt: Diagram of the i.MX6Q SabreAuto media pipeline topology
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) :align: center
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) Media pipeline graph on i.MX6Q SabreAuto
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) Entities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) --------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) imx6-mipi-csi2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) This is the MIPI CSI-2 receiver entity. It has one sink pad to receive
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) four source pads, corresponding to the four MIPI CSI-2 demuxed virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) channel outputs. Multiple source pads can be enabled to independently
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) stream from multiple virtual channels.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) This entity actually consists of two sub-blocks. One is the MIPI CSI-2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) four virtual channels streams, providing four separate parallel buses
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) containing each virtual channel that are routed to CSIs or video
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) multiplexers as described below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) On i.MX6 solo/dual-lite, all four virtual channel buses are routed to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) two video multiplexers. Both CSI0 and CSI1 can receive any virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) channel, as selected by the video multiplexers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) and IPU2-CSI0, respectively, and virtual channel 3 is routed to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) IPU2-CSI1 (again selected by a video mux).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162) ipuX_csiY_mux
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) -------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165) These are the video multiplexers. They have two or more sink pads to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) select from either camera sensors with a parallel interface, or from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) single source pad that routes to a CSI (ipuX_csiY entities).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) On i.MX6 solo/dual-lite, there are two video mux entities. One sits
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) in front of IPU1-CSI0 to select between a parallel sensor and any of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) the four MIPI CSI-2 virtual channels (a total of five sink pads). The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) other mux sits in front of IPU1-CSI1, and again has five sink pads to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) select between a parallel sensor and any of the four MIPI CSI-2 virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) channels.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) On i.MX6 Quad, there are two video mux entities. One sits in front of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) sink pads).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) ipuX_csiY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) These are the CSI entities. They have a single sink pad receiving from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) either a video mux or from a MIPI CSI-2 virtual channel as described
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) above.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) This entity has two source pads. The first source pad can link directly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) that require no IDMAC memory buffer transfer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) When the direct source pad is routed to the ipuX_ic_prp entity, frames
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) from the CSI can be processed by one or both of the IC pre-processing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) tasks.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) When the direct source pad is routed to the ipuX_vdic entity, the VDIC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) will carry out motion-compensated de-interlace using "high motion" mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) (see description of ipuX_vdic entity).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) The second source pad sends video frames directly to memory buffers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) via the SMFC and an IDMAC channel, bypassing IC pre-processing. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) source pad is routed to a capture device node, with a node name of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) format "ipuX_csiY capture".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) Note that since the IDMAC source pad makes use of an IDMAC channel,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) pixel reordering within the same colorspace can be carried out by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) IDMAC channel. For example, if the CSI sink pad is receiving in UYVY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) order, the capture device linked to the IDMAC source pad can capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) in YUYV order. Also, if the CSI sink pad is receiving a packed YUV
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) format, the capture device can capture a planar YUV format such as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) YUV420.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) The IDMAC channel at the IDMAC source pad also supports simple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) interweave without motion compensation, which is activated if the source
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) pad's field type is sequential top-bottom or bottom-top, and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) requested capture interface field type is set to interlaced (t-b, b-t,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) or unqualified interlaced). The capture interface will enforce the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) field order as the source pad field order (interlaced-bt if source pad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) is seq-bt, interlaced-tb if source pad is seq-tb).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) For events produced by ipuX_csiY, see ref:`imx_api_ipuX_csiY`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) Cropping in ipuX_csiY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) The CSI supports cropping the incoming raw sensor frames. This is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) implemented in the ipuX_csiY entities at the sink pad, using the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) crop selection subdev API.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) The CSI also supports fixed divide-by-two downscaling independently in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) width and height. This is implemented in the ipuX_csiY entities at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) the sink pad, using the compose selection subdev API.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) The output rectangle at the ipuX_csiY source pad is the same as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) the compose rectangle at the sink pad. So the source pad rectangle
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) cannot be negotiated, it must be set using the compose selection
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) API at sink pad (if /2 downscale is desired, otherwise source pad
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) rectangle is equal to incoming rectangle).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) To give an example of crop and /2 downscale, this will crop a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) 1280x960 input frame to 640x480, and then /2 downscale in both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) Frame Skipping in ipuX_csiY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) ---------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) The CSI supports frame rate decimation, via frame skipping. Frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) rate decimation is specified by setting the frame intervals at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) sink and source pads. The ipuX_csiY entity then applies the best
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) frame skip setting to the CSI to achieve the desired frame rate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) at the source pad.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) The following example reduces an assumed incoming 60 Hz frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) rate by half at the IDMAC output source pad:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) Frame Interval Monitor in ipuX_csiY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) -----------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) See ref:`imx_api_FIM`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) ipuX_vdic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275) ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) The VDIC carries out motion compensated de-interlacing, with three
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) motion compensation modes: low, medium, and high motion. The mode is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) specified with the menu control V4L2_CID_DEINTERLACING_MODE. The VDIC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280) has two sink pads and a single source pad.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) The direct sink pad receives from an ipuX_csiY direct pad. With this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) link the VDIC can only operate in high motion mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) When the IDMAC sink pad is activated, it receives from an output
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) or mem2mem device node. With this pipeline, the VDIC can also operate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) in low and medium modes, because these modes require receiving
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) frames from memory buffers. Note that an output or mem2mem device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) is not implemented yet, so this sink pad currently has no links.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) The source pad routes to the IC pre-processing entity ipuX_ic_prp.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) ipuX_ic_prp
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) This is the IC pre-processing entity. It acts as a router, routing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) data from its sink pad to one or both of its source pads.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299) This entity has a single sink pad. The sink pad can receive from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) ipuX_csiY direct pad, or from ipuX_vdic.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) This entity has two source pads. One source pad routes to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) pre-process encode task entity (ipuX_ic_prpenc), the other to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305) can be activated at the same time if the sink pad is receiving from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) ipuX_csiY. Only the source pad to the pre-process viewfinder task entity
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) can be activated if the sink pad is receiving from ipuX_vdic (frames
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) from the VDIC can only be processed by the pre-process viewfinder task).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) ipuX_ic_prpenc
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) --------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) This is the IC pre-processing encode entity. It has a single sink
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) pad from ipuX_ic_prp, and a single source pad. The source pad is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) routed to a capture device node, with a node name of the format
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316) "ipuX_ic_prpenc capture".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318) This entity performs the IC pre-process encode task operations:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) color-space conversion, resizing (downscaling and upscaling),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) horizontal and vertical flip, and 90/270 degree rotation. Flip
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) and rotation are provided via standard V4L2 controls.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) Like the ipuX_csiY IDMAC source, this entity also supports simple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) de-interlace without motion compensation, and pixel reordering.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) ipuX_ic_prpvf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) -------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) This is the IC pre-processing viewfinder entity. It has a single sink
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330) pad from ipuX_ic_prp, and a single source pad. The source pad is routed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) to a capture device node, with a node name of the format
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332) "ipuX_ic_prpvf capture".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) This entity is identical in operation to ipuX_ic_prpenc, with the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) resizing and CSC operations and flip/rotation controls. It will receive
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336) and process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337) receiving from ipuX_vdic.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339) Like the ipuX_csiY IDMAC source, this entity supports simple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340) interweaving without motion compensation. However, note that if the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341) ipuX_vdic is included in the pipeline (ipuX_ic_prp is receiving from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) ipuX_vdic), it's not possible to use interweave in ipuX_ic_prpvf,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343) since the ipuX_vdic has already carried out de-interlacing (with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) motion compensation) and therefore the field type output from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) ipuX_vdic can only be none (progressive).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347) Capture Pipelines
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348) -----------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350) The following describe the various use-cases supported by the pipelines.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352) The links shown do not include the backend sensor, video mux, or mipi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353) csi-2 receiver links. This depends on the type of sensor interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) (parallel or mipi csi-2). So these pipelines begin with:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356) sensor -> ipuX_csiY_mux -> ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) for parallel sensors, or:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360) sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ...
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362) for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363) to the video mux (ipuX_csiY_mux) before sending to the CSI, depending
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364) on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365) parenthesis.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) Unprocessed Video Capture:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368) --------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370) Send frames directly from sensor to camera device interface node, with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371) no conversions, via ipuX_csiY IDMAC source pad:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373) -> ipuX_csiY:2 -> ipuX_csiY capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375) IC Direct Conversions:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376) ----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378) This pipeline uses the preprocess encode entity to route frames directly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379) from the CSI to the IC, to carry out scaling up to 1024x1024 resolution,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380) CSC, flipping, and image rotation:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382) -> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384) Motion Compensated De-interlace:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385) --------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387) This pipeline routes frames from the CSI direct pad to the VDIC entity to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388) support motion-compensated de-interlacing (high motion mode only),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389) scaling up to 1024x1024, CSC, flip, and rotation:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391) -> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394) Usage Notes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395) -----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397) To aid in configuration and for backward compatibility with V4L2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398) applications that access controls only from video device nodes, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399) capture device interfaces inherit controls from the active entities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400) in the current pipeline, so controls can be accessed either directly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401) from the subdev or from the active capture device interface. For
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402) example, the FIM controls are available either from the ipuX_csiY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403) subdevs or from the active capture device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405) The following are specific usage notes for the Sabre* reference
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406) boards:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) i.MX6Q SabreLite with OV5642 and OV5640
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410) ---------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412) This platform requires the OmniVision OV5642 module with a parallel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 413) camera interface, and the OV5640 module with a MIPI CSI-2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 414) interface. Both modules are available from Boundary Devices:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 415)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 416) - https://boundarydevices.com/product/nit6x_5mp
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 417) - https://boundarydevices.com/product/nit6x_5mp_mipi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 418)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 419) Note that if only one camera module is available, the other sensor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 420) node can be disabled in the device tree.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 421)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 422) The OV5642 module is connected to the parallel bus input on the i.MX
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 423) internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 424)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 425) The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 426) receiver, and the four virtual channel outputs from the receiver are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 427) routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 428) vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 429) also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 430) OV5640 must not share the same i2c slave address.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 431)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 432) The following basic example configures unprocessed video capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 433) pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 434) the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 435) imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 436) configured to output 640x480, and the OV5642 outputs YUYV2X8, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 437) OV5640 UYVY2X8:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 438)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 439) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 440)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 441) # Setup links for OV5642
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 442) media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 443) media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 444) media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 445) # Setup links for OV5640
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 446) media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 447) media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 448) media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 449) # Configure pads for OV5642 pipeline
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 450) media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 451) media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 452) media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 453) # Configure pads for OV5640 pipeline
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 454) media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 455) media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 456) media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 457)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 458) Streaming can then begin independently on the capture device nodes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 459) "ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 460) be used to select any supported YUV pixelformat on the capture device
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 461) nodes, including planar.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 462)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 463) i.MX6Q SabreAuto with ADV7180 decoder
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 464) -------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 465)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 466) On the i.MX6Q SabreAuto, an on-board ADV7180 SD decoder is connected to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 467) parallel bus input on the internal video mux to IPU1 CSI0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 468)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 469) The following example configures a pipeline to capture from the ADV7180
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 470) video decoder, assuming NTSC 720x480 input signals, using simple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 471) interweave (unconverted and without motion compensation). The adv7180
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 472) must output sequential or alternating fields (field type 'seq-bt' for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 473) NTSC, or 'alternate'):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 474)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 475) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 476)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 477) # Setup links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 478) media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 479) media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 480) media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 481) # Configure pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 482) media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 483) media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 484) media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 485) # Configure "ipu1_csi0 capture" interface (assumed at /dev/video4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 486) v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 487)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 488) Streaming can then begin on /dev/video4. The v4l2-ctl tool can also be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 489) used to select any supported YUV pixelformat on /dev/video4.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 490)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 491) This example configures a pipeline to capture from the ADV7180
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 492) video decoder, assuming PAL 720x576 input signals, with Motion
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 493) Compensated de-interlacing. The adv7180 must output sequential or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 494) alternating fields (field type 'seq-tb' for PAL, or 'alternate').
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 495)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 496) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 497)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 498) # Setup links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 499) media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 500) media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 501) media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 502) media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 503) media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 504) media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 505) # Configure pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 506) media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 507) media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x576]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 508) media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 509) media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 510) media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 511) media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 512) # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 513) v4l2-ctl -d2 --set-fmt-video=field=none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 514)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 515) Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 516) used to select any supported YUV pixelformat on /dev/video2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 517)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 518) This platform accepts Composite Video analog inputs to the ADV7180 on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 519) Ain1 (connector J42).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 520)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 521) i.MX6DL SabreAuto with ADV7180 decoder
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 522) --------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 523)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 524) On the i.MX6DL SabreAuto, an on-board ADV7180 SD decoder is connected to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 525) parallel bus input on the internal video mux to IPU1 CSI0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 526)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 527) The following example configures a pipeline to capture from the ADV7180
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 528) video decoder, assuming NTSC 720x480 input signals, using simple
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 529) interweave (unconverted and without motion compensation). The adv7180
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 530) must output sequential or alternating fields (field type 'seq-bt' for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 531) NTSC, or 'alternate'):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 532)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 533) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 534)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 535) # Setup links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 536) media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 537) media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 538) media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 539) # Configure pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 540) media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 541) media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 542) media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 543) # Configure "ipu1_csi0 capture" interface (assumed at /dev/video0)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 544) v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 545)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 546) Streaming can then begin on /dev/video0. The v4l2-ctl tool can also be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 547) used to select any supported YUV pixelformat on /dev/video0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 548)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 549) This example configures a pipeline to capture from the ADV7180
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 550) video decoder, assuming PAL 720x576 input signals, with Motion
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 551) Compensated de-interlacing. The adv7180 must output sequential or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 552) alternating fields (field type 'seq-tb' for PAL, or 'alternate').
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 553)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 554) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 555)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 556) # Setup links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 557) media-ctl -l "'adv7180 4-0021':0 -> 'ipu1_csi0_mux':4[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 558) media-ctl -l "'ipu1_csi0_mux':5 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 559) media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 560) media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 561) media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 562) media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 563) # Configure pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 564) media-ctl -V "'adv7180 4-0021':0 [fmt:UYVY2X8/720x576 field:seq-tb]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 565) media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x576]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 566) media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x576]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 567) media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x576 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 568) media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 569) media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 570) # Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 571) v4l2-ctl -d2 --set-fmt-video=field=none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 572)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 573) Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 574) used to select any supported YUV pixelformat on /dev/video2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 575)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 576) This platform accepts Composite Video analog inputs to the ADV7180 on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 577) Ain1 (connector J42).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 578)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 579) i.MX6Q SabreSD with MIPI CSI-2 OV5640
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 580) -------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 581)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 582) Similarly to i.MX6Q SabreLite, the i.MX6Q SabreSD supports a parallel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 583) interface OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 584) module. The OV5642 connects to i2c bus 1 and the OV5640 to i2c bus 2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 585)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 586) The device tree for SabreSD includes OF graphs for both the parallel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 587) OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 588) CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 589) The OV5640 module connects to MIPI connector J5. The NXP part number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 590) for the OV5640 module that connects to the SabreSD board is H120729.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 591)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 592) The following example configures unprocessed video capture pipeline to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 593) capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 594)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 595) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 596)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 597) # Setup links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 598) media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 599) media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 600) media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 601) media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 602) # Configure pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 603) media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 604) media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 605) media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 606) media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 607)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 608) Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 609) tool can be used to select any supported pixelformat on the capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 610) device node.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 611)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 612) To determine what is the /dev/video node correspondent to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 613) "ipu1_csi0 capture":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 614)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 615) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 616)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 617) media-ctl -e "ipu1_csi0 capture"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 618) /dev/video0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 619)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 620) /dev/video0 is the streaming element in this case.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 621)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 622) Starting the streaming via v4l2-ctl:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 623)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 624) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 625)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 626) v4l2-ctl --stream-mmap -d /dev/video0
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 627)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 628) Starting the streaming via Gstreamer and sending the content to the display:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 629)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 630) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 631)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 632) gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 633)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 634) The following example configures a direct conversion pipeline to capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 635) from the OV5640, transmitting on MIPI CSI-2 virtual channel 0. It also
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 636) shows colorspace conversion and scaling at IC output.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 637)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 638) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 639)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 640) # Setup links
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 641) media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 642) media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 643) media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 644) media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 645) media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 646) media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 647) # Configure pads
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 648) media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 649) media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 650) media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 651) media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 652) media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 653) media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 654) # Set a format at the capture interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 655) v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 656)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 657) Streaming can then begin on "ipu1_ic_prpenc capture" node.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 658)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 659) To determine what is the /dev/video node correspondent to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 660) "ipu1_ic_prpenc capture":
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 661)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 662) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 663)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 664) media-ctl -e "ipu1_ic_prpenc capture"
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 665) /dev/video1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 666)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 667)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 668) /dev/video1 is the streaming element in this case.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 669)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 670) Starting the streaming via v4l2-ctl:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 671)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 672) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 673)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 674) v4l2-ctl --stream-mmap -d /dev/video1
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 675)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 676) Starting the streaming via Gstreamer and sending the content to the display:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 677)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 678) .. code-block:: none
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 679)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 680) gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 681)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 682) Known Issues
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 683) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 684)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 685) 1. When using 90 or 270 degree rotation control at capture resolutions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 686) near the IC resizer limit of 1024x1024, and combined with planar
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 687) pixel formats (YUV420, YUV422p), frame capture will often fail with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 688) no end-of-frame interrupts from the IDMAC channel. To work around
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 689) this, use lower resolution and/or packed formats (YUYV, RGB3, etc.)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 690) when 90 or 270 rotations are needed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 691)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 692)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 693) File list
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 694) ---------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 695)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 696) drivers/staging/media/imx/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 697) include/media/imx.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 698) include/linux/imx-media.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 699)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 700) References
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 701) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 702)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 703) .. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 704) .. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 705)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 706)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 707) Authors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 708) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 709)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 710) - Steve Longerbeam <steve_longerbeam@mentor.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 711) - Philipp Zabel <kernel@pengutronix.de>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 712) - Russell King <linux@armlinux.org.uk>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 713)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 714) Copyright (C) 2012-2017 Mentor Graphics Inc.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags