^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3) .. _encoder:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) *************************************************
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) Memory-to-Memory Stateful Video Encoder Interface
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) *************************************************
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) A stateful video encoder takes raw video frames in display order and encodes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10) them into a bytestream. It generates complete chunks of the bytestream, including
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) all metadata, headers, etc. The resulting bytestream does not require any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) further post-processing by the client.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) Performing software stream processing, header generation etc. in the driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) in order to support this interface is strongly discouraged. In case such
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16) operations are needed, use of the Stateless Video Encoder Interface (in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) development) is strongly advised.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19) Conventions and Notations Used in This Document
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) ===============================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) 1. The general V4L2 API rules apply if not specified in this document
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) otherwise.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) 2. The meaning of words "must", "may", "should", etc. is as per `RFC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) 2119 <https://tools.ietf.org/html/rfc2119>`_.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) 3. All steps not marked "optional" are required.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30) 4. :c:func:`VIDIOC_G_EXT_CTRLS` and :c:func:`VIDIOC_S_EXT_CTRLS` may be used
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) interchangeably with :c:func:`VIDIOC_G_CTRL` and :c:func:`VIDIOC_S_CTRL`,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) unless specified otherwise.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) 5. Single-planar API (see :ref:`planar-apis`) and applicable structures may be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) used interchangeably with multi-planar API, unless specified otherwise,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) depending on encoder capabilities and following the general V4L2 guidelines.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) 6. i = [a..b]: sequence of integers from a to b, inclusive, i.e. i =
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) [0..2]: i = 0, 1, 2.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41) 7. Given an ``OUTPUT`` buffer A, then A' represents a buffer on the ``CAPTURE``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) queue containing data that resulted from processing buffer A.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) Glossary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) Refer to :ref:`decoder-glossary`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) State Machine
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) .. kernel-render:: DOT
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) :alt: DOT digraph of encoder state machine
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) :caption: Encoder State Machine
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) digraph encoder_state_machine {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) node [shape = doublecircle, label="Encoding"] Encoding;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) node [shape = circle, label="Initialization"] Initialization;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) node [shape = circle, label="Stopped"] Stopped;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) node [shape = circle, label="Drain"] Drain;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) node [shape = circle, label="Reset"] Reset;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) node [shape = point]; qi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) qi -> Initialization [ label = "open()" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) Initialization -> Encoding [ label = "Both queues streaming" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) Encoding -> Drain [ label = "V4L2_ENC_CMD_STOP" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) Encoding -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) Encoding -> Stopped [ label = "VIDIOC_STREAMOFF(OUTPUT)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) Encoding -> Encoding;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) Drain -> Stopped [ label = "All CAPTURE\nbuffers dequeued\nor\nVIDIOC_STREAMOFF(OUTPUT)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) Drain -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) Reset -> Encoding [ label = "VIDIOC_STREAMON(CAPTURE)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) Reset -> Initialization [ label = "VIDIOC_REQBUFS(OUTPUT, 0)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) Stopped -> Encoding [ label = "V4L2_ENC_CMD_START\nor\nVIDIOC_STREAMON(OUTPUT)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) Stopped -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ];
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) Querying Capabilities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) =====================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) 1. To enumerate the set of coded formats supported by the encoder, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) client may call :c:func:`VIDIOC_ENUM_FMT` on ``CAPTURE``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90) * The full set of supported formats will be returned, regardless of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) format set on ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93) 2. To enumerate the set of supported raw formats, the client may call
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) :c:func:`VIDIOC_ENUM_FMT` on ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96) * Only the formats supported for the format currently active on ``CAPTURE``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) will be returned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) * In order to enumerate raw formats supported by a given coded format,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) the client must first set that coded format on ``CAPTURE`` and then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) enumerate the formats on ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) 3. The client may use :c:func:`VIDIOC_ENUM_FRAMESIZES` to detect supported
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) resolutions for a given format, passing the desired pixel format in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) :c:type:`v4l2_frmsizeenum` ``pixel_format``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) * Values returned by :c:func:`VIDIOC_ENUM_FRAMESIZES` for a coded pixel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) format will include all possible coded resolutions supported by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) encoder for the given coded pixel format.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) * Values returned by :c:func:`VIDIOC_ENUM_FRAMESIZES` for a raw pixel format
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) will include all possible frame buffer resolutions supported by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) encoder for the given raw pixel format and coded format currently set on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) ``CAPTURE``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) 4. The client may use :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` to detect supported
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117) frame intervals for a given format and resolution, passing the desired pixel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) format in :c:type:`v4l2_frmsizeenum` ``pixel_format`` and the resolution
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) in :c:type:`v4l2_frmsizeenum` ``width`` and :c:type:`v4l2_frmsizeenum`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) ``height``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) * Values returned by :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` for a coded pixel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) format and coded resolution will include all possible frame intervals
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) supported by the encoder for the given coded pixel format and resolution.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) * Values returned by :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` for a raw pixel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) format and resolution will include all possible frame intervals supported
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) by the encoder for the given raw pixel format and resolution and for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) coded format, coded resolution and coded frame interval currently set on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) ``CAPTURE``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) * Support for :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` is optional. If it is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) not implemented, then there are no special restrictions other than the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) limits of the codec itself.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) 5. Supported profiles and levels for the coded format currently set on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) ``CAPTURE``, if applicable, may be queried using their respective controls
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138) via :c:func:`VIDIOC_QUERYCTRL`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) 6. Any additional encoder capabilities may be discovered by querying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) their respective controls.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) Initialization
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) ==============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 1. Set the coded format on the ``CAPTURE`` queue via :c:func:`VIDIOC_S_FMT`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) ``pixelformat``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) the coded format to be produced.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) ``sizeimage``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) desired size of ``CAPTURE`` buffers; the encoder may adjust it to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) match hardware requirements.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) ``width``, ``height``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) ignored (read-only).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) ``sizeimage``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) adjusted size of ``CAPTURE`` buffers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171) ``width``, ``height``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) the coded size selected by the encoder based on current state, e.g.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) ``OUTPUT`` format, selection rectangles, etc. (read-only).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) .. important::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) Changing the ``CAPTURE`` format may change the currently set ``OUTPUT``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) format. How the new ``OUTPUT`` format is determined is up to the encoder
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) and the client must ensure it matches its needs afterwards.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) 2. **Optional.** Enumerate supported ``OUTPUT`` formats (raw formats for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) source) for the selected coded format via :c:func:`VIDIOC_ENUM_FMT`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) ``pixelformat``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) raw format supported for the coded format currently selected on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) the ``CAPTURE`` queue.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) 3. Set the raw source format on the ``OUTPUT`` queue via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) :c:func:`VIDIOC_S_FMT`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) ``pixelformat``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) raw format of the source.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) ``width``, ``height``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) source resolution.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) ``width``, ``height``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) may be adjusted to match encoder minimums, maximums and alignment
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) requirements, as required by the currently selected formats, as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) reported by :c:func:`VIDIOC_ENUM_FRAMESIZES`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) * Setting the ``OUTPUT`` format will reset the selection rectangles to their
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) default values, based on the new resolution, as described in the next
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) step.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) 4. Set the raw frame interval on the ``OUTPUT`` queue via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) :c:func:`VIDIOC_S_PARM`. This also sets the coded frame interval on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) ``CAPTURE`` queue to the same value.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) * ** Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) ``parm.output``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) set all fields except ``parm.output.timeperframe`` to 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) ``parm.output.timeperframe``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) the desired frame interval; the encoder may adjust it to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) match hardware requirements.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) ``parm.output.timeperframe``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) the adjusted frame interval.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) .. important::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) Changing the ``OUTPUT`` frame interval *also* sets the framerate that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) the encoder uses to encode the video. So setting the frame interval
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) to 1/24 (or 24 frames per second) will produce a coded video stream
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) that can be played back at that speed. The frame interval for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) ``OUTPUT`` queue is just a hint, the application may provide raw
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) frames at a different rate. It can be used by the driver to help
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) schedule multiple encoders running in parallel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) In the next step the ``CAPTURE`` frame interval can optionally be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) changed to a different value. This is useful for off-line encoding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) were the coded frame interval can be different from the rate at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) which raw frames are supplied.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) .. important::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) ``timeperframe`` deals with *frames*, not fields. So for interlaced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) formats this is the time per two fields, since a frame consists of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272) a top and a bottom field.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276) It is due to historical reasons that changing the ``OUTPUT`` frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277) interval also changes the coded frame interval on the ``CAPTURE``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) queue. Ideally these would be independent settings, but that would
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279) break the existing API.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281) 5. **Optional** Set the coded frame interval on the ``CAPTURE`` queue via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) :c:func:`VIDIOC_S_PARM`. This is only necessary if the coded frame
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) interval is different from the raw frame interval, which is typically
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) the case for off-line encoding. Support for this feature is signalled
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) by the :ref:`V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL <fmtdesc-flags>` format flag.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) * ** Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) ``parm.capture``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293) set all fields except ``parm.capture.timeperframe`` to 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) ``parm.capture.timeperframe``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) the desired coded frame interval; the encoder may adjust it to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297) match hardware requirements.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301) ``parm.capture.timeperframe``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302) the adjusted frame interval.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) .. important::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) Changing the ``CAPTURE`` frame interval sets the framerate for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307) coded video. It does *not* set the rate at which buffers arrive on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308) ``CAPTURE`` queue, that depends on how fast the encoder is and how
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309) fast raw frames are queued on the ``OUTPUT`` queue.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) .. important::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313) ``timeperframe`` deals with *frames*, not fields. So for interlaced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) formats this is the time per two fields, since a frame consists of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) a top and a bottom field.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) Not all drivers support this functionality, in that case just set
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) the desired coded frame interval for the ``OUTPUT`` queue.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) However, drivers that can schedule multiple encoders based on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) ``OUTPUT`` frame interval must support this optional feature.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) 6. **Optional.** Set the visible resolution for the stream metadata via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) :c:func:`VIDIOC_S_SELECTION` on the ``OUTPUT`` queue if it is desired
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) to be different than the full OUTPUT resolution.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) ``target``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335) set to ``V4L2_SEL_TGT_CROP``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337) ``r.left``, ``r.top``, ``r.width``, ``r.height``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338) visible rectangle; this must fit within the `V4L2_SEL_TGT_CROP_BOUNDS`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339) rectangle and may be subject to adjustment to match codec and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340) hardware constraints.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) ``r.left``, ``r.top``, ``r.width``, ``r.height``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) visible rectangle adjusted by the encoder.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347) * The following selection targets are supported on ``OUTPUT``:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349) ``V4L2_SEL_TGT_CROP_BOUNDS``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350) equal to the full source frame, matching the active ``OUTPUT``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351) format.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353) ``V4L2_SEL_TGT_CROP_DEFAULT``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354) equal to ``V4L2_SEL_TGT_CROP_BOUNDS``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356) ``V4L2_SEL_TGT_CROP``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357) rectangle within the source buffer to be encoded into the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358) ``CAPTURE`` stream; defaults to ``V4L2_SEL_TGT_CROP_DEFAULT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362) A common use case for this selection target is encoding a source
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363) video with a resolution that is not a multiple of a macroblock,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364) e.g. the common 1920x1080 resolution may require the source
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365) buffers to be aligned to 1920x1088 for codecs with 16x16 macroblock
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366) size. To avoid encoding the padding, the client needs to explicitly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) configure this selection target to 1920x1080.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369) .. warning::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371) The encoder may adjust the crop/compose rectangles to the nearest
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372) supported ones to meet codec and hardware requirements. The client needs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373) to check the adjusted rectangle returned by :c:func:`VIDIOC_S_SELECTION`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375) 7. Allocate buffers for both ``OUTPUT`` and ``CAPTURE`` via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376) :c:func:`VIDIOC_REQBUFS`. This may be performed in any order.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380) ``count``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381) requested number of buffers to allocate; greater than zero.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT`` or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385) ``CAPTURE``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392) ``count``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393) actual number of buffers allocated.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395) .. warning::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397) The actual number of allocated buffers may differ from the ``count``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398) given. The client must check the updated value of ``count`` after the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399) call returns.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403) To allocate more than the minimum number of OUTPUT buffers (for pipeline
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404) depth), the client may query the ``V4L2_CID_MIN_BUFFERS_FOR_OUTPUT``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405) control to get the minimum number of buffers required, and pass the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406) obtained value plus the number of additional buffers needed in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407) ``count`` field to :c:func:`VIDIOC_REQBUFS`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) Alternatively, :c:func:`VIDIOC_CREATE_BUFS` can be used to have more
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410) control over buffer allocation.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 413)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 414) ``count``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 415) requested number of buffers to allocate; greater than zero.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 416)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 417) ``type``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 418) a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 419)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 420) other fields
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 421) follow standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 422)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 423) * **Return fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 424)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 425) ``count``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 426) adjusted to the number of allocated buffers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 427)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 428) 8. Begin streaming on both ``OUTPUT`` and ``CAPTURE`` queues via
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 429) :c:func:`VIDIOC_STREAMON`. This may be performed in any order. The actual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 430) encoding process starts when both queues start streaming.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 431)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 432) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 433)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 434) If the client stops the ``CAPTURE`` queue during the encode process and then
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 435) restarts it again, the encoder will begin generating a stream independent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 436) from the stream generated before the stop. The exact constraints depend
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 437) on the coded format, but may include the following implications:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 438)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 439) * encoded frames produced after the restart must not reference any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 440) frames produced before the stop, e.g. no long term references for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 441) H.264/HEVC,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 442)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 443) * any headers that must be included in a standalone stream must be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 444) produced again, e.g. SPS and PPS for H.264/HEVC.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 445)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 446) Encoding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 447) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 448)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 449) This state is reached after the `Initialization` sequence finishes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 450) successfully. In this state, the client queues and dequeues buffers to both
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 451) queues via :c:func:`VIDIOC_QBUF` and :c:func:`VIDIOC_DQBUF`, following the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 452) standard semantics.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 453)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 454) The content of encoded ``CAPTURE`` buffers depends on the active coded pixel
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 455) format and may be affected by codec-specific extended controls, as stated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 456) in the documentation of each format.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 457)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 458) Both queues operate independently, following standard behavior of V4L2 buffer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 459) queues and memory-to-memory devices. In addition, the order of encoded frames
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 460) dequeued from the ``CAPTURE`` queue may differ from the order of queuing raw
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 461) frames to the ``OUTPUT`` queue, due to properties of the selected coded format,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 462) e.g. frame reordering.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 463)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 464) The client must not assume any direct relationship between ``CAPTURE`` and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 465) ``OUTPUT`` buffers and any specific timing of buffers becoming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 466) available to dequeue. Specifically:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 467)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 468) * a buffer queued to ``OUTPUT`` may result in more than one buffer produced on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 469) ``CAPTURE`` (for example, if returning an encoded frame allowed the encoder
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 470) to return a frame that preceded it in display, but succeeded it in the decode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 471) order; however, there may be other reasons for this as well),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 472)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 473) * a buffer queued to ``OUTPUT`` may result in a buffer being produced on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 474) ``CAPTURE`` later into encode process, and/or after processing further
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 475) ``OUTPUT`` buffers, or be returned out of order, e.g. if display
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 476) reordering is used,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 477)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 478) * buffers may become available on the ``CAPTURE`` queue without additional
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 479) buffers queued to ``OUTPUT`` (e.g. during drain or ``EOS``), because of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 480) ``OUTPUT`` buffers queued in the past whose encoding results are only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 481) available at later time, due to specifics of the encoding process,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 482)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 483) * buffers queued to ``OUTPUT`` may not become available to dequeue instantly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 484) after being encoded into a corresponding ``CAPTURE`` buffer, e.g. if the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 485) encoder needs to use the frame as a reference for encoding further frames.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 486)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 487) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 488)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 489) To allow matching encoded ``CAPTURE`` buffers with ``OUTPUT`` buffers they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 490) originated from, the client can set the ``timestamp`` field of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 491) :c:type:`v4l2_buffer` struct when queuing an ``OUTPUT`` buffer. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 492) ``CAPTURE`` buffer(s), which resulted from encoding that ``OUTPUT`` buffer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 493) will have their ``timestamp`` field set to the same value when dequeued.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 494)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 495) In addition to the straightforward case of one ``OUTPUT`` buffer producing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 496) one ``CAPTURE`` buffer, the following cases are defined:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 497)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 498) * one ``OUTPUT`` buffer generates multiple ``CAPTURE`` buffers: the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 499) ``OUTPUT`` timestamp will be copied to multiple ``CAPTURE`` buffers,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 500)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 501) * the encoding order differs from the presentation order (i.e. the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 502) ``CAPTURE`` buffers are out-of-order compared to the ``OUTPUT`` buffers):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 503) ``CAPTURE`` timestamps will not retain the order of ``OUTPUT`` timestamps.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 504)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 505) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 506)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 507) To let the client distinguish between frame types (keyframes, intermediate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 508) frames; the exact list of types depends on the coded format), the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 509) ``CAPTURE`` buffers will have corresponding flag bits set in their
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 510) :c:type:`v4l2_buffer` struct when dequeued. See the documentation of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 511) :c:type:`v4l2_buffer` and each coded pixel format for exact list of flags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 512) and their meanings.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 513)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 514) Should an encoding error occur, it will be reported to the client with the level
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 515) of details depending on the encoder capabilities. Specifically:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 516)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 517) * the ``CAPTURE`` buffer (if any) that contains the results of the failed encode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 518) operation will be returned with the ``V4L2_BUF_FLAG_ERROR`` flag set,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 519)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 520) * if the encoder is able to precisely report the ``OUTPUT`` buffer(s) that triggered
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 521) the error, such buffer(s) will be returned with the ``V4L2_BUF_FLAG_ERROR`` flag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 522) set.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 523)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 524) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 525)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 526) If a ``CAPTURE`` buffer is too small then it is just returned with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 527) ``V4L2_BUF_FLAG_ERROR`` flag set. More work is needed to detect that this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 528) error occurred because the buffer was too small, and to provide support to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 529) free existing buffers that were too small.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 530)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 531) In case of a fatal failure that does not allow the encoding to continue, any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 532) further operations on corresponding encoder file handle will return the -EIO
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 533) error code. The client may close the file handle and open a new one, or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 534) alternatively reinitialize the instance by stopping streaming on both queues,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 535) releasing all buffers and performing the Initialization sequence again.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 536)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 537) Encoding Parameter Changes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 538) ==========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 539)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 540) The client is allowed to use :c:func:`VIDIOC_S_CTRL` to change encoder
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 541) parameters at any time. The availability of parameters is encoder-specific
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 542) and the client must query the encoder to find the set of available controls.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 543)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 544) The ability to change each parameter during encoding is encoder-specific, as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 545) per the standard semantics of the V4L2 control interface. The client may
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 546) attempt to set a control during encoding and if the operation fails with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 547) -EBUSY error code, the ``CAPTURE`` queue needs to be stopped for the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 548) configuration change to be allowed. To do this, it may follow the `Drain`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 549) sequence to avoid losing the already queued/encoded frames.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 550)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 551) The timing of parameter updates is encoder-specific, as per the standard
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 552) semantics of the V4L2 control interface. If the client needs to apply the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 553) parameters exactly at specific frame, using the Request API
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 554) (:ref:`media-request-api`) should be considered, if supported by the encoder.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 555)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 556) Drain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 557) =====
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 558)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 559) To ensure that all the queued ``OUTPUT`` buffers have been processed and the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 560) related ``CAPTURE`` buffers are given to the client, the client must follow the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 561) drain sequence described below. After the drain sequence ends, the client has
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 562) received all encoded frames for all ``OUTPUT`` buffers queued before the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 563) sequence was started.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 564)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 565) 1. Begin the drain sequence by issuing :c:func:`VIDIOC_ENCODER_CMD`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 566)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 567) * **Required fields:**
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 568)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 569) ``cmd``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 570) set to ``V4L2_ENC_CMD_STOP``.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 571)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 572) ``flags``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 573) set to 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 574)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 575) ``pts``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 576) set to 0.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 577)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 578) .. warning::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 579)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 580) The sequence can be only initiated if both ``OUTPUT`` and ``CAPTURE``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 581) queues are streaming. For compatibility reasons, the call to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 582) :c:func:`VIDIOC_ENCODER_CMD` will not fail even if any of the queues is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 583) not streaming, but at the same time it will not initiate the `Drain`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 584) sequence and so the steps described below would not be applicable.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 585)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 586) 2. Any ``OUTPUT`` buffers queued by the client before the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 587) :c:func:`VIDIOC_ENCODER_CMD` was issued will be processed and encoded as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 588) normal. The client must continue to handle both queues independently,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 589) similarly to normal encode operation. This includes:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 590)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 591) * queuing and dequeuing ``CAPTURE`` buffers, until a buffer marked with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 592) ``V4L2_BUF_FLAG_LAST`` flag is dequeued,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 593)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 594) .. warning::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 595)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 596) The last buffer may be empty (with :c:type:`v4l2_buffer`
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 597) ``bytesused`` = 0) and in that case it must be ignored by the client,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 598) as it does not contain an encoded frame.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 599)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 600) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 601)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 602) Any attempt to dequeue more ``CAPTURE`` buffers beyond the buffer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 603) marked with ``V4L2_BUF_FLAG_LAST`` will result in a -EPIPE error from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 604) :c:func:`VIDIOC_DQBUF`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 605)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 606) * dequeuing processed ``OUTPUT`` buffers, until all the buffers queued
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 607) before the ``V4L2_ENC_CMD_STOP`` command are dequeued,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 608)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 609) * dequeuing the ``V4L2_EVENT_EOS`` event, if the client subscribes to it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 610)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 611) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 612)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 613) For backwards compatibility, the encoder will signal a ``V4L2_EVENT_EOS``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 614) event when the last frame has been encoded and all frames are ready to be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 615) dequeued. It is deprecated behavior and the client must not rely on it.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 616) The ``V4L2_BUF_FLAG_LAST`` buffer flag should be used instead.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 617)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 618) 3. Once all ``OUTPUT`` buffers queued before the ``V4L2_ENC_CMD_STOP`` call are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 619) dequeued and the last ``CAPTURE`` buffer is dequeued, the encoder is stopped
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 620) and it will accept, but not process any newly queued ``OUTPUT`` buffers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 621) until the client issues any of the following operations:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 622)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 623) * ``V4L2_ENC_CMD_START`` - the encoder will not be reset and will resume
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 624) operation normally, with all the state from before the drain,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 625)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 626) * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 627) ``CAPTURE`` queue - the encoder will be reset (see the `Reset` sequence)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 628) and then resume encoding,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 629)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 630) * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 631) ``OUTPUT`` queue - the encoder will resume operation normally, however any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 632) source frames queued to the ``OUTPUT`` queue between ``V4L2_ENC_CMD_STOP``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 633) and :c:func:`VIDIOC_STREAMOFF` will be discarded.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 634)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 635) .. note::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 636)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 637) Once the drain sequence is initiated, the client needs to drive it to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 638) completion, as described by the steps above, unless it aborts the process by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 639) issuing :c:func:`VIDIOC_STREAMOFF` on any of the ``OUTPUT`` or ``CAPTURE``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 640) queues. The client is not allowed to issue ``V4L2_ENC_CMD_START`` or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 641) ``V4L2_ENC_CMD_STOP`` again while the drain sequence is in progress and they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 642) will fail with -EBUSY error code if attempted.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 643)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 644) For reference, handling of various corner cases is described below:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 645)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 646) * In case of no buffer in the ``OUTPUT`` queue at the time the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 647) ``V4L2_ENC_CMD_STOP`` command was issued, the drain sequence completes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 648) immediately and the encoder returns an empty ``CAPTURE`` buffer with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 649) ``V4L2_BUF_FLAG_LAST`` flag set.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 650)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 651) * In case of no buffer in the ``CAPTURE`` queue at the time the drain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 652) sequence completes, the next time the client queues a ``CAPTURE`` buffer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 653) it is returned at once as an empty buffer with the ``V4L2_BUF_FLAG_LAST``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 654) flag set.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 655)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 656) * If :c:func:`VIDIOC_STREAMOFF` is called on the ``CAPTURE`` queue in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 657) middle of the drain sequence, the drain sequence is canceled and all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 658) ``CAPTURE`` buffers are implicitly returned to the client.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 659)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 660) * If :c:func:`VIDIOC_STREAMOFF` is called on the ``OUTPUT`` queue in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 661) middle of the drain sequence, the drain sequence completes immediately and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 662) next ``CAPTURE`` buffer will be returned empty with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 663) ``V4L2_BUF_FLAG_LAST`` flag set.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 664)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 665) Although not mandatory, the availability of encoder commands may be queried
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 666) using :c:func:`VIDIOC_TRY_ENCODER_CMD`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 667)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 668) Reset
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 669) =====
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 670)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 671) The client may want to request the encoder to reinitialize the encoding, so
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 672) that the following stream data becomes independent from the stream data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 673) generated before. Depending on the coded format, that may imply that:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 674)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 675) * encoded frames produced after the restart must not reference any frames
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 676) produced before the stop, e.g. no long term references for H.264/HEVC,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 677)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 678) * any headers that must be included in a standalone stream must be produced
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 679) again, e.g. SPS and PPS for H.264/HEVC.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 680)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 681) This can be achieved by performing the reset sequence.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 682)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 683) 1. Perform the `Drain` sequence to ensure all the in-flight encoding finishes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 684) and respective buffers are dequeued.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 685)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 686) 2. Stop streaming on the ``CAPTURE`` queue via :c:func:`VIDIOC_STREAMOFF`. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 687) will return all currently queued ``CAPTURE`` buffers to the client, without
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 688) valid frame data.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 689)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 690) 3. Start streaming on the ``CAPTURE`` queue via :c:func:`VIDIOC_STREAMON` and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 691) continue with regular encoding sequence. The encoded frames produced into
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 692) ``CAPTURE`` buffers from now on will contain a standalone stream that can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 693) decoded without the need for frames encoded before the reset sequence,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 694) starting at the first ``OUTPUT`` buffer queued after issuing the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 695) `V4L2_ENC_CMD_STOP` of the `Drain` sequence.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 696)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 697) This sequence may be also used to change encoding parameters for encoders
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 698) without the ability to change the parameters on the fly.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 699)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 700) Commit Points
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 701) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 702)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 703) Setting formats and allocating buffers triggers changes in the behavior of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 704) encoder.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 705)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 706) 1. Setting the format on the ``CAPTURE`` queue may change the set of formats
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 707) supported/advertised on the ``OUTPUT`` queue. In particular, it also means
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 708) that the ``OUTPUT`` format may be reset and the client must not rely on the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 709) previously set format being preserved.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 710)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 711) 2. Enumerating formats on the ``OUTPUT`` queue always returns only formats
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 712) supported for the current ``CAPTURE`` format.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 713)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 714) 3. Setting the format on the ``OUTPUT`` queue does not change the list of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 715) formats available on the ``CAPTURE`` queue. An attempt to set the ``OUTPUT``
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 716) format that is not supported for the currently selected ``CAPTURE`` format
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 717) will result in the encoder adjusting the requested ``OUTPUT`` format to a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 718) supported one.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 719)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 720) 4. Enumerating formats on the ``CAPTURE`` queue always returns the full set of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 721) supported coded formats, irrespective of the current ``OUTPUT`` format.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 722)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 723) 5. While buffers are allocated on any of the ``OUTPUT`` or ``CAPTURE`` queues,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 724) the client must not change the format on the ``CAPTURE`` queue. Drivers will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 725) return the -EBUSY error code for any such format change attempt.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 726)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 727) To summarize, setting formats and allocation must always start with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 728) ``CAPTURE`` queue and the ``CAPTURE`` queue is the master that governs the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 729) set of supported formats for the ``OUTPUT`` queue.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags