Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) =========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) ALSA Compress-Offload API
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) =========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) Pierre-Louis.Bossart <pierre-louis.bossart@linux.intel.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) Vinod Koul <vinod.koul@linux.intel.com>
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12) Since its early days, the ALSA API was defined with PCM support or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13) constant bitrates payloads such as IEC61937 in mind. Arguments and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14) returned values in frames are the norm, making it a challenge to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15) extend the existing API to compressed data streams.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) In recent years, audio digital signal processors (DSP) were integrated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18) in system-on-chip designs, and DSPs are also integrated in audio
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19) codecs. Processing compressed data on such DSPs results in a dramatic
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20) reduction of power consumption compared to host-based
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21) processing. Support for such hardware has not been very good in Linux,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22) mostly because of a lack of a generic API available in the mainline
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) kernel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25) Rather than requiring a compatibility break with an API change of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26) ALSA PCM interface, a new 'Compressed Data' API is introduced to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27) provide a control and data-streaming interface for audio DSPs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29) The design of this API was inspired by the 2-year experience with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30) Intel Moorestown SOC, with many corrections required to upstream the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31) API in the mainline kernel instead of the staging tree and make it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32) usable by others.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35) Requirements
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37) The main requirements are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) - separation between byte counts and time. Compressed formats may have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40)   a header per file, per frame, or no header at all. The payload size
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41)   may vary from frame-to-frame. As a result, it is not possible to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42)   estimate reliably the duration of audio buffers when handling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43)   compressed data. Dedicated mechanisms are required to allow for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44)   reliable audio-video synchronization, which requires precise
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45)   reporting of the number of samples rendered at any given time.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47) - Handling of multiple formats. PCM data only requires a specification
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48)   of the sampling rate, number of channels and bits per sample. In
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49)   contrast, compressed data comes in a variety of formats. Audio DSPs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50)   may also provide support for a limited number of audio encoders and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51)   decoders embedded in firmware, or may support more choices through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52)   dynamic download of libraries.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54) - Focus on main formats. This API provides support for the most
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55)   popular formats used for audio and video capture and playback. It is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56)   likely that as audio compression technology advances, new formats
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57)   will be added.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59) - Handling of multiple configurations. Even for a given format like
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60)   AAC, some implementations may support AAC multichannel but HE-AAC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61)   stereo. Likewise WMA10 level M3 may require too much memory and cpu
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62)   cycles. The new API needs to provide a generic way of listing these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63)   formats.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65) - Rendering/Grabbing only. This API does not provide any means of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66)   hardware acceleration, where PCM samples are provided back to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67)   user-space for additional processing. This API focuses instead on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68)   streaming compressed data to a DSP, with the assumption that the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69)   decoded samples are routed to a physical output or logical back-end.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71) - Complexity hiding. Existing user-space multimedia frameworks all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72)   have existing enums/structures for each compressed format. This new
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73)   API assumes the existence of a platform-specific compatibility layer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74)   to expose, translate and make use of the capabilities of the audio
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75)   DSP, eg. Android HAL or PulseAudio sinks. By construction, regular
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76)   applications are not supposed to make use of this API.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) Design
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80) ======
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81) The new API shares a number of concepts with the PCM API for flow
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82) control. Start, pause, resume, drain and stop commands have the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83) semantics no matter what the content is.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85) The concept of memory ring buffer divided in a set of fragments is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86) borrowed from the ALSA PCM API. However, only sizes in bytes can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) specified.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89) Seeks/trick modes are assumed to be handled by the host.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) The notion of rewinds/forwards is not supported. Data committed to the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92) ring buffer cannot be invalidated, except when dropping all buffers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94) The Compressed Data API does not make any assumptions on how the data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95) is transmitted to the audio DSP. DMA transfers from main memory to an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96) embedded audio cluster or to a SPI interface for external DSPs are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97) possible. As in the ALSA PCM case, a core set of routines is exposed;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98) each driver implementer will have to write support for a set of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99) mandatory routines and possibly make use of optional ones.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) The main additions are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) get_caps
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)   This routine returns the list of audio formats supported. Querying the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)   codecs on a capture stream will return encoders, decoders will be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)   listed for playback streams.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108) get_codec_caps
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)   For each codec, this routine returns a list of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110)   capabilities. The intent is to make sure all the capabilities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)   correspond to valid settings, and to minimize the risks of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)   configuration failures. For example, for a complex codec such as AAC,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113)   the number of channels supported may depend on a specific profile. If
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)   the capabilities were exposed with a single descriptor, it may happen
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)   that a specific combination of profiles/channels/formats may not be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)   supported. Likewise, embedded DSPs have limited memory and cpu cycles,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)   it is likely that some implementations make the list of capabilities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)   dynamic and dependent on existing workloads. In addition to codec
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)   settings, this routine returns the minimum buffer size handled by the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)   implementation. This information can be a function of the DMA buffer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)   sizes, the number of bytes required to synchronize, etc, and can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)   used by userspace to define how much needs to be written in the ring
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123)   buffer before playback can start.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) set_params
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)   This routine sets the configuration chosen for a specific codec. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)   most important field in the parameters is the codec type; in most
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)   cases decoders will ignore other fields, while encoders will strictly
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)   comply to the settings
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) get_params
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)   This routines returns the actual settings used by the DSP. Changes to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133)   the settings should remain the exception.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135) get_timestamp
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)   The timestamp becomes a multiple field structure. It lists the number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)   of bytes transferred, the number of samples processed and the number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)   of samples rendered/grabbed. All these values can be used to determine
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)   the average bitrate, figure out if the ring buffer needs to be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140)   refilled or the delay due to decoding/encoding/io on the DSP.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) Note that the list of codecs/profiles/modes was derived from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) OpenMAX AL specification instead of reinventing the wheel.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144) Modifications include:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) - Addition of FLAC and IEC formats
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) - Merge of encoder/decoder capabilities
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147) - Profiles/modes listed as bitmasks to make descriptors more compact
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) - Addition of set_params for decoders (missing in OpenMAX AL)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) - Addition of AMR/AMR-WB encoding modes (missing in OpenMAX AL)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) - Addition of format information for WMA
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) - Addition of encoding options when required (derived from OpenMAX IL)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) - Addition of rateControlSupported (missing in OpenMAX AL)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) State Machine
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) The compressed audio stream state machine is described below ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)                                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160)                                         |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)                                         |   OPEN   |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)                                         |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)                                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164)                                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)                                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)                                              | compr_set_params()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167)                                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)                                              v
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)          compr_free()                  +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)   +------------------------------------|          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)   |                                    |   SETUP  |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172)   |          +-------------------------|          |<-------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)   |          |       compr_write()     +----------+                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)   |          |                              ^                                |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)   |          |                              | compr_drain_notify()           |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)   |          |                              |        or                      |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177)   |          |                              |     compr_stop()               |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178)   |          |                              |                                |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179)   |          |                         +----------+                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)   |          |                         |          |                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181)   |          |                         |   DRAIN  |                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182)   |          |                         |          |                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183)   |          |                         +----------+                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184)   |          |                              ^                                |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)   |          |                              |                                |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186)   |          |                              | compr_drain()                  |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187)   |          |                              |                                |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)   |          v                              |                                |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189)   |    +----------+                    +----------+                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190)   |    |          |    compr_start()   |          |        compr_stop()      |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191)   |    | PREPARE  |------------------->|  RUNNING |--------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)   |    |          |                    |          |                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193)   |    +----------+                    +----------+                          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194)   |          |                            |    ^                             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195)   |          |compr_free()                |    |                             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196)   |          |              compr_pause() |    | compr_resume()              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197)   |          |                            |    |                             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198)   |          v                            v    |                             |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199)   |    +----------+                   +----------+                           |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200)   |    |          |                   |          |         compr_stop()      |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201)   +--->|   FREE   |                   |  PAUSE   |---------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202)        |          |                   |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203)        +----------+                   +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) Gapless Playback
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207) ================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) When playing thru an album, the decoders have the ability to skip the encoder
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) delay and padding and directly move from one track content to another. The end
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) user can perceive this as gapless playback as we don't have silence while
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) switching from one track to another
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) Also, there might be low-intensity noises due to encoding. Perfect gapless is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) difficult to reach with all types of compressed data, but works fine with most
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) music content. The decoder needs to know the encoder delay and encoder padding.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) So we need to pass this to DSP. This metadata is extracted from ID3/MP4 headers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) and are not present by default in the bitstream, hence the need for a new
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) interface to pass this information to the DSP. Also DSP and userspace needs to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) switch from one track to another and start using data for second track.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) The main additions are:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) set_metadata
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)   This routine sets the encoder delay and encoder padding. This can be used by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225)   decoder to strip the silence. This needs to be set before the data in the track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226)   is written.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) set_next_track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229)   This routine tells DSP that metadata and write operation sent after this would
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230)   correspond to subsequent track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) partial drain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233)   This is called when end of file is reached. The userspace can inform DSP that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234)   EOF is reached and now DSP can start skipping padding delay. Also next write
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)   data would belong to next track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) Sequence flow for gapless would be:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) - Open
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) - Get caps / codec caps
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240) - Set params
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) - Set metadata of the first track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) - Fill data of the first track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) - Trigger start
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) - User-space finished sending all,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) - Indicate next track data by sending set_next_track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) - Set metadata of the next track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) - then call partial_drain to flush most of buffer in DSP
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) - Fill data of the next track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) - DSP switches to second track
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) (note: order for partial_drain and write for next track can be reversed as well)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) Gapless Playback SM
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254) ===================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) For Gapless, we move from running state to partial drain and back, along
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) with setting of meta_data and signalling for next track ::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260)                                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261)                 compr_drain_notify()    |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262)               +------------------------>|  RUNNING |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263)               |                         |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264)               |                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265)               |                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266)               |                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267)               |                              | compr_next_track()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268)               |                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269)               |                              V
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270)               |                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271)               |                         |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272)               |                         |NEXT_TRACK|
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273)               |                         |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274)               |                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275)               |                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276)               |                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277)               |                              | compr_partial_drain()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278)               |                              |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279)               |                              V
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280)               |                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281)               |                         |          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282)               +------------------------ | PARTIAL_ |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283)                                         |  DRAIN   |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284)                                         +----------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286) Not supported
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287) =============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288) - Support for VoIP/circuit-switched calls is not the target of this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289)   API. Support for dynamic bit-rate changes would require a tight
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290)   coupling between the DSP and the host stack, limiting power savings.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292) - Packet-loss concealment is not supported. This would require an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293)   additional interface to let the decoder synthesize data when frames
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294)   are lost during transmission. This may be added in the future.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) - Volume control/routing is not handled by this API. Devices exposing a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297)   compressed data interface will be considered as regular ALSA devices;
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298)   volume changes and routing information will be provided with regular
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299)   ALSA kcontrols.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301) - Embedded audio effects. Such effects should be enabled in the same
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302)   manner, no matter if the input was PCM or compressed.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304) - multichannel IEC encoding. Unclear if this is required.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306) - Encoding/decoding acceleration is not supported as mentioned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307)   above. It is possible to route the output of a decoder to a capture
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308)   stream, or even implement transcoding capabilities. This routing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309)   would be enabled with ALSA kcontrols.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) - Audio policy/resource management. This API does not provide any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312)   hooks to query the utilization of the audio DSP, nor any preemption
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313)   mechanisms.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) - No notion of underrun/overrun. Since the bytes written are compressed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316)   in nature and data written/read doesn't translate directly to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317)   rendered output in time, this does not deal with underrun/overrun and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318)   maybe dealt in user-library
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) Credits
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322) =======
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323) - Mark Brown and Liam Girdwood for discussions on the need for this API
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324) - Harsha Priya for her work on intel_sst compressed API
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325) - Rakesh Ughreja for valuable feedback
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326) - Sing Nallasellan, Sikkandar Madar and Prasanna Samaga for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327)   demonstrating and quantifying the benefits of audio offload on a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328)   real platform.