^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) Buffer Sharing and Synchronization
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) ==================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) The dma-buf subsystem provides the framework for sharing buffers for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) hardware (DMA) access across multiple device drivers and subsystems, and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6) for synchronizing asynchronous hardware access.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) This is used, for example, by drm "prime" multi-GPU support, but is of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) course not limited to GPU use cases.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) The three main components of this are: (1) dma-buf, representing a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) sg_table and exposed to userspace as a file descriptor to allow passing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) between devices, (2) fence, which provides a mechanism to signal when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) one device has finished access, and (3) reservation, which manages the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15) shared or exclusive fence(s) associated with the buffer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) Shared DMA Buffers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) ------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) This document serves as a guide to device-driver writers on what is the dma-buf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) buffer sharing API, how to use it for exporting and using shared buffers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) Any device driver which wishes to be a part of DMA buffer sharing, can do so as
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24) either the 'exporter' of buffers, or the 'user' or 'importer' of buffers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) Say a driver A wants to use buffers created by driver B, then we call B as the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27) exporter, and A as buffer-user/importer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) The exporter
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) - implements and manages operations in :c:type:`struct dma_buf_ops
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) <dma_buf_ops>` for the buffer,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) - allows other users to share the buffer by using dma_buf sharing APIs,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34) - manages the details of buffer allocation, wrapped in a :c:type:`struct
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35) dma_buf <dma_buf>`,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) - decides about the actual backing storage where this allocation happens,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) - and takes care of any migration of scatterlist - for all (shared) users of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38) this buffer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) The buffer-user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) - is one of (many) sharing users of the buffer.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) - doesn't need to worry about how the buffer is allocated, or where.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) - and needs a mechanism to get access to the scatterlist that makes up this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45) buffer in memory, mapped into its own address space, so it can access the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) same area of memory. This interface is provided by :c:type:`struct
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) dma_buf_attachment <dma_buf_attachment>`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49) Any exporters or users of the dma-buf buffer sharing framework must have a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) 'select DMA_SHARED_BUFFER' in their respective Kconfigs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) Userspace Interface Notes
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) ~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55) Mostly a DMA buffer file descriptor is simply an opaque object for userspace,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) and hence the generic interface exposed is very minimal. There's a few things to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) consider though:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59) - Since kernel 3.12 the dma-buf FD supports the llseek system call, but only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) with offset=0 and whence=SEEK_END|SEEK_SET. SEEK_SET is supported to allow
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61) the usual size discover pattern size = SEEK_END(0); SEEK_SET(0). Every other
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) llseek operation will report -EINVAL.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) If llseek on dma-buf FDs isn't support the kernel will report -ESPIPE for all
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65) cases. Userspace can use this to detect support for discovering the dma-buf
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) size using llseek.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) - In order to avoid fd leaks on exec, the FD_CLOEXEC flag must be set
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69) on the file descriptor. This is not just a resource leak, but a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) potential security hole. It could give the newly exec'd application
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) access to buffers, via the leaked fd, to which it should otherwise
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) not be permitted access.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74) The problem with doing this via a separate fcntl() call, versus doing it
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) atomically when the fd is created, is that this is inherently racy in a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76) multi-threaded app[3]. The issue is made worse when it is library code
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) opening/creating the file descriptor, as the application may not even be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) aware of the fd's.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) To avoid this problem, userspace must have a way to request O_CLOEXEC
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81) flag be set when the dma-buf fd is created. So any API provided by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) the exporting driver to create a dmabuf fd must provide a way to let
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd().
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85) - Memory mapping the contents of the DMA buffer is also supported. See the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) discussion below on `CPU Access to DMA Buffer Objects`_ for the full details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) - The DMA buffer FD is also pollable, see `Implicit Fence Poll Support`_ below for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) details.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) Basic Operation and Device DMA Access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) .. kernel-doc:: drivers/dma-buf/dma-buf.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) :doc: dma buf device access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) CPU Access to DMA Buffer Objects
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100) .. kernel-doc:: drivers/dma-buf/dma-buf.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) :doc: cpu access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) Implicit Fence Poll Support
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104) ~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) .. kernel-doc:: drivers/dma-buf/dma-buf.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) :doc: implicit fence polling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) Kernel Functions and Structures Reference
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) .. kernel-doc:: drivers/dma-buf/dma-buf.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) :export:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) .. kernel-doc:: include/linux/dma-buf.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) :internal:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) Reservation Objects
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119) -------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121) .. kernel-doc:: drivers/dma-buf/dma-resv.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) :doc: Reservation Object Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) .. kernel-doc:: drivers/dma-buf/dma-resv.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) :export:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) .. kernel-doc:: include/linux/dma-resv.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) :internal:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) DMA Fences
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) .. kernel-doc:: drivers/dma-buf/dma-fence.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) :doc: DMA fences overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) DMA Fence Cross-Driver Contract
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) .. kernel-doc:: drivers/dma-buf/dma-fence.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140) :doc: fence cross-driver contract
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) DMA Fence Signalling Annotations
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) .. kernel-doc:: drivers/dma-buf/dma-fence.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) :doc: fence signalling annotation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148) DMA Fences Functions Reference
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) .. kernel-doc:: drivers/dma-buf/dma-fence.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) :export:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) .. kernel-doc:: include/linux/dma-fence.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) :internal:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) Seqno Hardware Fences
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) ~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160) .. kernel-doc:: include/linux/seqno-fence.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161) :internal:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) DMA Fence Array
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) ~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) .. kernel-doc:: drivers/dma-buf/dma-fence-array.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) :export:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) .. kernel-doc:: include/linux/dma-fence-array.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) :internal:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) DMA Fence uABI/Sync File
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) ~~~~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) .. kernel-doc:: drivers/dma-buf/sync_file.c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176) :export:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) .. kernel-doc:: include/linux/sync_file.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) :internal:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) Indefinite DMA Fences
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) ~~~~~~~~~~~~~~~~~~~~~
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) At various times &dma_fence with an indefinite time until dma_fence_wait()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) finishes have been proposed. Examples include:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) * Future fences, used in HWC1 to signal when a buffer isn't used by the display
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) any longer, and created with the screen update that makes the buffer visible.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) The time this fence completes is entirely under userspace's control.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) * Proxy fences, proposed to handle &drm_syncobj for which the fence has not yet
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) been set. Used to asynchronously delay command submission.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) * Userspace fences or gpu futexes, fine-grained locking within a command buffer
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) that userspace uses for synchronization across engines or with the CPU, which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) are then imported as a DMA fence for integration into existing winsys
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) protocols.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) * Long-running compute command buffers, while still using traditional end of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) batch DMA fences for memory management instead of context preemption DMA
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) fences which get reattached when the compute job is rescheduled.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) Common to all these schemes is that userspace controls the dependencies of these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) fences and controls when they fire. Mixing indefinite fences with normal
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) in-kernel DMA fences does not work, even when a fallback timeout is included to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) protect against malicious userspace:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) * Only the kernel knows about all DMA fence dependencies, userspace is not aware
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) of dependencies injected due to memory management or scheduler decisions.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211) * Only userspace knows about all dependencies in indefinite fences and when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) exactly they will complete, the kernel has no visibility.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) Furthermore the kernel has to be able to hold up userspace command submission
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) for memory management needs, which means we must support indefinite fences being
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216) dependent upon DMA fences. If the kernel also support indefinite fences in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217) kernel like a DMA fence, like any of the above proposal would, there is the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) potential for deadlocks.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) .. kernel-render:: DOT
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) :alt: Indefinite Fencing Dependency Cycle
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) :caption: Indefinite Fencing Dependency Cycle
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) digraph "Fencing Cycle" {
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) node [shape=box bgcolor=grey style=filled]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) kernel [label="Kernel DMA Fences"]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227) userspace [label="userspace controlled fences"]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) kernel -> userspace [label="memory management"]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) userspace -> kernel [label="Future fence, fence proxy, ..."]
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) { rank=same; kernel userspace }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) }
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) This means that the kernel might accidentally create deadlocks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) through memory management dependencies which userspace is unaware of, which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) randomly hangs workloads until the timeout kicks in. Workloads, which from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) userspace's perspective, do not contain a deadlock. In such a mixed fencing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) architecture there is no single entity with knowledge of all dependencies.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) Thefore preventing such deadlocks from within the kernel is not possible.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) The only solution to avoid dependencies loops is by not allowing indefinite
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) fences in the kernel. This means:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) * No future fences, proxy fences or userspace fences imported as DMA fences,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) with or without a timeout.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) * No DMA fences that signal end of batchbuffer for command submission where
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248) userspace is allowed to use userspace fencing or long running compute
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249) workloads. This also means no implicit fencing for shared buffers in these
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) cases.
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags