Orange Pi5 kernel

^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   1) ================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   2) Coherent Accelerator (CXL) Flash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   3) ================================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   4) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   5) Introduction
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   6) ============
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   7) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   8)     The IBM Power architecture provides support for CAPI (Coherent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300   9)     Accelerator Power Interface), which is available to certain PCIe slots
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  10)     on Power 8 systems. CAPI can be thought of as a special tunneling
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  11)     protocol through PCIe that allow PCIe adapters to look like special
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  12)     purpose co-processors which can read or write an application's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  13)     memory and generate page faults. As a result, the host interface to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  14)     an adapter running in CAPI mode does not require the data buffers to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  15)     be mapped to the device's memory (IOMMU bypass) nor does it require
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  16)     memory to be pinned.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  17) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  18)     On Linux, Coherent Accelerator (CXL) kernel services present CAPI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  19)     devices as a PCI device by implementing a virtual PCI host bridge.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  20)     This abstraction simplifies the infrastructure and programming
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  21)     model, allowing for drivers to look similar to other native PCI
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  22)     device drivers.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  23) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  24)     CXL provides a mechanism by which user space applications can
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  25)     directly talk to a device (network or storage) bypassing the typical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  26)     kernel/device driver stack. The CXL Flash Adapter Driver enables a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  27)     user space application direct access to Flash storage.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  28) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  29)     The CXL Flash Adapter Driver is a kernel module that sits in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  30)     SCSI stack as a low level device driver (below the SCSI disk and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  31)     protocol drivers) for the IBM CXL Flash Adapter. This driver is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  32)     responsible for the initialization of the adapter, setting up the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  33)     special path for user space access, and performing error recovery. It
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  34)     communicates directly the Flash Accelerator Functional Unit (AFU)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  35)     as described in Documentation/powerpc/cxl.rst.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  36) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  37)     The cxlflash driver supports two, mutually exclusive, modes of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  38)     operation at the device (LUN) level:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  39) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  40)         - Any flash device (LUN) can be configured to be accessed as a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  41)           regular disk device (i.e.: /dev/sdc). This is the default mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  42) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  43)         - Any flash device (LUN) can be configured to be accessed from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  44)           user space with a special block library. This mode further
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  45)           specifies the means of accessing the device and provides for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  46)           either raw access to the entire LUN (referred to as direct
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  47)           or physical LUN access) or access to a kernel/AFU-mediated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  48)           partition of the LUN (referred to as virtual LUN access). The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  49)           segmentation of a disk device into virtual LUNs is assisted
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  50)           by special translation services provided by the Flash AFU.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  51) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  52) Overview
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  53) ========
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  54) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  55)     The Coherent Accelerator Interface Architecture (CAIA) introduces a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  56)     concept of a master context. A master typically has special privileges
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  57)     granted to it by the kernel or hypervisor allowing it to perform AFU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  58)     wide management and control. The master may or may not be involved
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  59)     directly in each user I/O, but at the minimum is involved in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  60)     initial setup before the user application is allowed to send requests
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  61)     directly to the AFU.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  62) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  63)     The CXL Flash Adapter Driver establishes a master context with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  64)     AFU. It uses memory mapped I/O (MMIO) for this control and setup. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  65)     Adapter Problem Space Memory Map looks like this::
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  66) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  67)                      +-------------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  68)                      |    512 * 64 KB User MMIO      |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  69)                      |        (per context)          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  70)                      |       User Accessible         |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  71)                      +-------------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  72)                      |    512 * 128 B per context    |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  73)                      |    Provisioning and Control   |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  74)                      |   Trusted Process accessible  |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  75)                      +-------------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  76)                      |         64 KB Global          |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  77)                      |   Trusted Process accessible  |
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  78)                      +-------------------------------+
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  79) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  80)     This driver configures itself into the SCSI software stack as an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  81)     adapter driver. The driver is the only entity that is considered a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  82)     Trusted Process to program the Provisioning and Control and Global
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  83)     areas in the MMIO Space shown above.  The master context driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  84)     discovers all LUNs attached to the CXL Flash adapter and instantiates
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  85)     scsi block devices (/dev/sdb, /dev/sdc etc.) for each unique LUN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  86)     seen from each path.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  87) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  88)     Once these scsi block devices are instantiated, an application
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  89)     written to a specification provided by the block library may get
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  90)     access to the Flash from user space (without requiring a system call).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  91) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  92)     This master context driver also provides a series of ioctls for this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  93)     block library to enable this user space access.  The driver supports
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  94)     two modes for accessing the block device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  95) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  96)     The first mode is called a virtual mode. In this mode a single scsi
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  97)     block device (/dev/sdb) may be carved up into any number of distinct
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  98)     virtual LUNs. The virtual LUNs may be resized as long as the sum of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300  99)     the sizes of all the virtual LUNs, along with the meta-data associated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)     with it does not exceed the physical capacity.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102)     The second mode is called the physical mode. In this mode a single
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103)     block device (/dev/sdb) may be opened directly by the block library
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)     and the entire space for the LUN is available to the application.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106)     Only the physical mode provides persistence of the data.  i.e. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107)     data written to the block device will survive application exit and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)     restart and also reboot. The virtual LUNs do not persist (i.e. do
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109)     not survive after the application terminates or the system reboots).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112) Block library API
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) =================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115)     Applications intending to get access to the CXL Flash from user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116)     space should use the block library, as it abstracts the details of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)     interfacing directly with the cxlflash driver that are necessary for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118)     performing administrative actions (i.e.: setup, tear down, resize).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)     The block library can be thought of as a 'user' of services,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120)     implemented as IOCTLs, that are provided by the cxlflash driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)     specifically for devices (LUNs) operating in user space access
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122)     mode. While it is not a requirement that applications understand
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123)     the interface between the block library and the cxlflash driver,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124)     a high-level overview of each supported service (IOCTL) is provided
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125)     below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127)     The block library can be found on GitHub:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128)     http://github.com/open-power/capiflash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131) CXL Flash Driver LUN IOCTLs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) ===========================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134)     Users, such as the block library, that wish to interface with a flash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)     device (LUN) via user space access need to use the services provided
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136)     by the cxlflash driver. As these services are implemented as ioctls,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137)     a file descriptor handle must first be obtained in order to establish
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)     the communication channel between a user and the kernel.  This file
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139)     descriptor is obtained by opening the device special file associated
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140)     with the scsi disk device (/dev/sdb) that was created during LUN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141)     discovery. As per the location of the cxlflash driver within the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142)     SCSI protocol stack, this open is actually not seen by the cxlflash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143)     driver. Upon successful open, the user receives a file descriptor
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)     (herein referred to as fd1) that should be used for issuing the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145)     subsequent ioctls listed below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)     The structure definitions for these IOCTLs are available in:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)     uapi/scsi/cxlflash_ioctl.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) DK_CXLFLASH_ATTACH
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) ------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153)     This ioctl obtains, initializes, and starts a context using the CXL
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154)     kernel services. These services specify a context id (u16) by which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155)     to uniquely identify the context and its allocated resources. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)     services additionally provide a second file descriptor (herein
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157)     referred to as fd2) that is used by the block library to initiate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158)     memory mapped I/O (via mmap()) to the CXL flash device and poll for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159)     completion events. This file descriptor is intentionally installed by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160)     this driver and not the CXL kernel services to allow for intermediary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)     notification and access in the event of a non-user-initiated close(),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)     such as a killed process. This design point is described in further
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163)     detail in the description for the DK_CXLFLASH_DETACH ioctl.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)     There are a few important aspects regarding the "tokens" (context id
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166)     and fd2) that are provided back to the user:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168)         - These tokens are only valid for the process under which they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169)           were created. The child of a forked process cannot continue
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170)           to use the context id or file descriptor created by its parent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)           (see DK_CXLFLASH_VLUN_CLONE for further details).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173)         - These tokens are only valid for the lifetime of the context and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174)           the process under which they were created. Once either is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175)           destroyed, the tokens are to be considered stale and subsequent
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)           usage will result in errors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) 	- A valid adapter file descriptor (fd2 >= 0) is only returned on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) 	  the initial attach for a context. Subsequent attaches to an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) 	  existing context (DK_CXLFLASH_ATTACH_REUSE_CONTEXT flag present)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) 	  do not provide the adapter file descriptor as it was previously
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) 	  made known to the application.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184)         - When a context is no longer needed, the user shall detach from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185)           the context via the DK_CXLFLASH_DETACH ioctl. When this ioctl
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) 	  returns with a valid adapter file descriptor and the return flag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) 	  DK_CXLFLASH_APP_CLOSE_ADAP_FD is present, the application _must_
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188) 	  close the adapter file descriptor following a successful detach.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) 	- When this ioctl returns with a valid fd2 and the return flag
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) 	  DK_CXLFLASH_APP_CLOSE_ADAP_FD is present, the application _must_
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192) 	  close fd2 in the following circumstances:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) 		+ Following a successful detach of the last user of the context
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) 		+ Following a successful recovery on the context's original fd2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) 		+ In the child process of a fork(), following a clone ioctl,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) 		  on the fd2 associated with the source context
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199)         - At any time, a close on fd2 will invalidate the tokens. Applications
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) 	  should exercise caution to only close fd2 when appropriate (outlined
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) 	  in the previous bullet) to avoid premature loss of I/O.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) DK_CXLFLASH_USER_DIRECT
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) -----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205)     This ioctl is responsible for transitioning the LUN to direct
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206)     (physical) mode access and configuring the AFU for direct access from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207)     user space on a per-context basis. Additionally, the block size and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208)     last logical block address (LBA) are returned to the user.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210)     As mentioned previously, when operating in user space access mode,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211)     LUNs may be accessed in whole or in part. Only one mode is allowed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212)     at a time and if one mode is active (outstanding references exist),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213)     requests to use the LUN in a different mode are denied.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215)     The AFU is configured for direct access from user space by adding an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216)     entry to the AFU's resource handle table. The index of the entry is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217)     treated as a resource handle that is returned to the user. The user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218)     is then able to use the handle to reference the LUN during I/O.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220) DK_CXLFLASH_USER_VIRTUAL
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) ------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222)     This ioctl is responsible for transitioning the LUN to virtual mode
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223)     of access and configuring the AFU for virtual access from user space
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224)     on a per-context basis. Additionally, the block size and last logical
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225)     block address (LBA) are returned to the user.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227)     As mentioned previously, when operating in user space access mode,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228)     LUNs may be accessed in whole or in part. Only one mode is allowed
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229)     at a time and if one mode is active (outstanding references exist),
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230)     requests to use the LUN in a different mode are denied.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232)     The AFU is configured for virtual access from user space by adding
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233)     an entry to the AFU's resource handle table. The index of the entry
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234)     is treated as a resource handle that is returned to the user. The
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235)     user is then able to use the handle to reference the LUN during I/O.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237)     By default, the virtual LUN is created with a size of 0. The user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238)     would need to use the DK_CXLFLASH_VLUN_RESIZE ioctl to adjust the grow
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239)     the virtual LUN to a desired size. To avoid having to perform this
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240)     resize for the initial creation of the virtual LUN, the user has the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241)     option of specifying a size as part of the DK_CXLFLASH_USER_VIRTUAL
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242)     ioctl, such that when success is returned to the user, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243)     resource handle that is provided is already referencing provisioned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244)     storage. This is reflected by the last LBA being a non-zero value.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246)     When a LUN is accessible from more than one port, this ioctl will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247)     return with the DK_CXLFLASH_ALL_PORTS_ACTIVE return flag set. This
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248)     provides the user with a hint that I/O can be retried in the event
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249)     of an I/O error as the LUN can be reached over multiple paths.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) DK_CXLFLASH_VLUN_RESIZE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252) -----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253)     This ioctl is responsible for resizing a previously created virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254)     LUN and will fail if invoked upon a LUN that is not in virtual
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255)     mode. Upon success, an updated last LBA is returned to the user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256)     indicating the new size of the virtual LUN associated with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257)     resource handle.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259)     The partitioning of virtual LUNs is jointly mediated by the cxlflash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260)     driver and the AFU. An allocation table is kept for each LUN that is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261)     operating in the virtual mode and used to program a LUN translation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262)     table that the AFU references when provided with a resource handle.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264)     This ioctl can return -EAGAIN if an AFU sync operation takes too long.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265)     In addition to returning a failure to user, cxlflash will also schedule
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266)     an asynchronous AFU reset. Should the user choose to retry the operation,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267)     it is expected to succeed. If this ioctl fails with -EAGAIN, the user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268)     can either retry the operation or treat it as a failure.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 269) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 270) DK_CXLFLASH_RELEASE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 271) -------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 272)     This ioctl is responsible for releasing a previously obtained
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 273)     reference to either a physical or virtual LUN. This can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 274)     thought of as the inverse of the DK_CXLFLASH_USER_DIRECT or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 275)     DK_CXLFLASH_USER_VIRTUAL ioctls. Upon success, the resource handle
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 276)     is no longer valid and the entry in the resource handle table is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 277)     made available to be used again.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 278) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 279)     As part of the release process for virtual LUNs, the virtual LUN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 280)     is first resized to 0 to clear out and free the translation tables
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 281)     associated with the virtual LUN reference.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 282) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 283) DK_CXLFLASH_DETACH
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 284) ------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 285)     This ioctl is responsible for unregistering a context with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 286)     cxlflash driver and release outstanding resources that were
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 287)     not explicitly released via the DK_CXLFLASH_RELEASE ioctl. Upon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 288)     success, all "tokens" which had been provided to the user from the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 289)     DK_CXLFLASH_ATTACH onward are no longer valid.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 290) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 291)     When the DK_CXLFLASH_APP_CLOSE_ADAP_FD flag was returned on a successful
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 292)     attach, the application _must_ close the fd2 associated with the context
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 293)     following the detach of the final user of the context.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 294) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 295) DK_CXLFLASH_VLUN_CLONE
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 296) ----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 297)     This ioctl is responsible for cloning a previously created
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 298)     context to a more recently created context. It exists solely to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 299)     support maintaining user space access to storage after a process
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 300)     forks. Upon success, the child process (which invoked the ioctl)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 301)     will have access to the same LUNs via the same resource handle(s)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 302)     as the parent, but under a different context.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 303) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 304)     Context sharing across processes is not supported with CXL and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 305)     therefore each fork must be met with establishing a new context
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 306)     for the child process. This ioctl simplifies the state management
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 307)     and playback required by a user in such a scenario. When a process
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 308)     forks, child process can clone the parents context by first creating
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 309)     a context (via DK_CXLFLASH_ATTACH) and then using this ioctl to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 310)     perform the clone from the parent to the child.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 311) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 312)     The clone itself is fairly simple. The resource handle and lun
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 313)     translation tables are copied from the parent context to the child's
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 314)     and then synced with the AFU.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 315) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 316)     When the DK_CXLFLASH_APP_CLOSE_ADAP_FD flag was returned on a successful
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 317)     attach, the application _must_ close the fd2 associated with the source
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 318)     context (still resident/accessible in the parent process) following the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 319)     clone. This is to avoid a stale entry in the file descriptor table of the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 320)     child process.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 321) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 322)     This ioctl can return -EAGAIN if an AFU sync operation takes too long.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 323)     In addition to returning a failure to user, cxlflash will also schedule
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 324)     an asynchronous AFU reset. Should the user choose to retry the operation,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 325)     it is expected to succeed. If this ioctl fails with -EAGAIN, the user
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 326)     can either retry the operation or treat it as a failure.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 327) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 328) DK_CXLFLASH_VERIFY
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 329) ------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 330)     This ioctl is used to detect various changes such as the capacity of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 331)     the disk changing, the number of LUNs visible changing, etc. In cases
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 332)     where the changes affect the application (such as a LUN resize), the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 333)     cxlflash driver will report the changed state to the application.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 334) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 335)     The user calls in when they want to validate that a LUN hasn't been
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 336)     changed in response to a check condition. As the user is operating out
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 337)     of band from the kernel, they will see these types of events without
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 338)     the kernel's knowledge. When encountered, the user's architected
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 339)     behavior is to call in to this ioctl, indicating what they want to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 340)     verify and passing along any appropriate information. For now, only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 341)     verifying a LUN change (ie: size different) with sense data is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 342)     supported.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 343) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 344) DK_CXLFLASH_RECOVER_AFU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 345) -----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 346)     This ioctl is used to drive recovery (if such an action is warranted)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 347)     of a specified user context. Any state associated with the user context
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 348)     is re-established upon successful recovery.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 349) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 350)     User contexts are put into an error condition when the device needs to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 351)     be reset or is terminating. Users are notified of this error condition
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 352)     by seeing all 0xF's on an MMIO read. Upon encountering this, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 353)     architected behavior for a user is to call into this ioctl to recover
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 354)     their context. A user may also call into this ioctl at any time to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 355)     check if the device is operating normally. If a failure is returned
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 356)     from this ioctl, the user is expected to gracefully clean up their
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 357)     context via release/detach ioctls. Until they do, the context they
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 358)     hold is not relinquished. The user may also optionally exit the process
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 359)     at which time the context/resources they held will be freed as part of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 360)     the release fop.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 361) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 362)     When the DK_CXLFLASH_APP_CLOSE_ADAP_FD flag was returned on a successful
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 363)     attach, the application _must_ unmap and close the fd2 associated with the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 364)     original context following this ioctl returning success and indicating that
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 365)     the context was recovered (DK_CXLFLASH_RECOVER_AFU_CONTEXT_RESET).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 366) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 367) DK_CXLFLASH_MANAGE_LUN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 368) ----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 369)     This ioctl is used to switch a LUN from a mode where it is available
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 370)     for file-system access (legacy), to a mode where it is set aside for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 371)     exclusive user space access (superpipe). In case a LUN is visible
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 372)     across multiple ports and adapters, this ioctl is used to uniquely
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 373)     identify each LUN by its World Wide Node Name (WWNN).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 374) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 375) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 376) CXL Flash Driver Host IOCTLs
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 377) ============================
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 378) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 379)     Each host adapter instance that is supported by the cxlflash driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 380)     has a special character device associated with it to enable a set of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 381)     host management function. These character devices are hosted in a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 382)     class dedicated for cxlflash and can be accessed via `/dev/cxlflash/*`.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 383) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 384)     Applications can be written to perform various functions using the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 385)     host ioctl APIs below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 386) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 387)     The structure definitions for these IOCTLs are available in:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 388)     uapi/scsi/cxlflash_ioctl.h
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 389) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 390) HT_CXLFLASH_LUN_PROVISION
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 391) -------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 392)     This ioctl is used to create and delete persistent LUNs on cxlflash
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 393)     devices that lack an external LUN management interface. It is only
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 394)     valid when used with AFUs that support the LUN provision capability.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 395) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 396)     When sufficient space is available, LUNs can be created by specifying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 397)     the target port to host the LUN and a desired size in 4K blocks. Upon
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 398)     success, the LUN ID and WWID of the created LUN will be returned and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 399)     the SCSI bus can be scanned to detect the change in LUN topology. Note
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 400)     that partial allocations are not supported. Should a creation fail due
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 401)     to a space issue, the target port can be queried for its current LUN
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 402)     geometry.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 403) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 404)     To remove a LUN, the device must first be disassociated from the Linux
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 405)     SCSI subsystem. The LUN deletion can then be initiated by specifying a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 406)     target port and LUN ID. Upon success, the LUN geometry associated with
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 407)     the port will be updated to reflect new number of provisioned LUNs and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 408)     available capacity.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 409) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 410)     To query the LUN geometry of a port, the target port is specified and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 411)     upon success, the following information is presented:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 412) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 413)         - Maximum number of provisioned LUNs allowed for the port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 414)         - Current number of provisioned LUNs for the port
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 415)         - Maximum total capacity of provisioned LUNs for the port (4K blocks)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 416)         - Current total capacity of provisioned LUNs for the port (4K blocks)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 417) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 418)     With this information, the number of available LUNs and capacity can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 419)     can be calculated.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 420) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 421) HT_CXLFLASH_AFU_DEBUG
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 422) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 423)     This ioctl is used to debug AFUs by supporting a command pass-through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 424)     interface. It is only valid when used with AFUs that support the AFU
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 425)     debug capability.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 426) 
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 427)     With exception of buffer management, AFU debug commands are opaque to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 428)     cxlflash and treated as pass-through. For debug commands that do require
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 429)     data transfer, the user supplies an adequately sized data buffer and must
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 430)     specify the data transfer direction with respect to the host. There is a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 431)     maximum transfer size of 256K imposed. Note that partial read completions
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 432)     are not supported - when errors are experienced with a host read data
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 433)     transfer, the data buffer is not copied back to the user.