^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 1) Direct Access for files
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 2) -----------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 3)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 4) Motivation
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 5) ----------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 6)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 7) The page cache is usually used to buffer reads and writes to files.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 8) It is also used to provide the pages which are mapped into userspace
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 9) by a call to mmap.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 10)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 11) For block devices that are memory-like, the page cache pages would be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 12) unnecessary copies of the original storage. The DAX code removes the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 13) extra copy by performing reads and writes directly to the storage device.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 14) For file mappings, the storage device is mapped directly into userspace.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 15)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 16)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 17) Usage
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 18) -----
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 19)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 20) If you have a block device which supports DAX, you can make a filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 21) on it as usual. The DAX code currently only supports files with a block
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 22) size equal to your kernel's PAGE_SIZE, so you may need to specify a block
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 23) size when creating the filesystem.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 24)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 25) Currently 3 filesystems support DAX: ext2, ext4 and xfs. Enabling DAX on them
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 26) is different.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 27)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 28) Enabling DAX on ext2
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 29) -----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 30)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 31) When mounting the filesystem, use the "-o dax" option on the command line or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 32) add 'dax' to the options in /etc/fstab. This works to enable DAX on all files
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 33) within the filesystem. It is equivalent to the '-o dax=always' behavior below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 34)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 35)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 36) Enabling DAX on xfs and ext4
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 37) ----------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 38)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 39) Summary
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 40) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 41)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 42) 1. There exists an in-kernel file access mode flag S_DAX that corresponds to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 43) the statx flag STATX_ATTR_DAX. See the manpage for statx(2) for details
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 44) about this access mode.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 45)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 46) 2. There exists a persistent flag FS_XFLAG_DAX that can be applied to regular
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 47) files and directories. This advisory flag can be set or cleared at any
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 48) time, but doing so does not immediately affect the S_DAX state.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 49)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 50) 3. If the persistent FS_XFLAG_DAX flag is set on a directory, this flag will
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 51) be inherited by all regular files and subdirectories that are subsequently
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 52) created in this directory. Files and subdirectories that exist at the time
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 53) this flag is set or cleared on the parent directory are not modified by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 54) this modification of the parent directory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 55)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 56) 4. There exist dax mount options which can override FS_XFLAG_DAX in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 57) setting of the S_DAX flag. Given underlying storage which supports DAX the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 58) following hold:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 59)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 60) "-o dax=inode" means "follow FS_XFLAG_DAX" and is the default.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 61)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 62) "-o dax=never" means "never set S_DAX, ignore FS_XFLAG_DAX."
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 63)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 64) "-o dax=always" means "always set S_DAX ignore FS_XFLAG_DAX."
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 65)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 66) "-o dax" is a legacy option which is an alias for "dax=always".
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 67) This may be removed in the future so "-o dax=always" is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 68) the preferred method for specifying this behavior.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 69)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 70) NOTE: Modifications to and the inheritance behavior of FS_XFLAG_DAX remain
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 71) the same even when the filesystem is mounted with a dax option. However,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 72) in-core inode state (S_DAX) will be overridden until the filesystem is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 73) remounted with dax=inode and the inode is evicted from kernel memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 74)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 75) 5. The S_DAX policy can be changed via:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 76)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 77) a) Setting the parent directory FS_XFLAG_DAX as needed before files are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 78) created
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 79)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 80) b) Setting the appropriate dax="foo" mount option
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 81)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 82) c) Changing the FS_XFLAG_DAX flag on existing regular files and
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 83) directories. This has runtime constraints and limitations that are
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 84) described in 6) below.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 85)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 86) 6. When changing the S_DAX policy via toggling the persistent FS_XFLAG_DAX flag,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 87) the change in behaviour for existing regular files may not occur
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 88) immediately. If the change must take effect immediately, the administrator
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 89) needs to:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 90)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 91) a) stop the application so there are no active references to the data set
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 92) the policy change will affect
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 93)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 94) b) evict the data set from kernel caches so it will be re-instantiated when
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 95) the application is restarted. This can be achieved by:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 96)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 97) i. drop-caches
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 98) ii. a filesystem unmount and mount cycle
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 99) iii. a system reboot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 100)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 101)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 102) Details
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 103) -------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 104)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 105) There are 2 per-file dax flags. One is a persistent inode setting (FS_XFLAG_DAX)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 106) and the other is a volatile flag indicating the active state of the feature
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 107) (S_DAX).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 108)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 109) FS_XFLAG_DAX is preserved within the filesystem. This persistent config
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 110) setting can be set, cleared and/or queried using the FS_IOC_FS[GS]ETXATTR ioctl
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 111) (see ioctl_xfs_fsgetxattr(2)) or an utility such as 'xfs_io'.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 112)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 113) New files and directories automatically inherit FS_XFLAG_DAX from
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 114) their parent directory _when_ _created_. Therefore, setting FS_XFLAG_DAX at
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 115) directory creation time can be used to set a default behavior for an entire
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 116) sub-tree.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 117)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 118) To clarify inheritance, here are 3 examples:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 119)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 120) Example A:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 121)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 122) mkdir -p a/b/c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 123) xfs_io -c 'chattr +x' a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 124) mkdir a/b/c/d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 125) mkdir a/e
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 126)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 127) dax: a,e
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 128) no dax: b,c,d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 129)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 130) Example B:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 131)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 132) mkdir a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 133) xfs_io -c 'chattr +x' a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 134) mkdir -p a/b/c/d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 135)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 136) dax: a,b,c,d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 137) no dax:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 138)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 139) Example C:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 140)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 141) mkdir -p a/b/c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 142) xfs_io -c 'chattr +x' c
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 143) mkdir a/b/c/d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 144)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 145) dax: c,d
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 146) no dax: a,b
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 147)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 148)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 149) The current enabled state (S_DAX) is set when a file inode is instantiated in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 150) memory by the kernel. It is set based on the underlying media support, the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 151) value of FS_XFLAG_DAX and the filesystem's dax mount option.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 152)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 153) statx can be used to query S_DAX. NOTE that only regular files will ever have
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 154) S_DAX set and therefore statx will never indicate that S_DAX is set on
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 155) directories.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 156)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 157) Setting the FS_XFLAG_DAX flag (specifically or through inheritance) occurs even
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 158) if the underlying media does not support dax and/or the filesystem is
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 159) overridden with a mount option.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 160)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 161)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 162)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 163) Implementation Tips for Block Driver Writers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 164) --------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 165)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 166) To support DAX in your block driver, implement the 'direct_access'
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 167) block device operation. It is used to translate the sector number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 168) (expressed in units of 512-byte sectors) to a page frame number (pfn)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 169) that identifies the physical page for the memory. It also returns a
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 170) kernel virtual address that can be used to access the memory.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 171)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 172) The direct_access method takes a 'size' parameter that indicates the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 173) number of bytes being requested. The function should return the number
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 174) of bytes that can be contiguously accessed at that offset. It may also
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 175) return a negative errno if an error occurs.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 176)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 177) In order to support this method, the storage must be byte-accessible by
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 178) the CPU at all times. If your device uses paging techniques to expose
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 179) a large amount of memory through a smaller window, then you cannot
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 180) implement direct_access. Equally, if your device can occasionally
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 181) stall the CPU for an extended period, you should also not attempt to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 182) implement direct_access.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 183)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 184) These block devices may be used for inspiration:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 185) - brd: RAM backed block device driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 186) - dcssblk: s390 dcss block device driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 187) - pmem: NVDIMM persistent memory driver
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 188)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 189)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 190) Implementation Tips for Filesystem Writers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 191) ------------------------------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 192)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 193) Filesystem support consists of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 194) - adding support to mark inodes as being DAX by setting the S_DAX flag in
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 195) i_flags
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 196) - implementing ->read_iter and ->write_iter operations which use dax_iomap_rw()
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 197) when inode has S_DAX flag set
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 198) - implementing an mmap file operation for DAX files which sets the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 199) VM_MIXEDMAP and VM_HUGEPAGE flags on the VMA, and setting the vm_ops to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 200) include handlers for fault, pmd_fault, page_mkwrite, pfn_mkwrite. These
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 201) handlers should probably call dax_iomap_fault() passing the appropriate
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 202) fault size and iomap operations.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 203) - calling iomap_zero_range() passing appropriate iomap operations instead of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 204) block_truncate_page() for DAX files
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 205) - ensuring that there is sufficient locking between reads, writes,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 206) truncates and page faults
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 207)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 208) The iomap handlers for allocating blocks must make sure that allocated blocks
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 209) are zeroed out and converted to written extents before being returned to avoid
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 210) exposure of uninitialized data through mmap.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 211)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 212) These filesystems may be used for inspiration:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 213) - ext2: see Documentation/filesystems/ext2.rst
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 214) - ext4: see Documentation/filesystems/ext4/
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 215) - xfs: see Documentation/admin-guide/xfs.rst
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 216)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 217)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 218) Handling Media Errors
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 219) ---------------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 220)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 221) The libnvdimm subsystem stores a record of known media error locations for
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 222) each pmem block device (in gendisk->badblocks). If we fault at such location,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 223) or one with a latent error not yet discovered, the application can expect
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 224) to receive a SIGBUS. Libnvdimm also allows clearing of these errors by simply
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 225) writing the affected sectors (through the pmem driver, and if the underlying
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 226) NVDIMM supports the clear_poison DSM defined by ACPI).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 227)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 228) Since DAX IO normally doesn't go through the driver/bio path, applications or
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 229) sysadmins have an option to restore the lost data from a prior backup/inbuilt
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 230) redundancy in the following ways:
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 231)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 232) 1. Delete the affected file, and restore from a backup (sysadmin route):
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 233) This will free the filesystem blocks that were being used by the file,
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 234) and the next time they're allocated, they will be zeroed first, which
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 235) happens through the driver, and will clear bad sectors.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 236)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 237) 2. Truncate or hole-punch the part of the file that has a bad-block (at least
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 238) an entire aligned sector has to be hole-punched, but not necessarily an
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 239) entire filesystem block).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 240)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 241) These are the two basic paths that allow DAX filesystems to continue operating
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 242) in the presence of media errors. More robust error recovery mechanisms can be
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 243) built on top of this in the future, for example, involving redundancy/mirroring
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 244) provided at the block layer through DM, or additionally, at the filesystem
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 245) level. These would have to rely on the above two tenets, that error clearing
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 246) can happen either by sending an IO through the driver, or zeroing (also through
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 247) the driver).
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 248)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 249)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 250) Shortcomings
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 251) ------------
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 252)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 253) Even if the kernel or its modules are stored on a filesystem that supports
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 254) DAX on a block device that supports DAX, they will still be copied into RAM.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 255)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 256) The DAX code does not work correctly on architectures which have virtually
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 257) mapped caches such as ARM, MIPS and SPARC.
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 258)
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 259) Calling get_user_pages() on a range of user memory that has been mmaped
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 260) from a DAX file will fail when there are no 'struct page' to describe
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 261) those pages. This problem has been addressed in some device drivers
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 262) by adding optional struct page support for pages under the control of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 263) the driver (see CONFIG_NVDIMM_PFN in drivers/nvdimm for an example of
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 264) how to do this). In the non struct page cases O_DIRECT reads/writes to
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 265) those memory ranges from a non-DAX file will fail (note that O_DIRECT
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 266) reads/writes _of a DAX file_ do work, it is the memory that is being
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 267) accessed that is key here). Other things that will not work in the
^8f3ce5b39 (kx 2023-10-28 12:00:06 +0300 268) non struct page case include RDMA, sendfile() and splice().
Orange Pi5 kernel
Deprecated Linux kernel 5.10.110 for OrangePi 5/5B/5+ boards
3 Commits
0 Branches
0 Tags